Sep 16 00:43:04.033604 ip-10-0-130-207 systemd[1]: Starting Kubernetes Kubelet...
Sep 16 00:43:04.435384 ip-10-0-130-207 kubenswrapper[2556]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 16 00:43:04.435384 ip-10-0-130-207 kubenswrapper[2556]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 16 00:43:04.435384 ip-10-0-130-207 kubenswrapper[2556]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 16 00:43:04.435384 ip-10-0-130-207 kubenswrapper[2556]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 16 00:43:04.435384 ip-10-0-130-207 kubenswrapper[2556]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 16 00:43:04.437962 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.437892    2556 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 16 00:43:04.442443 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442428    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 00:43:04.442443 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442443    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 00:43:04.442526 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442449    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 00:43:04.442526 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442452    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 00:43:04.442526 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442454    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 00:43:04.442526 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442457    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 00:43:04.442526 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442460    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 00:43:04.442526 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442463    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 00:43:04.442526 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442466    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 00:43:04.442526 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442469    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 00:43:04.442526 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442471    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 00:43:04.442526 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442474    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 00:43:04.442526 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442477    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 00:43:04.442526 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442480    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 00:43:04.442526 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442482    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 00:43:04.442526 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442485    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 00:43:04.442526 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442487    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 00:43:04.442526 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442490    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 00:43:04.442526 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442493    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 00:43:04.442526 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442496    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 00:43:04.442526 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442503    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 00:43:04.443604 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442507    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 00:43:04.443604 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442510    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 00:43:04.443604 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442516    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 00:43:04.443604 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442518    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 00:43:04.443604 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442521    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 00:43:04.443604 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442523    2556 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 00:43:04.443604 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442527    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 00:43:04.443604 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442530    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 00:43:04.443604 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442532    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 00:43:04.443604 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442535    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 00:43:04.443604 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442538    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 00:43:04.443604 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442540    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 00:43:04.443604 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442543    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 00:43:04.443604 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442545    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 00:43:04.443604 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442548    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 00:43:04.443604 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442550    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 00:43:04.443604 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442552    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 00:43:04.443604 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442555    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 00:43:04.443604 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442559    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 00:43:04.444065 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442563    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 00:43:04.444065 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442566    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 00:43:04.444065 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442569    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 00:43:04.444065 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442572    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 00:43:04.444065 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442575    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 00:43:04.444065 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442577    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 00:43:04.444065 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442580    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 00:43:04.444065 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442583    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 00:43:04.444065 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442586    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 00:43:04.444065 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442589    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 00:43:04.444065 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442591    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 00:43:04.444065 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442594    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 00:43:04.444065 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442597    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 00:43:04.444065 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442599    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 00:43:04.444065 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442603    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 00:43:04.444065 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442606    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 00:43:04.444065 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442609    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 00:43:04.444065 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442612    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 00:43:04.444065 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442616    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 00:43:04.444065 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442619    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 00:43:04.444581 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442621    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 00:43:04.444581 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442624    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 00:43:04.444581 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442627    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 00:43:04.444581 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442629    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 00:43:04.444581 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442632    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 00:43:04.444581 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442634    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 00:43:04.444581 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442637    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 00:43:04.444581 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442639    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 00:43:04.444581 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442642    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 00:43:04.444581 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442645    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 00:43:04.444581 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442647    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 00:43:04.444581 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442650    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 00:43:04.444581 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442653    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 00:43:04.444581 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442656    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 00:43:04.444581 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442658    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 00:43:04.444581 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442661    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 00:43:04.444581 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442663    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 00:43:04.444581 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442666    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 00:43:04.444581 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442668    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 00:43:04.444581 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442671    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 00:43:04.445079 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442673    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 00:43:04.445079 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442676    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 00:43:04.445079 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442678    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 00:43:04.445079 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442681    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 00:43:04.445079 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442683    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 00:43:04.445079 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442686    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 00:43:04.445079 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442689    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 00:43:04.445079 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442692    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 00:43:04.445079 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442695    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 00:43:04.445079 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442697    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 00:43:04.445079 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442700    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 00:43:04.445079 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442702    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 00:43:04.445079 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442705    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 00:43:04.445079 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442707    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 00:43:04.445079 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442710    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 00:43:04.445079 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442712    2556 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 00:43:04.445079 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442715    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 00:43:04.445079 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442717    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 00:43:04.445079 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.442720    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 00:43:04.445079 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443397    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 00:43:04.445631 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443404    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 00:43:04.445631 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443407    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 00:43:04.445631 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443410    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 00:43:04.445631 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443413    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 00:43:04.445631 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443416    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 00:43:04.445631 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443419    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 00:43:04.445631 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443422    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 00:43:04.445631 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443425    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 00:43:04.445631 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443427    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 00:43:04.445631 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443430    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 00:43:04.445631 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443433    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 00:43:04.445631 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443435    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 00:43:04.445631 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443438    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 00:43:04.445631 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443440    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 00:43:04.445631 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443443    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 00:43:04.445631 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443446    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 00:43:04.445631 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443448    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 00:43:04.445631 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443453    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 00:43:04.445631 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443457    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 00:43:04.446104 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443460    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 00:43:04.446104 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443464    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 00:43:04.446104 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443467    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 00:43:04.446104 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443471    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 00:43:04.446104 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443475    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 00:43:04.446104 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443478    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 00:43:04.446104 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443482    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 00:43:04.446104 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443485    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 00:43:04.446104 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443488    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 00:43:04.446104 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443491    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 00:43:04.446104 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443494    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 00:43:04.446104 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443496    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 00:43:04.446104 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443499    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 00:43:04.446104 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443502    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 00:43:04.446104 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443505    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 00:43:04.446104 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443508    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 00:43:04.446104 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443510    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 00:43:04.446104 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443513    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 00:43:04.446104 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443522    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 00:43:04.446594 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443525    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 00:43:04.446594 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443527    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 00:43:04.446594 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443531    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 00:43:04.446594 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443535    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 00:43:04.446594 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443538    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 00:43:04.446594 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443541    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 00:43:04.446594 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443543    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 00:43:04.446594 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443546    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 00:43:04.446594 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443549    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 00:43:04.446594 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443552    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 00:43:04.446594 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443554    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 00:43:04.446594 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443557    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 00:43:04.446594 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443561    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 00:43:04.446594 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443564    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 00:43:04.446594 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443567    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 00:43:04.446594 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443569    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 00:43:04.446594 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443572    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 00:43:04.446594 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443574    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 00:43:04.446594 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443577    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 00:43:04.446594 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443579    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 00:43:04.447081 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443582    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 00:43:04.447081 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443584    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 00:43:04.447081 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443587    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 00:43:04.447081 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443590    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 00:43:04.447081 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443592    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 00:43:04.447081 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443596    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 00:43:04.447081 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443598    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 00:43:04.447081 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443601    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 00:43:04.447081 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443604    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 00:43:04.447081 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443606    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 00:43:04.447081 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443609    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 00:43:04.447081 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443612    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 00:43:04.447081 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443615    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 00:43:04.447081 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443618    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 00:43:04.447081 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443621    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 00:43:04.447081 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443624    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 00:43:04.447081 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443626    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 00:43:04.447081 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443629    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 00:43:04.447081 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443631    2556 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 00:43:04.447081 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443634    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 00:43:04.447595 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443637    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 00:43:04.447595 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443639    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 00:43:04.447595 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443642    2556 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 00:43:04.447595 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443645    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 00:43:04.447595 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443648    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 00:43:04.447595 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443651    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 00:43:04.447595 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443653    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 00:43:04.447595 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443656    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 00:43:04.447595 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443658    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 00:43:04.447595 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443661    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 00:43:04.447595 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443663    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 00:43:04.447595 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443666    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 00:43:04.447595 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443669    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 00:43:04.447595 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443671    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 00:43:04.447595 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443674    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 00:43:04.447595 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443676    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 00:43:04.447595 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443679    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 00:43:04.447595 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443682    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 00:43:04.447595 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443684    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 00:43:04.447595 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.443687    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 00:43:04.448086 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444024    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 00:43:04.448086 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444028    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 00:43:04.448086 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444031    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 00:43:04.448086 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444033    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 00:43:04.448086 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444036    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 00:43:04.448086 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444039    2556 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 00:43:04.448086 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444042    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 00:43:04.448086 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444044    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 00:43:04.448086 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444047    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 00:43:04.448086 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444049    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 00:43:04.448086 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444052    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 00:43:04.448086 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444055    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 00:43:04.448086 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444058    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 00:43:04.448086 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444060    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 00:43:04.448086 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444062    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 00:43:04.448086 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444065    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 00:43:04.448086 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444069    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 00:43:04.448086 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444072    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 00:43:04.448086 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444075    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 00:43:04.448086 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444078    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 00:43:04.448595 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444081    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 00:43:04.448595 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444083    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 00:43:04.448595 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444086    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 00:43:04.448595 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444089    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 00:43:04.448595 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444091    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 00:43:04.448595 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444094    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 00:43:04.448595 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444096    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 00:43:04.448595 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444099    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 00:43:04.448595 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444101    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 00:43:04.448595 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444104    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 00:43:04.448595 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444106    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 00:43:04.448595 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444109    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 00:43:04.448595 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444111    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 00:43:04.448595 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444114    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 00:43:04.448595 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444116    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 00:43:04.448595 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444119    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 00:43:04.448595 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444121    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 00:43:04.448595 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444124    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 00:43:04.448595 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444127    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 00:43:04.448595 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444129    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 00:43:04.449083 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444132    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 00:43:04.449083 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444134    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 00:43:04.449083 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444137    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 00:43:04.449083 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444139    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 00:43:04.449083 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444144    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 00:43:04.449083 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444147    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 00:43:04.449083 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444149    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 00:43:04.449083 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444152    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 00:43:04.449083 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444154    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 00:43:04.449083 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444157    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 00:43:04.449083 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444160    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 00:43:04.449083 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444162    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 00:43:04.449083 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444165    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 00:43:04.449083 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444167    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 00:43:04.449083 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444170    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 00:43:04.449083 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444174    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 00:43:04.449083 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444178    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 00:43:04.449083 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444182    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 00:43:04.449083 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444186    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 00:43:04.449553 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444189    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 00:43:04.449553 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444191    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 00:43:04.449553 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444194    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 00:43:04.449553 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444198    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 00:43:04.449553 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444201    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 00:43:04.449553 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444203    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 00:43:04.449553 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444206    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 00:43:04.449553 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444209    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 00:43:04.449553 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444211    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 00:43:04.449553 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444214    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 00:43:04.449553 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444217    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 00:43:04.449553 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444225    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 00:43:04.449553 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444228    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 00:43:04.449553 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444231    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 00:43:04.449553 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444234    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 00:43:04.449553 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444237    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 00:43:04.449553 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444240    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 00:43:04.449553 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444245    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 00:43:04.449553 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444247    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 00:43:04.450017 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444250    2556 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 00:43:04.450017 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444253    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 00:43:04.450017 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444256    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 00:43:04.450017 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444259    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 00:43:04.450017 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444261    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 00:43:04.450017 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444264    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 00:43:04.450017 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444266    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 00:43:04.450017 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444269    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 00:43:04.450017 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444271    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 00:43:04.450017 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444275    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 00:43:04.450017 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444277    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 00:43:04.450017 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444280    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 00:43:04.450017 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444283    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 00:43:04.450017 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444285    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 00:43:04.450017 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444288    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 00:43:04.450017 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444290    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 00:43:04.450017 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444293    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 00:43:04.450017 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444295    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 00:43:04.450017 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444298    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 00:43:04.450017 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444301    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 00:43:04.450523 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.444303    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 00:43:04.450523 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444381    2556 flags.go:64] FLAG: --address="0.0.0.0"
Sep 16 00:43:04.450523 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444417    2556 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 16 00:43:04.450523 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444426    2556 flags.go:64] FLAG: --anonymous-auth="true"
Sep 16 00:43:04.450523 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444431    2556 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 16 00:43:04.450523 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444446    2556 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 16 00:43:04.450523 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444449    2556 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 16 00:43:04.450523 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444453    2556 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 16 00:43:04.450523 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444458    2556 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 16 00:43:04.450523 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444461    2556 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 16 00:43:04.450523 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444465    2556 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 16 00:43:04.450523 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444469    2556 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 16 00:43:04.450523 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444472    2556 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 16 00:43:04.450523 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444475    2556 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 16 00:43:04.450523 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444478    2556 flags.go:64] FLAG: --cgroup-root=""
Sep 16 00:43:04.450523 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444481    2556 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 16 00:43:04.450523 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444484    2556 flags.go:64] FLAG: --client-ca-file=""
Sep 16 00:43:04.450523 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444486    2556 flags.go:64] FLAG: --cloud-provider="external"
Sep 16 00:43:04.450523 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444489    2556 flags.go:64] FLAG: --cluster-dns="[]"
Sep 16 00:43:04.450523 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444496    2556 flags.go:64] FLAG: --cluster-domain=""
Sep 16 00:43:04.450523 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444499    2556 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 16 00:43:04.450523 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444502    2556 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 16 00:43:04.450523 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444505    2556 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 16 00:43:04.450523 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444508    2556 flags.go:64] FLAG: --container-log-max-files="5"
Sep 16 00:43:04.451120 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444512    2556 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 16 00:43:04.451120 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444514    2556 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 16 00:43:04.451120 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444517    2556 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 16 00:43:04.451120 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444521    2556 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 16 00:43:04.451120 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444523    2556 flags.go:64] FLAG: --contention-profiling="false"
Sep 16 00:43:04.451120 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444527    2556 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 16 00:43:04.451120 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444529    2556 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 16 00:43:04.451120 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444532    2556 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 16 00:43:04.451120 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444535    2556 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 16 00:43:04.451120 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444539    2556 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 16 00:43:04.451120 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444542    2556 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 16 00:43:04.451120 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444545    2556 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 16 00:43:04.451120 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444548    2556 flags.go:64] FLAG: --enable-load-reader="false"
Sep 16 00:43:04.451120 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444551    2556 flags.go:64] FLAG: --enable-server="true"
Sep 16 00:43:04.451120 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444555    2556 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 16 00:43:04.451120 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444559    2556 flags.go:64] FLAG: --event-burst="100"
Sep 16 00:43:04.451120 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444562    2556 flags.go:64] FLAG: --event-qps="50"
Sep 16 00:43:04.451120 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444565    2556 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 16 00:43:04.451120 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444570    2556 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 16 00:43:04.451120 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444573    2556 flags.go:64] FLAG: --eviction-hard=""
Sep 16 00:43:04.451120 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444576    2556 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 16 00:43:04.451120 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444579    2556 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 16 00:43:04.451120 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444582    2556 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 16 00:43:04.451120 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444586    2556 flags.go:64] FLAG: --eviction-soft=""
Sep 16 00:43:04.451120 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444589    2556 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 16 00:43:04.452032 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444592    2556 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 16 00:43:04.452032 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444595    2556 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 16 00:43:04.452032 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444598    2556 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 16 00:43:04.452032 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444601    2556 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 16 00:43:04.452032 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444604    2556 flags.go:64] FLAG: --fail-swap-on="true"
Sep 16 00:43:04.452032 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444607    2556 flags.go:64] FLAG: --feature-gates=""
Sep 16 00:43:04.452032 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444610    2556 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 16 00:43:04.452032 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444613    2556 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 16 00:43:04.452032 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444616    2556 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 16 00:43:04.452032 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444619    2556 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 16 00:43:04.452032 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444622    2556 flags.go:64] FLAG: --healthz-port="10248"
Sep 16 00:43:04.452032 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444625    2556 flags.go:64] FLAG: --help="false"
Sep 16 00:43:04.452032 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444628    2556 flags.go:64] FLAG: --hostname-override="ip-10-0-130-207.ec2.internal"
Sep 16 00:43:04.452032 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444631    2556 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 16 00:43:04.452032 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444634    2556 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 16 00:43:04.452032 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444637    2556 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 16 00:43:04.452032 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444640    2556 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 16 00:43:04.452032 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444643    2556 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 16 00:43:04.452032 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444646    2556 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 16 00:43:04.452032 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444649    2556 flags.go:64] FLAG: --image-service-endpoint=""
Sep 16 00:43:04.452032 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444652    2556 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 16 00:43:04.452032 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444655    2556 flags.go:64] FLAG: --kube-api-burst="100"
Sep 16 00:43:04.452032 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444658    2556 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 16 00:43:04.453062 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444661    2556 flags.go:64] FLAG: --kube-api-qps="50"
Sep 16 00:43:04.453062 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444664    2556 flags.go:64] FLAG: --kube-reserved=""
Sep 16 00:43:04.453062 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444668    2556 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 16 00:43:04.453062 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444671    2556 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 16 00:43:04.453062 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444683    2556 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 16 00:43:04.453062 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444685    2556 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 16 00:43:04.453062 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444688    2556 flags.go:64] FLAG: --lock-file=""
Sep 16 00:43:04.453062 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444691    2556 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 16 00:43:04.453062 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444694    2556 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 16 00:43:04.453062 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444697    2556 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 16 00:43:04.453062 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444703    2556 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 16 00:43:04.453062 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444705    2556 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 16 00:43:04.453062 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444708    2556 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 16 00:43:04.453062 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444711    2556 flags.go:64] FLAG: --logging-format="text"
Sep 16 00:43:04.453062 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444714    2556 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 16 00:43:04.453062 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444717    2556 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 16 00:43:04.453062 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444720    2556 flags.go:64] FLAG: --manifest-url=""
Sep 16 00:43:04.453062 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444723    2556 flags.go:64] FLAG: --manifest-url-header=""
Sep 16 00:43:04.453062 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444727    2556 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 16 00:43:04.453062 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444730    2556 flags.go:64] FLAG: --max-open-files="1000000"
Sep 16 00:43:04.453062 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444733    2556 flags.go:64] FLAG: --max-pods="110"
Sep 16 00:43:04.453062 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444736    2556 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 16 00:43:04.453062 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444739    2556 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 16 00:43:04.453062 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444742    2556 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 16 00:43:04.453062 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444745    2556 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 16 00:43:04.454137 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444748    2556 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 16 00:43:04.454137 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444751    2556 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 16 00:43:04.454137 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444754    2556 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 16 00:43:04.454137 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444761    2556 flags.go:64] FLAG: --node-status-max-images="50"
Sep 16 00:43:04.454137 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444764    2556 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 16 00:43:04.454137 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444767    2556 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 16 00:43:04.454137 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444770    2556 flags.go:64] FLAG: --pod-cidr=""
Sep 16 00:43:04.454137 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444773    2556 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 16 00:43:04.454137 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444778    2556 flags.go:64] FLAG: --pod-manifest-path=""
Sep 16 00:43:04.454137 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444781    2556 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 16 00:43:04.454137 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444788    2556 flags.go:64] FLAG: --pods-per-core="0"
Sep 16 00:43:04.454137 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444791    2556 flags.go:64] FLAG: --port="10250"
Sep 16 00:43:04.454137 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444794    2556 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 16 00:43:04.454137 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444796    2556 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0b730405ff388650d"
Sep 16 00:43:04.454137 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444799    2556 flags.go:64] FLAG: --qos-reserved=""
Sep 16 00:43:04.454137 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444802    2556 flags.go:64] FLAG: --read-only-port="10255"
Sep 16 00:43:04.454137 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444805    2556 flags.go:64] FLAG: --register-node="true"
Sep 16 00:43:04.454137 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444808    2556 flags.go:64] FLAG: --register-with-taints=""
Sep 16 00:43:04.454137 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444812    2556 flags.go:64] FLAG: --registry-burst="10"
Sep 16 00:43:04.454137 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444815    2556 flags.go:64] FLAG: --registry-qps="5"
Sep 16 00:43:04.454137 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444817    2556 flags.go:64] FLAG: --reserved-cpus=""
Sep 16 00:43:04.454137 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444820    2556 flags.go:64] FLAG: --reserved-memory=""
Sep 16 00:43:04.454137 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444823    2556 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 16 00:43:04.454137 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444826    2556 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 16 00:43:04.454137 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444829    2556 flags.go:64] FLAG: --rotate-certificates="false"
Sep 16 00:43:04.455148 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444832    2556 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 16 00:43:04.455148 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444835    2556 flags.go:64] FLAG: --runonce="false"
Sep 16 00:43:04.455148 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444838    2556 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 16 00:43:04.455148 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444840    2556 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 16 00:43:04.455148 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444843    2556 flags.go:64] FLAG: --seccomp-default="false"
Sep 16 00:43:04.455148 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444846    2556 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 16 00:43:04.455148 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444849    2556 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 16 00:43:04.455148 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444852    2556 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 16 00:43:04.455148 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444855    2556 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 16 00:43:04.455148 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444858    2556 flags.go:64] FLAG: --storage-driver-password="root"
Sep 16 00:43:04.455148 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444861    2556 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 16 00:43:04.455148 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444865    2556 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 16 00:43:04.455148 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444868    2556 flags.go:64] FLAG: --storage-driver-user="root"
Sep 16 00:43:04.455148 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444870    2556 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 16 00:43:04.455148 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444873    2556 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 16 00:43:04.455148 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444882    2556 flags.go:64] FLAG: --system-cgroups=""
Sep 16 00:43:04.455148 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444886    2556 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 16 00:43:04.455148 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444892    2556 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 16 00:43:04.455148 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444895    2556 flags.go:64] FLAG: --tls-cert-file=""
Sep 16 00:43:04.455148 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444898    2556 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 16 00:43:04.455148 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444901    2556 flags.go:64] FLAG: --tls-min-version=""
Sep 16 00:43:04.455148 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444904    2556 flags.go:64] FLAG: --tls-private-key-file=""
Sep 16 00:43:04.455148 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444907    2556 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 16 00:43:04.455148 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444910    2556 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 16 00:43:04.455148 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444913    2556 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 16 00:43:04.455980 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444916    2556 flags.go:64] FLAG: --v="2"
Sep 16 00:43:04.455980 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444919    2556 flags.go:64] FLAG: --version="false"
Sep 16 00:43:04.455980 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444923    2556 flags.go:64] FLAG: --vmodule=""
Sep 16 00:43:04.455980 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444927    2556 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 16 00:43:04.455980 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.444930    2556 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 16 00:43:04.455980 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445044    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 00:43:04.455980 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445048    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 00:43:04.455980 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445051    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 00:43:04.455980 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445053    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 00:43:04.455980 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445056    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 00:43:04.455980 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445059    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 00:43:04.455980 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445062    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 00:43:04.455980 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445065    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 00:43:04.455980 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445068    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 00:43:04.455980 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445070    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 00:43:04.455980 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445073    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 00:43:04.455980 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445075    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 00:43:04.455980 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445078    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 00:43:04.455980 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445083    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 00:43:04.455980 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445085    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 00:43:04.455980 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445088    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 00:43:04.456901 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445091    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 00:43:04.456901 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445094    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 00:43:04.456901 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445097    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 00:43:04.456901 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445100    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 00:43:04.456901 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445103    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 00:43:04.456901 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445106    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 00:43:04.456901 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445108    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 00:43:04.456901 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445111    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 00:43:04.456901 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445115    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 00:43:04.456901 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445119    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 00:43:04.456901 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445122    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 00:43:04.456901 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445125    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 00:43:04.456901 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445128    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 00:43:04.456901 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445130    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 00:43:04.456901 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445133    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 00:43:04.456901 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445135    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 00:43:04.456901 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445138    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 00:43:04.456901 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445140    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 00:43:04.456901 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445143    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 00:43:04.457421 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445145    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 00:43:04.457421 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445148    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 00:43:04.457421 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445150    2556 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 00:43:04.457421 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445153    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 00:43:04.457421 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445155    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 00:43:04.457421 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445158    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 00:43:04.457421 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445160    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 00:43:04.457421 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445163    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 00:43:04.457421 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445166    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 00:43:04.457421 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445168    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 00:43:04.457421 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445172    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 00:43:04.457421 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445175    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 00:43:04.457421 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445177    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 00:43:04.457421 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445180    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 00:43:04.457421 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445183    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 00:43:04.457421 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445185    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 00:43:04.457421 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445188    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 00:43:04.457421 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445191    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 00:43:04.457421 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445194    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 00:43:04.457421 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445196    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 00:43:04.458064 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445199    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 00:43:04.458064 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445203    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 00:43:04.458064 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445206    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 00:43:04.458064 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445209    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 00:43:04.458064 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445211    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 00:43:04.458064 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445214    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 00:43:04.458064 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445216    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 00:43:04.458064 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445219    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 00:43:04.458064 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445221    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 00:43:04.458064 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445224    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 00:43:04.458064 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445227    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 00:43:04.458064 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445229    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 00:43:04.458064 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445231    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 00:43:04.458064 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445234    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 00:43:04.458064 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445237    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 00:43:04.458064 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445239    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 00:43:04.458064 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445242    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 00:43:04.458064 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445245    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 00:43:04.458064 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445247    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 00:43:04.458552 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445250    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 00:43:04.458552 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445253    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 00:43:04.458552 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445255    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 00:43:04.458552 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445259    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 00:43:04.458552 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445261    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 00:43:04.458552 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445264    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 00:43:04.458552 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445267    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 00:43:04.458552 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445271    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 00:43:04.458552 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445274    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 00:43:04.458552 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445277    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 00:43:04.458552 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445281    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 00:43:04.458552 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445284    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 00:43:04.458552 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445286    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 00:43:04.458552 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445289    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 00:43:04.458552 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445292    2556 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 00:43:04.458552 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445294    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 00:43:04.458552 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445297    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 00:43:04.458552 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445299    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 00:43:04.458552 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445302    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 00:43:04.459086 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445304    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 00:43:04.459086 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445307    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 00:43:04.459086 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445310    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 00:43:04.459086 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445325    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 00:43:04.459086 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445327    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 00:43:04.459086 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.445330    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 00:43:04.459086 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.446208    2556 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 16 00:43:04.459086 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.453541    2556 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 16 00:43:04.459086 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.453560    2556 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 16 00:43:04.459086 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.453588    2556 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 16 00:43:04.459086 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.453601    2556 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 16 00:43:04.459086 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.453701    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 00:43:04.459086 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.453710    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 00:43:04.459086 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.453716    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 00:43:04.459457 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.453722    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 00:43:04.459457 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.453727    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 00:43:04.459457 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.453731    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 00:43:04.459457 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.453736    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 00:43:04.459457 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.453740    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 00:43:04.459457 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.453744    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 00:43:04.459457 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.453748    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 00:43:04.459457 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.453752    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 00:43:04.459457 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.453757    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 00:43:04.459457 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.453762    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 00:43:04.459457 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.453766    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 00:43:04.459457 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.453770    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 00:43:04.459457 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.453774    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 00:43:04.459457 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.453778    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 00:43:04.459457 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.453782    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 00:43:04.459457 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.453786    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 00:43:04.459457 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.453790    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 00:43:04.459457 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.453796    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 00:43:04.459457 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.453803    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 00:43:04.459975 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.453807    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 00:43:04.459975 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.453812    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 00:43:04.459975 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.453816    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 00:43:04.459975 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.453820    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 00:43:04.459975 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.453824    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 00:43:04.459975 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.453828    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 00:43:04.459975 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.453833    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 00:43:04.459975 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.453838    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 00:43:04.459975 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.453842    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 00:43:04.459975 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.453847    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 00:43:04.459975 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.453851    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 00:43:04.459975 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.453855    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 00:43:04.459975 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.453859    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 00:43:04.459975 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.453863    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 00:43:04.459975 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.453868    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 00:43:04.459975 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.453872    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 00:43:04.459975 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.453876    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 00:43:04.459975 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.453880    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 00:43:04.459975 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.453884    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 00:43:04.459975 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.453888    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 00:43:04.460512 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.453892    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 00:43:04.460512 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.453897    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 00:43:04.460512 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.453901    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 00:43:04.460512 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.453905    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 00:43:04.460512 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.453909    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 00:43:04.460512 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.453913    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 00:43:04.460512 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.453917    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 00:43:04.460512 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.453921    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 00:43:04.460512 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.453925    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 00:43:04.460512 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.453929    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 00:43:04.460512 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.453934    2556 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 00:43:04.460512 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.453938    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 00:43:04.460512 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.453942    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 00:43:04.460512 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.453947    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 00:43:04.460512 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.453953    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 00:43:04.460512 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.453958    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 00:43:04.460512 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.453962    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 00:43:04.460512 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.453967    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 00:43:04.460512 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.453971    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 00:43:04.460512 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.453976    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 00:43:04.461037 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.453981    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 00:43:04.461037 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.453985    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 00:43:04.461037 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.453989    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 00:43:04.461037 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.453994    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 00:43:04.461037 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.453998    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 00:43:04.461037 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454003    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 00:43:04.461037 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454007    2556 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 00:43:04.461037 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454011    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 00:43:04.461037 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454015    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 00:43:04.461037 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454019    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 00:43:04.461037 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454023    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 00:43:04.461037 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454027    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 00:43:04.461037 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454032    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 00:43:04.461037 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454036    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 00:43:04.461037 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454040    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 00:43:04.461037 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454044    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 00:43:04.461037 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454048    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 00:43:04.461037 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454052    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 00:43:04.461037 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454056    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 00:43:04.461037 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454060    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 00:43:04.461560 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454064    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 00:43:04.461560 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454068    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 00:43:04.461560 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454073    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 00:43:04.461560 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454077    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 00:43:04.461560 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454081    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 00:43:04.461560 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454086    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 00:43:04.461560 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454090    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 00:43:04.461560 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454094    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 00:43:04.461560 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454098    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 00:43:04.461560 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454102    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 00:43:04.461560 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454108    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 00:43:04.461560 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454112    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 00:43:04.461560 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454116    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 00:43:04.461560 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454121    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 00:43:04.461560 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454125    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 00:43:04.461560 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454130    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 00:43:04.461560 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454133    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 00:43:04.462200 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.454141    2556 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 16 00:43:04.462200 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454306    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 00:43:04.462200 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454331    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 00:43:04.462200 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454336    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 00:43:04.462200 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454341    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 00:43:04.462200 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454345    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 00:43:04.462200 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454350    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 00:43:04.462200 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454354    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 00:43:04.462200 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454359    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 00:43:04.462200 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454363    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 00:43:04.462200 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454366    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 00:43:04.462200 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454371    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 00:43:04.462200 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454375    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 00:43:04.462200 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454379    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 00:43:04.462200 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454384    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 00:43:04.462711 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454388    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 00:43:04.462711 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454392    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 00:43:04.462711 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454396    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 00:43:04.462711 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454401    2556 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 00:43:04.462711 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454406    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 00:43:04.462711 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454410    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 00:43:04.462711 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454414    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 00:43:04.462711 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454420    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 00:43:04.462711 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454427    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 00:43:04.462711 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454431    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 00:43:04.462711 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454436    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 00:43:04.462711 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454439    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 00:43:04.462711 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454444    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 00:43:04.462711 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454448    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 00:43:04.462711 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454454    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 00:43:04.462711 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454458    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 00:43:04.462711 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454462    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 00:43:04.462711 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454466    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 00:43:04.462711 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454470    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 00:43:04.463258 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454474    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 00:43:04.463258 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454479    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 00:43:04.463258 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454483    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 00:43:04.463258 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454487    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 00:43:04.463258 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454491    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 00:43:04.463258 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454495    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 00:43:04.463258 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454499    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 00:43:04.463258 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454503    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 00:43:04.463258 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454507    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 00:43:04.463258 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454511    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 00:43:04.463258 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454515    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 00:43:04.463258 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454519    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 00:43:04.463258 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454523    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 00:43:04.463258 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454530    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 00:43:04.463258 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454535    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 00:43:04.463258 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454539    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 00:43:04.463258 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454544    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 00:43:04.463258 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454549    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 00:43:04.463258 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454553    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 00:43:04.463758 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454557    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 00:43:04.463758 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454561    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 00:43:04.463758 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454565    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 00:43:04.463758 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454570    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 00:43:04.463758 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454574    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 00:43:04.463758 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454578    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 00:43:04.463758 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454582    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 00:43:04.463758 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454586    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 00:43:04.463758 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454590    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 00:43:04.463758 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454595    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 00:43:04.463758 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454600    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 00:43:04.463758 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454604    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 00:43:04.463758 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454608    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 00:43:04.463758 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454612    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 00:43:04.463758 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454617    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 00:43:04.463758 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454620    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 00:43:04.463758 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454625    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 00:43:04.463758 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454629    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 00:43:04.463758 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454634    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 00:43:04.463758 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454638    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 00:43:04.464405 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454642    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 00:43:04.464405 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454648    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 00:43:04.464405 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454653    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 00:43:04.464405 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454657    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 00:43:04.464405 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454661    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 00:43:04.464405 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454665    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 00:43:04.464405 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454669    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 00:43:04.464405 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454673    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 00:43:04.464405 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454677    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 00:43:04.464405 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454682    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 00:43:04.464405 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454686    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 00:43:04.464405 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454689    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 00:43:04.464405 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454693    2556 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 00:43:04.464405 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454697    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 00:43:04.464405 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454701    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 00:43:04.464405 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454705    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 00:43:04.464405 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454709    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 00:43:04.464405 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454714    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 00:43:04.464405 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454718    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 00:43:04.464405 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454722    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 00:43:04.465033 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454725    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 00:43:04.465033 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454729    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 00:43:04.465033 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454734    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 00:43:04.465033 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454738    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 00:43:04.465033 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454742    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 00:43:04.465033 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454746    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 00:43:04.465033 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:04.454751    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 00:43:04.465033 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.454758    2556 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 16 00:43:04.465033 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.454900    2556 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 16 00:43:04.465033 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.457526    2556 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 16 00:43:04.465033 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.458673    2556 server.go:1019] "Starting client certificate rotation"
Sep 16 00:43:04.465033 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.458781    2556 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 16 00:43:04.465033 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.458815    2556 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 16 00:43:04.465532 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.465013    2556 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 16 00:43:04.467357 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.467339    2556 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 16 00:43:04.480943 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.480923    2556 log.go:25] "Validated CRI v1 runtime API"
Sep 16 00:43:04.483941 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.483926    2556 log.go:25] "Validated CRI v1 image API"
Sep 16 00:43:04.484743 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.484720    2556 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 16 00:43:04.487768 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.487746    2556 fs.go:135] Filesystem UUIDs: map[36842e40-7cb7-495f-902e-cd7e3fd1347d:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 d78a6555-df8d-4cee-862a-b0e104034e8b:/dev/nvme0n1p4]
Sep 16 00:43:04.487825 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.487768    2556 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 16 00:43:04.492770 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.492658    2556 manager.go:216] Machine: {Timestamp:2026-09-16 00:43:04.491180016 +0000 UTC m=+0.343834257 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099924 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec265c42e82e2bc2610ded553cd0a766 SystemUUID:ec265c42-e82e-2bc2-610d-ed553cd0a766 BootID:c21c3673-1a37-41fd-83a7-be7691131a6f Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:e4:84:68:35 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:e4:84:68:35 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:2e:ca:e5:c2:24:a3 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 16 00:43:04.492770 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.492765    2556 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 16 00:43:04.492879 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.492833    2556 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 16 00:43:04.493721 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.493708    2556 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 16 00:43:04.496909 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.496882    2556 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 16 00:43:04.497039 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.496912    2556 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-207.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 16 00:43:04.498289 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.498279    2556 topology_manager.go:138] "Creating topology manager with none policy"
Sep 16 00:43:04.498343 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.498292    2556 container_manager_linux.go:309] "Creating device plugin manager"
Sep 16 00:43:04.498343 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.498300    2556 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 16 00:43:04.498857 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.498847    2556 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 16 00:43:04.499729 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.499719    2556 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 16 00:43:04.503747 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.503736    2556 state_mem.go:36] "Initialized new in-memory state store"
Sep 16 00:43:04.503844 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.503835    2556 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 16 00:43:04.503873 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.503861    2556 kubelet.go:486] "Attempting to sync node with API server"
Sep 16 00:43:04.503873 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.503871    2556 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 16 00:43:04.503931 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.503881    2556 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 16 00:43:04.503931 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.503889    2556 kubelet.go:398] "Adding apiserver pod source"
Sep 16 00:43:04.503931 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.503899    2556 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 16 00:43:04.504206 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.504195    2556 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 16 00:43:04.504250 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.504217    2556 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 16 00:43:04.506577 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.506563    2556 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 16 00:43:04.508768 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.508755    2556 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 16 00:43:04.508816 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.508785    2556 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 16 00:43:04.510517 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.510506    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 16 00:43:04.510556 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.510522    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 16 00:43:04.510556 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.510528    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 16 00:43:04.510556 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.510533    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 16 00:43:04.510556 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.510543    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 16 00:43:04.510556 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.510549    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 16 00:43:04.510685 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.510563    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 16 00:43:04.510685 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.510568    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 16 00:43:04.510685 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.510574    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 16 00:43:04.510685 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.510579    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 16 00:43:04.510685 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.510594    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 16 00:43:04.510685 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.510600    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 16 00:43:04.512364 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.512354    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 16 00:43:04.512364 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.512364    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 16 00:43:04.513835 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:04.513806    2556 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-207.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 16 00:43:04.513979 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:04.513956    2556 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 16 00:43:04.516183 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.516165    2556 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-s9vp2"
Sep 16 00:43:04.516664 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.516647    2556 server.go:1268] "Started kubelet"
Sep 16 00:43:04.516803 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.516738    2556 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 16 00:43:04.516860 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.516827    2556 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 16 00:43:04.516917 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.516736    2556 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 16 00:43:04.517390 ip-10-0-130-207 systemd[1]: Started Kubernetes Kubelet.
Sep 16 00:43:04.517982 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.517962    2556 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 16 00:43:04.518381 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.518365    2556 server.go:310] "Adding debug handlers to kubelet server"
Sep 16 00:43:04.521956 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:04.521931    2556 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 16 00:43:04.522740 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.522720    2556 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-207.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 16 00:43:04.523415 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.523401    2556 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-s9vp2"
Sep 16 00:43:04.523735 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:04.522787    2556 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-207.ec2.internal.18d5a66620c981a7  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-207.ec2.internal,UID:ip-10-0-130-207.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-207.ec2.internal,},FirstTimestamp:2026-09-16 00:43:04.516592039 +0000 UTC m=+0.369246278,LastTimestamp:2026-09-16 00:43:04.516592039 +0000 UTC m=+0.369246278,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-207.ec2.internal,}"
Sep 16 00:43:04.525189 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.525173    2556 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 16 00:43:04.525640 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.525622    2556 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 16 00:43:04.525808 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:04.525791    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-207.ec2.internal\" not found"
Sep 16 00:43:04.526371 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.526355    2556 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 16 00:43:04.526371 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.526356    2556 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 16 00:43:04.526512 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.526380    2556 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 16 00:43:04.526512 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.526442    2556 reconstruct.go:97] "Volume reconstruction finished"
Sep 16 00:43:04.526512 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.526453    2556 reconciler.go:29] "Reconciler: start to sync state"
Sep 16 00:43:04.528052 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.528036    2556 factory.go:55] Registering systemd factory
Sep 16 00:43:04.528115 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.528074    2556 factory.go:223] Registration of the systemd container factory successfully
Sep 16 00:43:04.528291 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.528276    2556 factory.go:153] Registering CRI-O factory
Sep 16 00:43:04.528360 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.528293    2556 factory.go:223] Registration of the crio container factory successfully
Sep 16 00:43:04.528431 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.528405    2556 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 16 00:43:04.528497 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.528442    2556 factory.go:103] Registering Raw factory
Sep 16 00:43:04.528497 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.528468    2556 manager.go:1196] Started watching for new ooms in manager
Sep 16 00:43:04.528616 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.528577    2556 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 00:43:04.529929 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.529911    2556 manager.go:349] Starting recovery of all containers
Sep 16 00:43:04.532935 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:04.532913    2556 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-130-207.ec2.internal\" not found" node="ip-10-0-130-207.ec2.internal"
Sep 16 00:43:04.542174 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.542081    2556 manager.go:354] Recovery completed
Sep 16 00:43:04.546214 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.546200    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 00:43:04.547967 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.547948    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-207.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 00:43:04.548031 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.547979    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-207.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 00:43:04.548031 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.547992    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-207.ec2.internal" event="NodeHasSufficientPID"
Sep 16 00:43:04.548487 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.548473    2556 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 16 00:43:04.548487 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.548483    2556 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 16 00:43:04.548587 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.548490    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 00:43:04.548587 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.548498    2556 state_mem.go:36] "Initialized new in-memory state store"
Sep 16 00:43:04.549138 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.549123    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-207.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 00:43:04.549193 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.549145    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-207.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 00:43:04.549193 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.549154    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-207.ec2.internal" event="NodeHasSufficientPID"
Sep 16 00:43:04.550400 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.550388    2556 policy_none.go:49] "None policy: Start"
Sep 16 00:43:04.550452 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.550403    2556 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 16 00:43:04.550452 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.550429    2556 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 16 00:43:04.552235 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.552215    2556 policy_none.go:47] "Start"
Sep 16 00:43:04.590956 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.590943    2556 manager.go:337] "Starting Device Plugin manager"
Sep 16 00:43:04.606951 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:04.590980    2556 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 16 00:43:04.606951 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.590990    2556 server.go:87] "Starting device plugin registration server"
Sep 16 00:43:04.606951 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.591269    2556 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 16 00:43:04.606951 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.591282    2556 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 16 00:43:04.606951 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.591391    2556 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 16 00:43:04.606951 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.591468    2556 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 16 00:43:04.606951 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.591476    2556 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 16 00:43:04.606951 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:04.592036    2556 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 16 00:43:04.606951 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:04.592087    2556 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-207.ec2.internal\" not found"
Sep 16 00:43:04.657325 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.657293    2556 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 16 00:43:04.658458 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.658439    2556 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 16 00:43:04.658538 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.658481    2556 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 16 00:43:04.658538 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.658499    2556 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 16 00:43:04.658627 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:04.658557    2556 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 16 00:43:04.660902 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.660886    2556 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 00:43:04.691837 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.691785    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 00:43:04.692498 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.692485    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-207.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 00:43:04.692571 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.692509    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-207.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 00:43:04.692571 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.692519    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-207.ec2.internal" event="NodeHasSufficientPID"
Sep 16 00:43:04.692571 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.692537    2556 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-207.ec2.internal"
Sep 16 00:43:04.700759 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.700742    2556 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-207.ec2.internal"
Sep 16 00:43:04.700759 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:04.700760    2556 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-207.ec2.internal\": node \"ip-10-0-130-207.ec2.internal\" not found"
Sep 16 00:43:04.714238 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:04.714218    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-207.ec2.internal\" not found"
Sep 16 00:43:04.759446 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.759425    2556 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-207.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-207.ec2.internal"]
Sep 16 00:43:04.759513 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.759480    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 00:43:04.760183 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.760166    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-207.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 00:43:04.760279 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.760196    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-207.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 00:43:04.760279 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.760212    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-207.ec2.internal" event="NodeHasSufficientPID"
Sep 16 00:43:04.762394 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.762382    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 00:43:04.762517 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.762504    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-207.ec2.internal"
Sep 16 00:43:04.762578 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.762541    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 00:43:04.763104 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.763089    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-207.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 00:43:04.763168 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.763116    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-207.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 00:43:04.763168 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.763129    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-207.ec2.internal" event="NodeHasSufficientPID"
Sep 16 00:43:04.763168 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.763134    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-207.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 00:43:04.763168 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.763157    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-207.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 00:43:04.763168 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.763168    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-207.ec2.internal" event="NodeHasSufficientPID"
Sep 16 00:43:04.765293 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.765277    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-207.ec2.internal"
Sep 16 00:43:04.765385 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.765301    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 00:43:04.765923 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.765909    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-207.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 00:43:04.765984 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.765936    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-207.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 00:43:04.765984 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.765947    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-207.ec2.internal" event="NodeHasSufficientPID"
Sep 16 00:43:04.785925 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:04.785909    2556 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-207.ec2.internal\" not found" node="ip-10-0-130-207.ec2.internal"
Sep 16 00:43:04.790184 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:04.790170    2556 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-207.ec2.internal\" not found" node="ip-10-0-130-207.ec2.internal"
Sep 16 00:43:04.814995 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:04.814978    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-207.ec2.internal\" not found"
Sep 16 00:43:04.828340 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.828309    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a6be15198fb113e0b9a0291128436e59-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-207.ec2.internal\" (UID: \"a6be15198fb113e0b9a0291128436e59\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-207.ec2.internal"
Sep 16 00:43:04.828392 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.828352    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a6be15198fb113e0b9a0291128436e59-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-207.ec2.internal\" (UID: \"a6be15198fb113e0b9a0291128436e59\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-207.ec2.internal"
Sep 16 00:43:04.828434 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.828388    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4054c0090de8f54aed802a5d555be580-config\") pod \"kube-apiserver-proxy-ip-10-0-130-207.ec2.internal\" (UID: \"4054c0090de8f54aed802a5d555be580\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-207.ec2.internal"
Sep 16 00:43:04.915545 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:04.915525    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-207.ec2.internal\" not found"
Sep 16 00:43:04.928860 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.928839    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a6be15198fb113e0b9a0291128436e59-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-207.ec2.internal\" (UID: \"a6be15198fb113e0b9a0291128436e59\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-207.ec2.internal"
Sep 16 00:43:04.928913 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.928865    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a6be15198fb113e0b9a0291128436e59-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-207.ec2.internal\" (UID: \"a6be15198fb113e0b9a0291128436e59\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-207.ec2.internal"
Sep 16 00:43:04.928913 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.928881    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4054c0090de8f54aed802a5d555be580-config\") pod \"kube-apiserver-proxy-ip-10-0-130-207.ec2.internal\" (UID: \"4054c0090de8f54aed802a5d555be580\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-207.ec2.internal"
Sep 16 00:43:04.928985 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.928910    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4054c0090de8f54aed802a5d555be580-config\") pod \"kube-apiserver-proxy-ip-10-0-130-207.ec2.internal\" (UID: \"4054c0090de8f54aed802a5d555be580\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-207.ec2.internal"
Sep 16 00:43:04.928985 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.928928    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a6be15198fb113e0b9a0291128436e59-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-207.ec2.internal\" (UID: \"a6be15198fb113e0b9a0291128436e59\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-207.ec2.internal"
Sep 16 00:43:04.928985 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:04.928958    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a6be15198fb113e0b9a0291128436e59-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-207.ec2.internal\" (UID: \"a6be15198fb113e0b9a0291128436e59\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-207.ec2.internal"
Sep 16 00:43:05.016292 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:05.016237    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-207.ec2.internal\" not found"
Sep 16 00:43:05.088724 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:05.088688    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-207.ec2.internal"
Sep 16 00:43:05.093349 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:05.093306    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-207.ec2.internal"
Sep 16 00:43:05.116779 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:05.116753    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-207.ec2.internal\" not found"
Sep 16 00:43:05.217249 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:05.217220    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-207.ec2.internal\" not found"
Sep 16 00:43:05.317792 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:05.317746    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-207.ec2.internal\" not found"
Sep 16 00:43:06.489477 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:06.489452    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-207.ec2.internal\" not found"
Sep 16 00:43:06.529913 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:06.529893    2556 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 16 00:43:06.530300 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:06.530047    2556 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 16 00:43:06.530300 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:06.530047    2556 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 16 00:43:06.590387 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:06.590364    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-207.ec2.internal\" not found"
Sep 16 00:43:06.597498 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:06.597457    2556 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-15 00:38:05 +0000 UTC" deadline="2028-02-10 14:35:50.616196069 +0000 UTC"
Sep 16 00:43:06.597565 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:06.597498    2556 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12301h52m44.018702403s"
Sep 16 00:43:06.691272 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:06.691249    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-207.ec2.internal\" not found"
Sep 16 00:43:06.778856 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:06.778833    2556 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 00:43:06.789500 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:06.789460    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6be15198fb113e0b9a0291128436e59.slice/crio-a8e9ee13ce3a217185129180dd3d4db4b6da628612844a278520a54320e14a0b WatchSource:0}: Error finding container a8e9ee13ce3a217185129180dd3d4db4b6da628612844a278520a54320e14a0b: Status 404 returned error can't find the container with id a8e9ee13ce3a217185129180dd3d4db4b6da628612844a278520a54320e14a0b
Sep 16 00:43:06.791596 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:06.791580    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-207.ec2.internal\" not found"
Sep 16 00:43:06.794031 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:06.794018    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 00:43:06.826850 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:06.826831    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4054c0090de8f54aed802a5d555be580.slice/crio-d70faca41eb4bd2406c4bcaf2339615a2ae4f9e56bac1718adaab3ef541d210a WatchSource:0}: Error finding container d70faca41eb4bd2406c4bcaf2339615a2ae4f9e56bac1718adaab3ef541d210a: Status 404 returned error can't find the container with id d70faca41eb4bd2406c4bcaf2339615a2ae4f9e56bac1718adaab3ef541d210a
Sep 16 00:43:06.891646 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:06.891625    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-207.ec2.internal\" not found"
Sep 16 00:43:06.992120 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:06.992066    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-207.ec2.internal\" not found"
Sep 16 00:43:07.092872 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:07.092846    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-207.ec2.internal\" not found"
Sep 16 00:43:07.183053 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.183030    2556 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 00:43:07.197790 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.197773    2556 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-207.ec2.internal"
Sep 16 00:43:07.208823 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.208807    2556 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 16 00:43:07.209848 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.209837    2556 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-207.ec2.internal"
Sep 16 00:43:07.217848 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.217833    2556 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 16 00:43:07.485425 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.485362    2556 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 00:43:07.575542 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.575518    2556 apiserver.go:52] "Watching apiserver"
Sep 16 00:43:07.581996 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.581976    2556 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 16 00:43:07.582520 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.582497    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-m9fb7","openshift-network-operator/iptables-alerter-7jwf7","openshift-ovn-kubernetes/ovnkube-node-fvnth","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsf5q","openshift-image-registry/node-ca-jvxd7","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-207.ec2.internal","openshift-multus/multus-l7fbr","openshift-multus/network-metrics-daemon-hnw6l","openshift-network-diagnostics/network-check-target-4trxq","kube-system/global-pull-secret-syncer-wk9f6","kube-system/konnectivity-agent-sjrmq","kube-system/kube-apiserver-proxy-ip-10-0-130-207.ec2.internal","openshift-cluster-node-tuning-operator/tuned-pmg7l","openshift-dns/node-resolver-q29p2"]
Sep 16 00:43:07.585156 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.585136    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-l7fbr"
Sep 16 00:43:07.588082 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.587212    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 16 00:43:07.588082 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.587494    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 16 00:43:07.588082 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.587711    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 16 00:43:07.588082 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.587831    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 16 00:43:07.588082 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.587931    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-rtjm9\""
Sep 16 00:43:07.588894 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.588713    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-pmg7l"
Sep 16 00:43:07.590826 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.590642    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 16 00:43:07.592060 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.592041    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 16 00:43:07.592393 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.592252    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-n77q2\""
Sep 16 00:43:07.594912 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.594328    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-fvnth"
Sep 16 00:43:07.597089 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.596739    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 16 00:43:07.597089 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.596743    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-mhfhp\""
Sep 16 00:43:07.597089 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.596841    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 16 00:43:07.597089 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.596917    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 16 00:43:07.597089 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.597008    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 16 00:43:07.597089 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.597070    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 16 00:43:07.597441 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.597220    2556 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 16 00:43:07.598010 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.597993    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 16 00:43:07.599992 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.599694    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsf5q"
Sep 16 00:43:07.602242 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.601706    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 16 00:43:07.602242 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.601879    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 16 00:43:07.602242 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.602089    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-q7fk5\""
Sep 16 00:43:07.602242 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.602196    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-jvxd7"
Sep 16 00:43:07.602242 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.602221    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 16 00:43:07.604263 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.604116    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-q29p2"
Sep 16 00:43:07.604619 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.604457    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-r6rxl\""
Sep 16 00:43:07.604619 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.604521    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 16 00:43:07.604775 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.604757    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 16 00:43:07.604876 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.604834    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 16 00:43:07.607715 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.607695    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 16 00:43:07.607808 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.607778    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-fvhzp\""
Sep 16 00:43:07.607985 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.607962    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 16 00:43:07.608443 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.608418    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4187c8f2-c639-4f19-90aa-5c7dfad934f6-host-run-netns\") pod \"ovnkube-node-fvnth\" (UID: \"4187c8f2-c639-4f19-90aa-5c7dfad934f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fvnth"
Sep 16 00:43:07.608683 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.608657    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4trxq"
Sep 16 00:43:07.608821 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:07.608787    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4trxq" podUID="064c3719-509b-45f7-8367-ab4101e5d3bc"
Sep 16 00:43:07.609708 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.609687    2556 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 16 00:43:07.610753 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.608461    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4187c8f2-c639-4f19-90aa-5c7dfad934f6-host-cni-netd\") pod \"ovnkube-node-fvnth\" (UID: \"4187c8f2-c639-4f19-90aa-5c7dfad934f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fvnth"
Sep 16 00:43:07.610840 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.610783    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/94d2085e-48d2-4ae9-9112-6f4fa7897b43-system-cni-dir\") pod \"multus-l7fbr\" (UID: \"94d2085e-48d2-4ae9-9112-6f4fa7897b43\") " pod="openshift-multus/multus-l7fbr"
Sep 16 00:43:07.610840 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.610818    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/94d2085e-48d2-4ae9-9112-6f4fa7897b43-multus-socket-dir-parent\") pod \"multus-l7fbr\" (UID: \"94d2085e-48d2-4ae9-9112-6f4fa7897b43\") " pod="openshift-multus/multus-l7fbr"
Sep 16 00:43:07.610944 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.610850    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/94d2085e-48d2-4ae9-9112-6f4fa7897b43-host-var-lib-kubelet\") pod \"multus-l7fbr\" (UID: \"94d2085e-48d2-4ae9-9112-6f4fa7897b43\") " pod="openshift-multus/multus-l7fbr"
Sep 16 00:43:07.610944 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.610880    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f8bd281f-9067-462a-9e7c-d63c81f51837-etc-tuned\") pod \"tuned-pmg7l\" (UID: \"f8bd281f-9067-462a-9e7c-d63c81f51837\") " pod="openshift-cluster-node-tuning-operator/tuned-pmg7l"
Sep 16 00:43:07.611044 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.610943    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghsv6\" (UniqueName: \"kubernetes.io/projected/f8bd281f-9067-462a-9e7c-d63c81f51837-kube-api-access-ghsv6\") pod \"tuned-pmg7l\" (UID: \"f8bd281f-9067-462a-9e7c-d63c81f51837\") " pod="openshift-cluster-node-tuning-operator/tuned-pmg7l"
Sep 16 00:43:07.611044 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.610997    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bds95\" (UniqueName: \"kubernetes.io/projected/94d2085e-48d2-4ae9-9112-6f4fa7897b43-kube-api-access-bds95\") pod \"multus-l7fbr\" (UID: \"94d2085e-48d2-4ae9-9112-6f4fa7897b43\") " pod="openshift-multus/multus-l7fbr"
Sep 16 00:43:07.611044 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.611030    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f8bd281f-9067-462a-9e7c-d63c81f51837-etc-systemd\") pod \"tuned-pmg7l\" (UID: \"f8bd281f-9067-462a-9e7c-d63c81f51837\") " pod="openshift-cluster-node-tuning-operator/tuned-pmg7l"
Sep 16 00:43:07.611212 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.611061    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4187c8f2-c639-4f19-90aa-5c7dfad934f6-etc-openvswitch\") pod \"ovnkube-node-fvnth\" (UID: \"4187c8f2-c639-4f19-90aa-5c7dfad934f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fvnth"
Sep 16 00:43:07.611212 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.611087    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4187c8f2-c639-4f19-90aa-5c7dfad934f6-run-ovn\") pod \"ovnkube-node-fvnth\" (UID: \"4187c8f2-c639-4f19-90aa-5c7dfad934f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fvnth"
Sep 16 00:43:07.611212 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.611117    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4187c8f2-c639-4f19-90aa-5c7dfad934f6-log-socket\") pod \"ovnkube-node-fvnth\" (UID: \"4187c8f2-c639-4f19-90aa-5c7dfad934f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fvnth"
Sep 16 00:43:07.611212 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.611178    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/94d2085e-48d2-4ae9-9112-6f4fa7897b43-multus-daemon-config\") pod \"multus-l7fbr\" (UID: \"94d2085e-48d2-4ae9-9112-6f4fa7897b43\") " pod="openshift-multus/multus-l7fbr"
Sep 16 00:43:07.611400 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.611253    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f8bd281f-9067-462a-9e7c-d63c81f51837-etc-sysctl-d\") pod \"tuned-pmg7l\" (UID: \"f8bd281f-9067-462a-9e7c-d63c81f51837\") " pod="openshift-cluster-node-tuning-operator/tuned-pmg7l"
Sep 16 00:43:07.611400 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.611288    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f8bd281f-9067-462a-9e7c-d63c81f51837-run\") pod \"tuned-pmg7l\" (UID: \"f8bd281f-9067-462a-9e7c-d63c81f51837\") " pod="openshift-cluster-node-tuning-operator/tuned-pmg7l"
Sep 16 00:43:07.611400 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.611328    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4187c8f2-c639-4f19-90aa-5c7dfad934f6-env-overrides\") pod \"ovnkube-node-fvnth\" (UID: \"4187c8f2-c639-4f19-90aa-5c7dfad934f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fvnth"
Sep 16 00:43:07.611400 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.611375    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4187c8f2-c639-4f19-90aa-5c7dfad934f6-host-slash\") pod \"ovnkube-node-fvnth\" (UID: \"4187c8f2-c639-4f19-90aa-5c7dfad934f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fvnth"
Sep 16 00:43:07.611620 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.611419    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4187c8f2-c639-4f19-90aa-5c7dfad934f6-ovnkube-config\") pod \"ovnkube-node-fvnth\" (UID: \"4187c8f2-c639-4f19-90aa-5c7dfad934f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fvnth"
Sep 16 00:43:07.611620 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.611451    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwqcj\" (UniqueName: \"kubernetes.io/projected/4187c8f2-c639-4f19-90aa-5c7dfad934f6-kube-api-access-qwqcj\") pod \"ovnkube-node-fvnth\" (UID: \"4187c8f2-c639-4f19-90aa-5c7dfad934f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fvnth"
Sep 16 00:43:07.611620 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.611519    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/94d2085e-48d2-4ae9-9112-6f4fa7897b43-host-run-k8s-cni-cncf-io\") pod \"multus-l7fbr\" (UID: \"94d2085e-48d2-4ae9-9112-6f4fa7897b43\") " pod="openshift-multus/multus-l7fbr"
Sep 16 00:43:07.611620 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.611551    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/94d2085e-48d2-4ae9-9112-6f4fa7897b43-multus-conf-dir\") pod \"multus-l7fbr\" (UID: \"94d2085e-48d2-4ae9-9112-6f4fa7897b43\") " pod="openshift-multus/multus-l7fbr"
Sep 16 00:43:07.611809 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.611623    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f8bd281f-9067-462a-9e7c-d63c81f51837-etc-kubernetes\") pod \"tuned-pmg7l\" (UID: \"f8bd281f-9067-462a-9e7c-d63c81f51837\") " pod="openshift-cluster-node-tuning-operator/tuned-pmg7l"
Sep 16 00:43:07.611809 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.611673    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4187c8f2-c639-4f19-90aa-5c7dfad934f6-host-cni-bin\") pod \"ovnkube-node-fvnth\" (UID: \"4187c8f2-c639-4f19-90aa-5c7dfad934f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fvnth"
Sep 16 00:43:07.611809 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.611703    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/94d2085e-48d2-4ae9-9112-6f4fa7897b43-cnibin\") pod \"multus-l7fbr\" (UID: \"94d2085e-48d2-4ae9-9112-6f4fa7897b43\") " pod="openshift-multus/multus-l7fbr"
Sep 16 00:43:07.612059 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.611846    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-m9fb7"
Sep 16 00:43:07.612059 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.611908    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/94d2085e-48d2-4ae9-9112-6f4fa7897b43-cni-binary-copy\") pod \"multus-l7fbr\" (UID: \"94d2085e-48d2-4ae9-9112-6f4fa7897b43\") " pod="openshift-multus/multus-l7fbr"
Sep 16 00:43:07.612059 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.611925    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/94d2085e-48d2-4ae9-9112-6f4fa7897b43-host-run-netns\") pod \"multus-l7fbr\" (UID: \"94d2085e-48d2-4ae9-9112-6f4fa7897b43\") " pod="openshift-multus/multus-l7fbr"
Sep 16 00:43:07.612059 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.611973    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/94d2085e-48d2-4ae9-9112-6f4fa7897b43-hostroot\") pod \"multus-l7fbr\" (UID: \"94d2085e-48d2-4ae9-9112-6f4fa7897b43\") " pod="openshift-multus/multus-l7fbr"
Sep 16 00:43:07.612059 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.611990    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f8bd281f-9067-462a-9e7c-d63c81f51837-tmp\") pod \"tuned-pmg7l\" (UID: \"f8bd281f-9067-462a-9e7c-d63c81f51837\") " pod="openshift-cluster-node-tuning-operator/tuned-pmg7l"
Sep 16 00:43:07.612361 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.612089    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f8bd281f-9067-462a-9e7c-d63c81f51837-etc-sysctl-conf\") pod \"tuned-pmg7l\" (UID: \"f8bd281f-9067-462a-9e7c-d63c81f51837\") " pod="openshift-cluster-node-tuning-operator/tuned-pmg7l"
Sep 16 00:43:07.612361 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.612237    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f8bd281f-9067-462a-9e7c-d63c81f51837-var-lib-kubelet\") pod \"tuned-pmg7l\" (UID: \"f8bd281f-9067-462a-9e7c-d63c81f51837\") " pod="openshift-cluster-node-tuning-operator/tuned-pmg7l"
Sep 16 00:43:07.612361 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.612318    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4187c8f2-c639-4f19-90aa-5c7dfad934f6-host-kubelet\") pod \"ovnkube-node-fvnth\" (UID: \"4187c8f2-c639-4f19-90aa-5c7dfad934f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fvnth"
Sep 16 00:43:07.612515 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.612368    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rhnf\" (UniqueName: \"kubernetes.io/projected/9fab7ba2-df76-4d06-a48f-d6aee7211db7-kube-api-access-8rhnf\") pod \"aws-ebs-csi-driver-node-zsf5q\" (UID: \"9fab7ba2-df76-4d06-a48f-d6aee7211db7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsf5q"
Sep 16 00:43:07.612515 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.612431    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/94d2085e-48d2-4ae9-9112-6f4fa7897b43-host-var-lib-cni-multus\") pod \"multus-l7fbr\" (UID: \"94d2085e-48d2-4ae9-9112-6f4fa7897b43\") " pod="openshift-multus/multus-l7fbr"
Sep 16 00:43:07.612621 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.612558    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f8bd281f-9067-462a-9e7c-d63c81f51837-sys\") pod \"tuned-pmg7l\" (UID: \"f8bd281f-9067-462a-9e7c-d63c81f51837\") " pod="openshift-cluster-node-tuning-operator/tuned-pmg7l"
Sep 16 00:43:07.612706 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.612664    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4187c8f2-c639-4f19-90aa-5c7dfad934f6-var-lib-openvswitch\") pod \"ovnkube-node-fvnth\" (UID: \"4187c8f2-c639-4f19-90aa-5c7dfad934f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fvnth"
Sep 16 00:43:07.612785 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.612768    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/94d2085e-48d2-4ae9-9112-6f4fa7897b43-host-var-lib-cni-bin\") pod \"multus-l7fbr\" (UID: \"94d2085e-48d2-4ae9-9112-6f4fa7897b43\") " pod="openshift-multus/multus-l7fbr"
Sep 16 00:43:07.612861 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.612801    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f8bd281f-9067-462a-9e7c-d63c81f51837-lib-modules\") pod \"tuned-pmg7l\" (UID: \"f8bd281f-9067-462a-9e7c-d63c81f51837\") " pod="openshift-cluster-node-tuning-operator/tuned-pmg7l"
Sep 16 00:43:07.613267 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.613246    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4187c8f2-c639-4f19-90aa-5c7dfad934f6-node-log\") pod \"ovnkube-node-fvnth\" (UID: \"4187c8f2-c639-4f19-90aa-5c7dfad934f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fvnth"
Sep 16 00:43:07.613340 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.613308    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/94d2085e-48d2-4ae9-9112-6f4fa7897b43-etc-kubernetes\") pod \"multus-l7fbr\" (UID: \"94d2085e-48d2-4ae9-9112-6f4fa7897b43\") " pod="openshift-multus/multus-l7fbr"
Sep 16 00:43:07.613393 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.613339    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f8bd281f-9067-462a-9e7c-d63c81f51837-etc-modprobe-d\") pod \"tuned-pmg7l\" (UID: \"f8bd281f-9067-462a-9e7c-d63c81f51837\") " pod="openshift-cluster-node-tuning-operator/tuned-pmg7l"
Sep 16 00:43:07.613393 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.613362    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4187c8f2-c639-4f19-90aa-5c7dfad934f6-run-systemd\") pod \"ovnkube-node-fvnth\" (UID: \"4187c8f2-c639-4f19-90aa-5c7dfad934f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fvnth"
Sep 16 00:43:07.613526 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.613405    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4187c8f2-c639-4f19-90aa-5c7dfad934f6-ovnkube-script-lib\") pod \"ovnkube-node-fvnth\" (UID: \"4187c8f2-c639-4f19-90aa-5c7dfad934f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fvnth"
Sep 16 00:43:07.613526 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.613431    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9fab7ba2-df76-4d06-a48f-d6aee7211db7-sys-fs\") pod \"aws-ebs-csi-driver-node-zsf5q\" (UID: \"9fab7ba2-df76-4d06-a48f-d6aee7211db7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsf5q"
Sep 16 00:43:07.613526 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.613452    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4187c8f2-c639-4f19-90aa-5c7dfad934f6-run-openvswitch\") pod \"ovnkube-node-fvnth\" (UID: \"4187c8f2-c639-4f19-90aa-5c7dfad934f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fvnth"
Sep 16 00:43:07.613696 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.613567    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4187c8f2-c639-4f19-90aa-5c7dfad934f6-ovn-node-metrics-cert\") pod \"ovnkube-node-fvnth\" (UID: \"4187c8f2-c639-4f19-90aa-5c7dfad934f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fvnth"
Sep 16 00:43:07.613696 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.613624    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9fab7ba2-df76-4d06-a48f-d6aee7211db7-device-dir\") pod \"aws-ebs-csi-driver-node-zsf5q\" (UID: \"9fab7ba2-df76-4d06-a48f-d6aee7211db7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsf5q"
Sep 16 00:43:07.613816 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.613650    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4187c8f2-c639-4f19-90aa-5c7dfad934f6-host-run-ovn-kubernetes\") pod \"ovnkube-node-fvnth\" (UID: \"4187c8f2-c639-4f19-90aa-5c7dfad934f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fvnth"
Sep 16 00:43:07.613816 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.613732    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/94d2085e-48d2-4ae9-9112-6f4fa7897b43-multus-cni-dir\") pod \"multus-l7fbr\" (UID: \"94d2085e-48d2-4ae9-9112-6f4fa7897b43\") " pod="openshift-multus/multus-l7fbr"
Sep 16 00:43:07.613816 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.613755    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/94d2085e-48d2-4ae9-9112-6f4fa7897b43-host-run-multus-certs\") pod \"multus-l7fbr\" (UID: \"94d2085e-48d2-4ae9-9112-6f4fa7897b43\") " pod="openshift-multus/multus-l7fbr"
Sep 16 00:43:07.613816 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.613786    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4187c8f2-c639-4f19-90aa-5c7dfad934f6-systemd-units\") pod \"ovnkube-node-fvnth\" (UID: \"4187c8f2-c639-4f19-90aa-5c7dfad934f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fvnth"
Sep 16 00:43:07.613816 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.613811    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4187c8f2-c639-4f19-90aa-5c7dfad934f6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fvnth\" (UID: \"4187c8f2-c639-4f19-90aa-5c7dfad934f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fvnth"
Sep 16 00:43:07.614087 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.613836    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9fab7ba2-df76-4d06-a48f-d6aee7211db7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zsf5q\" (UID: \"9fab7ba2-df76-4d06-a48f-d6aee7211db7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsf5q"
Sep 16 00:43:07.614204 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.614185    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 16 00:43:07.614391 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.614377    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-pztrf\""
Sep 16 00:43:07.614579 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.614562    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9fab7ba2-df76-4d06-a48f-d6aee7211db7-socket-dir\") pod \"aws-ebs-csi-driver-node-zsf5q\" (UID: \"9fab7ba2-df76-4d06-a48f-d6aee7211db7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsf5q"
Sep 16 00:43:07.614645 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.614594    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9fab7ba2-df76-4d06-a48f-d6aee7211db7-registration-dir\") pod \"aws-ebs-csi-driver-node-zsf5q\" (UID: \"9fab7ba2-df76-4d06-a48f-d6aee7211db7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsf5q"
Sep 16 00:43:07.614645 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.614616    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9fab7ba2-df76-4d06-a48f-d6aee7211db7-etc-selinux\") pod \"aws-ebs-csi-driver-node-zsf5q\" (UID: \"9fab7ba2-df76-4d06-a48f-d6aee7211db7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsf5q"
Sep 16 00:43:07.614747 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.614649    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f8bd281f-9067-462a-9e7c-d63c81f51837-etc-sysconfig\") pod \"tuned-pmg7l\" (UID: \"f8bd281f-9067-462a-9e7c-d63c81f51837\") " pod="openshift-cluster-node-tuning-operator/tuned-pmg7l"
Sep 16 00:43:07.614747 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.614693    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f8bd281f-9067-462a-9e7c-d63c81f51837-host\") pod \"tuned-pmg7l\" (UID: \"f8bd281f-9067-462a-9e7c-d63c81f51837\") " pod="openshift-cluster-node-tuning-operator/tuned-pmg7l"
Sep 16 00:43:07.614747 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.614737    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 16 00:43:07.615914 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.615834    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-7jwf7"
Sep 16 00:43:07.617966 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.617947    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 16 00:43:07.618763 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.618489    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 16 00:43:07.618763 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.618600    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 16 00:43:07.619138 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.619120    2556 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 00:43:07.619528 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.619508    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wk9f6"
Sep 16 00:43:07.619624 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:07.619576    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wk9f6" podUID="c23418e8-6326-481d-9a85-2cbda835d5f6"
Sep 16 00:43:07.619834 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.618930    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-8sxqn\""
Sep 16 00:43:07.622189 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.622170    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-sjrmq"
Sep 16 00:43:07.624059 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.624038    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hnw6l"
Sep 16 00:43:07.624147 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:07.624126    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hnw6l" podUID="f2569a02-edf9-4b33-a7df-a8d0fdce0580"
Sep 16 00:43:07.624210 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.624175    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 16 00:43:07.624315 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.624299    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 16 00:43:07.624414 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.624400    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-qn7nn\""
Sep 16 00:43:07.630661 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.630642    2556 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-5nmz4"
Sep 16 00:43:07.638556 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.638537    2556 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-5nmz4"
Sep 16 00:43:07.699017 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.698994    2556 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 16 00:43:07.715031 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.715005    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4187c8f2-c639-4f19-90aa-5c7dfad934f6-etc-openvswitch\") pod \"ovnkube-node-fvnth\" (UID: \"4187c8f2-c639-4f19-90aa-5c7dfad934f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fvnth"
Sep 16 00:43:07.715127 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.715085    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4187c8f2-c639-4f19-90aa-5c7dfad934f6-etc-openvswitch\") pod \"ovnkube-node-fvnth\" (UID: \"4187c8f2-c639-4f19-90aa-5c7dfad934f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fvnth"
Sep 16 00:43:07.715127 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.715116    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4187c8f2-c639-4f19-90aa-5c7dfad934f6-run-ovn\") pod \"ovnkube-node-fvnth\" (UID: \"4187c8f2-c639-4f19-90aa-5c7dfad934f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fvnth"
Sep 16 00:43:07.715240 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.715152    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4187c8f2-c639-4f19-90aa-5c7dfad934f6-log-socket\") pod \"ovnkube-node-fvnth\" (UID: \"4187c8f2-c639-4f19-90aa-5c7dfad934f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fvnth"
Sep 16 00:43:07.715240 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.715192    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-929sb\" (UniqueName: \"kubernetes.io/projected/21802a49-7d39-41db-a85c-0b6c56472a56-kube-api-access-929sb\") pod \"node-resolver-q29p2\" (UID: \"21802a49-7d39-41db-a85c-0b6c56472a56\") " pod="openshift-dns/node-resolver-q29p2"
Sep 16 00:43:07.715240 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.715219    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4187c8f2-c639-4f19-90aa-5c7dfad934f6-log-socket\") pod \"ovnkube-node-fvnth\" (UID: \"4187c8f2-c639-4f19-90aa-5c7dfad934f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fvnth"
Sep 16 00:43:07.715240 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.715221    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/94d2085e-48d2-4ae9-9112-6f4fa7897b43-multus-daemon-config\") pod \"multus-l7fbr\" (UID: \"94d2085e-48d2-4ae9-9112-6f4fa7897b43\") " pod="openshift-multus/multus-l7fbr"
Sep 16 00:43:07.715433 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.715197    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4187c8f2-c639-4f19-90aa-5c7dfad934f6-run-ovn\") pod \"ovnkube-node-fvnth\" (UID: \"4187c8f2-c639-4f19-90aa-5c7dfad934f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fvnth"
Sep 16 00:43:07.715433 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.715258    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f8bd281f-9067-462a-9e7c-d63c81f51837-etc-sysctl-d\") pod \"tuned-pmg7l\" (UID: \"f8bd281f-9067-462a-9e7c-d63c81f51837\") " pod="openshift-cluster-node-tuning-operator/tuned-pmg7l"
Sep 16 00:43:07.715433 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.715282    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f8bd281f-9067-462a-9e7c-d63c81f51837-run\") pod \"tuned-pmg7l\" (UID: \"f8bd281f-9067-462a-9e7c-d63c81f51837\") " pod="openshift-cluster-node-tuning-operator/tuned-pmg7l"
Sep 16 00:43:07.715433 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.715321    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4187c8f2-c639-4f19-90aa-5c7dfad934f6-env-overrides\") pod \"ovnkube-node-fvnth\" (UID: \"4187c8f2-c639-4f19-90aa-5c7dfad934f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fvnth"
Sep 16 00:43:07.715433 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.715344    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4187c8f2-c639-4f19-90aa-5c7dfad934f6-host-slash\") pod \"ovnkube-node-fvnth\" (UID: \"4187c8f2-c639-4f19-90aa-5c7dfad934f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fvnth"
Sep 16 00:43:07.715433 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.715385    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4187c8f2-c639-4f19-90aa-5c7dfad934f6-ovnkube-config\") pod \"ovnkube-node-fvnth\" (UID: \"4187c8f2-c639-4f19-90aa-5c7dfad934f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fvnth"
Sep 16 00:43:07.715433 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.715408    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwqcj\" (UniqueName: \"kubernetes.io/projected/4187c8f2-c639-4f19-90aa-5c7dfad934f6-kube-api-access-qwqcj\") pod \"ovnkube-node-fvnth\" (UID: \"4187c8f2-c639-4f19-90aa-5c7dfad934f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fvnth"
Sep 16 00:43:07.715433 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.715400    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f8bd281f-9067-462a-9e7c-d63c81f51837-run\") pod \"tuned-pmg7l\" (UID: \"f8bd281f-9067-462a-9e7c-d63c81f51837\") " pod="openshift-cluster-node-tuning-operator/tuned-pmg7l"
Sep 16 00:43:07.715433 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.715422    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f8bd281f-9067-462a-9e7c-d63c81f51837-etc-sysctl-d\") pod \"tuned-pmg7l\" (UID: \"f8bd281f-9067-462a-9e7c-d63c81f51837\") " pod="openshift-cluster-node-tuning-operator/tuned-pmg7l"
Sep 16 00:43:07.715855 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.715490    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4187c8f2-c639-4f19-90aa-5c7dfad934f6-host-slash\") pod \"ovnkube-node-fvnth\" (UID: \"4187c8f2-c639-4f19-90aa-5c7dfad934f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fvnth"
Sep 16 00:43:07.715855 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.715555    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/94d2085e-48d2-4ae9-9112-6f4fa7897b43-host-run-k8s-cni-cncf-io\") pod \"multus-l7fbr\" (UID: \"94d2085e-48d2-4ae9-9112-6f4fa7897b43\") " pod="openshift-multus/multus-l7fbr"
Sep 16 00:43:07.715855 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.715582    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/94d2085e-48d2-4ae9-9112-6f4fa7897b43-host-run-k8s-cni-cncf-io\") pod \"multus-l7fbr\" (UID: \"94d2085e-48d2-4ae9-9112-6f4fa7897b43\") " pod="openshift-multus/multus-l7fbr"
Sep 16 00:43:07.715855 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.715628    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/94d2085e-48d2-4ae9-9112-6f4fa7897b43-multus-conf-dir\") pod \"multus-l7fbr\" (UID: \"94d2085e-48d2-4ae9-9112-6f4fa7897b43\") " pod="openshift-multus/multus-l7fbr"
Sep 16 00:43:07.715855 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.715653    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f8bd281f-9067-462a-9e7c-d63c81f51837-etc-kubernetes\") pod \"tuned-pmg7l\" (UID: \"f8bd281f-9067-462a-9e7c-d63c81f51837\") " pod="openshift-cluster-node-tuning-operator/tuned-pmg7l"
Sep 16 00:43:07.715855 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.715677    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4187c8f2-c639-4f19-90aa-5c7dfad934f6-host-cni-bin\") pod \"ovnkube-node-fvnth\" (UID: \"4187c8f2-c639-4f19-90aa-5c7dfad934f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fvnth"
Sep 16 00:43:07.715855 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.715715    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/94d2085e-48d2-4ae9-9112-6f4fa7897b43-multus-conf-dir\") pod \"multus-l7fbr\" (UID: \"94d2085e-48d2-4ae9-9112-6f4fa7897b43\") " pod="openshift-multus/multus-l7fbr"
Sep 16 00:43:07.715855 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.715718    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dnb28\" (UniqueName: \"kubernetes.io/projected/07a10807-5530-48f1-9aeb-8becb1380db5-kube-api-access-dnb28\") pod \"node-ca-jvxd7\" (UID: \"07a10807-5530-48f1-9aeb-8becb1380db5\") " pod="openshift-image-registry/node-ca-jvxd7"
Sep 16 00:43:07.715855 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.715768    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7tqn\" (UniqueName: \"kubernetes.io/projected/064c3719-509b-45f7-8367-ab4101e5d3bc-kube-api-access-d7tqn\") pod \"network-check-target-4trxq\" (UID: \"064c3719-509b-45f7-8367-ab4101e5d3bc\") " pod="openshift-network-diagnostics/network-check-target-4trxq"
Sep 16 00:43:07.715855 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.715795    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f8bd281f-9067-462a-9e7c-d63c81f51837-etc-kubernetes\") pod \"tuned-pmg7l\" (UID: \"f8bd281f-9067-462a-9e7c-d63c81f51837\") " pod="openshift-cluster-node-tuning-operator/tuned-pmg7l"
Sep 16 00:43:07.716213 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.715898    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5mt5\" (UniqueName: \"kubernetes.io/projected/f2569a02-edf9-4b33-a7df-a8d0fdce0580-kube-api-access-w5mt5\") pod \"network-metrics-daemon-hnw6l\" (UID: \"f2569a02-edf9-4b33-a7df-a8d0fdce0580\") " pod="openshift-multus/network-metrics-daemon-hnw6l"
Sep 16 00:43:07.716213 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.715921    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/94d2085e-48d2-4ae9-9112-6f4fa7897b43-multus-daemon-config\") pod \"multus-l7fbr\" (UID: \"94d2085e-48d2-4ae9-9112-6f4fa7897b43\") " pod="openshift-multus/multus-l7fbr"
Sep 16 00:43:07.716213 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.715929    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/94d2085e-48d2-4ae9-9112-6f4fa7897b43-cnibin\") pod \"multus-l7fbr\" (UID: \"94d2085e-48d2-4ae9-9112-6f4fa7897b43\") " pod="openshift-multus/multus-l7fbr"
Sep 16 00:43:07.716213 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.715996    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/94d2085e-48d2-4ae9-9112-6f4fa7897b43-cnibin\") pod \"multus-l7fbr\" (UID: \"94d2085e-48d2-4ae9-9112-6f4fa7897b43\") " pod="openshift-multus/multus-l7fbr"
Sep 16 00:43:07.716213 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.716028    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/94d2085e-48d2-4ae9-9112-6f4fa7897b43-cni-binary-copy\") pod \"multus-l7fbr\" (UID: \"94d2085e-48d2-4ae9-9112-6f4fa7897b43\") " pod="openshift-multus/multus-l7fbr"
Sep 16 00:43:07.716213 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.716026    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4187c8f2-c639-4f19-90aa-5c7dfad934f6-host-cni-bin\") pod \"ovnkube-node-fvnth\" (UID: \"4187c8f2-c639-4f19-90aa-5c7dfad934f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fvnth"
Sep 16 00:43:07.716213 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.716077    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/94d2085e-48d2-4ae9-9112-6f4fa7897b43-host-run-netns\") pod \"multus-l7fbr\" (UID: \"94d2085e-48d2-4ae9-9112-6f4fa7897b43\") " pod="openshift-multus/multus-l7fbr"
Sep 16 00:43:07.716213 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.716159    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/94d2085e-48d2-4ae9-9112-6f4fa7897b43-hostroot\") pod \"multus-l7fbr\" (UID: \"94d2085e-48d2-4ae9-9112-6f4fa7897b43\") " pod="openshift-multus/multus-l7fbr"
Sep 16 00:43:07.716213 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.716132    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/94d2085e-48d2-4ae9-9112-6f4fa7897b43-host-run-netns\") pod \"multus-l7fbr\" (UID: \"94d2085e-48d2-4ae9-9112-6f4fa7897b43\") " pod="openshift-multus/multus-l7fbr"
Sep 16 00:43:07.716213 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.716179    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f8bd281f-9067-462a-9e7c-d63c81f51837-tmp\") pod \"tuned-pmg7l\" (UID: \"f8bd281f-9067-462a-9e7c-d63c81f51837\") " pod="openshift-cluster-node-tuning-operator/tuned-pmg7l"
Sep 16 00:43:07.716599 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.716222    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/94d2085e-48d2-4ae9-9112-6f4fa7897b43-hostroot\") pod \"multus-l7fbr\" (UID: \"94d2085e-48d2-4ae9-9112-6f4fa7897b43\") " pod="openshift-multus/multus-l7fbr"
Sep 16 00:43:07.716599 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.716259    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/21802a49-7d39-41db-a85c-0b6c56472a56-tmp-dir\") pod \"node-resolver-q29p2\" (UID: \"21802a49-7d39-41db-a85c-0b6c56472a56\") " pod="openshift-dns/node-resolver-q29p2"
Sep 16 00:43:07.716599 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.716328    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f8bd281f-9067-462a-9e7c-d63c81f51837-etc-sysctl-conf\") pod \"tuned-pmg7l\" (UID: \"f8bd281f-9067-462a-9e7c-d63c81f51837\") " pod="openshift-cluster-node-tuning-operator/tuned-pmg7l"
Sep 16 00:43:07.716599 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.716352    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f8bd281f-9067-462a-9e7c-d63c81f51837-var-lib-kubelet\") pod \"tuned-pmg7l\" (UID: \"f8bd281f-9067-462a-9e7c-d63c81f51837\") " pod="openshift-cluster-node-tuning-operator/tuned-pmg7l"
Sep 16 00:43:07.716599 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.716394    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4187c8f2-c639-4f19-90aa-5c7dfad934f6-host-kubelet\") pod \"ovnkube-node-fvnth\" (UID: \"4187c8f2-c639-4f19-90aa-5c7dfad934f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fvnth"
Sep 16 00:43:07.716599 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.716424    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rhnf\" (UniqueName: \"kubernetes.io/projected/9fab7ba2-df76-4d06-a48f-d6aee7211db7-kube-api-access-8rhnf\") pod \"aws-ebs-csi-driver-node-zsf5q\" (UID: \"9fab7ba2-df76-4d06-a48f-d6aee7211db7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsf5q"
Sep 16 00:43:07.716599 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.716498    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/94d2085e-48d2-4ae9-9112-6f4fa7897b43-cni-binary-copy\") pod \"multus-l7fbr\" (UID: \"94d2085e-48d2-4ae9-9112-6f4fa7897b43\") " pod="openshift-multus/multus-l7fbr"
Sep 16 00:43:07.716599 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.716504    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/94d2085e-48d2-4ae9-9112-6f4fa7897b43-host-var-lib-cni-multus\") pod \"multus-l7fbr\" (UID: \"94d2085e-48d2-4ae9-9112-6f4fa7897b43\") " pod="openshift-multus/multus-l7fbr"
Sep 16 00:43:07.716599 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.716527    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f8bd281f-9067-462a-9e7c-d63c81f51837-etc-sysctl-conf\") pod \"tuned-pmg7l\" (UID: \"f8bd281f-9067-462a-9e7c-d63c81f51837\") " pod="openshift-cluster-node-tuning-operator/tuned-pmg7l"
Sep 16 00:43:07.716599 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.716576    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f8bd281f-9067-462a-9e7c-d63c81f51837-sys\") pod \"tuned-pmg7l\" (UID: \"f8bd281f-9067-462a-9e7c-d63c81f51837\") " pod="openshift-cluster-node-tuning-operator/tuned-pmg7l"
Sep 16 00:43:07.716599 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.716551    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/94d2085e-48d2-4ae9-9112-6f4fa7897b43-host-var-lib-cni-multus\") pod \"multus-l7fbr\" (UID: \"94d2085e-48d2-4ae9-9112-6f4fa7897b43\") " pod="openshift-multus/multus-l7fbr"
Sep 16 00:43:07.716599 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.716591    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4187c8f2-c639-4f19-90aa-5c7dfad934f6-host-kubelet\") pod \"ovnkube-node-fvnth\" (UID: \"4187c8f2-c639-4f19-90aa-5c7dfad934f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fvnth"
Sep 16 00:43:07.717175 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.716609    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4187c8f2-c639-4f19-90aa-5c7dfad934f6-var-lib-openvswitch\") pod \"ovnkube-node-fvnth\" (UID: \"4187c8f2-c639-4f19-90aa-5c7dfad934f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fvnth"
Sep 16 00:43:07.717175 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.716637    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/07a10807-5530-48f1-9aeb-8becb1380db5-host\") pod \"node-ca-jvxd7\" (UID: \"07a10807-5530-48f1-9aeb-8becb1380db5\") " pod="openshift-image-registry/node-ca-jvxd7"
Sep 16 00:43:07.717175 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.716651    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f8bd281f-9067-462a-9e7c-d63c81f51837-sys\") pod \"tuned-pmg7l\" (UID: \"f8bd281f-9067-462a-9e7c-d63c81f51837\") " pod="openshift-cluster-node-tuning-operator/tuned-pmg7l"
Sep 16 00:43:07.717175 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.716666    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/94d2085e-48d2-4ae9-9112-6f4fa7897b43-host-var-lib-cni-bin\") pod \"multus-l7fbr\" (UID: \"94d2085e-48d2-4ae9-9112-6f4fa7897b43\") " pod="openshift-multus/multus-l7fbr"
Sep 16 00:43:07.717175 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.716620    2556 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 16 00:43:07.717175 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.716668    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f8bd281f-9067-462a-9e7c-d63c81f51837-var-lib-kubelet\") pod \"tuned-pmg7l\" (UID: \"f8bd281f-9067-462a-9e7c-d63c81f51837\") " pod="openshift-cluster-node-tuning-operator/tuned-pmg7l"
Sep 16 00:43:07.717175 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.716708    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4187c8f2-c639-4f19-90aa-5c7dfad934f6-env-overrides\") pod \"ovnkube-node-fvnth\" (UID: \"4187c8f2-c639-4f19-90aa-5c7dfad934f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fvnth"
Sep 16 00:43:07.717175 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.716714    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4187c8f2-c639-4f19-90aa-5c7dfad934f6-ovnkube-config\") pod \"ovnkube-node-fvnth\" (UID: \"4187c8f2-c639-4f19-90aa-5c7dfad934f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fvnth"
Sep 16 00:43:07.717175 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.716717    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/94d2085e-48d2-4ae9-9112-6f4fa7897b43-host-var-lib-cni-bin\") pod \"multus-l7fbr\" (UID: \"94d2085e-48d2-4ae9-9112-6f4fa7897b43\") " pod="openshift-multus/multus-l7fbr"
Sep 16 00:43:07.717175 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.716690    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f8bd281f-9067-462a-9e7c-d63c81f51837-lib-modules\") pod \"tuned-pmg7l\" (UID: \"f8bd281f-9067-462a-9e7c-d63c81f51837\") " pod="openshift-cluster-node-tuning-operator/tuned-pmg7l"
Sep 16 00:43:07.717175 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.716687    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4187c8f2-c639-4f19-90aa-5c7dfad934f6-var-lib-openvswitch\") pod \"ovnkube-node-fvnth\" (UID: \"4187c8f2-c639-4f19-90aa-5c7dfad934f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fvnth"
Sep 16 00:43:07.717175 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.716753    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4187c8f2-c639-4f19-90aa-5c7dfad934f6-node-log\") pod \"ovnkube-node-fvnth\" (UID: \"4187c8f2-c639-4f19-90aa-5c7dfad934f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fvnth"
Sep 16 00:43:07.717175 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.716768    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/21802a49-7d39-41db-a85c-0b6c56472a56-hosts-file\") pod \"node-resolver-q29p2\" (UID: \"21802a49-7d39-41db-a85c-0b6c56472a56\") " pod="openshift-dns/node-resolver-q29p2"
Sep 16 00:43:07.717175 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.716791    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rx6zq\" (UniqueName: \"kubernetes.io/projected/e94d6aa4-58be-4a67-bd8b-5670201fc0af-kube-api-access-rx6zq\") pod \"multus-additional-cni-plugins-m9fb7\" (UID: \"e94d6aa4-58be-4a67-bd8b-5670201fc0af\") " pod="openshift-multus/multus-additional-cni-plugins-m9fb7"
Sep 16 00:43:07.717175 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.716822    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c23418e8-6326-481d-9a85-2cbda835d5f6-dbus\") pod \"global-pull-secret-syncer-wk9f6\" (UID: \"c23418e8-6326-481d-9a85-2cbda835d5f6\") " pod="kube-system/global-pull-secret-syncer-wk9f6"
Sep 16 00:43:07.717175 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.716815    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4187c8f2-c639-4f19-90aa-5c7dfad934f6-node-log\") pod \"ovnkube-node-fvnth\" (UID: \"4187c8f2-c639-4f19-90aa-5c7dfad934f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fvnth"
Sep 16 00:43:07.717175 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.716846    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f2569a02-edf9-4b33-a7df-a8d0fdce0580-metrics-certs\") pod \"network-metrics-daemon-hnw6l\" (UID: \"f2569a02-edf9-4b33-a7df-a8d0fdce0580\") " pod="openshift-multus/network-metrics-daemon-hnw6l"
Sep 16 00:43:07.717175 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.716890    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f8bd281f-9067-462a-9e7c-d63c81f51837-lib-modules\") pod \"tuned-pmg7l\" (UID: \"f8bd281f-9067-462a-9e7c-d63c81f51837\") " pod="openshift-cluster-node-tuning-operator/tuned-pmg7l"
Sep 16 00:43:07.718015 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.716940    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/94d2085e-48d2-4ae9-9112-6f4fa7897b43-etc-kubernetes\") pod \"multus-l7fbr\" (UID: \"94d2085e-48d2-4ae9-9112-6f4fa7897b43\") " pod="openshift-multus/multus-l7fbr"
Sep 16 00:43:07.718015 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.716990    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f8bd281f-9067-462a-9e7c-d63c81f51837-etc-modprobe-d\") pod \"tuned-pmg7l\" (UID: \"f8bd281f-9067-462a-9e7c-d63c81f51837\") " pod="openshift-cluster-node-tuning-operator/tuned-pmg7l"
Sep 16 00:43:07.718015 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.717019    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/94d2085e-48d2-4ae9-9112-6f4fa7897b43-etc-kubernetes\") pod \"multus-l7fbr\" (UID: \"94d2085e-48d2-4ae9-9112-6f4fa7897b43\") " pod="openshift-multus/multus-l7fbr"
Sep 16 00:43:07.718015 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.717037    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4187c8f2-c639-4f19-90aa-5c7dfad934f6-run-systemd\") pod \"ovnkube-node-fvnth\" (UID: \"4187c8f2-c639-4f19-90aa-5c7dfad934f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fvnth"
Sep 16 00:43:07.718015 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.717072    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4187c8f2-c639-4f19-90aa-5c7dfad934f6-ovnkube-script-lib\") pod \"ovnkube-node-fvnth\" (UID: \"4187c8f2-c639-4f19-90aa-5c7dfad934f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fvnth"
Sep 16 00:43:07.718015 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.717094    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9fab7ba2-df76-4d06-a48f-d6aee7211db7-sys-fs\") pod \"aws-ebs-csi-driver-node-zsf5q\" (UID: \"9fab7ba2-df76-4d06-a48f-d6aee7211db7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsf5q"
Sep 16 00:43:07.718015 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.717110    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f8bd281f-9067-462a-9e7c-d63c81f51837-etc-modprobe-d\") pod \"tuned-pmg7l\" (UID: \"f8bd281f-9067-462a-9e7c-d63c81f51837\") " pod="openshift-cluster-node-tuning-operator/tuned-pmg7l"
Sep 16 00:43:07.718015 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.717125    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4187c8f2-c639-4f19-90aa-5c7dfad934f6-run-systemd\") pod \"ovnkube-node-fvnth\" (UID: \"4187c8f2-c639-4f19-90aa-5c7dfad934f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fvnth"
Sep 16 00:43:07.718015 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.717148    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c23418e8-6326-481d-9a85-2cbda835d5f6-original-pull-secret\") pod \"global-pull-secret-syncer-wk9f6\" (UID: \"c23418e8-6326-481d-9a85-2cbda835d5f6\") " pod="kube-system/global-pull-secret-syncer-wk9f6"
Sep 16 00:43:07.718015 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.717187    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4187c8f2-c639-4f19-90aa-5c7dfad934f6-run-openvswitch\") pod \"ovnkube-node-fvnth\" (UID: \"4187c8f2-c639-4f19-90aa-5c7dfad934f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fvnth"
Sep 16 00:43:07.718015 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.717197    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9fab7ba2-df76-4d06-a48f-d6aee7211db7-sys-fs\") pod \"aws-ebs-csi-driver-node-zsf5q\" (UID: \"9fab7ba2-df76-4d06-a48f-d6aee7211db7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsf5q"
Sep 16 00:43:07.718015 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.717242    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4187c8f2-c639-4f19-90aa-5c7dfad934f6-ovn-node-metrics-cert\") pod \"ovnkube-node-fvnth\" (UID: \"4187c8f2-c639-4f19-90aa-5c7dfad934f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fvnth"
Sep 16 00:43:07.718015 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.717294    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4187c8f2-c639-4f19-90aa-5c7dfad934f6-run-openvswitch\") pod \"ovnkube-node-fvnth\" (UID: \"4187c8f2-c639-4f19-90aa-5c7dfad934f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fvnth"
Sep 16 00:43:07.718015 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.717325    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9fab7ba2-df76-4d06-a48f-d6aee7211db7-device-dir\") pod \"aws-ebs-csi-driver-node-zsf5q\" (UID: \"9fab7ba2-df76-4d06-a48f-d6aee7211db7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsf5q"
Sep 16 00:43:07.718015 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.717367    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f1c71941-4979-4074-beb1-cf727004f45d-agent-certs\") pod \"konnectivity-agent-sjrmq\" (UID: \"f1c71941-4979-4074-beb1-cf727004f45d\") " pod="kube-system/konnectivity-agent-sjrmq"
Sep 16 00:43:07.718015 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.717415    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4187c8f2-c639-4f19-90aa-5c7dfad934f6-host-run-ovn-kubernetes\") pod \"ovnkube-node-fvnth\" (UID: \"4187c8f2-c639-4f19-90aa-5c7dfad934f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fvnth"
Sep 16 00:43:07.718015 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.717443    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e94d6aa4-58be-4a67-bd8b-5670201fc0af-tuning-conf-dir\") pod \"multus-additional-cni-plugins-m9fb7\" (UID: \"e94d6aa4-58be-4a67-bd8b-5670201fc0af\") " pod="openshift-multus/multus-additional-cni-plugins-m9fb7"
Sep 16 00:43:07.718728 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.717505    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e94d6aa4-58be-4a67-bd8b-5670201fc0af-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-m9fb7\" (UID: \"e94d6aa4-58be-4a67-bd8b-5670201fc0af\") " pod="openshift-multus/multus-additional-cni-plugins-m9fb7"
Sep 16 00:43:07.718728 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.717549    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/94d2085e-48d2-4ae9-9112-6f4fa7897b43-multus-cni-dir\") pod \"multus-l7fbr\" (UID: \"94d2085e-48d2-4ae9-9112-6f4fa7897b43\") " pod="openshift-multus/multus-l7fbr"
Sep 16 00:43:07.718728 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.717575    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/94d2085e-48d2-4ae9-9112-6f4fa7897b43-host-run-multus-certs\") pod \"multus-l7fbr\" (UID: \"94d2085e-48d2-4ae9-9112-6f4fa7897b43\") " pod="openshift-multus/multus-l7fbr"
Sep 16 00:43:07.718728 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.717622    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4187c8f2-c639-4f19-90aa-5c7dfad934f6-systemd-units\") pod \"ovnkube-node-fvnth\" (UID: \"4187c8f2-c639-4f19-90aa-5c7dfad934f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fvnth"
Sep 16 00:43:07.718728 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.717645    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4187c8f2-c639-4f19-90aa-5c7dfad934f6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fvnth\" (UID: \"4187c8f2-c639-4f19-90aa-5c7dfad934f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fvnth"
Sep 16 00:43:07.718728 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.717670    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9fab7ba2-df76-4d06-a48f-d6aee7211db7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zsf5q\" (UID: \"9fab7ba2-df76-4d06-a48f-d6aee7211db7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsf5q"
Sep 16 00:43:07.718728 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.717692    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9fab7ba2-df76-4d06-a48f-d6aee7211db7-socket-dir\") pod \"aws-ebs-csi-driver-node-zsf5q\" (UID: \"9fab7ba2-df76-4d06-a48f-d6aee7211db7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsf5q"
Sep 16 00:43:07.718728 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.717714    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9fab7ba2-df76-4d06-a48f-d6aee7211db7-registration-dir\") pod \"aws-ebs-csi-driver-node-zsf5q\" (UID: \"9fab7ba2-df76-4d06-a48f-d6aee7211db7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsf5q"
Sep 16 00:43:07.718728 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.717720    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4187c8f2-c639-4f19-90aa-5c7dfad934f6-ovnkube-script-lib\") pod \"ovnkube-node-fvnth\" (UID: \"4187c8f2-c639-4f19-90aa-5c7dfad934f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fvnth"
Sep 16 00:43:07.718728 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.717759    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9fab7ba2-df76-4d06-a48f-d6aee7211db7-etc-selinux\") pod \"aws-ebs-csi-driver-node-zsf5q\" (UID: \"9fab7ba2-df76-4d06-a48f-d6aee7211db7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsf5q"
Sep 16 00:43:07.718728 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.717769    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9fab7ba2-df76-4d06-a48f-d6aee7211db7-registration-dir\") pod \"aws-ebs-csi-driver-node-zsf5q\" (UID: \"9fab7ba2-df76-4d06-a48f-d6aee7211db7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsf5q"
Sep 16 00:43:07.718728 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.717792    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f8bd281f-9067-462a-9e7c-d63c81f51837-etc-sysconfig\") pod \"tuned-pmg7l\" (UID: \"f8bd281f-9067-462a-9e7c-d63c81f51837\") " pod="openshift-cluster-node-tuning-operator/tuned-pmg7l"
Sep 16 00:43:07.718728 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.717795    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4187c8f2-c639-4f19-90aa-5c7dfad934f6-systemd-units\") pod \"ovnkube-node-fvnth\" (UID: \"4187c8f2-c639-4f19-90aa-5c7dfad934f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fvnth"
Sep 16 00:43:07.718728 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.717828    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4187c8f2-c639-4f19-90aa-5c7dfad934f6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fvnth\" (UID: \"4187c8f2-c639-4f19-90aa-5c7dfad934f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fvnth"
Sep 16 00:43:07.718728 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.717836    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/94d2085e-48d2-4ae9-9112-6f4fa7897b43-multus-cni-dir\") pod \"multus-l7fbr\" (UID: \"94d2085e-48d2-4ae9-9112-6f4fa7897b43\") " pod="openshift-multus/multus-l7fbr"
Sep 16 00:43:07.718728 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.717856    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f8bd281f-9067-462a-9e7c-d63c81f51837-host\") pod \"tuned-pmg7l\" (UID: \"f8bd281f-9067-462a-9e7c-d63c81f51837\") " pod="openshift-cluster-node-tuning-operator/tuned-pmg7l"
Sep 16 00:43:07.718728 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.717856    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9fab7ba2-df76-4d06-a48f-d6aee7211db7-device-dir\") pod \"aws-ebs-csi-driver-node-zsf5q\" (UID: \"9fab7ba2-df76-4d06-a48f-d6aee7211db7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsf5q"
Sep 16 00:43:07.719555 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.717867    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/94d2085e-48d2-4ae9-9112-6f4fa7897b43-host-run-multus-certs\") pod \"multus-l7fbr\" (UID: \"94d2085e-48d2-4ae9-9112-6f4fa7897b43\") " pod="openshift-multus/multus-l7fbr"
Sep 16 00:43:07.719555 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.717915    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4187c8f2-c639-4f19-90aa-5c7dfad934f6-host-run-ovn-kubernetes\") pod \"ovnkube-node-fvnth\" (UID: \"4187c8f2-c639-4f19-90aa-5c7dfad934f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fvnth"
Sep 16 00:43:07.719555 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.717925    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f8bd281f-9067-462a-9e7c-d63c81f51837-etc-sysconfig\") pod \"tuned-pmg7l\" (UID: \"f8bd281f-9067-462a-9e7c-d63c81f51837\") " pod="openshift-cluster-node-tuning-operator/tuned-pmg7l"
Sep 16 00:43:07.719555 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.717956    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9fab7ba2-df76-4d06-a48f-d6aee7211db7-etc-selinux\") pod \"aws-ebs-csi-driver-node-zsf5q\" (UID: \"9fab7ba2-df76-4d06-a48f-d6aee7211db7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsf5q"
Sep 16 00:43:07.719555 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.717973    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9fab7ba2-df76-4d06-a48f-d6aee7211db7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zsf5q\" (UID: \"9fab7ba2-df76-4d06-a48f-d6aee7211db7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsf5q"
Sep 16 00:43:07.719555 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.717990    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f8bd281f-9067-462a-9e7c-d63c81f51837-host\") pod \"tuned-pmg7l\" (UID: \"f8bd281f-9067-462a-9e7c-d63c81f51837\") " pod="openshift-cluster-node-tuning-operator/tuned-pmg7l"
Sep 16 00:43:07.719555 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.718006    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e94d6aa4-58be-4a67-bd8b-5670201fc0af-system-cni-dir\") pod \"multus-additional-cni-plugins-m9fb7\" (UID: \"e94d6aa4-58be-4a67-bd8b-5670201fc0af\") " pod="openshift-multus/multus-additional-cni-plugins-m9fb7"
Sep 16 00:43:07.719555 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.718034    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e94d6aa4-58be-4a67-bd8b-5670201fc0af-cnibin\") pod \"multus-additional-cni-plugins-m9fb7\" (UID: \"e94d6aa4-58be-4a67-bd8b-5670201fc0af\") " pod="openshift-multus/multus-additional-cni-plugins-m9fb7"
Sep 16 00:43:07.719555 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.718054    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e94d6aa4-58be-4a67-bd8b-5670201fc0af-cni-binary-copy\") pod \"multus-additional-cni-plugins-m9fb7\" (UID: \"e94d6aa4-58be-4a67-bd8b-5670201fc0af\") " pod="openshift-multus/multus-additional-cni-plugins-m9fb7"
Sep 16 00:43:07.719555 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.718072    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9fab7ba2-df76-4d06-a48f-d6aee7211db7-socket-dir\") pod \"aws-ebs-csi-driver-node-zsf5q\" (UID: \"9fab7ba2-df76-4d06-a48f-d6aee7211db7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsf5q"
Sep 16 00:43:07.719555 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.718080    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e94d6aa4-58be-4a67-bd8b-5670201fc0af-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-m9fb7\" (UID: \"e94d6aa4-58be-4a67-bd8b-5670201fc0af\") " pod="openshift-multus/multus-additional-cni-plugins-m9fb7"
Sep 16 00:43:07.719555 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.718110    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c23418e8-6326-481d-9a85-2cbda835d5f6-kubelet-config\") pod \"global-pull-secret-syncer-wk9f6\" (UID: \"c23418e8-6326-481d-9a85-2cbda835d5f6\") " pod="kube-system/global-pull-secret-syncer-wk9f6"
Sep 16 00:43:07.719555 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.718138    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/07a10807-5530-48f1-9aeb-8becb1380db5-serviceca\") pod \"node-ca-jvxd7\" (UID: \"07a10807-5530-48f1-9aeb-8becb1380db5\") " pod="openshift-image-registry/node-ca-jvxd7"
Sep 16 00:43:07.719555 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.718163    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4187c8f2-c639-4f19-90aa-5c7dfad934f6-host-run-netns\") pod \"ovnkube-node-fvnth\" (UID: \"4187c8f2-c639-4f19-90aa-5c7dfad934f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fvnth"
Sep 16 00:43:07.719555 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.718193    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4187c8f2-c639-4f19-90aa-5c7dfad934f6-host-cni-netd\") pod \"ovnkube-node-fvnth\" (UID: \"4187c8f2-c639-4f19-90aa-5c7dfad934f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fvnth"
Sep 16 00:43:07.719555 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.718242    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/94d2085e-48d2-4ae9-9112-6f4fa7897b43-system-cni-dir\") pod \"multus-l7fbr\" (UID: \"94d2085e-48d2-4ae9-9112-6f4fa7897b43\") " pod="openshift-multus/multus-l7fbr"
Sep 16 00:43:07.719555 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.718240    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4187c8f2-c639-4f19-90aa-5c7dfad934f6-host-run-netns\") pod \"ovnkube-node-fvnth\" (UID: \"4187c8f2-c639-4f19-90aa-5c7dfad934f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fvnth"
Sep 16 00:43:07.720343 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.718266    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/94d2085e-48d2-4ae9-9112-6f4fa7897b43-multus-socket-dir-parent\") pod \"multus-l7fbr\" (UID: \"94d2085e-48d2-4ae9-9112-6f4fa7897b43\") " pod="openshift-multus/multus-l7fbr"
Sep 16 00:43:07.720343 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.718297    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/94d2085e-48d2-4ae9-9112-6f4fa7897b43-host-var-lib-kubelet\") pod \"multus-l7fbr\" (UID: \"94d2085e-48d2-4ae9-9112-6f4fa7897b43\") " pod="openshift-multus/multus-l7fbr"
Sep 16 00:43:07.720343 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.718319    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f8bd281f-9067-462a-9e7c-d63c81f51837-etc-tuned\") pod \"tuned-pmg7l\" (UID: \"f8bd281f-9067-462a-9e7c-d63c81f51837\") " pod="openshift-cluster-node-tuning-operator/tuned-pmg7l"
Sep 16 00:43:07.720343 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.718337    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/94d2085e-48d2-4ae9-9112-6f4fa7897b43-system-cni-dir\") pod \"multus-l7fbr\" (UID: \"94d2085e-48d2-4ae9-9112-6f4fa7897b43\") " pod="openshift-multus/multus-l7fbr"
Sep 16 00:43:07.720343 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.718340    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghsv6\" (UniqueName: \"kubernetes.io/projected/f8bd281f-9067-462a-9e7c-d63c81f51837-kube-api-access-ghsv6\") pod \"tuned-pmg7l\" (UID: \"f8bd281f-9067-462a-9e7c-d63c81f51837\") " pod="openshift-cluster-node-tuning-operator/tuned-pmg7l"
Sep 16 00:43:07.720343 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.718373    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a949202d-9c40-4794-96da-77b18812a729-iptables-alerter-script\") pod \"iptables-alerter-7jwf7\" (UID: \"a949202d-9c40-4794-96da-77b18812a729\") " pod="openshift-network-operator/iptables-alerter-7jwf7"
Sep 16 00:43:07.720343 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.718401    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bds95\" (UniqueName: \"kubernetes.io/projected/94d2085e-48d2-4ae9-9112-6f4fa7897b43-kube-api-access-bds95\") pod \"multus-l7fbr\" (UID: \"94d2085e-48d2-4ae9-9112-6f4fa7897b43\") " pod="openshift-multus/multus-l7fbr"
Sep 16 00:43:07.720343 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.718435    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a949202d-9c40-4794-96da-77b18812a729-host-slash\") pod \"iptables-alerter-7jwf7\" (UID: \"a949202d-9c40-4794-96da-77b18812a729\") " pod="openshift-network-operator/iptables-alerter-7jwf7"
Sep 16 00:43:07.720343 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.718487    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pg7z9\" (UniqueName: \"kubernetes.io/projected/a949202d-9c40-4794-96da-77b18812a729-kube-api-access-pg7z9\") pod \"iptables-alerter-7jwf7\" (UID: \"a949202d-9c40-4794-96da-77b18812a729\") " pod="openshift-network-operator/iptables-alerter-7jwf7"
Sep 16 00:43:07.720343 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.718520    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f1c71941-4979-4074-beb1-cf727004f45d-konnectivity-ca\") pod \"konnectivity-agent-sjrmq\" (UID: \"f1c71941-4979-4074-beb1-cf727004f45d\") " pod="kube-system/konnectivity-agent-sjrmq"
Sep 16 00:43:07.720343 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.718548    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f8bd281f-9067-462a-9e7c-d63c81f51837-etc-systemd\") pod \"tuned-pmg7l\" (UID: \"f8bd281f-9067-462a-9e7c-d63c81f51837\") " pod="openshift-cluster-node-tuning-operator/tuned-pmg7l"
Sep 16 00:43:07.720343 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.718602    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/94d2085e-48d2-4ae9-9112-6f4fa7897b43-host-var-lib-kubelet\") pod \"multus-l7fbr\" (UID: \"94d2085e-48d2-4ae9-9112-6f4fa7897b43\") " pod="openshift-multus/multus-l7fbr"
Sep 16 00:43:07.720343 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.718296    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4187c8f2-c639-4f19-90aa-5c7dfad934f6-host-cni-netd\") pod \"ovnkube-node-fvnth\" (UID: \"4187c8f2-c639-4f19-90aa-5c7dfad934f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fvnth"
Sep 16 00:43:07.720343 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.718679    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f8bd281f-9067-462a-9e7c-d63c81f51837-etc-systemd\") pod \"tuned-pmg7l\" (UID: \"f8bd281f-9067-462a-9e7c-d63c81f51837\") " pod="openshift-cluster-node-tuning-operator/tuned-pmg7l"
Sep 16 00:43:07.720343 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.718747    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/94d2085e-48d2-4ae9-9112-6f4fa7897b43-multus-socket-dir-parent\") pod \"multus-l7fbr\" (UID: \"94d2085e-48d2-4ae9-9112-6f4fa7897b43\") " pod="openshift-multus/multus-l7fbr"
Sep 16 00:43:07.721090 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.720942    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4187c8f2-c639-4f19-90aa-5c7dfad934f6-ovn-node-metrics-cert\") pod \"ovnkube-node-fvnth\" (UID: \"4187c8f2-c639-4f19-90aa-5c7dfad934f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fvnth"
Sep 16 00:43:07.721090 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.721033    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f8bd281f-9067-462a-9e7c-d63c81f51837-tmp\") pod \"tuned-pmg7l\" (UID: \"f8bd281f-9067-462a-9e7c-d63c81f51837\") " pod="openshift-cluster-node-tuning-operator/tuned-pmg7l"
Sep 16 00:43:07.721190 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.721110    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f8bd281f-9067-462a-9e7c-d63c81f51837-etc-tuned\") pod \"tuned-pmg7l\" (UID: \"f8bd281f-9067-462a-9e7c-d63c81f51837\") " pod="openshift-cluster-node-tuning-operator/tuned-pmg7l"
Sep 16 00:43:07.723202 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.723183    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwqcj\" (UniqueName: \"kubernetes.io/projected/4187c8f2-c639-4f19-90aa-5c7dfad934f6-kube-api-access-qwqcj\") pod \"ovnkube-node-fvnth\" (UID: \"4187c8f2-c639-4f19-90aa-5c7dfad934f6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fvnth"
Sep 16 00:43:07.724121 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.723908    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rhnf\" (UniqueName: \"kubernetes.io/projected/9fab7ba2-df76-4d06-a48f-d6aee7211db7-kube-api-access-8rhnf\") pod \"aws-ebs-csi-driver-node-zsf5q\" (UID: \"9fab7ba2-df76-4d06-a48f-d6aee7211db7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsf5q"
Sep 16 00:43:07.732942 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.732916    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bds95\" (UniqueName: \"kubernetes.io/projected/94d2085e-48d2-4ae9-9112-6f4fa7897b43-kube-api-access-bds95\") pod \"multus-l7fbr\" (UID: \"94d2085e-48d2-4ae9-9112-6f4fa7897b43\") " pod="openshift-multus/multus-l7fbr"
Sep 16 00:43:07.734018 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.733996    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghsv6\" (UniqueName: \"kubernetes.io/projected/f8bd281f-9067-462a-9e7c-d63c81f51837-kube-api-access-ghsv6\") pod \"tuned-pmg7l\" (UID: \"f8bd281f-9067-462a-9e7c-d63c81f51837\") " pod="openshift-cluster-node-tuning-operator/tuned-pmg7l"
Sep 16 00:43:07.735570 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.735496    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-207.ec2.internal" event={"ID":"4054c0090de8f54aed802a5d555be580","Type":"ContainerStarted","Data":"d70faca41eb4bd2406c4bcaf2339615a2ae4f9e56bac1718adaab3ef541d210a"}
Sep 16 00:43:07.737408 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.737386    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-207.ec2.internal" event={"ID":"a6be15198fb113e0b9a0291128436e59","Type":"ContainerStarted","Data":"a8e9ee13ce3a217185129180dd3d4db4b6da628612844a278520a54320e14a0b"}
Sep 16 00:43:07.819586 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.819558    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dnb28\" (UniqueName: \"kubernetes.io/projected/07a10807-5530-48f1-9aeb-8becb1380db5-kube-api-access-dnb28\") pod \"node-ca-jvxd7\" (UID: \"07a10807-5530-48f1-9aeb-8becb1380db5\") " pod="openshift-image-registry/node-ca-jvxd7"
Sep 16 00:43:07.819738 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.819599    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7tqn\" (UniqueName: \"kubernetes.io/projected/064c3719-509b-45f7-8367-ab4101e5d3bc-kube-api-access-d7tqn\") pod \"network-check-target-4trxq\" (UID: \"064c3719-509b-45f7-8367-ab4101e5d3bc\") " pod="openshift-network-diagnostics/network-check-target-4trxq"
Sep 16 00:43:07.819738 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.819722    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5mt5\" (UniqueName: \"kubernetes.io/projected/f2569a02-edf9-4b33-a7df-a8d0fdce0580-kube-api-access-w5mt5\") pod \"network-metrics-daemon-hnw6l\" (UID: \"f2569a02-edf9-4b33-a7df-a8d0fdce0580\") " pod="openshift-multus/network-metrics-daemon-hnw6l"
Sep 16 00:43:07.819852 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.819769    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/21802a49-7d39-41db-a85c-0b6c56472a56-tmp-dir\") pod \"node-resolver-q29p2\" (UID: \"21802a49-7d39-41db-a85c-0b6c56472a56\") " pod="openshift-dns/node-resolver-q29p2"
Sep 16 00:43:07.819852 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.819801    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/07a10807-5530-48f1-9aeb-8becb1380db5-host\") pod \"node-ca-jvxd7\" (UID: \"07a10807-5530-48f1-9aeb-8becb1380db5\") " pod="openshift-image-registry/node-ca-jvxd7"
Sep 16 00:43:07.819852 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.819835    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/21802a49-7d39-41db-a85c-0b6c56472a56-hosts-file\") pod \"node-resolver-q29p2\" (UID: \"21802a49-7d39-41db-a85c-0b6c56472a56\") " pod="openshift-dns/node-resolver-q29p2"
Sep 16 00:43:07.819999 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.819864    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rx6zq\" (UniqueName: \"kubernetes.io/projected/e94d6aa4-58be-4a67-bd8b-5670201fc0af-kube-api-access-rx6zq\") pod \"multus-additional-cni-plugins-m9fb7\" (UID: \"e94d6aa4-58be-4a67-bd8b-5670201fc0af\") " pod="openshift-multus/multus-additional-cni-plugins-m9fb7"
Sep 16 00:43:07.819999 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.819888    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c23418e8-6326-481d-9a85-2cbda835d5f6-dbus\") pod \"global-pull-secret-syncer-wk9f6\" (UID: \"c23418e8-6326-481d-9a85-2cbda835d5f6\") " pod="kube-system/global-pull-secret-syncer-wk9f6"
Sep 16 00:43:07.819999 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.819912    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f2569a02-edf9-4b33-a7df-a8d0fdce0580-metrics-certs\") pod \"network-metrics-daemon-hnw6l\" (UID: \"f2569a02-edf9-4b33-a7df-a8d0fdce0580\") " pod="openshift-multus/network-metrics-daemon-hnw6l"
Sep 16 00:43:07.819999 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.819912    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/07a10807-5530-48f1-9aeb-8becb1380db5-host\") pod \"node-ca-jvxd7\" (UID: \"07a10807-5530-48f1-9aeb-8becb1380db5\") " pod="openshift-image-registry/node-ca-jvxd7"
Sep 16 00:43:07.819999 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.819926    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/21802a49-7d39-41db-a85c-0b6c56472a56-hosts-file\") pod \"node-resolver-q29p2\" (UID: \"21802a49-7d39-41db-a85c-0b6c56472a56\") " pod="openshift-dns/node-resolver-q29p2"
Sep 16 00:43:07.819999 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.819944    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c23418e8-6326-481d-9a85-2cbda835d5f6-original-pull-secret\") pod \"global-pull-secret-syncer-wk9f6\" (UID: \"c23418e8-6326-481d-9a85-2cbda835d5f6\") " pod="kube-system/global-pull-secret-syncer-wk9f6"
Sep 16 00:43:07.820264 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.820002    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c23418e8-6326-481d-9a85-2cbda835d5f6-dbus\") pod \"global-pull-secret-syncer-wk9f6\" (UID: \"c23418e8-6326-481d-9a85-2cbda835d5f6\") " pod="kube-system/global-pull-secret-syncer-wk9f6"
Sep 16 00:43:07.820264 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:07.820067    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 00:43:07.820264 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.820093    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f1c71941-4979-4074-beb1-cf727004f45d-agent-certs\") pod \"konnectivity-agent-sjrmq\" (UID: \"f1c71941-4979-4074-beb1-cf727004f45d\") " pod="kube-system/konnectivity-agent-sjrmq"
Sep 16 00:43:07.820264 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.820104    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/21802a49-7d39-41db-a85c-0b6c56472a56-tmp-dir\") pod \"node-resolver-q29p2\" (UID: \"21802a49-7d39-41db-a85c-0b6c56472a56\") " pod="openshift-dns/node-resolver-q29p2"
Sep 16 00:43:07.820264 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:07.820068    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 00:43:07.820264 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:07.820137    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c23418e8-6326-481d-9a85-2cbda835d5f6-original-pull-secret podName:c23418e8-6326-481d-9a85-2cbda835d5f6 nodeName:}" failed. No retries permitted until 2026-09-16 00:43:08.320112876 +0000 UTC m=+3.101614447 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c23418e8-6326-481d-9a85-2cbda835d5f6-original-pull-secret") pod "global-pull-secret-syncer-wk9f6" (UID: "c23418e8-6326-481d-9a85-2cbda835d5f6") : object "kube-system"/"original-pull-secret" not registered
Sep 16 00:43:07.820264 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:07.820229    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f2569a02-edf9-4b33-a7df-a8d0fdce0580-metrics-certs podName:f2569a02-edf9-4b33-a7df-a8d0fdce0580 nodeName:}" failed. No retries permitted until 2026-09-16 00:43:08.3202116 +0000 UTC m=+3.101713171 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f2569a02-edf9-4b33-a7df-a8d0fdce0580-metrics-certs") pod "network-metrics-daemon-hnw6l" (UID: "f2569a02-edf9-4b33-a7df-a8d0fdce0580") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 00:43:07.820264 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.820255    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e94d6aa4-58be-4a67-bd8b-5670201fc0af-tuning-conf-dir\") pod \"multus-additional-cni-plugins-m9fb7\" (UID: \"e94d6aa4-58be-4a67-bd8b-5670201fc0af\") " pod="openshift-multus/multus-additional-cni-plugins-m9fb7"
Sep 16 00:43:07.820667 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.820283    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e94d6aa4-58be-4a67-bd8b-5670201fc0af-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-m9fb7\" (UID: \"e94d6aa4-58be-4a67-bd8b-5670201fc0af\") " pod="openshift-multus/multus-additional-cni-plugins-m9fb7"
Sep 16 00:43:07.820667 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.820317    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e94d6aa4-58be-4a67-bd8b-5670201fc0af-system-cni-dir\") pod \"multus-additional-cni-plugins-m9fb7\" (UID: \"e94d6aa4-58be-4a67-bd8b-5670201fc0af\") " pod="openshift-multus/multus-additional-cni-plugins-m9fb7"
Sep 16 00:43:07.820667 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.820342    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e94d6aa4-58be-4a67-bd8b-5670201fc0af-cnibin\") pod \"multus-additional-cni-plugins-m9fb7\" (UID: \"e94d6aa4-58be-4a67-bd8b-5670201fc0af\") " pod="openshift-multus/multus-additional-cni-plugins-m9fb7"
Sep 16 00:43:07.820667 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.820368    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e94d6aa4-58be-4a67-bd8b-5670201fc0af-cni-binary-copy\") pod \"multus-additional-cni-plugins-m9fb7\" (UID: \"e94d6aa4-58be-4a67-bd8b-5670201fc0af\") " pod="openshift-multus/multus-additional-cni-plugins-m9fb7"
Sep 16 00:43:07.820667 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.820405    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e94d6aa4-58be-4a67-bd8b-5670201fc0af-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-m9fb7\" (UID: \"e94d6aa4-58be-4a67-bd8b-5670201fc0af\") " pod="openshift-multus/multus-additional-cni-plugins-m9fb7"
Sep 16 00:43:07.820667 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.820420    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e94d6aa4-58be-4a67-bd8b-5670201fc0af-tuning-conf-dir\") pod \"multus-additional-cni-plugins-m9fb7\" (UID: \"e94d6aa4-58be-4a67-bd8b-5670201fc0af\") " pod="openshift-multus/multus-additional-cni-plugins-m9fb7"
Sep 16 00:43:07.820667 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.820429    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c23418e8-6326-481d-9a85-2cbda835d5f6-kubelet-config\") pod \"global-pull-secret-syncer-wk9f6\" (UID: \"c23418e8-6326-481d-9a85-2cbda835d5f6\") " pod="kube-system/global-pull-secret-syncer-wk9f6"
Sep 16 00:43:07.820667 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.820486    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e94d6aa4-58be-4a67-bd8b-5670201fc0af-cnibin\") pod \"multus-additional-cni-plugins-m9fb7\" (UID: \"e94d6aa4-58be-4a67-bd8b-5670201fc0af\") " pod="openshift-multus/multus-additional-cni-plugins-m9fb7"
Sep 16 00:43:07.820667 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.820511    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c23418e8-6326-481d-9a85-2cbda835d5f6-kubelet-config\") pod \"global-pull-secret-syncer-wk9f6\" (UID: \"c23418e8-6326-481d-9a85-2cbda835d5f6\") " pod="kube-system/global-pull-secret-syncer-wk9f6"
Sep 16 00:43:07.820667 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.820517    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/07a10807-5530-48f1-9aeb-8becb1380db5-serviceca\") pod \"node-ca-jvxd7\" (UID: \"07a10807-5530-48f1-9aeb-8becb1380db5\") " pod="openshift-image-registry/node-ca-jvxd7"
Sep 16 00:43:07.820667 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.820546    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e94d6aa4-58be-4a67-bd8b-5670201fc0af-system-cni-dir\") pod \"multus-additional-cni-plugins-m9fb7\" (UID: \"e94d6aa4-58be-4a67-bd8b-5670201fc0af\") " pod="openshift-multus/multus-additional-cni-plugins-m9fb7"
Sep 16 00:43:07.820667 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.820560    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a949202d-9c40-4794-96da-77b18812a729-iptables-alerter-script\") pod \"iptables-alerter-7jwf7\" (UID: \"a949202d-9c40-4794-96da-77b18812a729\") " pod="openshift-network-operator/iptables-alerter-7jwf7"
Sep 16 00:43:07.820667 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.820590    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a949202d-9c40-4794-96da-77b18812a729-host-slash\") pod \"iptables-alerter-7jwf7\" (UID: \"a949202d-9c40-4794-96da-77b18812a729\") " pod="openshift-network-operator/iptables-alerter-7jwf7"
Sep 16 00:43:07.820667 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.820614    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pg7z9\" (UniqueName: \"kubernetes.io/projected/a949202d-9c40-4794-96da-77b18812a729-kube-api-access-pg7z9\") pod \"iptables-alerter-7jwf7\" (UID: \"a949202d-9c40-4794-96da-77b18812a729\") " pod="openshift-network-operator/iptables-alerter-7jwf7"
Sep 16 00:43:07.820667 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.820631    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f1c71941-4979-4074-beb1-cf727004f45d-konnectivity-ca\") pod \"konnectivity-agent-sjrmq\" (UID: \"f1c71941-4979-4074-beb1-cf727004f45d\") " pod="kube-system/konnectivity-agent-sjrmq"
Sep 16 00:43:07.820667 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.820650    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-929sb\" (UniqueName: \"kubernetes.io/projected/21802a49-7d39-41db-a85c-0b6c56472a56-kube-api-access-929sb\") pod \"node-resolver-q29p2\" (UID: \"21802a49-7d39-41db-a85c-0b6c56472a56\") " pod="openshift-dns/node-resolver-q29p2"
Sep 16 00:43:07.821516 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.820920    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e94d6aa4-58be-4a67-bd8b-5670201fc0af-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-m9fb7\" (UID: \"e94d6aa4-58be-4a67-bd8b-5670201fc0af\") " pod="openshift-multus/multus-additional-cni-plugins-m9fb7"
Sep 16 00:43:07.821516 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.820971    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/07a10807-5530-48f1-9aeb-8becb1380db5-serviceca\") pod \"node-ca-jvxd7\" (UID: \"07a10807-5530-48f1-9aeb-8becb1380db5\") " pod="openshift-image-registry/node-ca-jvxd7"
Sep 16 00:43:07.821516 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.820980    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e94d6aa4-58be-4a67-bd8b-5670201fc0af-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-m9fb7\" (UID: \"e94d6aa4-58be-4a67-bd8b-5670201fc0af\") " pod="openshift-multus/multus-additional-cni-plugins-m9fb7"
Sep 16 00:43:07.821516 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.820992    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e94d6aa4-58be-4a67-bd8b-5670201fc0af-cni-binary-copy\") pod \"multus-additional-cni-plugins-m9fb7\" (UID: \"e94d6aa4-58be-4a67-bd8b-5670201fc0af\") " pod="openshift-multus/multus-additional-cni-plugins-m9fb7"
Sep 16 00:43:07.821516 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.821052    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a949202d-9c40-4794-96da-77b18812a729-host-slash\") pod \"iptables-alerter-7jwf7\" (UID: \"a949202d-9c40-4794-96da-77b18812a729\") " pod="openshift-network-operator/iptables-alerter-7jwf7"
Sep 16 00:43:07.821516 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.821196    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a949202d-9c40-4794-96da-77b18812a729-iptables-alerter-script\") pod \"iptables-alerter-7jwf7\" (UID: \"a949202d-9c40-4794-96da-77b18812a729\") " pod="openshift-network-operator/iptables-alerter-7jwf7"
Sep 16 00:43:07.821516 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.821406    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f1c71941-4979-4074-beb1-cf727004f45d-konnectivity-ca\") pod \"konnectivity-agent-sjrmq\" (UID: \"f1c71941-4979-4074-beb1-cf727004f45d\") " pod="kube-system/konnectivity-agent-sjrmq"
Sep 16 00:43:07.823254 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.823233    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f1c71941-4979-4074-beb1-cf727004f45d-agent-certs\") pod \"konnectivity-agent-sjrmq\" (UID: \"f1c71941-4979-4074-beb1-cf727004f45d\") " pod="kube-system/konnectivity-agent-sjrmq"
Sep 16 00:43:07.828059 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:07.828039    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 00:43:07.828191 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:07.828064    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 00:43:07.828191 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:07.828077    2556 projected.go:196] Error preparing data for projected volume kube-api-access-d7tqn for pod openshift-network-diagnostics/network-check-target-4trxq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 00:43:07.828191 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:07.828137    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/064c3719-509b-45f7-8367-ab4101e5d3bc-kube-api-access-d7tqn podName:064c3719-509b-45f7-8367-ab4101e5d3bc nodeName:}" failed. No retries permitted until 2026-09-16 00:43:08.328119969 +0000 UTC m=+3.109621539 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-d7tqn" (UniqueName: "kubernetes.io/projected/064c3719-509b-45f7-8367-ab4101e5d3bc-kube-api-access-d7tqn") pod "network-check-target-4trxq" (UID: "064c3719-509b-45f7-8367-ab4101e5d3bc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 00:43:07.830606 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.830557    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rx6zq\" (UniqueName: \"kubernetes.io/projected/e94d6aa4-58be-4a67-bd8b-5670201fc0af-kube-api-access-rx6zq\") pod \"multus-additional-cni-plugins-m9fb7\" (UID: \"e94d6aa4-58be-4a67-bd8b-5670201fc0af\") " pod="openshift-multus/multus-additional-cni-plugins-m9fb7"
Sep 16 00:43:07.830708 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.830655    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5mt5\" (UniqueName: \"kubernetes.io/projected/f2569a02-edf9-4b33-a7df-a8d0fdce0580-kube-api-access-w5mt5\") pod \"network-metrics-daemon-hnw6l\" (UID: \"f2569a02-edf9-4b33-a7df-a8d0fdce0580\") " pod="openshift-multus/network-metrics-daemon-hnw6l"
Sep 16 00:43:07.830901 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.830874    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dnb28\" (UniqueName: \"kubernetes.io/projected/07a10807-5530-48f1-9aeb-8becb1380db5-kube-api-access-dnb28\") pod \"node-ca-jvxd7\" (UID: \"07a10807-5530-48f1-9aeb-8becb1380db5\") " pod="openshift-image-registry/node-ca-jvxd7"
Sep 16 00:43:07.831183 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.831165    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-929sb\" (UniqueName: \"kubernetes.io/projected/21802a49-7d39-41db-a85c-0b6c56472a56-kube-api-access-929sb\") pod \"node-resolver-q29p2\" (UID: \"21802a49-7d39-41db-a85c-0b6c56472a56\") " pod="openshift-dns/node-resolver-q29p2"
Sep 16 00:43:07.831360 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.831340    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pg7z9\" (UniqueName: \"kubernetes.io/projected/a949202d-9c40-4794-96da-77b18812a729-kube-api-access-pg7z9\") pod \"iptables-alerter-7jwf7\" (UID: \"a949202d-9c40-4794-96da-77b18812a729\") " pod="openshift-network-operator/iptables-alerter-7jwf7"
Sep 16 00:43:07.897555 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.897532    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-l7fbr"
Sep 16 00:43:07.908376 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.908358    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-pmg7l"
Sep 16 00:43:07.920078 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.920057    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-fvnth"
Sep 16 00:43:07.926880 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.926859    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsf5q"
Sep 16 00:43:07.935492 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.935457    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-jvxd7"
Sep 16 00:43:07.944111 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.944094    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-q29p2"
Sep 16 00:43:07.954795 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.954767    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-m9fb7"
Sep 16 00:43:07.962286 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.962267    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-7jwf7"
Sep 16 00:43:07.968866 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:07.968849    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-sjrmq"
Sep 16 00:43:08.322582 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:08.322552    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f2569a02-edf9-4b33-a7df-a8d0fdce0580-metrics-certs\") pod \"network-metrics-daemon-hnw6l\" (UID: \"f2569a02-edf9-4b33-a7df-a8d0fdce0580\") " pod="openshift-multus/network-metrics-daemon-hnw6l"
Sep 16 00:43:08.322757 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:08.322594    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c23418e8-6326-481d-9a85-2cbda835d5f6-original-pull-secret\") pod \"global-pull-secret-syncer-wk9f6\" (UID: \"c23418e8-6326-481d-9a85-2cbda835d5f6\") " pod="kube-system/global-pull-secret-syncer-wk9f6"
Sep 16 00:43:08.322757 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:08.322722    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 00:43:08.322853 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:08.322775    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f2569a02-edf9-4b33-a7df-a8d0fdce0580-metrics-certs podName:f2569a02-edf9-4b33-a7df-a8d0fdce0580 nodeName:}" failed. No retries permitted until 2026-09-16 00:43:09.322761689 +0000 UTC m=+4.104263272 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f2569a02-edf9-4b33-a7df-a8d0fdce0580-metrics-certs") pod "network-metrics-daemon-hnw6l" (UID: "f2569a02-edf9-4b33-a7df-a8d0fdce0580") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 00:43:08.322853 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:08.322722    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 00:43:08.322945 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:08.322892    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c23418e8-6326-481d-9a85-2cbda835d5f6-original-pull-secret podName:c23418e8-6326-481d-9a85-2cbda835d5f6 nodeName:}" failed. No retries permitted until 2026-09-16 00:43:09.32286932 +0000 UTC m=+4.104370910 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c23418e8-6326-481d-9a85-2cbda835d5f6-original-pull-secret") pod "global-pull-secret-syncer-wk9f6" (UID: "c23418e8-6326-481d-9a85-2cbda835d5f6") : object "kube-system"/"original-pull-secret" not registered
Sep 16 00:43:08.423356 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:08.423327    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7tqn\" (UniqueName: \"kubernetes.io/projected/064c3719-509b-45f7-8367-ab4101e5d3bc-kube-api-access-d7tqn\") pod \"network-check-target-4trxq\" (UID: \"064c3719-509b-45f7-8367-ab4101e5d3bc\") " pod="openshift-network-diagnostics/network-check-target-4trxq"
Sep 16 00:43:08.423510 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:08.423489    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 00:43:08.423510 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:08.423504    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 00:43:08.423581 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:08.423514    2556 projected.go:196] Error preparing data for projected volume kube-api-access-d7tqn for pod openshift-network-diagnostics/network-check-target-4trxq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 00:43:08.423581 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:08.423566    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/064c3719-509b-45f7-8367-ab4101e5d3bc-kube-api-access-d7tqn podName:064c3719-509b-45f7-8367-ab4101e5d3bc nodeName:}" failed. No retries permitted until 2026-09-16 00:43:09.423552982 +0000 UTC m=+4.205054551 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-d7tqn" (UniqueName: "kubernetes.io/projected/064c3719-509b-45f7-8367-ab4101e5d3bc-kube-api-access-d7tqn") pod "network-check-target-4trxq" (UID: "064c3719-509b-45f7-8367-ab4101e5d3bc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 00:43:08.641768 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:08.641689    2556 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-15 00:38:07 +0000 UTC" deadline="2028-06-12 11:15:41.972227782 +0000 UTC"
Sep 16 00:43:08.641768 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:08.641723    2556 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15250h32m33.330509993s"
Sep 16 00:43:08.797798 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:08.797759    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4187c8f2_c639_4f19_90aa_5c7dfad934f6.slice/crio-e0f364544e7a1395b3589d886c933d34e5b5123ae88aefdaaaac298b6b14d3d9 WatchSource:0}: Error finding container e0f364544e7a1395b3589d886c933d34e5b5123ae88aefdaaaac298b6b14d3d9: Status 404 returned error can't find the container with id e0f364544e7a1395b3589d886c933d34e5b5123ae88aefdaaaac298b6b14d3d9
Sep 16 00:43:08.802141 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:08.802105    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode94d6aa4_58be_4a67_bd8b_5670201fc0af.slice/crio-d7603df1a2074a1d3894d8b6e6472dec76cd6565239e7cb901dbac97bf7ea962 WatchSource:0}: Error finding container d7603df1a2074a1d3894d8b6e6472dec76cd6565239e7cb901dbac97bf7ea962: Status 404 returned error can't find the container with id d7603df1a2074a1d3894d8b6e6472dec76cd6565239e7cb901dbac97bf7ea962
Sep 16 00:43:08.802912 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:08.802814    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94d2085e_48d2_4ae9_9112_6f4fa7897b43.slice/crio-16308e913a97f48d241364c51a5e76570b7e44e9bb3d5848f29b443ff97f1442 WatchSource:0}: Error finding container 16308e913a97f48d241364c51a5e76570b7e44e9bb3d5848f29b443ff97f1442: Status 404 returned error can't find the container with id 16308e913a97f48d241364c51a5e76570b7e44e9bb3d5848f29b443ff97f1442
Sep 16 00:43:08.803800 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:08.803779    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07a10807_5530_48f1_9aeb_8becb1380db5.slice/crio-a22c900ac7b41ff42ed89f08ef669af0b25d745e1d7783e3320e51a7c9ddf627 WatchSource:0}: Error finding container a22c900ac7b41ff42ed89f08ef669af0b25d745e1d7783e3320e51a7c9ddf627: Status 404 returned error can't find the container with id a22c900ac7b41ff42ed89f08ef669af0b25d745e1d7783e3320e51a7c9ddf627
Sep 16 00:43:08.805053 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:08.805031    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fab7ba2_df76_4d06_a48f_d6aee7211db7.slice/crio-50d5b84b369c6baa9c1bffe48b03fd7a114e26a3a9e8efe4543bf9aaae862e0a WatchSource:0}: Error finding container 50d5b84b369c6baa9c1bffe48b03fd7a114e26a3a9e8efe4543bf9aaae862e0a: Status 404 returned error can't find the container with id 50d5b84b369c6baa9c1bffe48b03fd7a114e26a3a9e8efe4543bf9aaae862e0a
Sep 16 00:43:09.331061 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:09.330718    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f2569a02-edf9-4b33-a7df-a8d0fdce0580-metrics-certs\") pod \"network-metrics-daemon-hnw6l\" (UID: \"f2569a02-edf9-4b33-a7df-a8d0fdce0580\") " pod="openshift-multus/network-metrics-daemon-hnw6l"
Sep 16 00:43:09.331061 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:09.330996    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c23418e8-6326-481d-9a85-2cbda835d5f6-original-pull-secret\") pod \"global-pull-secret-syncer-wk9f6\" (UID: \"c23418e8-6326-481d-9a85-2cbda835d5f6\") " pod="kube-system/global-pull-secret-syncer-wk9f6"
Sep 16 00:43:09.331061 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:09.331017    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 00:43:09.331320 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:09.331093    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f2569a02-edf9-4b33-a7df-a8d0fdce0580-metrics-certs podName:f2569a02-edf9-4b33-a7df-a8d0fdce0580 nodeName:}" failed. No retries permitted until 2026-09-16 00:43:11.331072805 +0000 UTC m=+6.112574395 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f2569a02-edf9-4b33-a7df-a8d0fdce0580-metrics-certs") pod "network-metrics-daemon-hnw6l" (UID: "f2569a02-edf9-4b33-a7df-a8d0fdce0580") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 00:43:09.331320 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:09.331133    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 00:43:09.331320 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:09.331190    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c23418e8-6326-481d-9a85-2cbda835d5f6-original-pull-secret podName:c23418e8-6326-481d-9a85-2cbda835d5f6 nodeName:}" failed. No retries permitted until 2026-09-16 00:43:11.331172428 +0000 UTC m=+6.112674011 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c23418e8-6326-481d-9a85-2cbda835d5f6-original-pull-secret") pod "global-pull-secret-syncer-wk9f6" (UID: "c23418e8-6326-481d-9a85-2cbda835d5f6") : object "kube-system"/"original-pull-secret" not registered
Sep 16 00:43:09.431903 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:09.431868    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7tqn\" (UniqueName: \"kubernetes.io/projected/064c3719-509b-45f7-8367-ab4101e5d3bc-kube-api-access-d7tqn\") pod \"network-check-target-4trxq\" (UID: \"064c3719-509b-45f7-8367-ab4101e5d3bc\") " pod="openshift-network-diagnostics/network-check-target-4trxq"
Sep 16 00:43:09.432066 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:09.432023    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 00:43:09.432066 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:09.432045    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 00:43:09.432066 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:09.432055    2556 projected.go:196] Error preparing data for projected volume kube-api-access-d7tqn for pod openshift-network-diagnostics/network-check-target-4trxq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 00:43:09.432232 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:09.432111    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/064c3719-509b-45f7-8367-ab4101e5d3bc-kube-api-access-d7tqn podName:064c3719-509b-45f7-8367-ab4101e5d3bc nodeName:}" failed. No retries permitted until 2026-09-16 00:43:11.432093061 +0000 UTC m=+6.213594647 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-d7tqn" (UniqueName: "kubernetes.io/projected/064c3719-509b-45f7-8367-ab4101e5d3bc-kube-api-access-d7tqn") pod "network-check-target-4trxq" (UID: "064c3719-509b-45f7-8367-ab4101e5d3bc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 00:43:09.642414 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:09.642293    2556 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-15 00:38:07 +0000 UTC" deadline="2028-06-28 15:19:55.688486278 +0000 UTC"
Sep 16 00:43:09.642414 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:09.642349    2556 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15638h36m46.046141785s"
Sep 16 00:43:09.731120 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:09.730640    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wk9f6"
Sep 16 00:43:09.731120 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:09.730640    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4trxq"
Sep 16 00:43:09.731120 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:09.730772    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wk9f6" podUID="c23418e8-6326-481d-9a85-2cbda835d5f6"
Sep 16 00:43:09.731120 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:09.730784    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hnw6l"
Sep 16 00:43:09.731120 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:09.730863    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4trxq" podUID="064c3719-509b-45f7-8367-ab4101e5d3bc"
Sep 16 00:43:09.731120 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:09.730954    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hnw6l" podUID="f2569a02-edf9-4b33-a7df-a8d0fdce0580"
Sep 16 00:43:09.780317 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:09.780285    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-jvxd7" event={"ID":"07a10807-5530-48f1-9aeb-8becb1380db5","Type":"ContainerStarted","Data":"a22c900ac7b41ff42ed89f08ef669af0b25d745e1d7783e3320e51a7c9ddf627"}
Sep 16 00:43:09.781768 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:09.781740    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-l7fbr" event={"ID":"94d2085e-48d2-4ae9-9112-6f4fa7897b43","Type":"ContainerStarted","Data":"16308e913a97f48d241364c51a5e76570b7e44e9bb3d5848f29b443ff97f1442"}
Sep 16 00:43:09.789153 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:09.789103    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m9fb7" event={"ID":"e94d6aa4-58be-4a67-bd8b-5670201fc0af","Type":"ContainerStarted","Data":"d7603df1a2074a1d3894d8b6e6472dec76cd6565239e7cb901dbac97bf7ea962"}
Sep 16 00:43:09.804619 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:09.804235    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-207.ec2.internal" event={"ID":"4054c0090de8f54aed802a5d555be580","Type":"ContainerStarted","Data":"a9d53262d27dbe7ed3316639a29d734b49c7d4c745a9ed3fba9f0fb6ec785507"}
Sep 16 00:43:09.809740 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:09.809191    2556 generic.go:356] "Generic (PLEG): container finished" podID="a6be15198fb113e0b9a0291128436e59" containerID="288fbc97a4d4b635d1112ea1cfa51fd76fa97f0fb4c2ec273c39725e348cb7ca" exitCode=0
Sep 16 00:43:09.809740 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:09.809261    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-207.ec2.internal" event={"ID":"a6be15198fb113e0b9a0291128436e59","Type":"ContainerDied","Data":"288fbc97a4d4b635d1112ea1cfa51fd76fa97f0fb4c2ec273c39725e348cb7ca"}
Sep 16 00:43:09.824183 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:09.823779    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-207.ec2.internal" podStartSLOduration=2.823761615 podStartE2EDuration="2.823761615s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 00:43:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 00:43:09.823076813 +0000 UTC m=+4.604578408" watchObservedRunningTime="2026-09-16 00:43:09.823761615 +0000 UTC m=+4.605263224"
Sep 16 00:43:09.831242 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:09.831217    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-7jwf7" event={"ID":"a949202d-9c40-4794-96da-77b18812a729","Type":"ContainerStarted","Data":"5f713a405d6955e58d77411beceb61ef6fc2b35c3b0ef5dc422caffd8a0a549c"}
Sep 16 00:43:09.834871 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:09.834839    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-sjrmq" event={"ID":"f1c71941-4979-4074-beb1-cf727004f45d","Type":"ContainerStarted","Data":"6f6fb1200d540b97489fdecbb551c68a414c831ad48124980fa647afc7c91ede"}
Sep 16 00:43:09.844489 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:09.844451    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsf5q" event={"ID":"9fab7ba2-df76-4d06-a48f-d6aee7211db7","Type":"ContainerStarted","Data":"50d5b84b369c6baa9c1bffe48b03fd7a114e26a3a9e8efe4543bf9aaae862e0a"}
Sep 16 00:43:09.849895 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:09.849869    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fvnth" event={"ID":"4187c8f2-c639-4f19-90aa-5c7dfad934f6","Type":"ContainerStarted","Data":"e0f364544e7a1395b3589d886c933d34e5b5123ae88aefdaaaac298b6b14d3d9"}
Sep 16 00:43:09.865814 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:09.865779    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-pmg7l" event={"ID":"f8bd281f-9067-462a-9e7c-d63c81f51837","Type":"ContainerStarted","Data":"eb06e3eca0d3992e0b5c07489260c0f453be573f007a622e6775f58e3654dc45"}
Sep 16 00:43:09.883343 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:09.883278    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-q29p2" event={"ID":"21802a49-7d39-41db-a85c-0b6c56472a56","Type":"ContainerStarted","Data":"d92dc1154a20fc20a3e18f74182babf045f83dc42f1567f29ebcebace6ae81d1"}
Sep 16 00:43:10.921872 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:10.921838    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-207.ec2.internal" event={"ID":"a6be15198fb113e0b9a0291128436e59","Type":"ContainerStarted","Data":"fe0ef4344a9e1f2a43067855549add706739bba7866d5e7c22c3d85523b80378"}
Sep 16 00:43:10.937646 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:10.937594    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-207.ec2.internal" podStartSLOduration=3.937574261 podStartE2EDuration="3.937574261s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 00:43:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 00:43:10.935877046 +0000 UTC m=+5.717378641" watchObservedRunningTime="2026-09-16 00:43:10.937574261 +0000 UTC m=+5.719075857"
Sep 16 00:43:11.349879 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:11.349832    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c23418e8-6326-481d-9a85-2cbda835d5f6-original-pull-secret\") pod \"global-pull-secret-syncer-wk9f6\" (UID: \"c23418e8-6326-481d-9a85-2cbda835d5f6\") " pod="kube-system/global-pull-secret-syncer-wk9f6"
Sep 16 00:43:11.350060 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:11.349924    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f2569a02-edf9-4b33-a7df-a8d0fdce0580-metrics-certs\") pod \"network-metrics-daemon-hnw6l\" (UID: \"f2569a02-edf9-4b33-a7df-a8d0fdce0580\") " pod="openshift-multus/network-metrics-daemon-hnw6l"
Sep 16 00:43:11.350135 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:11.350084    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 00:43:11.350195 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:11.350141    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f2569a02-edf9-4b33-a7df-a8d0fdce0580-metrics-certs podName:f2569a02-edf9-4b33-a7df-a8d0fdce0580 nodeName:}" failed. No retries permitted until 2026-09-16 00:43:15.350122531 +0000 UTC m=+10.131624106 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f2569a02-edf9-4b33-a7df-a8d0fdce0580-metrics-certs") pod "network-metrics-daemon-hnw6l" (UID: "f2569a02-edf9-4b33-a7df-a8d0fdce0580") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 00:43:11.350566 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:11.350547    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 00:43:11.350661 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:11.350601    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c23418e8-6326-481d-9a85-2cbda835d5f6-original-pull-secret podName:c23418e8-6326-481d-9a85-2cbda835d5f6 nodeName:}" failed. No retries permitted until 2026-09-16 00:43:15.350587125 +0000 UTC m=+10.132088699 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c23418e8-6326-481d-9a85-2cbda835d5f6-original-pull-secret") pod "global-pull-secret-syncer-wk9f6" (UID: "c23418e8-6326-481d-9a85-2cbda835d5f6") : object "kube-system"/"original-pull-secret" not registered
Sep 16 00:43:11.451275 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:11.450703    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7tqn\" (UniqueName: \"kubernetes.io/projected/064c3719-509b-45f7-8367-ab4101e5d3bc-kube-api-access-d7tqn\") pod \"network-check-target-4trxq\" (UID: \"064c3719-509b-45f7-8367-ab4101e5d3bc\") " pod="openshift-network-diagnostics/network-check-target-4trxq"
Sep 16 00:43:11.451275 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:11.450865    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 00:43:11.451275 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:11.450889    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 00:43:11.451275 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:11.450901    2556 projected.go:196] Error preparing data for projected volume kube-api-access-d7tqn for pod openshift-network-diagnostics/network-check-target-4trxq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 00:43:11.451275 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:11.450954    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/064c3719-509b-45f7-8367-ab4101e5d3bc-kube-api-access-d7tqn podName:064c3719-509b-45f7-8367-ab4101e5d3bc nodeName:}" failed. No retries permitted until 2026-09-16 00:43:15.450937351 +0000 UTC m=+10.232438925 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-d7tqn" (UniqueName: "kubernetes.io/projected/064c3719-509b-45f7-8367-ab4101e5d3bc-kube-api-access-d7tqn") pod "network-check-target-4trxq" (UID: "064c3719-509b-45f7-8367-ab4101e5d3bc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 00:43:11.733164 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:11.732279    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4trxq"
Sep 16 00:43:11.733164 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:11.732401    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4trxq" podUID="064c3719-509b-45f7-8367-ab4101e5d3bc"
Sep 16 00:43:11.733164 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:11.732789    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hnw6l"
Sep 16 00:43:11.733164 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:11.732893    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hnw6l" podUID="f2569a02-edf9-4b33-a7df-a8d0fdce0580"
Sep 16 00:43:11.733164 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:11.732937    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wk9f6"
Sep 16 00:43:11.733164 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:11.733008    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wk9f6" podUID="c23418e8-6326-481d-9a85-2cbda835d5f6"
Sep 16 00:43:13.732379 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:13.731904    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hnw6l"
Sep 16 00:43:13.732379 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:13.731904    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4trxq"
Sep 16 00:43:13.732379 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:13.732061    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hnw6l" podUID="f2569a02-edf9-4b33-a7df-a8d0fdce0580"
Sep 16 00:43:13.732379 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:13.732129    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wk9f6"
Sep 16 00:43:13.732379 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:13.732225    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wk9f6" podUID="c23418e8-6326-481d-9a85-2cbda835d5f6"
Sep 16 00:43:13.732379 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:13.732303    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4trxq" podUID="064c3719-509b-45f7-8367-ab4101e5d3bc"
Sep 16 00:43:15.384304 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:15.384270    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f2569a02-edf9-4b33-a7df-a8d0fdce0580-metrics-certs\") pod \"network-metrics-daemon-hnw6l\" (UID: \"f2569a02-edf9-4b33-a7df-a8d0fdce0580\") " pod="openshift-multus/network-metrics-daemon-hnw6l"
Sep 16 00:43:15.384766 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:15.384319    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c23418e8-6326-481d-9a85-2cbda835d5f6-original-pull-secret\") pod \"global-pull-secret-syncer-wk9f6\" (UID: \"c23418e8-6326-481d-9a85-2cbda835d5f6\") " pod="kube-system/global-pull-secret-syncer-wk9f6"
Sep 16 00:43:15.384766 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:15.384437    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 00:43:15.384766 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:15.384454    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 00:43:15.384766 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:15.384521    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f2569a02-edf9-4b33-a7df-a8d0fdce0580-metrics-certs podName:f2569a02-edf9-4b33-a7df-a8d0fdce0580 nodeName:}" failed. No retries permitted until 2026-09-16 00:43:23.384499027 +0000 UTC m=+18.166000617 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f2569a02-edf9-4b33-a7df-a8d0fdce0580-metrics-certs") pod "network-metrics-daemon-hnw6l" (UID: "f2569a02-edf9-4b33-a7df-a8d0fdce0580") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 00:43:15.384766 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:15.384542    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c23418e8-6326-481d-9a85-2cbda835d5f6-original-pull-secret podName:c23418e8-6326-481d-9a85-2cbda835d5f6 nodeName:}" failed. No retries permitted until 2026-09-16 00:43:23.384532783 +0000 UTC m=+18.166034356 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c23418e8-6326-481d-9a85-2cbda835d5f6-original-pull-secret") pod "global-pull-secret-syncer-wk9f6" (UID: "c23418e8-6326-481d-9a85-2cbda835d5f6") : object "kube-system"/"original-pull-secret" not registered
Sep 16 00:43:15.485458 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:15.485237    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7tqn\" (UniqueName: \"kubernetes.io/projected/064c3719-509b-45f7-8367-ab4101e5d3bc-kube-api-access-d7tqn\") pod \"network-check-target-4trxq\" (UID: \"064c3719-509b-45f7-8367-ab4101e5d3bc\") " pod="openshift-network-diagnostics/network-check-target-4trxq"
Sep 16 00:43:15.485458 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:15.485401    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 00:43:15.485458 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:15.485420    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 00:43:15.485458 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:15.485432    2556 projected.go:196] Error preparing data for projected volume kube-api-access-d7tqn for pod openshift-network-diagnostics/network-check-target-4trxq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 00:43:15.485838 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:15.485504    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/064c3719-509b-45f7-8367-ab4101e5d3bc-kube-api-access-d7tqn podName:064c3719-509b-45f7-8367-ab4101e5d3bc nodeName:}" failed. No retries permitted until 2026-09-16 00:43:23.485485993 +0000 UTC m=+18.266987566 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-d7tqn" (UniqueName: "kubernetes.io/projected/064c3719-509b-45f7-8367-ab4101e5d3bc-kube-api-access-d7tqn") pod "network-check-target-4trxq" (UID: "064c3719-509b-45f7-8367-ab4101e5d3bc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 00:43:15.731628 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:15.731552    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4trxq"
Sep 16 00:43:15.731628 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:15.731568    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hnw6l"
Sep 16 00:43:15.731834 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:15.731678    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4trxq" podUID="064c3719-509b-45f7-8367-ab4101e5d3bc"
Sep 16 00:43:15.732935 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:15.732717    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wk9f6"
Sep 16 00:43:15.732935 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:15.732802    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wk9f6" podUID="c23418e8-6326-481d-9a85-2cbda835d5f6"
Sep 16 00:43:15.732935 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:15.732898    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hnw6l" podUID="f2569a02-edf9-4b33-a7df-a8d0fdce0580"
Sep 16 00:43:17.730938 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:17.730907    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wk9f6"
Sep 16 00:43:17.731388 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:17.730969    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4trxq"
Sep 16 00:43:17.731388 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:17.730981    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hnw6l"
Sep 16 00:43:17.731388 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:17.731066    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4trxq" podUID="064c3719-509b-45f7-8367-ab4101e5d3bc"
Sep 16 00:43:17.731388 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:17.731145    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hnw6l" podUID="f2569a02-edf9-4b33-a7df-a8d0fdce0580"
Sep 16 00:43:17.731388 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:17.731204    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wk9f6" podUID="c23418e8-6326-481d-9a85-2cbda835d5f6"
Sep 16 00:43:19.729991 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:19.729941    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wk9f6"
Sep 16 00:43:19.730399 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:19.730064    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wk9f6" podUID="c23418e8-6326-481d-9a85-2cbda835d5f6"
Sep 16 00:43:19.730460 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:19.730431    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4trxq"
Sep 16 00:43:19.730549 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:19.730459    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hnw6l"
Sep 16 00:43:19.730596 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:19.730576    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4trxq" podUID="064c3719-509b-45f7-8367-ab4101e5d3bc"
Sep 16 00:43:19.730700 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:19.730674    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hnw6l" podUID="f2569a02-edf9-4b33-a7df-a8d0fdce0580"
Sep 16 00:43:21.730958 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:21.730921    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wk9f6"
Sep 16 00:43:21.731390 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:21.730932    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4trxq"
Sep 16 00:43:21.731390 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:21.731057    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wk9f6" podUID="c23418e8-6326-481d-9a85-2cbda835d5f6"
Sep 16 00:43:21.731390 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:21.731067    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hnw6l"
Sep 16 00:43:21.731390 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:21.731164    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4trxq" podUID="064c3719-509b-45f7-8367-ab4101e5d3bc"
Sep 16 00:43:21.731390 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:21.731244    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hnw6l" podUID="f2569a02-edf9-4b33-a7df-a8d0fdce0580"
Sep 16 00:43:23.444815 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:23.444603    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f2569a02-edf9-4b33-a7df-a8d0fdce0580-metrics-certs\") pod \"network-metrics-daemon-hnw6l\" (UID: \"f2569a02-edf9-4b33-a7df-a8d0fdce0580\") " pod="openshift-multus/network-metrics-daemon-hnw6l"
Sep 16 00:43:23.445291 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:23.444841    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c23418e8-6326-481d-9a85-2cbda835d5f6-original-pull-secret\") pod \"global-pull-secret-syncer-wk9f6\" (UID: \"c23418e8-6326-481d-9a85-2cbda835d5f6\") " pod="kube-system/global-pull-secret-syncer-wk9f6"
Sep 16 00:43:23.445291 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:23.444777    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 00:43:23.445291 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:23.444962    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f2569a02-edf9-4b33-a7df-a8d0fdce0580-metrics-certs podName:f2569a02-edf9-4b33-a7df-a8d0fdce0580 nodeName:}" failed. No retries permitted until 2026-09-16 00:43:39.444941331 +0000 UTC m=+34.226442905 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f2569a02-edf9-4b33-a7df-a8d0fdce0580-metrics-certs") pod "network-metrics-daemon-hnw6l" (UID: "f2569a02-edf9-4b33-a7df-a8d0fdce0580") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 00:43:23.445291 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:23.444987    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 00:43:23.445291 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:23.445033    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c23418e8-6326-481d-9a85-2cbda835d5f6-original-pull-secret podName:c23418e8-6326-481d-9a85-2cbda835d5f6 nodeName:}" failed. No retries permitted until 2026-09-16 00:43:39.445019028 +0000 UTC m=+34.226520618 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c23418e8-6326-481d-9a85-2cbda835d5f6-original-pull-secret") pod "global-pull-secret-syncer-wk9f6" (UID: "c23418e8-6326-481d-9a85-2cbda835d5f6") : object "kube-system"/"original-pull-secret" not registered
Sep 16 00:43:23.545711 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:23.545682    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7tqn\" (UniqueName: \"kubernetes.io/projected/064c3719-509b-45f7-8367-ab4101e5d3bc-kube-api-access-d7tqn\") pod \"network-check-target-4trxq\" (UID: \"064c3719-509b-45f7-8367-ab4101e5d3bc\") " pod="openshift-network-diagnostics/network-check-target-4trxq"
Sep 16 00:43:23.545859 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:23.545813    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 00:43:23.545859 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:23.545829    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 00:43:23.545859 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:23.545838    2556 projected.go:196] Error preparing data for projected volume kube-api-access-d7tqn for pod openshift-network-diagnostics/network-check-target-4trxq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 00:43:23.546002 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:23.545881    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/064c3719-509b-45f7-8367-ab4101e5d3bc-kube-api-access-d7tqn podName:064c3719-509b-45f7-8367-ab4101e5d3bc nodeName:}" failed. No retries permitted until 2026-09-16 00:43:39.545868192 +0000 UTC m=+34.327369762 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-d7tqn" (UniqueName: "kubernetes.io/projected/064c3719-509b-45f7-8367-ab4101e5d3bc-kube-api-access-d7tqn") pod "network-check-target-4trxq" (UID: "064c3719-509b-45f7-8367-ab4101e5d3bc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 00:43:23.730069 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:23.729993    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4trxq"
Sep 16 00:43:23.730069 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:23.730029    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wk9f6"
Sep 16 00:43:23.730279 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:23.729999    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hnw6l"
Sep 16 00:43:23.730279 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:23.730111    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4trxq" podUID="064c3719-509b-45f7-8367-ab4101e5d3bc"
Sep 16 00:43:23.730279 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:23.730165    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hnw6l" podUID="f2569a02-edf9-4b33-a7df-a8d0fdce0580"
Sep 16 00:43:23.730279 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:23.730252    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wk9f6" podUID="c23418e8-6326-481d-9a85-2cbda835d5f6"
Sep 16 00:43:25.731538 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:25.731506    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wk9f6"
Sep 16 00:43:25.732005 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:25.731585    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hnw6l"
Sep 16 00:43:25.732005 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:25.731603    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wk9f6" podUID="c23418e8-6326-481d-9a85-2cbda835d5f6"
Sep 16 00:43:25.732005 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:25.731707    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hnw6l" podUID="f2569a02-edf9-4b33-a7df-a8d0fdce0580"
Sep 16 00:43:25.732005 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:25.731745    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4trxq"
Sep 16 00:43:25.732005 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:25.731892    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4trxq" podUID="064c3719-509b-45f7-8367-ab4101e5d3bc"
Sep 16 00:43:27.730923 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:27.730659    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hnw6l"
Sep 16 00:43:27.731585 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:27.730659    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4trxq"
Sep 16 00:43:27.731585 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:27.731007    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hnw6l" podUID="f2569a02-edf9-4b33-a7df-a8d0fdce0580"
Sep 16 00:43:27.731585 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:27.730689    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wk9f6"
Sep 16 00:43:27.731585 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:27.731062    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4trxq" podUID="064c3719-509b-45f7-8367-ab4101e5d3bc"
Sep 16 00:43:27.731585 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:27.731138    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wk9f6" podUID="c23418e8-6326-481d-9a85-2cbda835d5f6"
Sep 16 00:43:27.952965 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:27.952934    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-jvxd7" event={"ID":"07a10807-5530-48f1-9aeb-8becb1380db5","Type":"ContainerStarted","Data":"a941d6fa7474bf5bee4b10fffd4f3edbd4d00430613084c5dfce584a2277ec6d"}
Sep 16 00:43:27.954370 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:27.954349    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-l7fbr" event={"ID":"94d2085e-48d2-4ae9-9112-6f4fa7897b43","Type":"ContainerStarted","Data":"afa61613364b4a3b6479401bdab6bb6d2aae971f61422aa1afb6188bed152c9b"}
Sep 16 00:43:27.955949 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:27.955930    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m9fb7" event={"ID":"e94d6aa4-58be-4a67-bd8b-5670201fc0af","Type":"ContainerDied","Data":"23692f2db538203faafb27877e7b31502ada1f5e0951ed7ded99d13599833bfa"}
Sep 16 00:43:27.956073 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:27.956049    2556 generic.go:356] "Generic (PLEG): container finished" podID="e94d6aa4-58be-4a67-bd8b-5670201fc0af" containerID="23692f2db538203faafb27877e7b31502ada1f5e0951ed7ded99d13599833bfa" exitCode=0
Sep 16 00:43:27.957689 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:27.957669    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-sjrmq" event={"ID":"f1c71941-4979-4074-beb1-cf727004f45d","Type":"ContainerStarted","Data":"de875159c8b977de7272062264b1d22c707df67548989453268423a254cca5e3"}
Sep 16 00:43:27.959040 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:27.959020    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsf5q" event={"ID":"9fab7ba2-df76-4d06-a48f-d6aee7211db7","Type":"ContainerStarted","Data":"a96862d0f71a1e25dc43608fc0136a4d681f130bf6051aad0fd213408332b611"}
Sep 16 00:43:27.961998 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:27.961980    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fvnth" event={"ID":"4187c8f2-c639-4f19-90aa-5c7dfad934f6","Type":"ContainerStarted","Data":"f8a9a774f825f2b7717946bf4cc6a4730a668e700232c351e84878961c7cc15f"}
Sep 16 00:43:27.962079 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:27.962041    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fvnth" event={"ID":"4187c8f2-c639-4f19-90aa-5c7dfad934f6","Type":"ContainerStarted","Data":"a1eb6f70cd3b27935ffd1587f6fd0e7bf8822b6088d8e4527fc041a011dd3e2b"}
Sep 16 00:43:27.962079 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:27.962071    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fvnth" event={"ID":"4187c8f2-c639-4f19-90aa-5c7dfad934f6","Type":"ContainerStarted","Data":"e0e3e6d953ea3402b94c232fe635ad9f9f472e10ca537204ea6ee898c51a9582"}
Sep 16 00:43:27.962160 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:27.962101    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fvnth" event={"ID":"4187c8f2-c639-4f19-90aa-5c7dfad934f6","Type":"ContainerStarted","Data":"04db858490bc1a186be69b9c8f26c899e217cc6250eb4779074ad786267615a1"}
Sep 16 00:43:27.962160 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:27.962135    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fvnth" event={"ID":"4187c8f2-c639-4f19-90aa-5c7dfad934f6","Type":"ContainerStarted","Data":"3c0ac31abd03d19fc1f5768d6d91ec038b15a05e625323225606eb63c04fe4bc"}
Sep 16 00:43:27.962243 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:27.962163    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fvnth" event={"ID":"4187c8f2-c639-4f19-90aa-5c7dfad934f6","Type":"ContainerStarted","Data":"08d46a6f9cc86cbb0a2fe05818967586e4e7072b9535b87492d6910586aceb6b"}
Sep 16 00:43:27.963368 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:27.963346    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-pmg7l" event={"ID":"f8bd281f-9067-462a-9e7c-d63c81f51837","Type":"ContainerStarted","Data":"71286b0b58e080a4204b162b9c6c1e25adeae4d569f215e4f23de2ef2ed15556"}
Sep 16 00:43:27.964659 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:27.964641    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-q29p2" event={"ID":"21802a49-7d39-41db-a85c-0b6c56472a56","Type":"ContainerStarted","Data":"7bbeeeda6bbf27926c3e836a7cdb200703b6fe0b35341634ac738eb71e9454bd"}
Sep 16 00:43:27.966154 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:27.966119    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-jvxd7" podStartSLOduration=4.920987079 podStartE2EDuration="22.966108413s" totalImagesPullingTime="18.045121334s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 00:43:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 00:43:27.965821449 +0000 UTC m=+22.747323041" watchObservedRunningTime="2026-09-16 00:43:27.966108413 +0000 UTC m=+22.747610033"
Sep 16 00:43:27.980141 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:27.980104    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-l7fbr" podStartSLOduration=4.695299565 podStartE2EDuration="22.98009309s" totalImagesPullingTime="18.284793525s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 00:43:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 00:43:27.980045803 +0000 UTC m=+22.761547406" watchObservedRunningTime="2026-09-16 00:43:27.98009309 +0000 UTC m=+22.761594682"
Sep 16 00:43:27.994966 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:27.994910    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-pmg7l" podStartSLOduration=4.955160211 podStartE2EDuration="22.994899848s" totalImagesPullingTime="18.039739637s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 00:43:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 00:43:27.994729583 +0000 UTC m=+22.776231176" watchObservedRunningTime="2026-09-16 00:43:27.994899848 +0000 UTC m=+22.776401452"
Sep 16 00:43:28.028978 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:28.028942    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-q29p2" podStartSLOduration=4.987123361 podStartE2EDuration="23.02893174s" totalImagesPullingTime="18.041808379s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 00:43:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 00:43:28.02860942 +0000 UTC m=+22.810111011" watchObservedRunningTime="2026-09-16 00:43:28.02893174 +0000 UTC m=+22.810433332"
Sep 16 00:43:28.042637 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:28.042603    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-sjrmq" podStartSLOduration=8.945791347 podStartE2EDuration="23.042592944s" totalImagesPullingTime="14.096801597s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 00:43:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 00:43:28.042157716 +0000 UTC m=+22.823659308" watchObservedRunningTime="2026-09-16 00:43:28.042592944 +0000 UTC m=+22.824094549"
Sep 16 00:43:28.359879 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:28.359851    2556 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 16 00:43:28.675749 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:28.675609    2556 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-16T00:43:28.359874685Z","UUID":"9c81ebf3-2223-44bd-8ffd-a5211aad7f01","Handler":null,"Name":"","Endpoint":""}
Sep 16 00:43:28.677147 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:28.677120    2556 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 16 00:43:28.677147 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:28.677151    2556 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 16 00:43:28.968945 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:28.968875    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-7jwf7" event={"ID":"a949202d-9c40-4794-96da-77b18812a729","Type":"ContainerStarted","Data":"7888bc59729b232120ad9862ca7a5ddd0db7e005d469fc823d746beff05007a6"}
Sep 16 00:43:28.970824 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:28.970792    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsf5q" event={"ID":"9fab7ba2-df76-4d06-a48f-d6aee7211db7","Type":"ContainerStarted","Data":"8d7633d2ead71b74b1d430120a46733ba0a777e5dfb0a2f587afa8586addab3e"}
Sep 16 00:43:29.364644 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:29.364615    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-sjrmq"
Sep 16 00:43:29.365311 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:29.365290    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-sjrmq"
Sep 16 00:43:29.379617 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:29.379233    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-7jwf7" podStartSLOduration=6.341243064 podStartE2EDuration="24.37921506s" totalImagesPullingTime="18.037971996s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 00:43:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 00:43:28.982254126 +0000 UTC m=+23.763755720" watchObservedRunningTime="2026-09-16 00:43:29.37921506 +0000 UTC m=+24.160716653"
Sep 16 00:43:29.730218 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:29.730196    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wk9f6"
Sep 16 00:43:29.730334 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:29.730292    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wk9f6" podUID="c23418e8-6326-481d-9a85-2cbda835d5f6"
Sep 16 00:43:29.730601 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:29.730583    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4trxq"
Sep 16 00:43:29.730693 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:29.730664    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4trxq" podUID="064c3719-509b-45f7-8367-ab4101e5d3bc"
Sep 16 00:43:29.730754 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:29.730732    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hnw6l"
Sep 16 00:43:29.730849 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:29.730827    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hnw6l" podUID="f2569a02-edf9-4b33-a7df-a8d0fdce0580"
Sep 16 00:43:29.978401 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:29.978309    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsf5q" event={"ID":"9fab7ba2-df76-4d06-a48f-d6aee7211db7","Type":"ContainerStarted","Data":"e8ebe2fd9bc1b4151f782fecb1b09b5787316fa174bbc9354cc5273fa466b7a7"}
Sep 16 00:43:29.982367 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:29.982337    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fvnth" event={"ID":"4187c8f2-c639-4f19-90aa-5c7dfad934f6","Type":"ContainerStarted","Data":"300e16d1ad6ab4cfc8698a25a660a96a70c80292052a8f3772808c5274703caa"}
Sep 16 00:43:29.982641 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:29.982619    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-sjrmq"
Sep 16 00:43:29.983078 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:29.983058    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-sjrmq"
Sep 16 00:43:29.994298 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:29.994254    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsf5q" podStartSLOduration=4.800107448 podStartE2EDuration="24.99424243s" totalImagesPullingTime="20.194134982s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 00:43:05 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 00:43:29.993625996 +0000 UTC m=+24.775127587" watchObservedRunningTime="2026-09-16 00:43:29.99424243 +0000 UTC m=+24.775744023"
Sep 16 00:43:31.730848 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:31.730816    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hnw6l"
Sep 16 00:43:31.731304 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:31.730816    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4trxq"
Sep 16 00:43:31.731304 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:31.730953    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wk9f6"
Sep 16 00:43:31.731304 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:31.730950    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hnw6l" podUID="f2569a02-edf9-4b33-a7df-a8d0fdce0580"
Sep 16 00:43:31.731304 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:31.731048    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4trxq" podUID="064c3719-509b-45f7-8367-ab4101e5d3bc"
Sep 16 00:43:31.731304 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:31.731138    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wk9f6" podUID="c23418e8-6326-481d-9a85-2cbda835d5f6"
Sep 16 00:43:32.990501 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:32.990320    2556 generic.go:356] "Generic (PLEG): container finished" podID="e94d6aa4-58be-4a67-bd8b-5670201fc0af" containerID="6d2da6f70b712b05e422a6b9ace8b3180af0963b600ceaf47330cce28daa6756" exitCode=0
Sep 16 00:43:32.990986 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:32.990380    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m9fb7" event={"ID":"e94d6aa4-58be-4a67-bd8b-5670201fc0af","Type":"ContainerDied","Data":"6d2da6f70b712b05e422a6b9ace8b3180af0963b600ceaf47330cce28daa6756"}
Sep 16 00:43:32.994436 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:32.994410    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fvnth" event={"ID":"4187c8f2-c639-4f19-90aa-5c7dfad934f6","Type":"ContainerStarted","Data":"add167f4736bef00435739d9b34f8dbfda8e4937299bc696e642f266ee684f00"}
Sep 16 00:43:32.994731 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:32.994713    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fvnth"
Sep 16 00:43:33.009566 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:33.009550    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fvnth"
Sep 16 00:43:33.039081 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:33.039042    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-fvnth" podStartSLOduration=9.967507637 podStartE2EDuration="28.039026844s" totalImagesPullingTime="18.071519207s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 00:43:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 00:43:33.03741137 +0000 UTC m=+27.818912973" watchObservedRunningTime="2026-09-16 00:43:33.039026844 +0000 UTC m=+27.820528437"
Sep 16 00:43:33.730885 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:33.730853    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4trxq"
Sep 16 00:43:33.731140 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:33.730891    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wk9f6"
Sep 16 00:43:33.731140 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:33.730855    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hnw6l"
Sep 16 00:43:33.731140 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:33.730993    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4trxq" podUID="064c3719-509b-45f7-8367-ab4101e5d3bc"
Sep 16 00:43:33.731140 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:33.731102    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hnw6l" podUID="f2569a02-edf9-4b33-a7df-a8d0fdce0580"
Sep 16 00:43:33.731360 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:33.731183    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wk9f6" podUID="c23418e8-6326-481d-9a85-2cbda835d5f6"
Sep 16 00:43:33.999933 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:33.999909    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fvnth"
Sep 16 00:43:34.000373 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:33.999971    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fvnth"
Sep 16 00:43:34.013703 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:34.013683    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fvnth"
Sep 16 00:43:34.075693 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:34.075666    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-hnw6l"]
Sep 16 00:43:34.075800 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:34.075776    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hnw6l"
Sep 16 00:43:34.075903 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:34.075881    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hnw6l" podUID="f2569a02-edf9-4b33-a7df-a8d0fdce0580"
Sep 16 00:43:34.077910 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:34.077889    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wk9f6"]
Sep 16 00:43:34.077999 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:34.077959    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wk9f6"
Sep 16 00:43:34.078038 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:34.078027    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wk9f6" podUID="c23418e8-6326-481d-9a85-2cbda835d5f6"
Sep 16 00:43:34.086326 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:34.086306    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4trxq"]
Sep 16 00:43:34.086420 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:34.086382    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4trxq"
Sep 16 00:43:34.086524 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:34.086484    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4trxq" podUID="064c3719-509b-45f7-8367-ab4101e5d3bc"
Sep 16 00:43:35.003892 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:35.003667    2556 generic.go:356] "Generic (PLEG): container finished" podID="e94d6aa4-58be-4a67-bd8b-5670201fc0af" containerID="52127ef53f9d23c68b2c79910f5520182823a8c42f9503951db3cfb5c20423e3" exitCode=0
Sep 16 00:43:35.004274 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:35.003702    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m9fb7" event={"ID":"e94d6aa4-58be-4a67-bd8b-5670201fc0af","Type":"ContainerDied","Data":"52127ef53f9d23c68b2c79910f5520182823a8c42f9503951db3cfb5c20423e3"}
Sep 16 00:43:35.731054 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:35.731023    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4trxq"
Sep 16 00:43:35.731226 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:35.731097    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wk9f6"
Sep 16 00:43:35.731226 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:35.731115    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4trxq" podUID="064c3719-509b-45f7-8367-ab4101e5d3bc"
Sep 16 00:43:35.731226 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:35.731198    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wk9f6" podUID="c23418e8-6326-481d-9a85-2cbda835d5f6"
Sep 16 00:43:35.731404 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:35.731234    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hnw6l"
Sep 16 00:43:35.731404 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:35.731321    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hnw6l" podUID="f2569a02-edf9-4b33-a7df-a8d0fdce0580"
Sep 16 00:43:37.009965 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:37.009931    2556 generic.go:356] "Generic (PLEG): container finished" podID="e94d6aa4-58be-4a67-bd8b-5670201fc0af" containerID="8ffc1ff8e173323334022e82a808b2e914ee17364776bbdf3778cf4e7a9c4e82" exitCode=0
Sep 16 00:43:37.010365 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:37.009973    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m9fb7" event={"ID":"e94d6aa4-58be-4a67-bd8b-5670201fc0af","Type":"ContainerDied","Data":"8ffc1ff8e173323334022e82a808b2e914ee17364776bbdf3778cf4e7a9c4e82"}
Sep 16 00:43:37.730748 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:37.730715    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hnw6l"
Sep 16 00:43:37.730880 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:37.730715    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4trxq"
Sep 16 00:43:37.730880 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:37.730811    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wk9f6"
Sep 16 00:43:37.730880 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:37.730835    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hnw6l" podUID="f2569a02-edf9-4b33-a7df-a8d0fdce0580"
Sep 16 00:43:37.730997 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:37.730881    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wk9f6" podUID="c23418e8-6326-481d-9a85-2cbda835d5f6"
Sep 16 00:43:37.730997 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:37.730944    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4trxq" podUID="064c3719-509b-45f7-8367-ab4101e5d3bc"
Sep 16 00:43:39.468486 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:39.468430    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f2569a02-edf9-4b33-a7df-a8d0fdce0580-metrics-certs\") pod \"network-metrics-daemon-hnw6l\" (UID: \"f2569a02-edf9-4b33-a7df-a8d0fdce0580\") " pod="openshift-multus/network-metrics-daemon-hnw6l"
Sep 16 00:43:39.469002 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:39.468494    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c23418e8-6326-481d-9a85-2cbda835d5f6-original-pull-secret\") pod \"global-pull-secret-syncer-wk9f6\" (UID: \"c23418e8-6326-481d-9a85-2cbda835d5f6\") " pod="kube-system/global-pull-secret-syncer-wk9f6"
Sep 16 00:43:39.469002 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:39.468581    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 00:43:39.469002 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:39.468599    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 00:43:39.469002 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:39.468644    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f2569a02-edf9-4b33-a7df-a8d0fdce0580-metrics-certs podName:f2569a02-edf9-4b33-a7df-a8d0fdce0580 nodeName:}" failed. No retries permitted until 2026-09-16 00:44:11.468628306 +0000 UTC m=+66.250129895 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f2569a02-edf9-4b33-a7df-a8d0fdce0580-metrics-certs") pod "network-metrics-daemon-hnw6l" (UID: "f2569a02-edf9-4b33-a7df-a8d0fdce0580") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 00:43:39.469002 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:39.468658    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c23418e8-6326-481d-9a85-2cbda835d5f6-original-pull-secret podName:c23418e8-6326-481d-9a85-2cbda835d5f6 nodeName:}" failed. No retries permitted until 2026-09-16 00:44:11.468651832 +0000 UTC m=+66.250153402 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c23418e8-6326-481d-9a85-2cbda835d5f6-original-pull-secret") pod "global-pull-secret-syncer-wk9f6" (UID: "c23418e8-6326-481d-9a85-2cbda835d5f6") : object "kube-system"/"original-pull-secret" not registered
Sep 16 00:43:39.569416 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:39.569386    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7tqn\" (UniqueName: \"kubernetes.io/projected/064c3719-509b-45f7-8367-ab4101e5d3bc-kube-api-access-d7tqn\") pod \"network-check-target-4trxq\" (UID: \"064c3719-509b-45f7-8367-ab4101e5d3bc\") " pod="openshift-network-diagnostics/network-check-target-4trxq"
Sep 16 00:43:39.569598 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:39.569505    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 00:43:39.569598 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:39.569526    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 00:43:39.569598 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:39.569536    2556 projected.go:196] Error preparing data for projected volume kube-api-access-d7tqn for pod openshift-network-diagnostics/network-check-target-4trxq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 00:43:39.569598 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:39.569587    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/064c3719-509b-45f7-8367-ab4101e5d3bc-kube-api-access-d7tqn podName:064c3719-509b-45f7-8367-ab4101e5d3bc nodeName:}" failed. No retries permitted until 2026-09-16 00:44:11.569573148 +0000 UTC m=+66.351074717 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-d7tqn" (UniqueName: "kubernetes.io/projected/064c3719-509b-45f7-8367-ab4101e5d3bc-kube-api-access-d7tqn") pod "network-check-target-4trxq" (UID: "064c3719-509b-45f7-8367-ab4101e5d3bc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 00:43:39.730694 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:39.730626    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hnw6l"
Sep 16 00:43:39.730840 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:39.730626    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4trxq"
Sep 16 00:43:39.730840 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:39.730754    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hnw6l" podUID="f2569a02-edf9-4b33-a7df-a8d0fdce0580"
Sep 16 00:43:39.730966 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:39.730835    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4trxq" podUID="064c3719-509b-45f7-8367-ab4101e5d3bc"
Sep 16 00:43:39.730966 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:39.730894    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wk9f6"
Sep 16 00:43:39.731085 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:39.730978    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wk9f6" podUID="c23418e8-6326-481d-9a85-2cbda835d5f6"
Sep 16 00:43:41.029915 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:41.029886    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-207.ec2.internal" event="NodeReady"
Sep 16 00:43:41.030334 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:41.029985    2556 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 16 00:43:41.079675 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:41.079646    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-rsqrc"]
Sep 16 00:43:41.084416 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:41.084397    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-jt2l9"]
Sep 16 00:43:41.084564 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:41.084545    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rsqrc"
Sep 16 00:43:41.086862 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:41.086841    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 16 00:43:41.087145 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:41.087128    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 16 00:43:41.087385 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:41.087368    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-n5l29\""
Sep 16 00:43:41.087648 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:41.087536    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jt2l9"
Sep 16 00:43:41.090781 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:41.090719    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 16 00:43:41.090922 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:41.090903    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 16 00:43:41.090986 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:41.090966    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-tzjjl\""
Sep 16 00:43:41.091281 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:41.091264    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 16 00:43:41.101281 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:41.101254    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jt2l9"]
Sep 16 00:43:41.102048 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:41.102030    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-rsqrc"]
Sep 16 00:43:41.183453 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:41.183429    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80-metrics-tls\") pod \"dns-default-rsqrc\" (UID: \"d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80\") " pod="openshift-dns/dns-default-rsqrc"
Sep 16 00:43:41.183622 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:41.183482    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aa0a0489-cc60-43ea-89dc-cb3d97276b9d-cert\") pod \"ingress-canary-jt2l9\" (UID: \"aa0a0489-cc60-43ea-89dc-cb3d97276b9d\") " pod="openshift-ingress-canary/ingress-canary-jt2l9"
Sep 16 00:43:41.183622 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:41.183503    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80-tmp-dir\") pod \"dns-default-rsqrc\" (UID: \"d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80\") " pod="openshift-dns/dns-default-rsqrc"
Sep 16 00:43:41.183622 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:41.183562    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80-config-volume\") pod \"dns-default-rsqrc\" (UID: \"d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80\") " pod="openshift-dns/dns-default-rsqrc"
Sep 16 00:43:41.183622 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:41.183607    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cx5qg\" (UniqueName: \"kubernetes.io/projected/aa0a0489-cc60-43ea-89dc-cb3d97276b9d-kube-api-access-cx5qg\") pod \"ingress-canary-jt2l9\" (UID: \"aa0a0489-cc60-43ea-89dc-cb3d97276b9d\") " pod="openshift-ingress-canary/ingress-canary-jt2l9"
Sep 16 00:43:41.183750 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:41.183631    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kv9v\" (UniqueName: \"kubernetes.io/projected/d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80-kube-api-access-9kv9v\") pod \"dns-default-rsqrc\" (UID: \"d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80\") " pod="openshift-dns/dns-default-rsqrc"
Sep 16 00:43:41.284233 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:41.284207    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80-metrics-tls\") pod \"dns-default-rsqrc\" (UID: \"d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80\") " pod="openshift-dns/dns-default-rsqrc"
Sep 16 00:43:41.284342 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:41.284247    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aa0a0489-cc60-43ea-89dc-cb3d97276b9d-cert\") pod \"ingress-canary-jt2l9\" (UID: \"aa0a0489-cc60-43ea-89dc-cb3d97276b9d\") " pod="openshift-ingress-canary/ingress-canary-jt2l9"
Sep 16 00:43:41.284342 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:41.284260    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80-tmp-dir\") pod \"dns-default-rsqrc\" (UID: \"d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80\") " pod="openshift-dns/dns-default-rsqrc"
Sep 16 00:43:41.284342 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:41.284279    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80-config-volume\") pod \"dns-default-rsqrc\" (UID: \"d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80\") " pod="openshift-dns/dns-default-rsqrc"
Sep 16 00:43:41.284549 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:41.284381    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 00:43:41.284549 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:41.284448    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80-metrics-tls podName:d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80 nodeName:}" failed. No retries permitted until 2026-09-16 00:43:41.784426935 +0000 UTC m=+36.565928520 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80-metrics-tls") pod "dns-default-rsqrc" (UID: "d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80") : secret "dns-default-metrics-tls" not found
Sep 16 00:43:41.284653 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:41.284612    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cx5qg\" (UniqueName: \"kubernetes.io/projected/aa0a0489-cc60-43ea-89dc-cb3d97276b9d-kube-api-access-cx5qg\") pod \"ingress-canary-jt2l9\" (UID: \"aa0a0489-cc60-43ea-89dc-cb3d97276b9d\") " pod="openshift-ingress-canary/ingress-canary-jt2l9"
Sep 16 00:43:41.284698 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:41.284670    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kv9v\" (UniqueName: \"kubernetes.io/projected/d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80-kube-api-access-9kv9v\") pod \"dns-default-rsqrc\" (UID: \"d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80\") " pod="openshift-dns/dns-default-rsqrc"
Sep 16 00:43:41.285303 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:41.285139    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80-config-volume\") pod \"dns-default-rsqrc\" (UID: \"d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80\") " pod="openshift-dns/dns-default-rsqrc"
Sep 16 00:43:41.285393 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:41.285204    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80-tmp-dir\") pod \"dns-default-rsqrc\" (UID: \"d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80\") " pod="openshift-dns/dns-default-rsqrc"
Sep 16 00:43:41.288328 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:41.288310    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aa0a0489-cc60-43ea-89dc-cb3d97276b9d-cert\") pod \"ingress-canary-jt2l9\" (UID: \"aa0a0489-cc60-43ea-89dc-cb3d97276b9d\") " pod="openshift-ingress-canary/ingress-canary-jt2l9"
Sep 16 00:43:41.294380 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:41.294340    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kv9v\" (UniqueName: \"kubernetes.io/projected/d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80-kube-api-access-9kv9v\") pod \"dns-default-rsqrc\" (UID: \"d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80\") " pod="openshift-dns/dns-default-rsqrc"
Sep 16 00:43:41.294696 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:41.294677    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cx5qg\" (UniqueName: \"kubernetes.io/projected/aa0a0489-cc60-43ea-89dc-cb3d97276b9d-kube-api-access-cx5qg\") pod \"ingress-canary-jt2l9\" (UID: \"aa0a0489-cc60-43ea-89dc-cb3d97276b9d\") " pod="openshift-ingress-canary/ingress-canary-jt2l9"
Sep 16 00:43:41.402717 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:41.402690    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jt2l9"
Sep 16 00:43:41.673770 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:41.673743    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jt2l9"]
Sep 16 00:43:41.676994 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:41.676962    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa0a0489_cc60_43ea_89dc_cb3d97276b9d.slice/crio-1ea721dadd06986d16ca0fcbdaeb44994bb8827b9485b9e513a1f9cf2b2575b7 WatchSource:0}: Error finding container 1ea721dadd06986d16ca0fcbdaeb44994bb8827b9485b9e513a1f9cf2b2575b7: Status 404 returned error can't find the container with id 1ea721dadd06986d16ca0fcbdaeb44994bb8827b9485b9e513a1f9cf2b2575b7
Sep 16 00:43:41.729967 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:41.729934    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wk9f6"
Sep 16 00:43:41.729967 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:41.729952    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hnw6l"
Sep 16 00:43:41.730085 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:41.729934    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4trxq"
Sep 16 00:43:41.733042 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:41.733023    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 16 00:43:41.733170 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:41.733155    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 16 00:43:41.733385 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:41.733189    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ssxtd\""
Sep 16 00:43:41.733385 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:41.733213    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6btht\""
Sep 16 00:43:41.733385 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:41.733227    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 16 00:43:41.733385 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:41.733233    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 16 00:43:41.788578 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:41.788556    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80-metrics-tls\") pod \"dns-default-rsqrc\" (UID: \"d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80\") " pod="openshift-dns/dns-default-rsqrc"
Sep 16 00:43:41.788658 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:41.788648    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 00:43:41.788732 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:41.788722    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80-metrics-tls podName:d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80 nodeName:}" failed. No retries permitted until 2026-09-16 00:43:42.788708651 +0000 UTC m=+37.570210220 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80-metrics-tls") pod "dns-default-rsqrc" (UID: "d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80") : secret "dns-default-metrics-tls" not found
Sep 16 00:43:42.023267 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:42.023203    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jt2l9" event={"ID":"aa0a0489-cc60-43ea-89dc-cb3d97276b9d","Type":"ContainerStarted","Data":"1ea721dadd06986d16ca0fcbdaeb44994bb8827b9485b9e513a1f9cf2b2575b7"}
Sep 16 00:43:42.025892 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:42.025871    2556 generic.go:356] "Generic (PLEG): container finished" podID="e94d6aa4-58be-4a67-bd8b-5670201fc0af" containerID="3d6b478119ebf6b18d3c7c1733dfc1b1e47f947d8fa7869e90c88e06b290c253" exitCode=0
Sep 16 00:43:42.025968 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:42.025908    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m9fb7" event={"ID":"e94d6aa4-58be-4a67-bd8b-5670201fc0af","Type":"ContainerDied","Data":"3d6b478119ebf6b18d3c7c1733dfc1b1e47f947d8fa7869e90c88e06b290c253"}
Sep 16 00:43:42.796215 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:42.796181    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80-metrics-tls\") pod \"dns-default-rsqrc\" (UID: \"d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80\") " pod="openshift-dns/dns-default-rsqrc"
Sep 16 00:43:42.796893 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:42.796309    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 00:43:42.796893 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:42.796375    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80-metrics-tls podName:d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80 nodeName:}" failed. No retries permitted until 2026-09-16 00:43:44.796359819 +0000 UTC m=+39.577861393 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80-metrics-tls") pod "dns-default-rsqrc" (UID: "d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80") : secret "dns-default-metrics-tls" not found
Sep 16 00:43:43.033129 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:43.033095    2556 generic.go:356] "Generic (PLEG): container finished" podID="e94d6aa4-58be-4a67-bd8b-5670201fc0af" containerID="788115348c423db3a0a9d0634c49727b953d17714e0ef751ba146822dd8765a9" exitCode=0
Sep 16 00:43:43.033295 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:43.033176    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m9fb7" event={"ID":"e94d6aa4-58be-4a67-bd8b-5670201fc0af","Type":"ContainerDied","Data":"788115348c423db3a0a9d0634c49727b953d17714e0ef751ba146822dd8765a9"}
Sep 16 00:43:44.038684 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:44.038461    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m9fb7" event={"ID":"e94d6aa4-58be-4a67-bd8b-5670201fc0af","Type":"ContainerStarted","Data":"d38f20f82cd420338dc4a5b9599664a162908d6de398a5c2dde79d2f98311bb2"}
Sep 16 00:43:44.061124 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:44.061079    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-m9fb7" podStartSLOduration=10.468117596999999 podStartE2EDuration="39.061061757s" totalImagesPullingTime="28.59294416s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 00:43:05 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 00:43:44.059524277 +0000 UTC m=+38.841025868" watchObservedRunningTime="2026-09-16 00:43:44.061061757 +0000 UTC m=+38.842563395"
Sep 16 00:43:44.810801 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:44.810769    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80-metrics-tls\") pod \"dns-default-rsqrc\" (UID: \"d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80\") " pod="openshift-dns/dns-default-rsqrc"
Sep 16 00:43:44.810999 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:44.810884    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 00:43:44.810999 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:44.810932    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80-metrics-tls podName:d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80 nodeName:}" failed. No retries permitted until 2026-09-16 00:43:48.810920373 +0000 UTC m=+43.592421947 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80-metrics-tls") pod "dns-default-rsqrc" (UID: "d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80") : secret "dns-default-metrics-tls" not found
Sep 16 00:43:45.042248 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:45.042216    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jt2l9" event={"ID":"aa0a0489-cc60-43ea-89dc-cb3d97276b9d","Type":"ContainerStarted","Data":"de88ed05175c7a4002b07393ac033d29b5214076eeec576c8decfa9b16acc356"}
Sep 16 00:43:45.068892 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:45.068814    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-jt2l9" podStartSLOduration=1.7065199340000001 podStartE2EDuration="4.06880005s" totalImagesPullingTime="2.362280116s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 00:43:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 00:43:45.068254082 +0000 UTC m=+39.849755673" watchObservedRunningTime="2026-09-16 00:43:45.06880005 +0000 UTC m=+39.850301690"
Sep 16 00:43:48.835549 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:48.835507    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80-metrics-tls\") pod \"dns-default-rsqrc\" (UID: \"d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80\") " pod="openshift-dns/dns-default-rsqrc"
Sep 16 00:43:48.835994 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:48.835643    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 00:43:48.835994 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:48.835718    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80-metrics-tls podName:d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80 nodeName:}" failed. No retries permitted until 2026-09-16 00:43:56.83570195 +0000 UTC m=+51.617203521 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80-metrics-tls") pod "dns-default-rsqrc" (UID: "d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80") : secret "dns-default-metrics-tls" not found
Sep 16 00:43:56.886483 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:56.886444    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80-metrics-tls\") pod \"dns-default-rsqrc\" (UID: \"d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80\") " pod="openshift-dns/dns-default-rsqrc"
Sep 16 00:43:56.886914 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:56.886588    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 00:43:56.886914 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:43:56.886664    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80-metrics-tls podName:d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80 nodeName:}" failed. No retries permitted until 2026-09-16 00:44:12.886647717 +0000 UTC m=+67.668149302 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80-metrics-tls") pod "dns-default-rsqrc" (UID: "d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80") : secret "dns-default-metrics-tls" not found
Sep 16 00:43:58.085697 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:58.085666    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-765c74974-b6ckj"]
Sep 16 00:43:58.106563 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:58.106538    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d57768899-dfrv6"]
Sep 16 00:43:58.106697 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:58.106656    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-765c74974-b6ckj"
Sep 16 00:43:58.109115 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:58.109087    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 16 00:43:58.110135 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:58.110112    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 16 00:43:58.110249 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:58.110153    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Sep 16 00:43:58.110310 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:58.110269    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 16 00:43:58.110365 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:58.110356    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-whjb2\""
Sep 16 00:43:58.122703 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:58.122679    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d57768899-dfrv6"
Sep 16 00:43:58.122809 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:58.122683    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-765c74974-b6ckj"]
Sep 16 00:43:58.122809 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:58.122793    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d57768899-dfrv6"]
Sep 16 00:43:58.124928 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:58.124912    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Sep 16 00:43:58.136554 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:58.136534    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-574f8cdd64-nwwz4"]
Sep 16 00:43:58.160655 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:58.160630    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-574f8cdd64-nwwz4"]
Sep 16 00:43:58.160757 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:58.160729    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-574f8cdd64-nwwz4"
Sep 16 00:43:58.163339 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:58.163318    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Sep 16 00:43:58.163456 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:58.163320    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Sep 16 00:43:58.163456 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:58.163381    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Sep 16 00:43:58.163637 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:58.163566    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Sep 16 00:43:58.195813 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:58.195792    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/7d7818d0-fcc3-4e11-9f5a-cbf7ab5592e5-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-574f8cdd64-nwwz4\" (UID: \"7d7818d0-fcc3-4e11-9f5a-cbf7ab5592e5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-574f8cdd64-nwwz4"
Sep 16 00:43:58.195931 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:58.195817    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-429kl\" (UniqueName: \"kubernetes.io/projected/7d7818d0-fcc3-4e11-9f5a-cbf7ab5592e5-kube-api-access-429kl\") pod \"cluster-proxy-proxy-agent-574f8cdd64-nwwz4\" (UID: \"7d7818d0-fcc3-4e11-9f5a-cbf7ab5592e5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-574f8cdd64-nwwz4"
Sep 16 00:43:58.195931 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:58.195836    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a59cb3a3-3ac9-45c0-983b-de7cd8810a74-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-765c74974-b6ckj\" (UID: \"a59cb3a3-3ac9-45c0-983b-de7cd8810a74\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-765c74974-b6ckj"
Sep 16 00:43:58.195931 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:58.195852    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a81e6da0-2824-495c-a0d4-6c65b9177170-klusterlet-config\") pod \"klusterlet-addon-workmgr-5d57768899-dfrv6\" (UID: \"a81e6da0-2824-495c-a0d4-6c65b9177170\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d57768899-dfrv6"
Sep 16 00:43:58.195931 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:58.195904    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7d7818d0-fcc3-4e11-9f5a-cbf7ab5592e5-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-574f8cdd64-nwwz4\" (UID: \"7d7818d0-fcc3-4e11-9f5a-cbf7ab5592e5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-574f8cdd64-nwwz4"
Sep 16 00:43:58.195931 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:58.195932    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/7d7818d0-fcc3-4e11-9f5a-cbf7ab5592e5-hub\") pod \"cluster-proxy-proxy-agent-574f8cdd64-nwwz4\" (UID: \"7d7818d0-fcc3-4e11-9f5a-cbf7ab5592e5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-574f8cdd64-nwwz4"
Sep 16 00:43:58.196121 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:58.195950    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wtbw\" (UniqueName: \"kubernetes.io/projected/a81e6da0-2824-495c-a0d4-6c65b9177170-kube-api-access-8wtbw\") pod \"klusterlet-addon-workmgr-5d57768899-dfrv6\" (UID: \"a81e6da0-2824-495c-a0d4-6c65b9177170\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d57768899-dfrv6"
Sep 16 00:43:58.196121 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:58.195967    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/7d7818d0-fcc3-4e11-9f5a-cbf7ab5592e5-ca\") pod \"cluster-proxy-proxy-agent-574f8cdd64-nwwz4\" (UID: \"7d7818d0-fcc3-4e11-9f5a-cbf7ab5592e5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-574f8cdd64-nwwz4"
Sep 16 00:43:58.196121 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:58.196023    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d7818d0-fcc3-4e11-9f5a-cbf7ab5592e5-additional-service-ca\") pod \"cluster-proxy-proxy-agent-574f8cdd64-nwwz4\" (UID: \"7d7818d0-fcc3-4e11-9f5a-cbf7ab5592e5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-574f8cdd64-nwwz4"
Sep 16 00:43:58.196121 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:58.196040    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hn5d9\" (UniqueName: \"kubernetes.io/projected/a59cb3a3-3ac9-45c0-983b-de7cd8810a74-kube-api-access-hn5d9\") pod \"managed-serviceaccount-addon-agent-765c74974-b6ckj\" (UID: \"a59cb3a3-3ac9-45c0-983b-de7cd8810a74\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-765c74974-b6ckj"
Sep 16 00:43:58.196121 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:58.196057    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a81e6da0-2824-495c-a0d4-6c65b9177170-tmp\") pod \"klusterlet-addon-workmgr-5d57768899-dfrv6\" (UID: \"a81e6da0-2824-495c-a0d4-6c65b9177170\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d57768899-dfrv6"
Sep 16 00:43:58.297024 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:58.296992    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d7818d0-fcc3-4e11-9f5a-cbf7ab5592e5-additional-service-ca\") pod \"cluster-proxy-proxy-agent-574f8cdd64-nwwz4\" (UID: \"7d7818d0-fcc3-4e11-9f5a-cbf7ab5592e5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-574f8cdd64-nwwz4"
Sep 16 00:43:58.297024 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:58.297026    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hn5d9\" (UniqueName: \"kubernetes.io/projected/a59cb3a3-3ac9-45c0-983b-de7cd8810a74-kube-api-access-hn5d9\") pod \"managed-serviceaccount-addon-agent-765c74974-b6ckj\" (UID: \"a59cb3a3-3ac9-45c0-983b-de7cd8810a74\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-765c74974-b6ckj"
Sep 16 00:43:58.297210 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:58.297045    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a81e6da0-2824-495c-a0d4-6c65b9177170-tmp\") pod \"klusterlet-addon-workmgr-5d57768899-dfrv6\" (UID: \"a81e6da0-2824-495c-a0d4-6c65b9177170\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d57768899-dfrv6"
Sep 16 00:43:58.297210 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:58.297171    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/7d7818d0-fcc3-4e11-9f5a-cbf7ab5592e5-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-574f8cdd64-nwwz4\" (UID: \"7d7818d0-fcc3-4e11-9f5a-cbf7ab5592e5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-574f8cdd64-nwwz4"
Sep 16 00:43:58.297312 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:58.297210    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-429kl\" (UniqueName: \"kubernetes.io/projected/7d7818d0-fcc3-4e11-9f5a-cbf7ab5592e5-kube-api-access-429kl\") pod \"cluster-proxy-proxy-agent-574f8cdd64-nwwz4\" (UID: \"7d7818d0-fcc3-4e11-9f5a-cbf7ab5592e5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-574f8cdd64-nwwz4"
Sep 16 00:43:58.297312 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:58.297243    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a59cb3a3-3ac9-45c0-983b-de7cd8810a74-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-765c74974-b6ckj\" (UID: \"a59cb3a3-3ac9-45c0-983b-de7cd8810a74\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-765c74974-b6ckj"
Sep 16 00:43:58.297312 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:58.297270    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a81e6da0-2824-495c-a0d4-6c65b9177170-klusterlet-config\") pod \"klusterlet-addon-workmgr-5d57768899-dfrv6\" (UID: \"a81e6da0-2824-495c-a0d4-6c65b9177170\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d57768899-dfrv6"
Sep 16 00:43:58.297453 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:58.297321    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7d7818d0-fcc3-4e11-9f5a-cbf7ab5592e5-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-574f8cdd64-nwwz4\" (UID: \"7d7818d0-fcc3-4e11-9f5a-cbf7ab5592e5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-574f8cdd64-nwwz4"
Sep 16 00:43:58.297453 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:58.297354    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/7d7818d0-fcc3-4e11-9f5a-cbf7ab5592e5-hub\") pod \"cluster-proxy-proxy-agent-574f8cdd64-nwwz4\" (UID: \"7d7818d0-fcc3-4e11-9f5a-cbf7ab5592e5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-574f8cdd64-nwwz4"
Sep 16 00:43:58.297453 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:58.297381    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wtbw\" (UniqueName: \"kubernetes.io/projected/a81e6da0-2824-495c-a0d4-6c65b9177170-kube-api-access-8wtbw\") pod \"klusterlet-addon-workmgr-5d57768899-dfrv6\" (UID: \"a81e6da0-2824-495c-a0d4-6c65b9177170\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d57768899-dfrv6"
Sep 16 00:43:58.297453 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:58.297386    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a81e6da0-2824-495c-a0d4-6c65b9177170-tmp\") pod \"klusterlet-addon-workmgr-5d57768899-dfrv6\" (UID: \"a81e6da0-2824-495c-a0d4-6c65b9177170\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d57768899-dfrv6"
Sep 16 00:43:58.297453 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:58.297409    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/7d7818d0-fcc3-4e11-9f5a-cbf7ab5592e5-ca\") pod \"cluster-proxy-proxy-agent-574f8cdd64-nwwz4\" (UID: \"7d7818d0-fcc3-4e11-9f5a-cbf7ab5592e5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-574f8cdd64-nwwz4"
Sep 16 00:43:58.297928 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:58.297905    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d7818d0-fcc3-4e11-9f5a-cbf7ab5592e5-additional-service-ca\") pod \"cluster-proxy-proxy-agent-574f8cdd64-nwwz4\" (UID: \"7d7818d0-fcc3-4e11-9f5a-cbf7ab5592e5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-574f8cdd64-nwwz4"
Sep 16 00:43:58.301039 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:58.301013    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/7d7818d0-fcc3-4e11-9f5a-cbf7ab5592e5-hub\") pod \"cluster-proxy-proxy-agent-574f8cdd64-nwwz4\" (UID: \"7d7818d0-fcc3-4e11-9f5a-cbf7ab5592e5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-574f8cdd64-nwwz4"
Sep 16 00:43:58.301133 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:58.301048    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/7d7818d0-fcc3-4e11-9f5a-cbf7ab5592e5-ca\") pod \"cluster-proxy-proxy-agent-574f8cdd64-nwwz4\" (UID: \"7d7818d0-fcc3-4e11-9f5a-cbf7ab5592e5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-574f8cdd64-nwwz4"
Sep 16 00:43:58.301133 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:58.301020    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a81e6da0-2824-495c-a0d4-6c65b9177170-klusterlet-config\") pod \"klusterlet-addon-workmgr-5d57768899-dfrv6\" (UID: \"a81e6da0-2824-495c-a0d4-6c65b9177170\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d57768899-dfrv6"
Sep 16 00:43:58.301258 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:58.301178    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/7d7818d0-fcc3-4e11-9f5a-cbf7ab5592e5-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-574f8cdd64-nwwz4\" (UID: \"7d7818d0-fcc3-4e11-9f5a-cbf7ab5592e5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-574f8cdd64-nwwz4"
Sep 16 00:43:58.301325 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:58.301296    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a59cb3a3-3ac9-45c0-983b-de7cd8810a74-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-765c74974-b6ckj\" (UID: \"a59cb3a3-3ac9-45c0-983b-de7cd8810a74\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-765c74974-b6ckj"
Sep 16 00:43:58.301417 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:58.301401    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7d7818d0-fcc3-4e11-9f5a-cbf7ab5592e5-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-574f8cdd64-nwwz4\" (UID: \"7d7818d0-fcc3-4e11-9f5a-cbf7ab5592e5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-574f8cdd64-nwwz4"
Sep 16 00:43:58.305406 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:58.305379    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hn5d9\" (UniqueName: \"kubernetes.io/projected/a59cb3a3-3ac9-45c0-983b-de7cd8810a74-kube-api-access-hn5d9\") pod \"managed-serviceaccount-addon-agent-765c74974-b6ckj\" (UID: \"a59cb3a3-3ac9-45c0-983b-de7cd8810a74\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-765c74974-b6ckj"
Sep 16 00:43:58.305586 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:58.305564    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wtbw\" (UniqueName: \"kubernetes.io/projected/a81e6da0-2824-495c-a0d4-6c65b9177170-kube-api-access-8wtbw\") pod \"klusterlet-addon-workmgr-5d57768899-dfrv6\" (UID: \"a81e6da0-2824-495c-a0d4-6c65b9177170\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d57768899-dfrv6"
Sep 16 00:43:58.305731 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:58.305713    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-429kl\" (UniqueName: \"kubernetes.io/projected/7d7818d0-fcc3-4e11-9f5a-cbf7ab5592e5-kube-api-access-429kl\") pod \"cluster-proxy-proxy-agent-574f8cdd64-nwwz4\" (UID: \"7d7818d0-fcc3-4e11-9f5a-cbf7ab5592e5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-574f8cdd64-nwwz4"
Sep 16 00:43:58.430355 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:58.430281    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-765c74974-b6ckj"
Sep 16 00:43:58.436524 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:58.436495    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d57768899-dfrv6"
Sep 16 00:43:58.469974 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:58.469281    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-574f8cdd64-nwwz4"
Sep 16 00:43:58.700085 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:58.700015    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-765c74974-b6ckj"]
Sep 16 00:43:58.703122 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:58.703061    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d57768899-dfrv6"]
Sep 16 00:43:58.703283 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:58.703255    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda59cb3a3_3ac9_45c0_983b_de7cd8810a74.slice/crio-63273f116f957eab3436153f86a815bf8c37520480bd109a8ebe88eb99bebbd7 WatchSource:0}: Error finding container 63273f116f957eab3436153f86a815bf8c37520480bd109a8ebe88eb99bebbd7: Status 404 returned error can't find the container with id 63273f116f957eab3436153f86a815bf8c37520480bd109a8ebe88eb99bebbd7
Sep 16 00:43:58.705285 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:58.705265    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda81e6da0_2824_495c_a0d4_6c65b9177170.slice/crio-d9ee8eb2184d5337b4ae28daafdf45c7f9ef6a7fbbd3aebb1c853100a900d105 WatchSource:0}: Error finding container d9ee8eb2184d5337b4ae28daafdf45c7f9ef6a7fbbd3aebb1c853100a900d105: Status 404 returned error can't find the container with id d9ee8eb2184d5337b4ae28daafdf45c7f9ef6a7fbbd3aebb1c853100a900d105
Sep 16 00:43:58.746615 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:58.746591    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-574f8cdd64-nwwz4"]
Sep 16 00:43:58.749051 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:43:58.749028    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7d7818d0_fcc3_4e11_9f5a_cbf7ab5592e5.slice/crio-06882278f9dfebe0cf4e72487057e4423241d417da25ac3f1905750616730f18 WatchSource:0}: Error finding container 06882278f9dfebe0cf4e72487057e4423241d417da25ac3f1905750616730f18: Status 404 returned error can't find the container with id 06882278f9dfebe0cf4e72487057e4423241d417da25ac3f1905750616730f18
Sep 16 00:43:59.074593 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:59.074522    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d57768899-dfrv6" event={"ID":"a81e6da0-2824-495c-a0d4-6c65b9177170","Type":"ContainerStarted","Data":"d9ee8eb2184d5337b4ae28daafdf45c7f9ef6a7fbbd3aebb1c853100a900d105"}
Sep 16 00:43:59.075601 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:59.075572    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-574f8cdd64-nwwz4" event={"ID":"7d7818d0-fcc3-4e11-9f5a-cbf7ab5592e5","Type":"ContainerStarted","Data":"06882278f9dfebe0cf4e72487057e4423241d417da25ac3f1905750616730f18"}
Sep 16 00:43:59.076516 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:43:59.076491    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-765c74974-b6ckj" event={"ID":"a59cb3a3-3ac9-45c0-983b-de7cd8810a74","Type":"ContainerStarted","Data":"63273f116f957eab3436153f86a815bf8c37520480bd109a8ebe88eb99bebbd7"}
Sep 16 00:44:06.017048 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:44:06.016977    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fvnth"
Sep 16 00:44:06.101615 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:44:06.101576    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-574f8cdd64-nwwz4" event={"ID":"7d7818d0-fcc3-4e11-9f5a-cbf7ab5592e5","Type":"ContainerStarted","Data":"a5ed3a7b456416abdd208a0976bb8f8553b7902fa54d5fa4e220fe3ea92c23de"}
Sep 16 00:44:06.101795 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:44:06.101633    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-574f8cdd64-nwwz4" event={"ID":"7d7818d0-fcc3-4e11-9f5a-cbf7ab5592e5","Type":"ContainerStarted","Data":"bc7ffb8775c99b9de5dd5e57aae8485d50aa9dc6651a225f2b2c23cba429b04d"}
Sep 16 00:44:06.101795 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:44:06.101650    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-574f8cdd64-nwwz4" event={"ID":"7d7818d0-fcc3-4e11-9f5a-cbf7ab5592e5","Type":"ContainerStarted","Data":"a3a58f3592e5a3401cee09c0945006f2986fb52558ac6062010f1f7ce1564c51"}
Sep 16 00:44:06.103204 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:44:06.103177    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-765c74974-b6ckj" event={"ID":"a59cb3a3-3ac9-45c0-983b-de7cd8810a74","Type":"ContainerStarted","Data":"f37690cac5088187bd5157c43b4ea0f310dd37d6736352ca15bc8dd5b3c9e614"}
Sep 16 00:44:06.104553 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:44:06.104532    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d57768899-dfrv6" event={"ID":"a81e6da0-2824-495c-a0d4-6c65b9177170","Type":"ContainerStarted","Data":"366896c0487c4bda2e113e74f23ee6879ddb0e1352e31e555996e9027e8c5214"}
Sep 16 00:44:06.104705 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:44:06.104688    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d57768899-dfrv6"
Sep 16 00:44:06.106410 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:44:06.106394    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d57768899-dfrv6"
Sep 16 00:44:06.122246 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:44:06.122205    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-574f8cdd64-nwwz4" podStartSLOduration=1.270054075 podStartE2EDuration="8.122193132s" totalImagesPullingTime="6.852139057s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 00:43:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 00:44:06.121168211 +0000 UTC m=+60.902669802" watchObservedRunningTime="2026-09-16 00:44:06.122193132 +0000 UTC m=+60.903694718"
Sep 16 00:44:06.134734 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:44:06.134696    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-765c74974-b6ckj" podStartSLOduration=1.251661306 podStartE2EDuration="8.134683507s" totalImagesPullingTime="6.883022201s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 00:43:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 00:44:06.134392947 +0000 UTC m=+60.915894539" watchObservedRunningTime="2026-09-16 00:44:06.134683507 +0000 UTC m=+60.916185155"
Sep 16 00:44:06.152530 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:44:06.152491    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d57768899-dfrv6" podStartSLOduration=1.2711752619999999 podStartE2EDuration="8.152458366s" totalImagesPullingTime="6.881283104s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 00:43:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 00:44:06.151427062 +0000 UTC m=+60.932928667" watchObservedRunningTime="2026-09-16 00:44:06.152458366 +0000 UTC m=+60.933959958"
Sep 16 00:44:11.492837 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:44:11.492780    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f2569a02-edf9-4b33-a7df-a8d0fdce0580-metrics-certs\") pod \"network-metrics-daemon-hnw6l\" (UID: \"f2569a02-edf9-4b33-a7df-a8d0fdce0580\") " pod="openshift-multus/network-metrics-daemon-hnw6l"
Sep 16 00:44:11.493229 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:44:11.492865    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c23418e8-6326-481d-9a85-2cbda835d5f6-original-pull-secret\") pod \"global-pull-secret-syncer-wk9f6\" (UID: \"c23418e8-6326-481d-9a85-2cbda835d5f6\") " pod="kube-system/global-pull-secret-syncer-wk9f6"
Sep 16 00:44:11.495628 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:44:11.495611    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 16 00:44:11.495682 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:44:11.495663    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 16 00:44:11.503583 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:44:11.503560    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 16 00:44:11.503656 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:44:11.503644    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f2569a02-edf9-4b33-a7df-a8d0fdce0580-metrics-certs podName:f2569a02-edf9-4b33-a7df-a8d0fdce0580 nodeName:}" failed. No retries permitted until 2026-09-16 00:45:15.503624673 +0000 UTC m=+130.285126249 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f2569a02-edf9-4b33-a7df-a8d0fdce0580-metrics-certs") pod "network-metrics-daemon-hnw6l" (UID: "f2569a02-edf9-4b33-a7df-a8d0fdce0580") : secret "metrics-daemon-secret" not found
Sep 16 00:44:11.505771 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:44:11.505747    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c23418e8-6326-481d-9a85-2cbda835d5f6-original-pull-secret\") pod \"global-pull-secret-syncer-wk9f6\" (UID: \"c23418e8-6326-481d-9a85-2cbda835d5f6\") " pod="kube-system/global-pull-secret-syncer-wk9f6"
Sep 16 00:44:11.593189 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:44:11.593156    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7tqn\" (UniqueName: \"kubernetes.io/projected/064c3719-509b-45f7-8367-ab4101e5d3bc-kube-api-access-d7tqn\") pod \"network-check-target-4trxq\" (UID: \"064c3719-509b-45f7-8367-ab4101e5d3bc\") " pod="openshift-network-diagnostics/network-check-target-4trxq"
Sep 16 00:44:11.595991 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:44:11.595971    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 16 00:44:11.605774 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:44:11.605758    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 16 00:44:11.616825 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:44:11.616799    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7tqn\" (UniqueName: \"kubernetes.io/projected/064c3719-509b-45f7-8367-ab4101e5d3bc-kube-api-access-d7tqn\") pod \"network-check-target-4trxq\" (UID: \"064c3719-509b-45f7-8367-ab4101e5d3bc\") " pod="openshift-network-diagnostics/network-check-target-4trxq"
Sep 16 00:44:11.740461 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:44:11.740434    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wk9f6"
Sep 16 00:44:11.748344 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:44:11.748290    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ssxtd\""
Sep 16 00:44:11.757033 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:44:11.757014    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4trxq"
Sep 16 00:44:12.035911 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:44:12.035814    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wk9f6"]
Sep 16 00:44:12.038458 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:44:12.038432    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc23418e8_6326_481d_9a85_2cbda835d5f6.slice/crio-dd0b1040b54ee1c5354344cf9a0f44f753fd1ff5270929a1334122b8888da37e WatchSource:0}: Error finding container dd0b1040b54ee1c5354344cf9a0f44f753fd1ff5270929a1334122b8888da37e: Status 404 returned error can't find the container with id dd0b1040b54ee1c5354344cf9a0f44f753fd1ff5270929a1334122b8888da37e
Sep 16 00:44:12.058676 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:44:12.058652    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4trxq"]
Sep 16 00:44:12.061102 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:44:12.061079    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod064c3719_509b_45f7_8367_ab4101e5d3bc.slice/crio-25069eeeeb446c57db1632cdf6b23e5dd538551b885082c8a32f7ef3a4bc3543 WatchSource:0}: Error finding container 25069eeeeb446c57db1632cdf6b23e5dd538551b885082c8a32f7ef3a4bc3543: Status 404 returned error can't find the container with id 25069eeeeb446c57db1632cdf6b23e5dd538551b885082c8a32f7ef3a4bc3543
Sep 16 00:44:12.125229 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:44:12.125193    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4trxq" event={"ID":"064c3719-509b-45f7-8367-ab4101e5d3bc","Type":"ContainerStarted","Data":"25069eeeeb446c57db1632cdf6b23e5dd538551b885082c8a32f7ef3a4bc3543"}
Sep 16 00:44:12.126358 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:44:12.126327    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wk9f6" event={"ID":"c23418e8-6326-481d-9a85-2cbda835d5f6","Type":"ContainerStarted","Data":"dd0b1040b54ee1c5354344cf9a0f44f753fd1ff5270929a1334122b8888da37e"}
Sep 16 00:44:12.904193 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:44:12.904080    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80-metrics-tls\") pod \"dns-default-rsqrc\" (UID: \"d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80\") " pod="openshift-dns/dns-default-rsqrc"
Sep 16 00:44:12.904682 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:44:12.904230    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 00:44:12.904682 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:44:12.904297    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80-metrics-tls podName:d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80 nodeName:}" failed. No retries permitted until 2026-09-16 00:44:44.904281185 +0000 UTC m=+99.685782755 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80-metrics-tls") pod "dns-default-rsqrc" (UID: "d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80") : secret "dns-default-metrics-tls" not found
Sep 16 00:44:17.147880 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:44:17.147821    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4trxq" event={"ID":"064c3719-509b-45f7-8367-ab4101e5d3bc","Type":"ContainerStarted","Data":"9203c6318a5848c2751e56d5ffcc2d604541dac7bcfa6a94c6f6182daed3226d"}
Sep 16 00:44:17.148422 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:44:17.147954    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-4trxq"
Sep 16 00:44:17.149692 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:44:17.149660    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wk9f6" event={"ID":"c23418e8-6326-481d-9a85-2cbda835d5f6","Type":"ContainerStarted","Data":"230d7a8879f690fb58dceb8e760c16b0931fcd2a27b9e76a6e98eafca73e8b09"}
Sep 16 00:44:17.163460 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:44:17.163414    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-4trxq" podStartSLOduration=67.722537443 podStartE2EDuration="1m12.163400026s" totalImagesPullingTime="4.440862583s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 00:43:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 00:44:17.162837974 +0000 UTC m=+71.944339565" watchObservedRunningTime="2026-09-16 00:44:17.163400026 +0000 UTC m=+71.944901617"
Sep 16 00:44:17.176568 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:44:17.176517    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-wk9f6" podStartSLOduration=67.70835394 podStartE2EDuration="1m12.176501788s" totalImagesPullingTime="4.468147848s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 00:43:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 00:44:17.176349557 +0000 UTC m=+71.957851149" watchObservedRunningTime="2026-09-16 00:44:17.176501788 +0000 UTC m=+71.958003378"
Sep 16 00:44:44.913154 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:44:44.913117    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80-metrics-tls\") pod \"dns-default-rsqrc\" (UID: \"d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80\") " pod="openshift-dns/dns-default-rsqrc"
Sep 16 00:44:44.913561 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:44:44.913225    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 00:44:44.913561 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:44:44.913280    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80-metrics-tls podName:d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80 nodeName:}" failed. No retries permitted until 2026-09-16 00:45:48.913265952 +0000 UTC m=+163.694767527 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80-metrics-tls") pod "dns-default-rsqrc" (UID: "d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80") : secret "dns-default-metrics-tls" not found
Sep 16 00:44:48.154987 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:44:48.154952    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-4trxq"
Sep 16 00:45:15.518071 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:45:15.518032    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f2569a02-edf9-4b33-a7df-a8d0fdce0580-metrics-certs\") pod \"network-metrics-daemon-hnw6l\" (UID: \"f2569a02-edf9-4b33-a7df-a8d0fdce0580\") " pod="openshift-multus/network-metrics-daemon-hnw6l"
Sep 16 00:45:15.518578 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:45:15.518170    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 16 00:45:15.518578 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:45:15.518243    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f2569a02-edf9-4b33-a7df-a8d0fdce0580-metrics-certs podName:f2569a02-edf9-4b33-a7df-a8d0fdce0580 nodeName:}" failed. No retries permitted until 2026-09-16 00:47:17.518227869 +0000 UTC m=+252.299729439 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f2569a02-edf9-4b33-a7df-a8d0fdce0580-metrics-certs") pod "network-metrics-daemon-hnw6l" (UID: "f2569a02-edf9-4b33-a7df-a8d0fdce0580") : secret "metrics-daemon-secret" not found
Sep 16 00:45:44.096518 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:45:44.096478    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-rsqrc" podUID="d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80"
Sep 16 00:45:44.399291 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:45:44.399202    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rsqrc"
Sep 16 00:45:44.750130 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:45:44.750059    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-hnw6l" podUID="f2569a02-edf9-4b33-a7df-a8d0fdce0580"
Sep 16 00:45:48.935182 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:45:48.935139    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80-metrics-tls\") pod \"dns-default-rsqrc\" (UID: \"d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80\") " pod="openshift-dns/dns-default-rsqrc"
Sep 16 00:45:48.935668 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:45:48.935275    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 00:45:48.935668 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:45:48.935336    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80-metrics-tls podName:d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80 nodeName:}" failed. No retries permitted until 2026-09-16 00:47:50.935321438 +0000 UTC m=+285.716823025 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80-metrics-tls") pod "dns-default-rsqrc" (UID: "d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80") : secret "dns-default-metrics-tls" not found
Sep 16 00:45:57.730295 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:45:57.730259    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hnw6l"
Sep 16 00:46:06.105825 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:46:06.105774    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d57768899-dfrv6" podUID="a81e6da0-2824-495c-a0d4-6c65b9177170" containerName="acm-agent" probeResult="failure" output="Get \"http://10.133.0.9:8000/readyz\": dial tcp 10.133.0.9:8000: connect: connection refused"
Sep 16 00:46:06.466085 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:46:06.465995    2556 generic.go:356] "Generic (PLEG): container finished" podID="a81e6da0-2824-495c-a0d4-6c65b9177170" containerID="366896c0487c4bda2e113e74f23ee6879ddb0e1352e31e555996e9027e8c5214" exitCode=1
Sep 16 00:46:06.466085 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:46:06.466067    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d57768899-dfrv6" event={"ID":"a81e6da0-2824-495c-a0d4-6c65b9177170","Type":"ContainerDied","Data":"366896c0487c4bda2e113e74f23ee6879ddb0e1352e31e555996e9027e8c5214"}
Sep 16 00:46:06.466432 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:46:06.466415    2556 scope.go:117] "RemoveContainer" containerID="366896c0487c4bda2e113e74f23ee6879ddb0e1352e31e555996e9027e8c5214"
Sep 16 00:46:06.467575 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:46:06.467551    2556 generic.go:356] "Generic (PLEG): container finished" podID="a59cb3a3-3ac9-45c0-983b-de7cd8810a74" containerID="f37690cac5088187bd5157c43b4ea0f310dd37d6736352ca15bc8dd5b3c9e614" exitCode=255
Sep 16 00:46:06.467684 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:46:06.467604    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-765c74974-b6ckj" event={"ID":"a59cb3a3-3ac9-45c0-983b-de7cd8810a74","Type":"ContainerDied","Data":"f37690cac5088187bd5157c43b4ea0f310dd37d6736352ca15bc8dd5b3c9e614"}
Sep 16 00:46:06.467846 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:46:06.467832    2556 scope.go:117] "RemoveContainer" containerID="f37690cac5088187bd5157c43b4ea0f310dd37d6736352ca15bc8dd5b3c9e614"
Sep 16 00:46:07.476297 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:46:07.476261    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d57768899-dfrv6" event={"ID":"a81e6da0-2824-495c-a0d4-6c65b9177170","Type":"ContainerStarted","Data":"c6923ee3489a1bce87b2334603ced6cd89250a027bdfe7ca4a74f8d5753a6244"}
Sep 16 00:46:07.476764 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:46:07.476546    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d57768899-dfrv6"
Sep 16 00:46:07.477231 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:46:07.477213    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d57768899-dfrv6"
Sep 16 00:46:07.478108 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:46:07.478088    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-765c74974-b6ckj" event={"ID":"a59cb3a3-3ac9-45c0-983b-de7cd8810a74","Type":"ContainerStarted","Data":"3435f44427e63cd6115688472fc04fb6e37028995762e2d45b10b2b112c8fbe3"}
Sep 16 00:47:17.519085 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:17.519046    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f2569a02-edf9-4b33-a7df-a8d0fdce0580-metrics-certs\") pod \"network-metrics-daemon-hnw6l\" (UID: \"f2569a02-edf9-4b33-a7df-a8d0fdce0580\") " pod="openshift-multus/network-metrics-daemon-hnw6l"
Sep 16 00:47:17.519568 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:47:17.519190    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 16 00:47:17.519568 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:47:17.519266    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f2569a02-edf9-4b33-a7df-a8d0fdce0580-metrics-certs podName:f2569a02-edf9-4b33-a7df-a8d0fdce0580 nodeName:}" failed. No retries permitted until 2026-09-16 00:49:19.519247089 +0000 UTC m=+374.300748667 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f2569a02-edf9-4b33-a7df-a8d0fdce0580-metrics-certs") pod "network-metrics-daemon-hnw6l" (UID: "f2569a02-edf9-4b33-a7df-a8d0fdce0580") : secret "metrics-daemon-secret" not found
Sep 16 00:47:21.385496 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:21.385451    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-mkbc5"]
Sep 16 00:47:21.388397 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:21.388381    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mkbc5"
Sep 16 00:47:21.390596 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:21.390569    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 16 00:47:21.390719 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:21.390698    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 16 00:47:21.390778 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:21.390733    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 16 00:47:21.390835 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:21.390733    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 16 00:47:21.391359 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:21.391342    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-lt8cw\""
Sep 16 00:47:21.398442 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:21.398420    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mkbc5"]
Sep 16 00:47:21.543010 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:21.542971    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/10b542a2-e2cd-48f8-bb46-9fe3b0711e83-crio-socket\") pod \"insights-runtime-extractor-mkbc5\" (UID: \"10b542a2-e2cd-48f8-bb46-9fe3b0711e83\") " pod="openshift-insights/insights-runtime-extractor-mkbc5"
Sep 16 00:47:21.543010 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:21.543004    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/10b542a2-e2cd-48f8-bb46-9fe3b0711e83-data-volume\") pod \"insights-runtime-extractor-mkbc5\" (UID: \"10b542a2-e2cd-48f8-bb46-9fe3b0711e83\") " pod="openshift-insights/insights-runtime-extractor-mkbc5"
Sep 16 00:47:21.543209 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:21.543035    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfb28\" (UniqueName: \"kubernetes.io/projected/10b542a2-e2cd-48f8-bb46-9fe3b0711e83-kube-api-access-mfb28\") pod \"insights-runtime-extractor-mkbc5\" (UID: \"10b542a2-e2cd-48f8-bb46-9fe3b0711e83\") " pod="openshift-insights/insights-runtime-extractor-mkbc5"
Sep 16 00:47:21.543209 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:21.543102    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/10b542a2-e2cd-48f8-bb46-9fe3b0711e83-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mkbc5\" (UID: \"10b542a2-e2cd-48f8-bb46-9fe3b0711e83\") " pod="openshift-insights/insights-runtime-extractor-mkbc5"
Sep 16 00:47:21.543209 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:21.543141    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/10b542a2-e2cd-48f8-bb46-9fe3b0711e83-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mkbc5\" (UID: \"10b542a2-e2cd-48f8-bb46-9fe3b0711e83\") " pod="openshift-insights/insights-runtime-extractor-mkbc5"
Sep 16 00:47:21.643666 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:21.643586    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/10b542a2-e2cd-48f8-bb46-9fe3b0711e83-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mkbc5\" (UID: \"10b542a2-e2cd-48f8-bb46-9fe3b0711e83\") " pod="openshift-insights/insights-runtime-extractor-mkbc5"
Sep 16 00:47:21.643666 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:21.643637    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/10b542a2-e2cd-48f8-bb46-9fe3b0711e83-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mkbc5\" (UID: \"10b542a2-e2cd-48f8-bb46-9fe3b0711e83\") " pod="openshift-insights/insights-runtime-extractor-mkbc5"
Sep 16 00:47:21.643790 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:21.643682    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/10b542a2-e2cd-48f8-bb46-9fe3b0711e83-crio-socket\") pod \"insights-runtime-extractor-mkbc5\" (UID: \"10b542a2-e2cd-48f8-bb46-9fe3b0711e83\") " pod="openshift-insights/insights-runtime-extractor-mkbc5"
Sep 16 00:47:21.643790 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:21.643698    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/10b542a2-e2cd-48f8-bb46-9fe3b0711e83-data-volume\") pod \"insights-runtime-extractor-mkbc5\" (UID: \"10b542a2-e2cd-48f8-bb46-9fe3b0711e83\") " pod="openshift-insights/insights-runtime-extractor-mkbc5"
Sep 16 00:47:21.643790 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:47:21.643709    2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 00:47:21.643790 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:21.643726    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfb28\" (UniqueName: \"kubernetes.io/projected/10b542a2-e2cd-48f8-bb46-9fe3b0711e83-kube-api-access-mfb28\") pod \"insights-runtime-extractor-mkbc5\" (UID: \"10b542a2-e2cd-48f8-bb46-9fe3b0711e83\") " pod="openshift-insights/insights-runtime-extractor-mkbc5"
Sep 16 00:47:21.643915 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:21.643801    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/10b542a2-e2cd-48f8-bb46-9fe3b0711e83-crio-socket\") pod \"insights-runtime-extractor-mkbc5\" (UID: \"10b542a2-e2cd-48f8-bb46-9fe3b0711e83\") " pod="openshift-insights/insights-runtime-extractor-mkbc5"
Sep 16 00:47:21.643915 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:47:21.643806    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/10b542a2-e2cd-48f8-bb46-9fe3b0711e83-kube-rbac-proxy-cm podName:10b542a2-e2cd-48f8-bb46-9fe3b0711e83 nodeName:}" failed. No retries permitted until 2026-09-16 00:47:22.143785878 +0000 UTC m=+256.925287461 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/10b542a2-e2cd-48f8-bb46-9fe3b0711e83-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mkbc5" (UID: "10b542a2-e2cd-48f8-bb46-9fe3b0711e83") : configmap "kube-rbac-proxy" not found
Sep 16 00:47:21.643915 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:47:21.643806    2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 00:47:21.643915 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:47:21.643891    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10b542a2-e2cd-48f8-bb46-9fe3b0711e83-insights-runtime-extractor-tls podName:10b542a2-e2cd-48f8-bb46-9fe3b0711e83 nodeName:}" failed. No retries permitted until 2026-09-16 00:47:22.143872246 +0000 UTC m=+256.925373828 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/10b542a2-e2cd-48f8-bb46-9fe3b0711e83-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mkbc5" (UID: "10b542a2-e2cd-48f8-bb46-9fe3b0711e83") : secret "insights-runtime-extractor-tls" not found
Sep 16 00:47:21.644056 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:21.644016    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/10b542a2-e2cd-48f8-bb46-9fe3b0711e83-data-volume\") pod \"insights-runtime-extractor-mkbc5\" (UID: \"10b542a2-e2cd-48f8-bb46-9fe3b0711e83\") " pod="openshift-insights/insights-runtime-extractor-mkbc5"
Sep 16 00:47:21.654030 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:21.653999    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfb28\" (UniqueName: \"kubernetes.io/projected/10b542a2-e2cd-48f8-bb46-9fe3b0711e83-kube-api-access-mfb28\") pod \"insights-runtime-extractor-mkbc5\" (UID: \"10b542a2-e2cd-48f8-bb46-9fe3b0711e83\") " pod="openshift-insights/insights-runtime-extractor-mkbc5"
Sep 16 00:47:22.149582 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:22.149550    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/10b542a2-e2cd-48f8-bb46-9fe3b0711e83-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mkbc5\" (UID: \"10b542a2-e2cd-48f8-bb46-9fe3b0711e83\") " pod="openshift-insights/insights-runtime-extractor-mkbc5"
Sep 16 00:47:22.149740 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:22.149631    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/10b542a2-e2cd-48f8-bb46-9fe3b0711e83-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mkbc5\" (UID: \"10b542a2-e2cd-48f8-bb46-9fe3b0711e83\") " pod="openshift-insights/insights-runtime-extractor-mkbc5"
Sep 16 00:47:22.149740 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:47:22.149693    2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 00:47:22.149740 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:47:22.149707    2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 00:47:22.149740 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:47:22.149741    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/10b542a2-e2cd-48f8-bb46-9fe3b0711e83-kube-rbac-proxy-cm podName:10b542a2-e2cd-48f8-bb46-9fe3b0711e83 nodeName:}" failed. No retries permitted until 2026-09-16 00:47:23.149728589 +0000 UTC m=+257.931230159 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/10b542a2-e2cd-48f8-bb46-9fe3b0711e83-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mkbc5" (UID: "10b542a2-e2cd-48f8-bb46-9fe3b0711e83") : configmap "kube-rbac-proxy" not found
Sep 16 00:47:22.149876 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:47:22.149774    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10b542a2-e2cd-48f8-bb46-9fe3b0711e83-insights-runtime-extractor-tls podName:10b542a2-e2cd-48f8-bb46-9fe3b0711e83 nodeName:}" failed. No retries permitted until 2026-09-16 00:47:23.149759616 +0000 UTC m=+257.931261186 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/10b542a2-e2cd-48f8-bb46-9fe3b0711e83-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mkbc5" (UID: "10b542a2-e2cd-48f8-bb46-9fe3b0711e83") : secret "insights-runtime-extractor-tls" not found
Sep 16 00:47:23.157315 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:23.157266    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/10b542a2-e2cd-48f8-bb46-9fe3b0711e83-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mkbc5\" (UID: \"10b542a2-e2cd-48f8-bb46-9fe3b0711e83\") " pod="openshift-insights/insights-runtime-extractor-mkbc5"
Sep 16 00:47:23.157315 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:23.157321    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/10b542a2-e2cd-48f8-bb46-9fe3b0711e83-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mkbc5\" (UID: \"10b542a2-e2cd-48f8-bb46-9fe3b0711e83\") " pod="openshift-insights/insights-runtime-extractor-mkbc5"
Sep 16 00:47:23.157882 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:47:23.157386    2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 00:47:23.157882 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:47:23.157447    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/10b542a2-e2cd-48f8-bb46-9fe3b0711e83-kube-rbac-proxy-cm podName:10b542a2-e2cd-48f8-bb46-9fe3b0711e83 nodeName:}" failed. No retries permitted until 2026-09-16 00:47:25.157430971 +0000 UTC m=+259.938932545 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/10b542a2-e2cd-48f8-bb46-9fe3b0711e83-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mkbc5" (UID: "10b542a2-e2cd-48f8-bb46-9fe3b0711e83") : configmap "kube-rbac-proxy" not found
Sep 16 00:47:23.157882 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:47:23.157492    2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 00:47:23.157882 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:47:23.157561    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10b542a2-e2cd-48f8-bb46-9fe3b0711e83-insights-runtime-extractor-tls podName:10b542a2-e2cd-48f8-bb46-9fe3b0711e83 nodeName:}" failed. No retries permitted until 2026-09-16 00:47:25.157544535 +0000 UTC m=+259.939046112 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/10b542a2-e2cd-48f8-bb46-9fe3b0711e83-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mkbc5" (UID: "10b542a2-e2cd-48f8-bb46-9fe3b0711e83") : secret "insights-runtime-extractor-tls" not found
Sep 16 00:47:24.141550 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:24.141521    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-q29p2_21802a49-7d39-41db-a85c-0b6c56472a56/dns-node-resolver/0.log"
Sep 16 00:47:25.171088 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:25.171052    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/10b542a2-e2cd-48f8-bb46-9fe3b0711e83-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mkbc5\" (UID: \"10b542a2-e2cd-48f8-bb46-9fe3b0711e83\") " pod="openshift-insights/insights-runtime-extractor-mkbc5"
Sep 16 00:47:25.171088 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:25.171092    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/10b542a2-e2cd-48f8-bb46-9fe3b0711e83-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mkbc5\" (UID: \"10b542a2-e2cd-48f8-bb46-9fe3b0711e83\") " pod="openshift-insights/insights-runtime-extractor-mkbc5"
Sep 16 00:47:25.171535 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:47:25.171167    2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 00:47:25.171535 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:47:25.171181    2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 00:47:25.171535 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:47:25.171226    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/10b542a2-e2cd-48f8-bb46-9fe3b0711e83-kube-rbac-proxy-cm podName:10b542a2-e2cd-48f8-bb46-9fe3b0711e83 nodeName:}" failed. No retries permitted until 2026-09-16 00:47:29.171211254 +0000 UTC m=+263.952712823 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/10b542a2-e2cd-48f8-bb46-9fe3b0711e83-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mkbc5" (UID: "10b542a2-e2cd-48f8-bb46-9fe3b0711e83") : configmap "kube-rbac-proxy" not found
Sep 16 00:47:25.171535 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:47:25.171241    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10b542a2-e2cd-48f8-bb46-9fe3b0711e83-insights-runtime-extractor-tls podName:10b542a2-e2cd-48f8-bb46-9fe3b0711e83 nodeName:}" failed. No retries permitted until 2026-09-16 00:47:29.171235572 +0000 UTC m=+263.952737142 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/10b542a2-e2cd-48f8-bb46-9fe3b0711e83-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mkbc5" (UID: "10b542a2-e2cd-48f8-bb46-9fe3b0711e83") : secret "insights-runtime-extractor-tls" not found
Sep 16 00:47:25.341206 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:25.341183    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-jvxd7_07a10807-5530-48f1-9aeb-8becb1380db5/node-ca/0.log"
Sep 16 00:47:26.141977 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:26.141946    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jt2l9_aa0a0489-cc60-43ea-89dc-cb3d97276b9d/serve-healthcheck-canary/0.log"
Sep 16 00:47:29.198422 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:29.198385    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/10b542a2-e2cd-48f8-bb46-9fe3b0711e83-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mkbc5\" (UID: \"10b542a2-e2cd-48f8-bb46-9fe3b0711e83\") " pod="openshift-insights/insights-runtime-extractor-mkbc5"
Sep 16 00:47:29.198867 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:29.198427    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/10b542a2-e2cd-48f8-bb46-9fe3b0711e83-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mkbc5\" (UID: \"10b542a2-e2cd-48f8-bb46-9fe3b0711e83\") " pod="openshift-insights/insights-runtime-extractor-mkbc5"
Sep 16 00:47:29.198867 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:47:29.198513    2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 00:47:29.198867 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:47:29.198586    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/10b542a2-e2cd-48f8-bb46-9fe3b0711e83-kube-rbac-proxy-cm podName:10b542a2-e2cd-48f8-bb46-9fe3b0711e83 nodeName:}" failed. No retries permitted until 2026-09-16 00:47:37.198567569 +0000 UTC m=+271.980069140 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/10b542a2-e2cd-48f8-bb46-9fe3b0711e83-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mkbc5" (UID: "10b542a2-e2cd-48f8-bb46-9fe3b0711e83") : configmap "kube-rbac-proxy" not found
Sep 16 00:47:29.198867 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:47:29.198611    2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 00:47:29.198867 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:47:29.198654    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10b542a2-e2cd-48f8-bb46-9fe3b0711e83-insights-runtime-extractor-tls podName:10b542a2-e2cd-48f8-bb46-9fe3b0711e83 nodeName:}" failed. No retries permitted until 2026-09-16 00:47:37.198643152 +0000 UTC m=+271.980144737 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/10b542a2-e2cd-48f8-bb46-9fe3b0711e83-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mkbc5" (UID: "10b542a2-e2cd-48f8-bb46-9fe3b0711e83") : secret "insights-runtime-extractor-tls" not found
Sep 16 00:47:37.250526 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:37.250492    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/10b542a2-e2cd-48f8-bb46-9fe3b0711e83-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mkbc5\" (UID: \"10b542a2-e2cd-48f8-bb46-9fe3b0711e83\") " pod="openshift-insights/insights-runtime-extractor-mkbc5"
Sep 16 00:47:37.250526 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:37.250528    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/10b542a2-e2cd-48f8-bb46-9fe3b0711e83-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mkbc5\" (UID: \"10b542a2-e2cd-48f8-bb46-9fe3b0711e83\") " pod="openshift-insights/insights-runtime-extractor-mkbc5"
Sep 16 00:47:37.250998 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:37.250976    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/10b542a2-e2cd-48f8-bb46-9fe3b0711e83-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mkbc5\" (UID: \"10b542a2-e2cd-48f8-bb46-9fe3b0711e83\") " pod="openshift-insights/insights-runtime-extractor-mkbc5"
Sep 16 00:47:37.252766 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:37.252749    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/10b542a2-e2cd-48f8-bb46-9fe3b0711e83-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mkbc5\" (UID: \"10b542a2-e2cd-48f8-bb46-9fe3b0711e83\") " pod="openshift-insights/insights-runtime-extractor-mkbc5"
Sep 16 00:47:37.296764 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:37.296730    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mkbc5"
Sep 16 00:47:37.561560 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:37.561486    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mkbc5"]
Sep 16 00:47:37.564682 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:47:37.564656    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10b542a2_e2cd_48f8_bb46_9fe3b0711e83.slice/crio-0519fe7eed89c5e82de5538d29c833474225e31195c784b5906f986536edcea2 WatchSource:0}: Error finding container 0519fe7eed89c5e82de5538d29c833474225e31195c784b5906f986536edcea2: Status 404 returned error can't find the container with id 0519fe7eed89c5e82de5538d29c833474225e31195c784b5906f986536edcea2
Sep 16 00:47:37.746252 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:37.746222    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mkbc5" event={"ID":"10b542a2-e2cd-48f8-bb46-9fe3b0711e83","Type":"ContainerStarted","Data":"ac01515adfe49d4065e56085d3ae5d513c86051dcbde4f1996f8558889cbfc32"}
Sep 16 00:47:37.746392 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:37.746274    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mkbc5" event={"ID":"10b542a2-e2cd-48f8-bb46-9fe3b0711e83","Type":"ContainerStarted","Data":"0519fe7eed89c5e82de5538d29c833474225e31195c784b5906f986536edcea2"}
Sep 16 00:47:38.751185 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:38.751145    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mkbc5" event={"ID":"10b542a2-e2cd-48f8-bb46-9fe3b0711e83","Type":"ContainerStarted","Data":"271d4fb7e9b84a8c45ed071ac804879b4faff273a4c6e51011c06fed10c7dfc7"}
Sep 16 00:47:39.756677 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:39.756645    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mkbc5" event={"ID":"10b542a2-e2cd-48f8-bb46-9fe3b0711e83","Type":"ContainerStarted","Data":"1ebac9938e565c2bcc2e15a91fcc2e122cd88dc7c31efff06e009c3830070bef"}
Sep 16 00:47:39.772335 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:39.772295    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-mkbc5" podStartSLOduration=16.786617877 podStartE2EDuration="18.772280502s" totalImagesPullingTime="1.985662625s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 00:47:21 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 00:47:39.771237706 +0000 UTC m=+274.552739298" watchObservedRunningTime="2026-09-16 00:47:39.772280502 +0000 UTC m=+274.553782091"
Sep 16 00:47:47.400276 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:47:47.400237    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-rsqrc" podUID="d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80"
Sep 16 00:47:47.779731 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:47.779648    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rsqrc"
Sep 16 00:47:49.201222 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:49.201190    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-xvms9"]
Sep 16 00:47:49.204336 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:49.204318    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-xvms9"
Sep 16 00:47:49.206664 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:49.206635    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 16 00:47:49.206768 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:49.206638    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-ldbkq\""
Sep 16 00:47:49.206768 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:49.206685    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 16 00:47:49.207360 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:49.207344    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 16 00:47:49.208302 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:49.207757    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 16 00:47:49.208302 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:49.207991    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 16 00:47:49.208302 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:49.208063    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 16 00:47:49.230646 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:49.230623    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dfce583b-e647-4ca3-bbec-34ce267f5b56-metrics-client-ca\") pod \"node-exporter-xvms9\" (UID: \"dfce583b-e647-4ca3-bbec-34ce267f5b56\") " pod="openshift-monitoring/node-exporter-xvms9"
Sep 16 00:47:49.230744 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:49.230649    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dfce583b-e647-4ca3-bbec-34ce267f5b56-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xvms9\" (UID: \"dfce583b-e647-4ca3-bbec-34ce267f5b56\") " pod="openshift-monitoring/node-exporter-xvms9"
Sep 16 00:47:49.230744 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:49.230667    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/dfce583b-e647-4ca3-bbec-34ce267f5b56-node-exporter-wtmp\") pod \"node-exporter-xvms9\" (UID: \"dfce583b-e647-4ca3-bbec-34ce267f5b56\") " pod="openshift-monitoring/node-exporter-xvms9"
Sep 16 00:47:49.230744 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:49.230684    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/dfce583b-e647-4ca3-bbec-34ce267f5b56-node-exporter-accelerators-collector-config\") pod \"node-exporter-xvms9\" (UID: \"dfce583b-e647-4ca3-bbec-34ce267f5b56\") " pod="openshift-monitoring/node-exporter-xvms9"
Sep 16 00:47:49.230744 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:49.230728    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/dfce583b-e647-4ca3-bbec-34ce267f5b56-node-exporter-tls\") pod \"node-exporter-xvms9\" (UID: \"dfce583b-e647-4ca3-bbec-34ce267f5b56\") " pod="openshift-monitoring/node-exporter-xvms9"
Sep 16 00:47:49.230744 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:49.230742    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgc64\" (UniqueName: \"kubernetes.io/projected/dfce583b-e647-4ca3-bbec-34ce267f5b56-kube-api-access-zgc64\") pod \"node-exporter-xvms9\" (UID: \"dfce583b-e647-4ca3-bbec-34ce267f5b56\") " pod="openshift-monitoring/node-exporter-xvms9"
Sep 16 00:47:49.230968 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:49.230768    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/dfce583b-e647-4ca3-bbec-34ce267f5b56-root\") pod \"node-exporter-xvms9\" (UID: \"dfce583b-e647-4ca3-bbec-34ce267f5b56\") " pod="openshift-monitoring/node-exporter-xvms9"
Sep 16 00:47:49.230968 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:49.230832    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/dfce583b-e647-4ca3-bbec-34ce267f5b56-node-exporter-textfile\") pod \"node-exporter-xvms9\" (UID: \"dfce583b-e647-4ca3-bbec-34ce267f5b56\") " pod="openshift-monitoring/node-exporter-xvms9"
Sep 16 00:47:49.230968 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:49.230933    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dfce583b-e647-4ca3-bbec-34ce267f5b56-sys\") pod \"node-exporter-xvms9\" (UID: \"dfce583b-e647-4ca3-bbec-34ce267f5b56\") " pod="openshift-monitoring/node-exporter-xvms9"
Sep 16 00:47:49.331411 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:49.331384    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/dfce583b-e647-4ca3-bbec-34ce267f5b56-root\") pod \"node-exporter-xvms9\" (UID: \"dfce583b-e647-4ca3-bbec-34ce267f5b56\") " pod="openshift-monitoring/node-exporter-xvms9"
Sep 16 00:47:49.331539 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:49.331420    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/dfce583b-e647-4ca3-bbec-34ce267f5b56-node-exporter-textfile\") pod \"node-exporter-xvms9\" (UID: \"dfce583b-e647-4ca3-bbec-34ce267f5b56\") " pod="openshift-monitoring/node-exporter-xvms9"
Sep 16 00:47:49.331539 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:49.331502    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/dfce583b-e647-4ca3-bbec-34ce267f5b56-root\") pod \"node-exporter-xvms9\" (UID: \"dfce583b-e647-4ca3-bbec-34ce267f5b56\") " pod="openshift-monitoring/node-exporter-xvms9"
Sep 16 00:47:49.331664 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:49.331539    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dfce583b-e647-4ca3-bbec-34ce267f5b56-sys\") pod \"node-exporter-xvms9\" (UID: \"dfce583b-e647-4ca3-bbec-34ce267f5b56\") " pod="openshift-monitoring/node-exporter-xvms9"
Sep 16 00:47:49.331664 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:49.331574    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dfce583b-e647-4ca3-bbec-34ce267f5b56-metrics-client-ca\") pod \"node-exporter-xvms9\" (UID: \"dfce583b-e647-4ca3-bbec-34ce267f5b56\") " pod="openshift-monitoring/node-exporter-xvms9"
Sep 16 00:47:49.331664 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:49.331591    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dfce583b-e647-4ca3-bbec-34ce267f5b56-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xvms9\" (UID: \"dfce583b-e647-4ca3-bbec-34ce267f5b56\") " pod="openshift-monitoring/node-exporter-xvms9"
Sep 16 00:47:49.331664 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:49.331610    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/dfce583b-e647-4ca3-bbec-34ce267f5b56-node-exporter-wtmp\") pod \"node-exporter-xvms9\" (UID: \"dfce583b-e647-4ca3-bbec-34ce267f5b56\") " pod="openshift-monitoring/node-exporter-xvms9"
Sep 16 00:47:49.331664 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:49.331626    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/dfce583b-e647-4ca3-bbec-34ce267f5b56-node-exporter-accelerators-collector-config\") pod \"node-exporter-xvms9\" (UID: \"dfce583b-e647-4ca3-bbec-34ce267f5b56\") " pod="openshift-monitoring/node-exporter-xvms9"
Sep 16 00:47:49.331664 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:49.331635    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dfce583b-e647-4ca3-bbec-34ce267f5b56-sys\") pod \"node-exporter-xvms9\" (UID: \"dfce583b-e647-4ca3-bbec-34ce267f5b56\") " pod="openshift-monitoring/node-exporter-xvms9"
Sep 16 00:47:49.331664 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:49.331667    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/dfce583b-e647-4ca3-bbec-34ce267f5b56-node-exporter-tls\") pod \"node-exporter-xvms9\" (UID: \"dfce583b-e647-4ca3-bbec-34ce267f5b56\") " pod="openshift-monitoring/node-exporter-xvms9"
Sep 16 00:47:49.331946 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:49.331682    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgc64\" (UniqueName: \"kubernetes.io/projected/dfce583b-e647-4ca3-bbec-34ce267f5b56-kube-api-access-zgc64\") pod \"node-exporter-xvms9\" (UID: \"dfce583b-e647-4ca3-bbec-34ce267f5b56\") " pod="openshift-monitoring/node-exporter-xvms9"
Sep 16 00:47:49.331946 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:49.331758    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/dfce583b-e647-4ca3-bbec-34ce267f5b56-node-exporter-wtmp\") pod \"node-exporter-xvms9\" (UID: \"dfce583b-e647-4ca3-bbec-34ce267f5b56\") " pod="openshift-monitoring/node-exporter-xvms9"
Sep 16 00:47:49.331946 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:49.331766    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/dfce583b-e647-4ca3-bbec-34ce267f5b56-node-exporter-textfile\") pod \"node-exporter-xvms9\" (UID: \"dfce583b-e647-4ca3-bbec-34ce267f5b56\") " pod="openshift-monitoring/node-exporter-xvms9"
Sep 16 00:47:49.331946 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:47:49.331800    2556 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 16 00:47:49.331946 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:47:49.331874    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dfce583b-e647-4ca3-bbec-34ce267f5b56-node-exporter-tls podName:dfce583b-e647-4ca3-bbec-34ce267f5b56 nodeName:}" failed. No retries permitted until 2026-09-16 00:47:49.831854231 +0000 UTC m=+284.613355817 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/dfce583b-e647-4ca3-bbec-34ce267f5b56-node-exporter-tls") pod "node-exporter-xvms9" (UID: "dfce583b-e647-4ca3-bbec-34ce267f5b56") : secret "node-exporter-tls" not found
Sep 16 00:47:49.332138 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:49.332059    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dfce583b-e647-4ca3-bbec-34ce267f5b56-metrics-client-ca\") pod \"node-exporter-xvms9\" (UID: \"dfce583b-e647-4ca3-bbec-34ce267f5b56\") " pod="openshift-monitoring/node-exporter-xvms9"
Sep 16 00:47:49.332138 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:49.332117    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/dfce583b-e647-4ca3-bbec-34ce267f5b56-node-exporter-accelerators-collector-config\") pod \"node-exporter-xvms9\" (UID: \"dfce583b-e647-4ca3-bbec-34ce267f5b56\") " pod="openshift-monitoring/node-exporter-xvms9"
Sep 16 00:47:49.333790 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:49.333771    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dfce583b-e647-4ca3-bbec-34ce267f5b56-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xvms9\" (UID: \"dfce583b-e647-4ca3-bbec-34ce267f5b56\") " pod="openshift-monitoring/node-exporter-xvms9"
Sep 16 00:47:49.340385 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:49.340363    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgc64\" (UniqueName: \"kubernetes.io/projected/dfce583b-e647-4ca3-bbec-34ce267f5b56-kube-api-access-zgc64\") pod \"node-exporter-xvms9\" (UID: \"dfce583b-e647-4ca3-bbec-34ce267f5b56\") " pod="openshift-monitoring/node-exporter-xvms9"
Sep 16 00:47:49.834665 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:49.834632    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/dfce583b-e647-4ca3-bbec-34ce267f5b56-node-exporter-tls\") pod \"node-exporter-xvms9\" (UID: \"dfce583b-e647-4ca3-bbec-34ce267f5b56\") " pod="openshift-monitoring/node-exporter-xvms9"
Sep 16 00:47:49.836713 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:49.836694    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/dfce583b-e647-4ca3-bbec-34ce267f5b56-node-exporter-tls\") pod \"node-exporter-xvms9\" (UID: \"dfce583b-e647-4ca3-bbec-34ce267f5b56\") " pod="openshift-monitoring/node-exporter-xvms9"
Sep 16 00:47:50.114518 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:50.114426    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-xvms9"
Sep 16 00:47:50.124230 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:47:50.124204    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfce583b_e647_4ca3_bbec_34ce267f5b56.slice/crio-3f371aef856491c96562c3eec2c08b550483213362b45a090315697d0810737c WatchSource:0}: Error finding container 3f371aef856491c96562c3eec2c08b550483213362b45a090315697d0810737c: Status 404 returned error can't find the container with id 3f371aef856491c96562c3eec2c08b550483213362b45a090315697d0810737c
Sep 16 00:47:50.789026 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:50.788994    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xvms9" event={"ID":"dfce583b-e647-4ca3-bbec-34ce267f5b56","Type":"ContainerStarted","Data":"3f371aef856491c96562c3eec2c08b550483213362b45a090315697d0810737c"}
Sep 16 00:47:50.943999 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:50.943959    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80-metrics-tls\") pod \"dns-default-rsqrc\" (UID: \"d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80\") " pod="openshift-dns/dns-default-rsqrc"
Sep 16 00:47:50.946230 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:50.946206    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80-metrics-tls\") pod \"dns-default-rsqrc\" (UID: \"d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80\") " pod="openshift-dns/dns-default-rsqrc"
Sep 16 00:47:51.082865 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:51.082834    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-n5l29\""
Sep 16 00:47:51.091545 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:51.091523    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rsqrc"
Sep 16 00:47:51.363840 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:51.363788    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-rsqrc"]
Sep 16 00:47:51.366628 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:47:51.366604    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2f6824d_c5ca_4cde_9c75_c7e4d4a63f80.slice/crio-4ff2aad605c6865d891bd733e5ee4ba0935a8914a38eb81e046e8ccdf8d4c8f8 WatchSource:0}: Error finding container 4ff2aad605c6865d891bd733e5ee4ba0935a8914a38eb81e046e8ccdf8d4c8f8: Status 404 returned error can't find the container with id 4ff2aad605c6865d891bd733e5ee4ba0935a8914a38eb81e046e8ccdf8d4c8f8
Sep 16 00:47:51.793561 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:51.793457    2556 generic.go:356] "Generic (PLEG): container finished" podID="dfce583b-e647-4ca3-bbec-34ce267f5b56" containerID="f35208c609d7ba009d8282e979dfb9123365cd8a5ae737dd4d72ac0d39444bb9" exitCode=0
Sep 16 00:47:51.793561 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:51.793494    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xvms9" event={"ID":"dfce583b-e647-4ca3-bbec-34ce267f5b56","Type":"ContainerDied","Data":"f35208c609d7ba009d8282e979dfb9123365cd8a5ae737dd4d72ac0d39444bb9"}
Sep 16 00:47:51.794755 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:51.794729    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rsqrc" event={"ID":"d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80","Type":"ContainerStarted","Data":"4ff2aad605c6865d891bd733e5ee4ba0935a8914a38eb81e046e8ccdf8d4c8f8"}
Sep 16 00:47:52.801286 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:52.801253    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xvms9" event={"ID":"dfce583b-e647-4ca3-bbec-34ce267f5b56","Type":"ContainerStarted","Data":"69cf3a14317aac6e8d84c1b271e5a326f00de14f5d37bcc0b8de0a9d51dd9fdc"}
Sep 16 00:47:52.801760 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:52.801312    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xvms9" event={"ID":"dfce583b-e647-4ca3-bbec-34ce267f5b56","Type":"ContainerStarted","Data":"473e22a9f0bc14b986a3f33c036bc95c75e05f4e853e23360b76d1bfb1b1f17a"}
Sep 16 00:47:52.819885 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:52.819841    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-xvms9" podStartSLOduration=2.7367941140000003 podStartE2EDuration="3.819824762s" totalImagesPullingTime="1.024819108s" totalInitContainerRuntime="58.21154ms" isStatefulPod=true podCreationTimestamp="2026-09-16 00:47:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 00:47:52.818800207 +0000 UTC m=+287.600301803" watchObservedRunningTime="2026-09-16 00:47:52.819824762 +0000 UTC m=+287.601326386"
Sep 16 00:47:53.806146 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:53.806108    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rsqrc" event={"ID":"d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80","Type":"ContainerStarted","Data":"34753044d572ed30764d187447c2a334b01fd429ad3436a067e67e12e629f94a"}
Sep 16 00:47:53.806554 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:53.806164    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rsqrc" event={"ID":"d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80","Type":"ContainerStarted","Data":"54d7e2087df03bee4fad0bfd26aa5f8d52c9820597653c5eb420b2437ce732bf"}
Sep 16 00:47:53.824276 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:53.824230    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-rsqrc" podStartSLOduration=251.006519108 podStartE2EDuration="4m12.824217228s" totalImagesPullingTime="1.81769812s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 00:43:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 00:47:53.82306881 +0000 UTC m=+288.604570403" watchObservedRunningTime="2026-09-16 00:47:53.824217228 +0000 UTC m=+288.605718820"
Sep 16 00:47:54.809947 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:47:54.809920    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-rsqrc"
Sep 16 00:48:00.730693 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:48:00.730654    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-hnw6l" podUID="f2569a02-edf9-4b33-a7df-a8d0fdce0580"
Sep 16 00:48:04.814525 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:04.814491    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-rsqrc"
Sep 16 00:48:05.614944 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:05.614918    2556 kubelet.go:1648] "Image garbage collection succeeded"
Sep 16 00:48:06.848079 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:06.848045    2556 generic.go:356] "Generic (PLEG): container finished" podID="a59cb3a3-3ac9-45c0-983b-de7cd8810a74" containerID="3435f44427e63cd6115688472fc04fb6e37028995762e2d45b10b2b112c8fbe3" exitCode=255
Sep 16 00:48:06.848587 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:06.848116    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-765c74974-b6ckj" event={"ID":"a59cb3a3-3ac9-45c0-983b-de7cd8810a74","Type":"ContainerDied","Data":"3435f44427e63cd6115688472fc04fb6e37028995762e2d45b10b2b112c8fbe3"}
Sep 16 00:48:06.848587 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:06.848189    2556 scope.go:117] "RemoveContainer" containerID="f37690cac5088187bd5157c43b4ea0f310dd37d6736352ca15bc8dd5b3c9e614"
Sep 16 00:48:06.848587 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:06.848519    2556 scope.go:117] "RemoveContainer" containerID="3435f44427e63cd6115688472fc04fb6e37028995762e2d45b10b2b112c8fbe3"
Sep 16 00:48:06.848730 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:48:06.848714    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"addon-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=addon-agent pod=managed-serviceaccount-addon-agent-765c74974-b6ckj_open-cluster-management-agent-addon(a59cb3a3-3ac9-45c0-983b-de7cd8810a74)\"" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-765c74974-b6ckj" podUID="a59cb3a3-3ac9-45c0-983b-de7cd8810a74"
Sep 16 00:48:06.850329 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:06.850305    2556 generic.go:356] "Generic (PLEG): container finished" podID="a81e6da0-2824-495c-a0d4-6c65b9177170" containerID="c6923ee3489a1bce87b2334603ced6cd89250a027bdfe7ca4a74f8d5753a6244" exitCode=1
Sep 16 00:48:06.850429 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:06.850368    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d57768899-dfrv6" event={"ID":"a81e6da0-2824-495c-a0d4-6c65b9177170","Type":"ContainerDied","Data":"c6923ee3489a1bce87b2334603ced6cd89250a027bdfe7ca4a74f8d5753a6244"}
Sep 16 00:48:06.850677 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:06.850662    2556 scope.go:117] "RemoveContainer" containerID="c6923ee3489a1bce87b2334603ced6cd89250a027bdfe7ca4a74f8d5753a6244"
Sep 16 00:48:06.850831 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:48:06.850814    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"acm-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=acm-agent pod=klusterlet-addon-workmgr-5d57768899-dfrv6_open-cluster-management-agent-addon(a81e6da0-2824-495c-a0d4-6c65b9177170)\"" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d57768899-dfrv6" podUID="a81e6da0-2824-495c-a0d4-6c65b9177170"
Sep 16 00:48:06.858347 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:06.858329    2556 scope.go:117] "RemoveContainer" containerID="366896c0487c4bda2e113e74f23ee6879ddb0e1352e31e555996e9027e8c5214"
Sep 16 00:48:07.476570 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:07.476538    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d57768899-dfrv6"
Sep 16 00:48:07.855883 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:07.855861    2556 scope.go:117] "RemoveContainer" containerID="c6923ee3489a1bce87b2334603ced6cd89250a027bdfe7ca4a74f8d5753a6244"
Sep 16 00:48:07.856224 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:48:07.856007    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"acm-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=acm-agent pod=klusterlet-addon-workmgr-5d57768899-dfrv6_open-cluster-management-agent-addon(a81e6da0-2824-495c-a0d4-6c65b9177170)\"" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d57768899-dfrv6" podUID="a81e6da0-2824-495c-a0d4-6c65b9177170"
Sep 16 00:48:08.431281 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:08.431251    2556 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-765c74974-b6ckj"
Sep 16 00:48:08.431599 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:08.431585    2556 scope.go:117] "RemoveContainer" containerID="3435f44427e63cd6115688472fc04fb6e37028995762e2d45b10b2b112c8fbe3"
Sep 16 00:48:08.431742 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:48:08.431724    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"addon-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=addon-agent pod=managed-serviceaccount-addon-agent-765c74974-b6ckj_open-cluster-management-agent-addon(a59cb3a3-3ac9-45c0-983b-de7cd8810a74)\"" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-765c74974-b6ckj" podUID="a59cb3a3-3ac9-45c0-983b-de7cd8810a74"
Sep 16 00:48:08.437424 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:08.437405    2556 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d57768899-dfrv6"
Sep 16 00:48:08.859621 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:08.859595    2556 scope.go:117] "RemoveContainer" containerID="c6923ee3489a1bce87b2334603ced6cd89250a027bdfe7ca4a74f8d5753a6244"
Sep 16 00:48:08.859976 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:48:08.859752    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"acm-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=acm-agent pod=klusterlet-addon-workmgr-5d57768899-dfrv6_open-cluster-management-agent-addon(a81e6da0-2824-495c-a0d4-6c65b9177170)\"" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d57768899-dfrv6" podUID="a81e6da0-2824-495c-a0d4-6c65b9177170"
Sep 16 00:48:14.730365 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:14.730329    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hnw6l"
Sep 16 00:48:22.730344 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:22.730317    2556 scope.go:117] "RemoveContainer" containerID="c6923ee3489a1bce87b2334603ced6cd89250a027bdfe7ca4a74f8d5753a6244"
Sep 16 00:48:22.903113 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:22.903073    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d57768899-dfrv6" event={"ID":"a81e6da0-2824-495c-a0d4-6c65b9177170","Type":"ContainerStarted","Data":"beb4c707a3421a47d2ccf3115bfe8713d6cd0393215e055097c012717e0fca7d"}
Sep 16 00:48:22.903354 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:22.903334    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d57768899-dfrv6"
Sep 16 00:48:22.904571 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:22.904549    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d57768899-dfrv6"
Sep 16 00:48:23.013202 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:23.013137    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5dffdc758d-bd9qh"]
Sep 16 00:48:23.016122 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:23.016089    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5dffdc758d-bd9qh"
Sep 16 00:48:23.020370 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:23.020332    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 16 00:48:23.020519 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:23.020373    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 16 00:48:23.020519 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:23.020340    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 16 00:48:23.020672 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:23.020524    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 16 00:48:23.022741 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:23.022719    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-m679f\""
Sep 16 00:48:23.023013 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:23.022992    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 16 00:48:23.023123 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:23.023052    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 16 00:48:23.023317 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:23.023296    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 16 00:48:23.027511 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:23.027491    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5dffdc758d-bd9qh"]
Sep 16 00:48:23.032836 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:23.032818    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 16 00:48:23.069686 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:23.069660    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d7a7db8c-34ff-40b4-9e90-a8887addbb03-oauth-serving-cert\") pod \"console-5dffdc758d-bd9qh\" (UID: \"d7a7db8c-34ff-40b4-9e90-a8887addbb03\") " pod="openshift-console/console-5dffdc758d-bd9qh"
Sep 16 00:48:23.069789 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:23.069703    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7a7db8c-34ff-40b4-9e90-a8887addbb03-trusted-ca-bundle\") pod \"console-5dffdc758d-bd9qh\" (UID: \"d7a7db8c-34ff-40b4-9e90-a8887addbb03\") " pod="openshift-console/console-5dffdc758d-bd9qh"
Sep 16 00:48:23.069789 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:23.069734    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7pzk\" (UniqueName: \"kubernetes.io/projected/d7a7db8c-34ff-40b4-9e90-a8887addbb03-kube-api-access-b7pzk\") pod \"console-5dffdc758d-bd9qh\" (UID: \"d7a7db8c-34ff-40b4-9e90-a8887addbb03\") " pod="openshift-console/console-5dffdc758d-bd9qh"
Sep 16 00:48:23.069906 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:23.069790    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d7a7db8c-34ff-40b4-9e90-a8887addbb03-console-config\") pod \"console-5dffdc758d-bd9qh\" (UID: \"d7a7db8c-34ff-40b4-9e90-a8887addbb03\") " pod="openshift-console/console-5dffdc758d-bd9qh"
Sep 16 00:48:23.069906 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:23.069849    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d7a7db8c-34ff-40b4-9e90-a8887addbb03-console-oauth-config\") pod \"console-5dffdc758d-bd9qh\" (UID: \"d7a7db8c-34ff-40b4-9e90-a8887addbb03\") " pod="openshift-console/console-5dffdc758d-bd9qh"
Sep 16 00:48:23.069994 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:23.069962    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d7a7db8c-34ff-40b4-9e90-a8887addbb03-console-serving-cert\") pod \"console-5dffdc758d-bd9qh\" (UID: \"d7a7db8c-34ff-40b4-9e90-a8887addbb03\") " pod="openshift-console/console-5dffdc758d-bd9qh"
Sep 16 00:48:23.070046 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:23.070008    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d7a7db8c-34ff-40b4-9e90-a8887addbb03-service-ca\") pod \"console-5dffdc758d-bd9qh\" (UID: \"d7a7db8c-34ff-40b4-9e90-a8887addbb03\") " pod="openshift-console/console-5dffdc758d-bd9qh"
Sep 16 00:48:23.171242 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:23.171216    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d7a7db8c-34ff-40b4-9e90-a8887addbb03-service-ca\") pod \"console-5dffdc758d-bd9qh\" (UID: \"d7a7db8c-34ff-40b4-9e90-a8887addbb03\") " pod="openshift-console/console-5dffdc758d-bd9qh"
Sep 16 00:48:23.171333 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:23.171251    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d7a7db8c-34ff-40b4-9e90-a8887addbb03-oauth-serving-cert\") pod \"console-5dffdc758d-bd9qh\" (UID: \"d7a7db8c-34ff-40b4-9e90-a8887addbb03\") " pod="openshift-console/console-5dffdc758d-bd9qh"
Sep 16 00:48:23.171333 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:23.171273    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7a7db8c-34ff-40b4-9e90-a8887addbb03-trusted-ca-bundle\") pod \"console-5dffdc758d-bd9qh\" (UID: \"d7a7db8c-34ff-40b4-9e90-a8887addbb03\") " pod="openshift-console/console-5dffdc758d-bd9qh"
Sep 16 00:48:23.171333 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:23.171295    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7pzk\" (UniqueName: \"kubernetes.io/projected/d7a7db8c-34ff-40b4-9e90-a8887addbb03-kube-api-access-b7pzk\") pod \"console-5dffdc758d-bd9qh\" (UID: \"d7a7db8c-34ff-40b4-9e90-a8887addbb03\") " pod="openshift-console/console-5dffdc758d-bd9qh"
Sep 16 00:48:23.171333 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:23.171322    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d7a7db8c-34ff-40b4-9e90-a8887addbb03-console-config\") pod \"console-5dffdc758d-bd9qh\" (UID: \"d7a7db8c-34ff-40b4-9e90-a8887addbb03\") " pod="openshift-console/console-5dffdc758d-bd9qh"
Sep 16 00:48:23.171544 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:23.171363    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d7a7db8c-34ff-40b4-9e90-a8887addbb03-console-oauth-config\") pod \"console-5dffdc758d-bd9qh\" (UID: \"d7a7db8c-34ff-40b4-9e90-a8887addbb03\") " pod="openshift-console/console-5dffdc758d-bd9qh"
Sep 16 00:48:23.171544 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:23.171406    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d7a7db8c-34ff-40b4-9e90-a8887addbb03-console-serving-cert\") pod \"console-5dffdc758d-bd9qh\" (UID: \"d7a7db8c-34ff-40b4-9e90-a8887addbb03\") " pod="openshift-console/console-5dffdc758d-bd9qh"
Sep 16 00:48:23.172074 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:23.172054    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d7a7db8c-34ff-40b4-9e90-a8887addbb03-service-ca\") pod \"console-5dffdc758d-bd9qh\" (UID: \"d7a7db8c-34ff-40b4-9e90-a8887addbb03\") " pod="openshift-console/console-5dffdc758d-bd9qh"
Sep 16 00:48:23.172198 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:23.172178    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d7a7db8c-34ff-40b4-9e90-a8887addbb03-oauth-serving-cert\") pod \"console-5dffdc758d-bd9qh\" (UID: \"d7a7db8c-34ff-40b4-9e90-a8887addbb03\") " pod="openshift-console/console-5dffdc758d-bd9qh"
Sep 16 00:48:23.172259 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:23.172239    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d7a7db8c-34ff-40b4-9e90-a8887addbb03-console-config\") pod \"console-5dffdc758d-bd9qh\" (UID: \"d7a7db8c-34ff-40b4-9e90-a8887addbb03\") " pod="openshift-console/console-5dffdc758d-bd9qh"
Sep 16 00:48:23.172259 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:23.172251    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7a7db8c-34ff-40b4-9e90-a8887addbb03-trusted-ca-bundle\") pod \"console-5dffdc758d-bd9qh\" (UID: \"d7a7db8c-34ff-40b4-9e90-a8887addbb03\") " pod="openshift-console/console-5dffdc758d-bd9qh"
Sep 16 00:48:23.173875 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:23.173851    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d7a7db8c-34ff-40b4-9e90-a8887addbb03-console-oauth-config\") pod \"console-5dffdc758d-bd9qh\" (UID: \"d7a7db8c-34ff-40b4-9e90-a8887addbb03\") " pod="openshift-console/console-5dffdc758d-bd9qh"
Sep 16 00:48:23.173972 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:23.173958    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d7a7db8c-34ff-40b4-9e90-a8887addbb03-console-serving-cert\") pod \"console-5dffdc758d-bd9qh\" (UID: \"d7a7db8c-34ff-40b4-9e90-a8887addbb03\") " pod="openshift-console/console-5dffdc758d-bd9qh"
Sep 16 00:48:23.178598 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:23.178578    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7pzk\" (UniqueName: \"kubernetes.io/projected/d7a7db8c-34ff-40b4-9e90-a8887addbb03-kube-api-access-b7pzk\") pod \"console-5dffdc758d-bd9qh\" (UID: \"d7a7db8c-34ff-40b4-9e90-a8887addbb03\") " pod="openshift-console/console-5dffdc758d-bd9qh"
Sep 16 00:48:23.325540 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:23.325515    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5dffdc758d-bd9qh"
Sep 16 00:48:23.603343 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:23.603271    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5dffdc758d-bd9qh"]
Sep 16 00:48:23.606226 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:48:23.606196    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7a7db8c_34ff_40b4_9e90_a8887addbb03.slice/crio-c4a18ed3a8cc32173221b8d8305668f1a36b9b4bc0a573651c9c0b53a5d47d19 WatchSource:0}: Error finding container c4a18ed3a8cc32173221b8d8305668f1a36b9b4bc0a573651c9c0b53a5d47d19: Status 404 returned error can't find the container with id c4a18ed3a8cc32173221b8d8305668f1a36b9b4bc0a573651c9c0b53a5d47d19
Sep 16 00:48:23.608188 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:23.608174    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 00:48:23.731009 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:23.730988    2556 scope.go:117] "RemoveContainer" containerID="3435f44427e63cd6115688472fc04fb6e37028995762e2d45b10b2b112c8fbe3"
Sep 16 00:48:23.907295 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:23.907221    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5dffdc758d-bd9qh" event={"ID":"d7a7db8c-34ff-40b4-9e90-a8887addbb03","Type":"ContainerStarted","Data":"c4a18ed3a8cc32173221b8d8305668f1a36b9b4bc0a573651c9c0b53a5d47d19"}
Sep 16 00:48:23.913101 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:23.913056    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-765c74974-b6ckj" event={"ID":"a59cb3a3-3ac9-45c0-983b-de7cd8810a74","Type":"ContainerStarted","Data":"f51fe8cbfe3005be8f42052cd08a70c330bf13467670ff0216e39eaa5c29ac52"}
Sep 16 00:48:26.925795 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:26.925758    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5dffdc758d-bd9qh" event={"ID":"d7a7db8c-34ff-40b4-9e90-a8887addbb03","Type":"ContainerStarted","Data":"5f668d2c50dcc70a132700bfcd3861893173c864ae200aa63a9d9303d634f1c8"}
Sep 16 00:48:26.941719 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:26.941659    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5dffdc758d-bd9qh" podStartSLOduration=2.125481372 podStartE2EDuration="4.941644559s" totalImagesPullingTime="2.816163187s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 00:48:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 00:48:26.941233064 +0000 UTC m=+321.722734656" watchObservedRunningTime="2026-09-16 00:48:26.941644559 +0000 UTC m=+321.723146152"
Sep 16 00:48:33.325704 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:33.325674    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5dffdc758d-bd9qh"
Sep 16 00:48:33.326150 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:33.325733    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5dffdc758d-bd9qh"
Sep 16 00:48:33.330434 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:33.330415    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5dffdc758d-bd9qh"
Sep 16 00:48:33.951115 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:33.951090    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5dffdc758d-bd9qh"
Sep 16 00:48:38.470810 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:38.470769    2556 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-574f8cdd64-nwwz4" podUID="7d7818d0-fcc3-4e11-9f5a-cbf7ab5592e5" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Sep 16 00:48:48.471266 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:48.471228    2556 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-574f8cdd64-nwwz4" podUID="7d7818d0-fcc3-4e11-9f5a-cbf7ab5592e5" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Sep 16 00:48:58.470772 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:58.470689    2556 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-574f8cdd64-nwwz4" podUID="7d7818d0-fcc3-4e11-9f5a-cbf7ab5592e5" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Sep 16 00:48:58.470772 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:58.470772    2556 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-574f8cdd64-nwwz4"
Sep 16 00:48:58.471254 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:58.471217    2556 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"a5ed3a7b456416abdd208a0976bb8f8553b7902fa54d5fa4e220fe3ea92c23de"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-574f8cdd64-nwwz4" containerMessage="Container service-proxy failed liveness probe, will be restarted"
Sep 16 00:48:58.471293 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:58.471272    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-574f8cdd64-nwwz4" podUID="7d7818d0-fcc3-4e11-9f5a-cbf7ab5592e5" containerName="service-proxy" containerID="cri-o://a5ed3a7b456416abdd208a0976bb8f8553b7902fa54d5fa4e220fe3ea92c23de" gracePeriod=30
Sep 16 00:48:59.029909 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:59.029877    2556 generic.go:356] "Generic (PLEG): container finished" podID="7d7818d0-fcc3-4e11-9f5a-cbf7ab5592e5" containerID="a5ed3a7b456416abdd208a0976bb8f8553b7902fa54d5fa4e220fe3ea92c23de" exitCode=2
Sep 16 00:48:59.030082 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:59.029926    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-574f8cdd64-nwwz4" event={"ID":"7d7818d0-fcc3-4e11-9f5a-cbf7ab5592e5","Type":"ContainerDied","Data":"a5ed3a7b456416abdd208a0976bb8f8553b7902fa54d5fa4e220fe3ea92c23de"}
Sep 16 00:48:59.030082 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:48:59.029964    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-574f8cdd64-nwwz4" event={"ID":"7d7818d0-fcc3-4e11-9f5a-cbf7ab5592e5","Type":"ContainerStarted","Data":"94a0e8028b1587834c2c35f17ab2c7d2fd026afd6c84a9e2e2ccefd9ca02f89d"}
Sep 16 00:49:19.555913 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:49:19.555870    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f2569a02-edf9-4b33-a7df-a8d0fdce0580-metrics-certs\") pod \"network-metrics-daemon-hnw6l\" (UID: \"f2569a02-edf9-4b33-a7df-a8d0fdce0580\") " pod="openshift-multus/network-metrics-daemon-hnw6l"
Sep 16 00:49:19.558366 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:49:19.558342    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f2569a02-edf9-4b33-a7df-a8d0fdce0580-metrics-certs\") pod \"network-metrics-daemon-hnw6l\" (UID: \"f2569a02-edf9-4b33-a7df-a8d0fdce0580\") " pod="openshift-multus/network-metrics-daemon-hnw6l"
Sep 16 00:49:19.834122 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:49:19.834098    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6btht\""
Sep 16 00:49:19.841808 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:49:19.841786    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hnw6l"
Sep 16 00:49:20.114547 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:49:20.114525    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-hnw6l"]
Sep 16 00:49:20.116930 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:49:20.116900    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2569a02_edf9_4b33_a7df_a8d0fdce0580.slice/crio-2a8d8a9fe1ee50a5a238af00846c42f050137e6a4ddd8ff6f1ee617b55557a9e WatchSource:0}: Error finding container 2a8d8a9fe1ee50a5a238af00846c42f050137e6a4ddd8ff6f1ee617b55557a9e: Status 404 returned error can't find the container with id 2a8d8a9fe1ee50a5a238af00846c42f050137e6a4ddd8ff6f1ee617b55557a9e
Sep 16 00:49:21.104116 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:49:21.104079    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hnw6l" event={"ID":"f2569a02-edf9-4b33-a7df-a8d0fdce0580","Type":"ContainerStarted","Data":"5b8d37f68c6f1c51b769c004f1260e603ff6c7ed6ee18258091fa475912b862b"}
Sep 16 00:49:21.104451 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:49:21.104143    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hnw6l" event={"ID":"f2569a02-edf9-4b33-a7df-a8d0fdce0580","Type":"ContainerStarted","Data":"2a8d8a9fe1ee50a5a238af00846c42f050137e6a4ddd8ff6f1ee617b55557a9e"}
Sep 16 00:49:22.110493 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:49:22.109829    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hnw6l" event={"ID":"f2569a02-edf9-4b33-a7df-a8d0fdce0580","Type":"ContainerStarted","Data":"7d76cd2c47e8969607c18347af36640096a218b047fd86fc116e93334b380acf"}
Sep 16 00:49:22.125340 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:49:22.125283    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-hnw6l" podStartSLOduration=376.252932482 podStartE2EDuration="6m17.125267044s" totalImagesPullingTime="872.334562ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 00:43:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 00:49:22.124095967 +0000 UTC m=+376.905597559" watchObservedRunningTime="2026-09-16 00:49:22.125267044 +0000 UTC m=+376.906768636"
Sep 16 00:49:27.451977 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:49:27.451931    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5dffdc758d-bd9qh"]
Sep 16 00:49:52.471091 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:49:52.471051    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5dffdc758d-bd9qh" podUID="d7a7db8c-34ff-40b4-9e90-a8887addbb03" containerName="console" containerID="cri-o://5f668d2c50dcc70a132700bfcd3861893173c864ae200aa63a9d9303d634f1c8" gracePeriod=15
Sep 16 00:49:52.700174 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:49:52.700152    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5dffdc758d-bd9qh"
Sep 16 00:49:52.767942 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:49:52.767880    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d7a7db8c-34ff-40b4-9e90-a8887addbb03-service-ca\") pod \"d7a7db8c-34ff-40b4-9e90-a8887addbb03\" (UID: \"d7a7db8c-34ff-40b4-9e90-a8887addbb03\") "
Sep 16 00:49:52.768038 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:49:52.767922    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7a7db8c-34ff-40b4-9e90-a8887addbb03-trusted-ca-bundle\") pod \"d7a7db8c-34ff-40b4-9e90-a8887addbb03\" (UID: \"d7a7db8c-34ff-40b4-9e90-a8887addbb03\") "
Sep 16 00:49:52.768038 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:49:52.767984    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b7pzk\" (UniqueName: \"kubernetes.io/projected/d7a7db8c-34ff-40b4-9e90-a8887addbb03-kube-api-access-b7pzk\") pod \"d7a7db8c-34ff-40b4-9e90-a8887addbb03\" (UID: \"d7a7db8c-34ff-40b4-9e90-a8887addbb03\") "
Sep 16 00:49:52.768038 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:49:52.768001    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d7a7db8c-34ff-40b4-9e90-a8887addbb03-console-oauth-config\") pod \"d7a7db8c-34ff-40b4-9e90-a8887addbb03\" (UID: \"d7a7db8c-34ff-40b4-9e90-a8887addbb03\") "
Sep 16 00:49:52.768038 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:49:52.768018    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d7a7db8c-34ff-40b4-9e90-a8887addbb03-console-serving-cert\") pod \"d7a7db8c-34ff-40b4-9e90-a8887addbb03\" (UID: \"d7a7db8c-34ff-40b4-9e90-a8887addbb03\") "
Sep 16 00:49:52.768038 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:49:52.768033    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d7a7db8c-34ff-40b4-9e90-a8887addbb03-oauth-serving-cert\") pod \"d7a7db8c-34ff-40b4-9e90-a8887addbb03\" (UID: \"d7a7db8c-34ff-40b4-9e90-a8887addbb03\") "
Sep 16 00:49:52.768272 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:49:52.768138    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d7a7db8c-34ff-40b4-9e90-a8887addbb03-console-config\") pod \"d7a7db8c-34ff-40b4-9e90-a8887addbb03\" (UID: \"d7a7db8c-34ff-40b4-9e90-a8887addbb03\") "
Sep 16 00:49:52.768352 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:49:52.768330    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7a7db8c-34ff-40b4-9e90-a8887addbb03-service-ca" (OuterVolumeSpecName: "service-ca") pod "d7a7db8c-34ff-40b4-9e90-a8887addbb03" (UID: "d7a7db8c-34ff-40b4-9e90-a8887addbb03"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 00:49:52.768569 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:49:52.768545    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7a7db8c-34ff-40b4-9e90-a8887addbb03-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "d7a7db8c-34ff-40b4-9e90-a8887addbb03" (UID: "d7a7db8c-34ff-40b4-9e90-a8887addbb03"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 00:49:52.768629 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:49:52.768585    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7a7db8c-34ff-40b4-9e90-a8887addbb03-console-config" (OuterVolumeSpecName: "console-config") pod "d7a7db8c-34ff-40b4-9e90-a8887addbb03" (UID: "d7a7db8c-34ff-40b4-9e90-a8887addbb03"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 00:49:52.768684 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:49:52.768670    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7a7db8c-34ff-40b4-9e90-a8887addbb03-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d7a7db8c-34ff-40b4-9e90-a8887addbb03" (UID: "d7a7db8c-34ff-40b4-9e90-a8887addbb03"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 00:49:52.770165 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:49:52.770144    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7a7db8c-34ff-40b4-9e90-a8887addbb03-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d7a7db8c-34ff-40b4-9e90-a8887addbb03" (UID: "d7a7db8c-34ff-40b4-9e90-a8887addbb03"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 00:49:52.770528 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:49:52.770507    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7a7db8c-34ff-40b4-9e90-a8887addbb03-kube-api-access-b7pzk" (OuterVolumeSpecName: "kube-api-access-b7pzk") pod "d7a7db8c-34ff-40b4-9e90-a8887addbb03" (UID: "d7a7db8c-34ff-40b4-9e90-a8887addbb03"). InnerVolumeSpecName "kube-api-access-b7pzk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 00:49:52.770592 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:49:52.770509    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7a7db8c-34ff-40b4-9e90-a8887addbb03-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d7a7db8c-34ff-40b4-9e90-a8887addbb03" (UID: "d7a7db8c-34ff-40b4-9e90-a8887addbb03"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 00:49:52.869539 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:49:52.869516    2556 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d7a7db8c-34ff-40b4-9e90-a8887addbb03-service-ca\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 00:49:52.869539 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:49:52.869536    2556 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7a7db8c-34ff-40b4-9e90-a8887addbb03-trusted-ca-bundle\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 00:49:52.869657 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:49:52.869545    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b7pzk\" (UniqueName: \"kubernetes.io/projected/d7a7db8c-34ff-40b4-9e90-a8887addbb03-kube-api-access-b7pzk\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 00:49:52.869657 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:49:52.869555    2556 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d7a7db8c-34ff-40b4-9e90-a8887addbb03-console-oauth-config\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 00:49:52.869657 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:49:52.869563    2556 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d7a7db8c-34ff-40b4-9e90-a8887addbb03-console-serving-cert\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 00:49:52.869657 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:49:52.869572    2556 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d7a7db8c-34ff-40b4-9e90-a8887addbb03-oauth-serving-cert\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 00:49:52.869657 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:49:52.869580    2556 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d7a7db8c-34ff-40b4-9e90-a8887addbb03-console-config\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 00:49:53.215434 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:49:53.215401    2556 generic.go:356] "Generic (PLEG): container finished" podID="d7a7db8c-34ff-40b4-9e90-a8887addbb03" containerID="5f668d2c50dcc70a132700bfcd3861893173c864ae200aa63a9d9303d634f1c8" exitCode=0
Sep 16 00:49:53.215609 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:49:53.215437    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5dffdc758d-bd9qh" event={"ID":"d7a7db8c-34ff-40b4-9e90-a8887addbb03","Type":"ContainerDied","Data":"5f668d2c50dcc70a132700bfcd3861893173c864ae200aa63a9d9303d634f1c8"}
Sep 16 00:49:53.215609 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:49:53.215449    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5dffdc758d-bd9qh"
Sep 16 00:49:53.215609 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:49:53.215488    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5dffdc758d-bd9qh" event={"ID":"d7a7db8c-34ff-40b4-9e90-a8887addbb03","Type":"ContainerDied","Data":"c4a18ed3a8cc32173221b8d8305668f1a36b9b4bc0a573651c9c0b53a5d47d19"}
Sep 16 00:49:53.215609 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:49:53.215526    2556 scope.go:117] "RemoveContainer" containerID="5f668d2c50dcc70a132700bfcd3861893173c864ae200aa63a9d9303d634f1c8"
Sep 16 00:49:53.223757 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:49:53.223735    2556 scope.go:117] "RemoveContainer" containerID="5f668d2c50dcc70a132700bfcd3861893173c864ae200aa63a9d9303d634f1c8"
Sep 16 00:49:53.224048 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:49:53.224023    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f668d2c50dcc70a132700bfcd3861893173c864ae200aa63a9d9303d634f1c8\": container with ID starting with 5f668d2c50dcc70a132700bfcd3861893173c864ae200aa63a9d9303d634f1c8 not found: ID does not exist" containerID="5f668d2c50dcc70a132700bfcd3861893173c864ae200aa63a9d9303d634f1c8"
Sep 16 00:49:53.224138 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:49:53.224054    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f668d2c50dcc70a132700bfcd3861893173c864ae200aa63a9d9303d634f1c8"} err="failed to get container status \"5f668d2c50dcc70a132700bfcd3861893173c864ae200aa63a9d9303d634f1c8\": rpc error: code = NotFound desc = could not find container \"5f668d2c50dcc70a132700bfcd3861893173c864ae200aa63a9d9303d634f1c8\": container with ID starting with 5f668d2c50dcc70a132700bfcd3861893173c864ae200aa63a9d9303d634f1c8 not found: ID does not exist"
Sep 16 00:49:53.235059 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:49:53.235038    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5dffdc758d-bd9qh"]
Sep 16 00:49:53.238348 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:49:53.238326    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5dffdc758d-bd9qh"]
Sep 16 00:49:53.734649 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:49:53.734625    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7a7db8c-34ff-40b4-9e90-a8887addbb03" path="/var/lib/kubelet/pods/d7a7db8c-34ff-40b4-9e90-a8887addbb03/volumes"
Sep 16 00:51:35.380150 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:51:35.380118    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnxgtl"]
Sep 16 00:51:35.380598 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:51:35.380355    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7a7db8c-34ff-40b4-9e90-a8887addbb03" containerName="console"
Sep 16 00:51:35.380598 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:51:35.380365    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7a7db8c-34ff-40b4-9e90-a8887addbb03" containerName="console"
Sep 16 00:51:35.380598 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:51:35.380424    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7a7db8c-34ff-40b4-9e90-a8887addbb03" containerName="console"
Sep 16 00:51:35.383278 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:51:35.383262    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnxgtl"
Sep 16 00:51:35.385481 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:51:35.385444    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 16 00:51:35.386299 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:51:35.386280    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-t4f99\""
Sep 16 00:51:35.386411 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:51:35.386302    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 16 00:51:35.391064 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:51:35.391044    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnxgtl"]
Sep 16 00:51:35.432854 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:51:35.432832    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3a56ff2b-d9e4-43bd-b5f0-8063f2db39e0-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnxgtl\" (UID: \"3a56ff2b-d9e4-43bd-b5f0-8063f2db39e0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnxgtl"
Sep 16 00:51:35.432957 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:51:35.432867    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nr4g4\" (UniqueName: \"kubernetes.io/projected/3a56ff2b-d9e4-43bd-b5f0-8063f2db39e0-kube-api-access-nr4g4\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnxgtl\" (UID: \"3a56ff2b-d9e4-43bd-b5f0-8063f2db39e0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnxgtl"
Sep 16 00:51:35.432957 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:51:35.432890    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3a56ff2b-d9e4-43bd-b5f0-8063f2db39e0-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnxgtl\" (UID: \"3a56ff2b-d9e4-43bd-b5f0-8063f2db39e0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnxgtl"
Sep 16 00:51:35.533697 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:51:35.533675    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nr4g4\" (UniqueName: \"kubernetes.io/projected/3a56ff2b-d9e4-43bd-b5f0-8063f2db39e0-kube-api-access-nr4g4\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnxgtl\" (UID: \"3a56ff2b-d9e4-43bd-b5f0-8063f2db39e0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnxgtl"
Sep 16 00:51:35.533807 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:51:35.533718    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3a56ff2b-d9e4-43bd-b5f0-8063f2db39e0-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnxgtl\" (UID: \"3a56ff2b-d9e4-43bd-b5f0-8063f2db39e0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnxgtl"
Sep 16 00:51:35.533807 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:51:35.533781    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3a56ff2b-d9e4-43bd-b5f0-8063f2db39e0-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnxgtl\" (UID: \"3a56ff2b-d9e4-43bd-b5f0-8063f2db39e0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnxgtl"
Sep 16 00:51:35.534093 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:51:35.534073    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3a56ff2b-d9e4-43bd-b5f0-8063f2db39e0-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnxgtl\" (UID: \"3a56ff2b-d9e4-43bd-b5f0-8063f2db39e0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnxgtl"
Sep 16 00:51:35.534164 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:51:35.534125    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3a56ff2b-d9e4-43bd-b5f0-8063f2db39e0-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnxgtl\" (UID: \"3a56ff2b-d9e4-43bd-b5f0-8063f2db39e0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnxgtl"
Sep 16 00:51:35.541506 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:51:35.541462    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nr4g4\" (UniqueName: \"kubernetes.io/projected/3a56ff2b-d9e4-43bd-b5f0-8063f2db39e0-kube-api-access-nr4g4\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnxgtl\" (UID: \"3a56ff2b-d9e4-43bd-b5f0-8063f2db39e0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnxgtl"
Sep 16 00:51:35.692833 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:51:35.692785    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnxgtl"
Sep 16 00:51:35.959875 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:51:35.959846    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnxgtl"]
Sep 16 00:51:35.964015 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:51:35.963987    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a56ff2b_d9e4_43bd_b5f0_8063f2db39e0.slice/crio-28af322f5d3bf88e9bba4955500b87f4c11e83197319bbc670e202ca9dcc85ab WatchSource:0}: Error finding container 28af322f5d3bf88e9bba4955500b87f4c11e83197319bbc670e202ca9dcc85ab: Status 404 returned error can't find the container with id 28af322f5d3bf88e9bba4955500b87f4c11e83197319bbc670e202ca9dcc85ab
Sep 16 00:51:36.546572 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:51:36.546533    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnxgtl" event={"ID":"3a56ff2b-d9e4-43bd-b5f0-8063f2db39e0","Type":"ContainerStarted","Data":"28af322f5d3bf88e9bba4955500b87f4c11e83197319bbc670e202ca9dcc85ab"}
Sep 16 00:51:42.570076 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:51:42.570007    2556 generic.go:356] "Generic (PLEG): container finished" podID="3a56ff2b-d9e4-43bd-b5f0-8063f2db39e0" containerID="4422f65ac3db24638e6cfa7591237f16f2f5e0654d1b4ea716bb3d85c91c6f0f" exitCode=0
Sep 16 00:51:42.570389 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:51:42.570079    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnxgtl" event={"ID":"3a56ff2b-d9e4-43bd-b5f0-8063f2db39e0","Type":"ContainerDied","Data":"4422f65ac3db24638e6cfa7591237f16f2f5e0654d1b4ea716bb3d85c91c6f0f"}
Sep 16 00:51:45.580866 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:51:45.580834    2556 generic.go:356] "Generic (PLEG): container finished" podID="3a56ff2b-d9e4-43bd-b5f0-8063f2db39e0" containerID="23bee8871819524de155cdd5d17be88c316f14ec9be6d116deb9f9bf1f1992fe" exitCode=0
Sep 16 00:51:45.581212 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:51:45.580916    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnxgtl" event={"ID":"3a56ff2b-d9e4-43bd-b5f0-8063f2db39e0","Type":"ContainerDied","Data":"23bee8871819524de155cdd5d17be88c316f14ec9be6d116deb9f9bf1f1992fe"}
Sep 16 00:51:52.606641 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:51:52.606605    2556 generic.go:356] "Generic (PLEG): container finished" podID="3a56ff2b-d9e4-43bd-b5f0-8063f2db39e0" containerID="567e302685b007fbcbe7fedabf0ed65c0465c63c30645d01af53ca1a89dc9092" exitCode=0
Sep 16 00:51:52.607007 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:51:52.606690    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnxgtl" event={"ID":"3a56ff2b-d9e4-43bd-b5f0-8063f2db39e0","Type":"ContainerDied","Data":"567e302685b007fbcbe7fedabf0ed65c0465c63c30645d01af53ca1a89dc9092"}
Sep 16 00:51:53.731295 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:51:53.731269    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnxgtl"
Sep 16 00:51:53.767077 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:51:53.767052    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nr4g4\" (UniqueName: \"kubernetes.io/projected/3a56ff2b-d9e4-43bd-b5f0-8063f2db39e0-kube-api-access-nr4g4\") pod \"3a56ff2b-d9e4-43bd-b5f0-8063f2db39e0\" (UID: \"3a56ff2b-d9e4-43bd-b5f0-8063f2db39e0\") "
Sep 16 00:51:53.767177 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:51:53.767093    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3a56ff2b-d9e4-43bd-b5f0-8063f2db39e0-util\") pod \"3a56ff2b-d9e4-43bd-b5f0-8063f2db39e0\" (UID: \"3a56ff2b-d9e4-43bd-b5f0-8063f2db39e0\") "
Sep 16 00:51:53.767177 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:51:53.767127    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3a56ff2b-d9e4-43bd-b5f0-8063f2db39e0-bundle\") pod \"3a56ff2b-d9e4-43bd-b5f0-8063f2db39e0\" (UID: \"3a56ff2b-d9e4-43bd-b5f0-8063f2db39e0\") "
Sep 16 00:51:53.767564 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:51:53.767538    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a56ff2b-d9e4-43bd-b5f0-8063f2db39e0-bundle" (OuterVolumeSpecName: "bundle") pod "3a56ff2b-d9e4-43bd-b5f0-8063f2db39e0" (UID: "3a56ff2b-d9e4-43bd-b5f0-8063f2db39e0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 00:51:53.769109 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:51:53.769087    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a56ff2b-d9e4-43bd-b5f0-8063f2db39e0-kube-api-access-nr4g4" (OuterVolumeSpecName: "kube-api-access-nr4g4") pod "3a56ff2b-d9e4-43bd-b5f0-8063f2db39e0" (UID: "3a56ff2b-d9e4-43bd-b5f0-8063f2db39e0"). InnerVolumeSpecName "kube-api-access-nr4g4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 00:51:53.771222 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:51:53.771198    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a56ff2b-d9e4-43bd-b5f0-8063f2db39e0-util" (OuterVolumeSpecName: "util") pod "3a56ff2b-d9e4-43bd-b5f0-8063f2db39e0" (UID: "3a56ff2b-d9e4-43bd-b5f0-8063f2db39e0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 00:51:53.868568 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:51:53.868509    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nr4g4\" (UniqueName: \"kubernetes.io/projected/3a56ff2b-d9e4-43bd-b5f0-8063f2db39e0-kube-api-access-nr4g4\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 00:51:53.868568 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:51:53.868537    2556 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3a56ff2b-d9e4-43bd-b5f0-8063f2db39e0-util\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 00:51:53.868568 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:51:53.868547    2556 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3a56ff2b-d9e4-43bd-b5f0-8063f2db39e0-bundle\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 00:51:54.615290 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:51:54.615261    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnxgtl"
Sep 16 00:51:54.615449 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:51:54.615256    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnxgtl" event={"ID":"3a56ff2b-d9e4-43bd-b5f0-8063f2db39e0","Type":"ContainerDied","Data":"28af322f5d3bf88e9bba4955500b87f4c11e83197319bbc670e202ca9dcc85ab"}
Sep 16 00:51:54.615449 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:51:54.615379    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="28af322f5d3bf88e9bba4955500b87f4c11e83197319bbc670e202ca9dcc85ab"
Sep 16 00:51:58.960706 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:51:58.960624    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hpbj5"]
Sep 16 00:51:58.961146 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:51:58.960914    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a56ff2b-d9e4-43bd-b5f0-8063f2db39e0" containerName="extract"
Sep 16 00:51:58.961146 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:51:58.960931    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a56ff2b-d9e4-43bd-b5f0-8063f2db39e0" containerName="extract"
Sep 16 00:51:58.961146 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:51:58.960950    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a56ff2b-d9e4-43bd-b5f0-8063f2db39e0" containerName="pull"
Sep 16 00:51:58.961146 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:51:58.960959    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a56ff2b-d9e4-43bd-b5f0-8063f2db39e0" containerName="pull"
Sep 16 00:51:58.961146 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:51:58.960984    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a56ff2b-d9e4-43bd-b5f0-8063f2db39e0" containerName="util"
Sep 16 00:51:58.961146 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:51:58.960994    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a56ff2b-d9e4-43bd-b5f0-8063f2db39e0" containerName="util"
Sep 16 00:51:58.961146 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:51:58.961053    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a56ff2b-d9e4-43bd-b5f0-8063f2db39e0" containerName="extract"
Sep 16 00:51:59.005009 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:51:59.004979    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hpbj5"]
Sep 16 00:51:59.005142 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:51:59.005059    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hpbj5"
Sep 16 00:51:59.007579 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:51:59.007552    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 16 00:51:59.007579 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:51:59.007569    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-sx7zw\""
Sep 16 00:51:59.007746 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:51:59.007587    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 16 00:51:59.103146 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:51:59.103117    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmcdb\" (UniqueName: \"kubernetes.io/projected/97e3a04a-9639-4970-8fc2-f92dd1cd9274-kube-api-access-rmcdb\") pod \"openshift-lws-operator-fd8ccff4c-hpbj5\" (UID: \"97e3a04a-9639-4970-8fc2-f92dd1cd9274\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hpbj5"
Sep 16 00:51:59.103146 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:51:59.103145    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/97e3a04a-9639-4970-8fc2-f92dd1cd9274-tmp\") pod \"openshift-lws-operator-fd8ccff4c-hpbj5\" (UID: \"97e3a04a-9639-4970-8fc2-f92dd1cd9274\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hpbj5"
Sep 16 00:51:59.205088 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:51:59.204384    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmcdb\" (UniqueName: \"kubernetes.io/projected/97e3a04a-9639-4970-8fc2-f92dd1cd9274-kube-api-access-rmcdb\") pod \"openshift-lws-operator-fd8ccff4c-hpbj5\" (UID: \"97e3a04a-9639-4970-8fc2-f92dd1cd9274\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hpbj5"
Sep 16 00:51:59.205088 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:51:59.204482    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/97e3a04a-9639-4970-8fc2-f92dd1cd9274-tmp\") pod \"openshift-lws-operator-fd8ccff4c-hpbj5\" (UID: \"97e3a04a-9639-4970-8fc2-f92dd1cd9274\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hpbj5"
Sep 16 00:51:59.205088 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:51:59.205041    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/97e3a04a-9639-4970-8fc2-f92dd1cd9274-tmp\") pod \"openshift-lws-operator-fd8ccff4c-hpbj5\" (UID: \"97e3a04a-9639-4970-8fc2-f92dd1cd9274\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hpbj5"
Sep 16 00:51:59.212284 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:51:59.212231    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmcdb\" (UniqueName: \"kubernetes.io/projected/97e3a04a-9639-4970-8fc2-f92dd1cd9274-kube-api-access-rmcdb\") pod \"openshift-lws-operator-fd8ccff4c-hpbj5\" (UID: \"97e3a04a-9639-4970-8fc2-f92dd1cd9274\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hpbj5"
Sep 16 00:51:59.313811 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:51:59.313784    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hpbj5"
Sep 16 00:51:59.595170 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:51:59.595065    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hpbj5"]
Sep 16 00:51:59.597890 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:51:59.597863    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod97e3a04a_9639_4970_8fc2_f92dd1cd9274.slice/crio-13e07033ec7948c8676b87e1d39c251c02bb3218c65394249a6f5bcac81a5a61 WatchSource:0}: Error finding container 13e07033ec7948c8676b87e1d39c251c02bb3218c65394249a6f5bcac81a5a61: Status 404 returned error can't find the container with id 13e07033ec7948c8676b87e1d39c251c02bb3218c65394249a6f5bcac81a5a61
Sep 16 00:51:59.633712 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:51:59.633688    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hpbj5" event={"ID":"97e3a04a-9639-4970-8fc2-f92dd1cd9274","Type":"ContainerStarted","Data":"13e07033ec7948c8676b87e1d39c251c02bb3218c65394249a6f5bcac81a5a61"}
Sep 16 00:52:02.647131 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:02.647096    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hpbj5" event={"ID":"97e3a04a-9639-4970-8fc2-f92dd1cd9274","Type":"ContainerStarted","Data":"4a50c59c9a2494a9b352aa171314356bc48ff808a9a82e98611ba00af9178dc5"}
Sep 16 00:52:02.662526 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:02.662481    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hpbj5" podStartSLOduration=2.075485653 podStartE2EDuration="4.662450242s" totalImagesPullingTime="2.586964589s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 00:51:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 00:52:02.660945679 +0000 UTC m=+537.442447271" watchObservedRunningTime="2026-09-16 00:52:02.662450242 +0000 UTC m=+537.443951834"
Sep 16 00:52:22.206078 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:22.206043    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b05gmpp"]
Sep 16 00:52:22.209834 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:22.209805    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b05gmpp"
Sep 16 00:52:22.213226 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:22.213195    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-t4f99\""
Sep 16 00:52:22.213226 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:22.213219    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 16 00:52:22.213427 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:22.213207    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 16 00:52:22.217128 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:22.217105    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b05gmpp"]
Sep 16 00:52:22.361579 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:22.361553    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8de955bf-f99b-439d-b694-c4c7740d88a7-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b05gmpp\" (UID: \"8de955bf-f99b-439d-b694-c4c7740d88a7\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b05gmpp"
Sep 16 00:52:22.361715 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:22.361590    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7djl9\" (UniqueName: \"kubernetes.io/projected/8de955bf-f99b-439d-b694-c4c7740d88a7-kube-api-access-7djl9\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b05gmpp\" (UID: \"8de955bf-f99b-439d-b694-c4c7740d88a7\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b05gmpp"
Sep 16 00:52:22.361715 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:22.361671    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8de955bf-f99b-439d-b694-c4c7740d88a7-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b05gmpp\" (UID: \"8de955bf-f99b-439d-b694-c4c7740d88a7\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b05gmpp"
Sep 16 00:52:22.410077 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:22.410051    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9778fc5bd-rvf5d"]
Sep 16 00:52:22.411921 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:22.411906    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-9778fc5bd-rvf5d"
Sep 16 00:52:22.414027 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:22.414005    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 16 00:52:22.414118 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:22.414033    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 16 00:52:22.414166 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:22.414138    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 16 00:52:22.414166 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:22.414147    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 16 00:52:22.414292 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:22.414156    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-cqdqc\""
Sep 16 00:52:22.428490 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:22.428459    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9778fc5bd-rvf5d"]
Sep 16 00:52:22.462657 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:22.462608    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8de955bf-f99b-439d-b694-c4c7740d88a7-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b05gmpp\" (UID: \"8de955bf-f99b-439d-b694-c4c7740d88a7\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b05gmpp"
Sep 16 00:52:22.462657 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:22.462646    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8de955bf-f99b-439d-b694-c4c7740d88a7-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b05gmpp\" (UID: \"8de955bf-f99b-439d-b694-c4c7740d88a7\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b05gmpp"
Sep 16 00:52:22.462807 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:22.462663    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7djl9\" (UniqueName: \"kubernetes.io/projected/8de955bf-f99b-439d-b694-c4c7740d88a7-kube-api-access-7djl9\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b05gmpp\" (UID: \"8de955bf-f99b-439d-b694-c4c7740d88a7\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b05gmpp"
Sep 16 00:52:22.462964 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:22.462944    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8de955bf-f99b-439d-b694-c4c7740d88a7-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b05gmpp\" (UID: \"8de955bf-f99b-439d-b694-c4c7740d88a7\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b05gmpp"
Sep 16 00:52:22.463023 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:22.462978    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8de955bf-f99b-439d-b694-c4c7740d88a7-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b05gmpp\" (UID: \"8de955bf-f99b-439d-b694-c4c7740d88a7\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b05gmpp"
Sep 16 00:52:22.478514 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:22.478487    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7djl9\" (UniqueName: \"kubernetes.io/projected/8de955bf-f99b-439d-b694-c4c7740d88a7-kube-api-access-7djl9\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b05gmpp\" (UID: \"8de955bf-f99b-439d-b694-c4c7740d88a7\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b05gmpp"
Sep 16 00:52:22.520934 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:22.520914    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b05gmpp"
Sep 16 00:52:22.564088 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:22.563997    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e20f5a6a-518f-498c-bcb0-1abe83414bb9-manifests\") pod \"opendatahub-operator-controller-manager-9778fc5bd-rvf5d\" (UID: \"e20f5a6a-518f-498c-bcb0-1abe83414bb9\") " pod="opendatahub/opendatahub-operator-controller-manager-9778fc5bd-rvf5d"
Sep 16 00:52:22.564088 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:22.564056    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e20f5a6a-518f-498c-bcb0-1abe83414bb9-webhook-cert\") pod \"opendatahub-operator-controller-manager-9778fc5bd-rvf5d\" (UID: \"e20f5a6a-518f-498c-bcb0-1abe83414bb9\") " pod="opendatahub/opendatahub-operator-controller-manager-9778fc5bd-rvf5d"
Sep 16 00:52:22.564295 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:22.564148    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e20f5a6a-518f-498c-bcb0-1abe83414bb9-tmp\") pod \"opendatahub-operator-controller-manager-9778fc5bd-rvf5d\" (UID: \"e20f5a6a-518f-498c-bcb0-1abe83414bb9\") " pod="opendatahub/opendatahub-operator-controller-manager-9778fc5bd-rvf5d"
Sep 16 00:52:22.564295 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:22.564235    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e20f5a6a-518f-498c-bcb0-1abe83414bb9-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9778fc5bd-rvf5d\" (UID: \"e20f5a6a-518f-498c-bcb0-1abe83414bb9\") " pod="opendatahub/opendatahub-operator-controller-manager-9778fc5bd-rvf5d"
Sep 16 00:52:22.564295 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:22.564267    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqbfb\" (UniqueName: \"kubernetes.io/projected/e20f5a6a-518f-498c-bcb0-1abe83414bb9-kube-api-access-vqbfb\") pod \"opendatahub-operator-controller-manager-9778fc5bd-rvf5d\" (UID: \"e20f5a6a-518f-498c-bcb0-1abe83414bb9\") " pod="opendatahub/opendatahub-operator-controller-manager-9778fc5bd-rvf5d"
Sep 16 00:52:22.664927 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:22.664888    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e20f5a6a-518f-498c-bcb0-1abe83414bb9-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9778fc5bd-rvf5d\" (UID: \"e20f5a6a-518f-498c-bcb0-1abe83414bb9\") " pod="opendatahub/opendatahub-operator-controller-manager-9778fc5bd-rvf5d"
Sep 16 00:52:22.664927 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:22.664932    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqbfb\" (UniqueName: \"kubernetes.io/projected/e20f5a6a-518f-498c-bcb0-1abe83414bb9-kube-api-access-vqbfb\") pod \"opendatahub-operator-controller-manager-9778fc5bd-rvf5d\" (UID: \"e20f5a6a-518f-498c-bcb0-1abe83414bb9\") " pod="opendatahub/opendatahub-operator-controller-manager-9778fc5bd-rvf5d"
Sep 16 00:52:22.665132 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:22.664952    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e20f5a6a-518f-498c-bcb0-1abe83414bb9-manifests\") pod \"opendatahub-operator-controller-manager-9778fc5bd-rvf5d\" (UID: \"e20f5a6a-518f-498c-bcb0-1abe83414bb9\") " pod="opendatahub/opendatahub-operator-controller-manager-9778fc5bd-rvf5d"
Sep 16 00:52:22.665132 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:22.664991    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e20f5a6a-518f-498c-bcb0-1abe83414bb9-webhook-cert\") pod \"opendatahub-operator-controller-manager-9778fc5bd-rvf5d\" (UID: \"e20f5a6a-518f-498c-bcb0-1abe83414bb9\") " pod="opendatahub/opendatahub-operator-controller-manager-9778fc5bd-rvf5d"
Sep 16 00:52:22.665132 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:22.665017    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e20f5a6a-518f-498c-bcb0-1abe83414bb9-tmp\") pod \"opendatahub-operator-controller-manager-9778fc5bd-rvf5d\" (UID: \"e20f5a6a-518f-498c-bcb0-1abe83414bb9\") " pod="opendatahub/opendatahub-operator-controller-manager-9778fc5bd-rvf5d"
Sep 16 00:52:22.665325 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:22.665298    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e20f5a6a-518f-498c-bcb0-1abe83414bb9-manifests\") pod \"opendatahub-operator-controller-manager-9778fc5bd-rvf5d\" (UID: \"e20f5a6a-518f-498c-bcb0-1abe83414bb9\") " pod="opendatahub/opendatahub-operator-controller-manager-9778fc5bd-rvf5d"
Sep 16 00:52:22.667152 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:22.667130    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e20f5a6a-518f-498c-bcb0-1abe83414bb9-tmp\") pod \"opendatahub-operator-controller-manager-9778fc5bd-rvf5d\" (UID: \"e20f5a6a-518f-498c-bcb0-1abe83414bb9\") " pod="opendatahub/opendatahub-operator-controller-manager-9778fc5bd-rvf5d"
Sep 16 00:52:22.667372 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:22.667352    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e20f5a6a-518f-498c-bcb0-1abe83414bb9-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9778fc5bd-rvf5d\" (UID: \"e20f5a6a-518f-498c-bcb0-1abe83414bb9\") " pod="opendatahub/opendatahub-operator-controller-manager-9778fc5bd-rvf5d"
Sep 16 00:52:22.667703 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:22.667686    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e20f5a6a-518f-498c-bcb0-1abe83414bb9-webhook-cert\") pod \"opendatahub-operator-controller-manager-9778fc5bd-rvf5d\" (UID: \"e20f5a6a-518f-498c-bcb0-1abe83414bb9\") " pod="opendatahub/opendatahub-operator-controller-manager-9778fc5bd-rvf5d"
Sep 16 00:52:22.677504 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:22.677484    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqbfb\" (UniqueName: \"kubernetes.io/projected/e20f5a6a-518f-498c-bcb0-1abe83414bb9-kube-api-access-vqbfb\") pod \"opendatahub-operator-controller-manager-9778fc5bd-rvf5d\" (UID: \"e20f5a6a-518f-498c-bcb0-1abe83414bb9\") " pod="opendatahub/opendatahub-operator-controller-manager-9778fc5bd-rvf5d"
Sep 16 00:52:22.720830 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:22.720781    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-9778fc5bd-rvf5d"
Sep 16 00:52:22.790188 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:22.790157    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b05gmpp"]
Sep 16 00:52:22.792422 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:52:22.792397    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8de955bf_f99b_439d_b694_c4c7740d88a7.slice/crio-e4937628e0574551a76a1ecc95646efa5383ed0bbc46977c19d1e7abd1f69789 WatchSource:0}: Error finding container e4937628e0574551a76a1ecc95646efa5383ed0bbc46977c19d1e7abd1f69789: Status 404 returned error can't find the container with id e4937628e0574551a76a1ecc95646efa5383ed0bbc46977c19d1e7abd1f69789
Sep 16 00:52:22.989560 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:22.989502    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9778fc5bd-rvf5d"]
Sep 16 00:52:22.992126 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:52:22.992100    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode20f5a6a_518f_498c_bcb0_1abe83414bb9.slice/crio-16e4ef8acb60973d313931223ef39888bc6ef14f55617f0cce6c6edd406b65da WatchSource:0}: Error finding container 16e4ef8acb60973d313931223ef39888bc6ef14f55617f0cce6c6edd406b65da: Status 404 returned error can't find the container with id 16e4ef8acb60973d313931223ef39888bc6ef14f55617f0cce6c6edd406b65da
Sep 16 00:52:23.719450 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:23.719375    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9778fc5bd-rvf5d" event={"ID":"e20f5a6a-518f-498c-bcb0-1abe83414bb9","Type":"ContainerStarted","Data":"16e4ef8acb60973d313931223ef39888bc6ef14f55617f0cce6c6edd406b65da"}
Sep 16 00:52:23.721564 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:23.721492    2556 generic.go:356] "Generic (PLEG): container finished" podID="8de955bf-f99b-439d-b694-c4c7740d88a7" containerID="aeb7885fee8ee1f7a9c455b563235c0f3ff7363e323736f7c5fdaf1e04469e5b" exitCode=0
Sep 16 00:52:23.721564 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:23.721529    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b05gmpp" event={"ID":"8de955bf-f99b-439d-b694-c4c7740d88a7","Type":"ContainerDied","Data":"aeb7885fee8ee1f7a9c455b563235c0f3ff7363e323736f7c5fdaf1e04469e5b"}
Sep 16 00:52:23.721564 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:23.721565    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b05gmpp" event={"ID":"8de955bf-f99b-439d-b694-c4c7740d88a7","Type":"ContainerStarted","Data":"e4937628e0574551a76a1ecc95646efa5383ed0bbc46977c19d1e7abd1f69789"}
Sep 16 00:52:24.727158 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:24.727112    2556 generic.go:356] "Generic (PLEG): container finished" podID="8de955bf-f99b-439d-b694-c4c7740d88a7" containerID="46a20b0346d349c903ea9e40bdebb080457736d27a2a1e1e335ac645cd66313c" exitCode=0
Sep 16 00:52:24.727602 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:24.727175    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b05gmpp" event={"ID":"8de955bf-f99b-439d-b694-c4c7740d88a7","Type":"ContainerDied","Data":"46a20b0346d349c903ea9e40bdebb080457736d27a2a1e1e335ac645cd66313c"}
Sep 16 00:52:26.738766 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:26.738731    2556 generic.go:356] "Generic (PLEG): container finished" podID="8de955bf-f99b-439d-b694-c4c7740d88a7" containerID="eaa179419eb2b1a15a1ea8519a810e1988f62babf4bf9884e68b3613dc09af00" exitCode=0
Sep 16 00:52:26.739189 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:26.738808    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b05gmpp" event={"ID":"8de955bf-f99b-439d-b694-c4c7740d88a7","Type":"ContainerDied","Data":"eaa179419eb2b1a15a1ea8519a810e1988f62babf4bf9884e68b3613dc09af00"}
Sep 16 00:52:26.740433 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:26.740411    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9778fc5bd-rvf5d" event={"ID":"e20f5a6a-518f-498c-bcb0-1abe83414bb9","Type":"ContainerStarted","Data":"eba9e14b6a25f62f06cf6567db98a3f3de202911122ab041461b4426af684b67"}
Sep 16 00:52:27.745087 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:27.745051    2556 generic.go:356] "Generic (PLEG): container finished" podID="e20f5a6a-518f-498c-bcb0-1abe83414bb9" containerID="eba9e14b6a25f62f06cf6567db98a3f3de202911122ab041461b4426af684b67" exitCode=0
Sep 16 00:52:27.745579 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:27.745129    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9778fc5bd-rvf5d" event={"ID":"e20f5a6a-518f-498c-bcb0-1abe83414bb9","Type":"ContainerDied","Data":"eba9e14b6a25f62f06cf6567db98a3f3de202911122ab041461b4426af684b67"}
Sep 16 00:52:27.866007 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:27.865983    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b05gmpp"
Sep 16 00:52:28.007612 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:28.007535    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8de955bf-f99b-439d-b694-c4c7740d88a7-bundle\") pod \"8de955bf-f99b-439d-b694-c4c7740d88a7\" (UID: \"8de955bf-f99b-439d-b694-c4c7740d88a7\") "
Sep 16 00:52:28.007612 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:28.007592    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7djl9\" (UniqueName: \"kubernetes.io/projected/8de955bf-f99b-439d-b694-c4c7740d88a7-kube-api-access-7djl9\") pod \"8de955bf-f99b-439d-b694-c4c7740d88a7\" (UID: \"8de955bf-f99b-439d-b694-c4c7740d88a7\") "
Sep 16 00:52:28.007819 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:28.007625    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8de955bf-f99b-439d-b694-c4c7740d88a7-util\") pod \"8de955bf-f99b-439d-b694-c4c7740d88a7\" (UID: \"8de955bf-f99b-439d-b694-c4c7740d88a7\") "
Sep 16 00:52:28.008281 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:28.008256    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8de955bf-f99b-439d-b694-c4c7740d88a7-bundle" (OuterVolumeSpecName: "bundle") pod "8de955bf-f99b-439d-b694-c4c7740d88a7" (UID: "8de955bf-f99b-439d-b694-c4c7740d88a7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 00:52:28.009660 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:28.009631    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8de955bf-f99b-439d-b694-c4c7740d88a7-kube-api-access-7djl9" (OuterVolumeSpecName: "kube-api-access-7djl9") pod "8de955bf-f99b-439d-b694-c4c7740d88a7" (UID: "8de955bf-f99b-439d-b694-c4c7740d88a7"). InnerVolumeSpecName "kube-api-access-7djl9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 00:52:28.013311 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:28.013288    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8de955bf-f99b-439d-b694-c4c7740d88a7-util" (OuterVolumeSpecName: "util") pod "8de955bf-f99b-439d-b694-c4c7740d88a7" (UID: "8de955bf-f99b-439d-b694-c4c7740d88a7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 00:52:28.108685 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:28.108663    2556 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8de955bf-f99b-439d-b694-c4c7740d88a7-bundle\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 00:52:28.108685 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:28.108682    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7djl9\" (UniqueName: \"kubernetes.io/projected/8de955bf-f99b-439d-b694-c4c7740d88a7-kube-api-access-7djl9\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 00:52:28.108800 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:28.108693    2556 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8de955bf-f99b-439d-b694-c4c7740d88a7-util\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 00:52:28.750996 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:28.750967    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b05gmpp"
Sep 16 00:52:28.751442 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:28.750993    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b05gmpp" event={"ID":"8de955bf-f99b-439d-b694-c4c7740d88a7","Type":"ContainerDied","Data":"e4937628e0574551a76a1ecc95646efa5383ed0bbc46977c19d1e7abd1f69789"}
Sep 16 00:52:28.751442 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:28.751052    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e4937628e0574551a76a1ecc95646efa5383ed0bbc46977c19d1e7abd1f69789"
Sep 16 00:52:28.753120 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:28.753095    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9778fc5bd-rvf5d" event={"ID":"e20f5a6a-518f-498c-bcb0-1abe83414bb9","Type":"ContainerStarted","Data":"d7363ea92d8a88ef59e0f7638f208e1e59fb2eb76835f7fb6fc372c205c682ea"}
Sep 16 00:52:28.753331 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:28.753314    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-9778fc5bd-rvf5d"
Sep 16 00:52:28.773795 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:28.773620    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-9778fc5bd-rvf5d" podStartSLOduration=2.390690246 podStartE2EDuration="6.773565844s" totalImagesPullingTime="3.367029535s" totalInitContainerRuntime="1.015846063s" isStatefulPod=false podCreationTimestamp="2026-09-16 00:52:22 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 00:52:28.772098737 +0000 UTC m=+563.553600329" watchObservedRunningTime="2026-09-16 00:52:28.773565844 +0000 UTC m=+563.555067437"
Sep 16 00:52:39.759124 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:39.759090    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-9778fc5bd-rvf5d"
Sep 16 00:52:41.946044 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:41.946013    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5bfdbc6f8d-j5r2k"]
Sep 16 00:52:41.946380 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:41.946230    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8de955bf-f99b-439d-b694-c4c7740d88a7" containerName="util"
Sep 16 00:52:41.946380 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:41.946241    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="8de955bf-f99b-439d-b694-c4c7740d88a7" containerName="util"
Sep 16 00:52:41.946380 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:41.946254    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8de955bf-f99b-439d-b694-c4c7740d88a7" containerName="extract"
Sep 16 00:52:41.946380 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:41.946259    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="8de955bf-f99b-439d-b694-c4c7740d88a7" containerName="extract"
Sep 16 00:52:41.946380 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:41.946268    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8de955bf-f99b-439d-b694-c4c7740d88a7" containerName="pull"
Sep 16 00:52:41.946380 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:41.946273    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="8de955bf-f99b-439d-b694-c4c7740d88a7" containerName="pull"
Sep 16 00:52:41.946380 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:41.946325    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="8de955bf-f99b-439d-b694-c4c7740d88a7" containerName="extract"
Sep 16 00:52:41.954218 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:41.954199    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5bfdbc6f8d-j5r2k"
Sep 16 00:52:41.959875 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:41.959847    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 16 00:52:41.960365 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:41.960111    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 16 00:52:41.961229 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:41.961208    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 16 00:52:41.964692 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:41.964676    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 16 00:52:41.972961 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:41.972934    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5bfdbc6f8d-j5r2k"]
Sep 16 00:52:42.095869 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:42.095837    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c3689323-4bf5-447c-9f4e-2d5054c5ff85-tmp\") pod \"kube-auth-proxy-5bfdbc6f8d-j5r2k\" (UID: \"c3689323-4bf5-447c-9f4e-2d5054c5ff85\") " pod="openshift-ingress/kube-auth-proxy-5bfdbc6f8d-j5r2k"
Sep 16 00:52:42.096028 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:42.095878    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c3689323-4bf5-447c-9f4e-2d5054c5ff85-tls-certs\") pod \"kube-auth-proxy-5bfdbc6f8d-j5r2k\" (UID: \"c3689323-4bf5-447c-9f4e-2d5054c5ff85\") " pod="openshift-ingress/kube-auth-proxy-5bfdbc6f8d-j5r2k"
Sep 16 00:52:42.096028 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:42.095920    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzp6f\" (UniqueName: \"kubernetes.io/projected/c3689323-4bf5-447c-9f4e-2d5054c5ff85-kube-api-access-jzp6f\") pod \"kube-auth-proxy-5bfdbc6f8d-j5r2k\" (UID: \"c3689323-4bf5-447c-9f4e-2d5054c5ff85\") " pod="openshift-ingress/kube-auth-proxy-5bfdbc6f8d-j5r2k"
Sep 16 00:52:42.196616 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:42.196553    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzp6f\" (UniqueName: \"kubernetes.io/projected/c3689323-4bf5-447c-9f4e-2d5054c5ff85-kube-api-access-jzp6f\") pod \"kube-auth-proxy-5bfdbc6f8d-j5r2k\" (UID: \"c3689323-4bf5-447c-9f4e-2d5054c5ff85\") " pod="openshift-ingress/kube-auth-proxy-5bfdbc6f8d-j5r2k"
Sep 16 00:52:42.196616 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:42.196589    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c3689323-4bf5-447c-9f4e-2d5054c5ff85-tmp\") pod \"kube-auth-proxy-5bfdbc6f8d-j5r2k\" (UID: \"c3689323-4bf5-447c-9f4e-2d5054c5ff85\") " pod="openshift-ingress/kube-auth-proxy-5bfdbc6f8d-j5r2k"
Sep 16 00:52:42.196616 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:42.196612    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c3689323-4bf5-447c-9f4e-2d5054c5ff85-tls-certs\") pod \"kube-auth-proxy-5bfdbc6f8d-j5r2k\" (UID: \"c3689323-4bf5-447c-9f4e-2d5054c5ff85\") " pod="openshift-ingress/kube-auth-proxy-5bfdbc6f8d-j5r2k"
Sep 16 00:52:42.196806 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:52:42.196712    2556 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Sep 16 00:52:42.196806 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:52:42.196773    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3689323-4bf5-447c-9f4e-2d5054c5ff85-tls-certs podName:c3689323-4bf5-447c-9f4e-2d5054c5ff85 nodeName:}" failed. No retries permitted until 2026-09-16 00:52:42.696753434 +0000 UTC m=+577.478255023 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/c3689323-4bf5-447c-9f4e-2d5054c5ff85-tls-certs") pod "kube-auth-proxy-5bfdbc6f8d-j5r2k" (UID: "c3689323-4bf5-447c-9f4e-2d5054c5ff85") : secret "kube-auth-proxy-tls" not found
Sep 16 00:52:42.198659 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:42.198640    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c3689323-4bf5-447c-9f4e-2d5054c5ff85-tmp\") pod \"kube-auth-proxy-5bfdbc6f8d-j5r2k\" (UID: \"c3689323-4bf5-447c-9f4e-2d5054c5ff85\") " pod="openshift-ingress/kube-auth-proxy-5bfdbc6f8d-j5r2k"
Sep 16 00:52:42.216666 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:42.216645    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzp6f\" (UniqueName: \"kubernetes.io/projected/c3689323-4bf5-447c-9f4e-2d5054c5ff85-kube-api-access-jzp6f\") pod \"kube-auth-proxy-5bfdbc6f8d-j5r2k\" (UID: \"c3689323-4bf5-447c-9f4e-2d5054c5ff85\") " pod="openshift-ingress/kube-auth-proxy-5bfdbc6f8d-j5r2k"
Sep 16 00:52:42.700527 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:42.700491    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c3689323-4bf5-447c-9f4e-2d5054c5ff85-tls-certs\") pod \"kube-auth-proxy-5bfdbc6f8d-j5r2k\" (UID: \"c3689323-4bf5-447c-9f4e-2d5054c5ff85\") " pod="openshift-ingress/kube-auth-proxy-5bfdbc6f8d-j5r2k"
Sep 16 00:52:42.703031 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:42.703004    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c3689323-4bf5-447c-9f4e-2d5054c5ff85-tls-certs\") pod \"kube-auth-proxy-5bfdbc6f8d-j5r2k\" (UID: \"c3689323-4bf5-447c-9f4e-2d5054c5ff85\") " pod="openshift-ingress/kube-auth-proxy-5bfdbc6f8d-j5r2k"
Sep 16 00:52:42.863165 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:42.863137    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5bfdbc6f8d-j5r2k"
Sep 16 00:52:43.136889 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:43.136814    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5bfdbc6f8d-j5r2k"]
Sep 16 00:52:43.139009 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:52:43.138981    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3689323_4bf5_447c_9f4e_2d5054c5ff85.slice/crio-0e746e12f2ee9498a7c470aec3fb7d93f8fd5cdd297cf4a873ffe923fa1082db WatchSource:0}: Error finding container 0e746e12f2ee9498a7c470aec3fb7d93f8fd5cdd297cf4a873ffe923fa1082db: Status 404 returned error can't find the container with id 0e746e12f2ee9498a7c470aec3fb7d93f8fd5cdd297cf4a873ffe923fa1082db
Sep 16 00:52:43.377775 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:43.377741    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sng4t"]
Sep 16 00:52:43.382619 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:43.382597    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sng4t"
Sep 16 00:52:43.384779 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:43.384735    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 16 00:52:43.384903 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:43.384814    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 16 00:52:43.385037 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:43.385023    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 16 00:52:43.385037 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:43.385032    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 16 00:52:43.385153 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:43.385041    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-5fr8j\""
Sep 16 00:52:43.393028 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:43.393010    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sng4t"]
Sep 16 00:52:43.507268 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:43.507246    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/52113c58-dc00-4270-bc9d-58d2cd30b9b6-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-sng4t\" (UID: \"52113c58-dc00-4270-bc9d-58d2cd30b9b6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sng4t"
Sep 16 00:52:43.507364 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:43.507276    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52113c58-dc00-4270-bc9d-58d2cd30b9b6-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-sng4t\" (UID: \"52113c58-dc00-4270-bc9d-58d2cd30b9b6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sng4t"
Sep 16 00:52:43.507364 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:43.507293    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/52113c58-dc00-4270-bc9d-58d2cd30b9b6-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-sng4t\" (UID: \"52113c58-dc00-4270-bc9d-58d2cd30b9b6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sng4t"
Sep 16 00:52:43.507364 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:43.507314    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/52113c58-dc00-4270-bc9d-58d2cd30b9b6-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-sng4t\" (UID: \"52113c58-dc00-4270-bc9d-58d2cd30b9b6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sng4t"
Sep 16 00:52:43.507364 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:43.507330    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxvfr\" (UniqueName: \"kubernetes.io/projected/52113c58-dc00-4270-bc9d-58d2cd30b9b6-kube-api-access-kxvfr\") pod \"istiod-openshift-gateway-7b7995d5cd-sng4t\" (UID: \"52113c58-dc00-4270-bc9d-58d2cd30b9b6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sng4t"
Sep 16 00:52:43.507529 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:43.507374    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/52113c58-dc00-4270-bc9d-58d2cd30b9b6-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-sng4t\" (UID: \"52113c58-dc00-4270-bc9d-58d2cd30b9b6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sng4t"
Sep 16 00:52:43.507529 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:43.507393    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/52113c58-dc00-4270-bc9d-58d2cd30b9b6-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-sng4t\" (UID: \"52113c58-dc00-4270-bc9d-58d2cd30b9b6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sng4t"
Sep 16 00:52:43.607674 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:43.607653    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/52113c58-dc00-4270-bc9d-58d2cd30b9b6-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-sng4t\" (UID: \"52113c58-dc00-4270-bc9d-58d2cd30b9b6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sng4t"
Sep 16 00:52:43.607768 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:43.607681    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52113c58-dc00-4270-bc9d-58d2cd30b9b6-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-sng4t\" (UID: \"52113c58-dc00-4270-bc9d-58d2cd30b9b6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sng4t"
Sep 16 00:52:43.607768 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:43.607695    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/52113c58-dc00-4270-bc9d-58d2cd30b9b6-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-sng4t\" (UID: \"52113c58-dc00-4270-bc9d-58d2cd30b9b6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sng4t"
Sep 16 00:52:43.607768 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:43.607719    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/52113c58-dc00-4270-bc9d-58d2cd30b9b6-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-sng4t\" (UID: \"52113c58-dc00-4270-bc9d-58d2cd30b9b6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sng4t"
Sep 16 00:52:43.607768 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:43.607734    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxvfr\" (UniqueName: \"kubernetes.io/projected/52113c58-dc00-4270-bc9d-58d2cd30b9b6-kube-api-access-kxvfr\") pod \"istiod-openshift-gateway-7b7995d5cd-sng4t\" (UID: \"52113c58-dc00-4270-bc9d-58d2cd30b9b6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sng4t"
Sep 16 00:52:43.607768 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:43.607762    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/52113c58-dc00-4270-bc9d-58d2cd30b9b6-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-sng4t\" (UID: \"52113c58-dc00-4270-bc9d-58d2cd30b9b6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sng4t"
Sep 16 00:52:43.608030 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:43.607800    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/52113c58-dc00-4270-bc9d-58d2cd30b9b6-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-sng4t\" (UID: \"52113c58-dc00-4270-bc9d-58d2cd30b9b6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sng4t"
Sep 16 00:52:43.608504 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:43.608422    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/52113c58-dc00-4270-bc9d-58d2cd30b9b6-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-sng4t\" (UID: \"52113c58-dc00-4270-bc9d-58d2cd30b9b6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sng4t"
Sep 16 00:52:43.610061 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:43.610042    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/52113c58-dc00-4270-bc9d-58d2cd30b9b6-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-sng4t\" (UID: \"52113c58-dc00-4270-bc9d-58d2cd30b9b6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sng4t"
Sep 16 00:52:43.610156 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:43.610073    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/52113c58-dc00-4270-bc9d-58d2cd30b9b6-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-sng4t\" (UID: \"52113c58-dc00-4270-bc9d-58d2cd30b9b6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sng4t"
Sep 16 00:52:43.610156 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:43.610113    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/52113c58-dc00-4270-bc9d-58d2cd30b9b6-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-sng4t\" (UID: \"52113c58-dc00-4270-bc9d-58d2cd30b9b6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sng4t"
Sep 16 00:52:43.610242 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:43.610192    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/52113c58-dc00-4270-bc9d-58d2cd30b9b6-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-sng4t\" (UID: \"52113c58-dc00-4270-bc9d-58d2cd30b9b6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sng4t"
Sep 16 00:52:43.614719 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:43.614702    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52113c58-dc00-4270-bc9d-58d2cd30b9b6-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-sng4t\" (UID: \"52113c58-dc00-4270-bc9d-58d2cd30b9b6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sng4t"
Sep 16 00:52:43.614985 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:43.614967    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxvfr\" (UniqueName: \"kubernetes.io/projected/52113c58-dc00-4270-bc9d-58d2cd30b9b6-kube-api-access-kxvfr\") pod \"istiod-openshift-gateway-7b7995d5cd-sng4t\" (UID: \"52113c58-dc00-4270-bc9d-58d2cd30b9b6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sng4t"
Sep 16 00:52:43.691567 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:43.691520    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sng4t"
Sep 16 00:52:43.808574 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:43.808418    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5bfdbc6f8d-j5r2k" event={"ID":"c3689323-4bf5-447c-9f4e-2d5054c5ff85","Type":"ContainerStarted","Data":"0e746e12f2ee9498a7c470aec3fb7d93f8fd5cdd297cf4a873ffe923fa1082db"}
Sep 16 00:52:43.969084 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:43.968974    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sng4t"]
Sep 16 00:52:43.972622 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:52:43.972592    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52113c58_dc00_4270_bc9d_58d2cd30b9b6.slice/crio-37963abcc7c63c7ac18a1170e12907b2d9c48c4d2993d531570d0f4cde4100f8 WatchSource:0}: Error finding container 37963abcc7c63c7ac18a1170e12907b2d9c48c4d2993d531570d0f4cde4100f8: Status 404 returned error can't find the container with id 37963abcc7c63c7ac18a1170e12907b2d9c48c4d2993d531570d0f4cde4100f8
Sep 16 00:52:44.815365 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:44.815331    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sng4t" event={"ID":"52113c58-dc00-4270-bc9d-58d2cd30b9b6","Type":"ContainerStarted","Data":"37963abcc7c63c7ac18a1170e12907b2d9c48c4d2993d531570d0f4cde4100f8"}
Sep 16 00:52:46.825492 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:46.825443    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5bfdbc6f8d-j5r2k" event={"ID":"c3689323-4bf5-447c-9f4e-2d5054c5ff85","Type":"ContainerStarted","Data":"a64bead098377cf3010c3ec3d523b5fe7b60cdfe5b6f281f453b834b2676bbc8"}
Sep 16 00:52:46.840676 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:46.840623    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5bfdbc6f8d-j5r2k" podStartSLOduration=3.1669209 podStartE2EDuration="5.84060184s" totalImagesPullingTime="2.67368094s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 00:52:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 00:52:46.840413511 +0000 UTC m=+581.621915104" watchObservedRunningTime="2026-09-16 00:52:46.84060184 +0000 UTC m=+581.622103432"
Sep 16 00:52:47.689638 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:47.689602    2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 00:52:47.689748 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:47.689670    2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 00:52:47.831485 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:47.831431    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sng4t" event={"ID":"52113c58-dc00-4270-bc9d-58d2cd30b9b6","Type":"ContainerStarted","Data":"c6f9a6c85ed6f58b1bf79ae31fb0bcf766c728f6cb7d166cba3b1731368aa604"}
Sep 16 00:52:47.831836 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:47.831766    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sng4t"
Sep 16 00:52:47.850097 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:47.850054    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sng4t" podStartSLOduration=1.136035754 podStartE2EDuration="4.850040632s" totalImagesPullingTime="3.714004878s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 00:52:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 00:52:47.848190661 +0000 UTC m=+582.629692252" watchObservedRunningTime="2026-09-16 00:52:47.850040632 +0000 UTC m=+582.631542223"
Sep 16 00:52:48.837450 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:48.837416    2556 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-sng4t container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 16 00:52:48.837906 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:48.837495    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sng4t" podUID="52113c58-dc00-4270-bc9d-58d2cd30b9b6" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 00:52:51.836854 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:51.836820    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sng4t"
Sep 16 00:52:59.204025 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:59.203996    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-7v9tv"]
Sep 16 00:52:59.209930 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:59.209913    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7v9tv"
Sep 16 00:52:59.219589 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:59.219564    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-7v9tv"]
Sep 16 00:52:59.315217 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:59.315190    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mstqt\" (UniqueName: \"kubernetes.io/projected/3e004a11-dc0e-4c29-ade3-ad5b9afbbab2-kube-api-access-mstqt\") pod \"istiod-openshift-gateway-5677ccb49b-7v9tv\" (UID: \"3e004a11-dc0e-4c29-ade3-ad5b9afbbab2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7v9tv"
Sep 16 00:52:59.315217 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:59.315221    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3e004a11-dc0e-4c29-ade3-ad5b9afbbab2-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-7v9tv\" (UID: \"3e004a11-dc0e-4c29-ade3-ad5b9afbbab2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7v9tv"
Sep 16 00:52:59.315421 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:59.315241    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e004a11-dc0e-4c29-ade3-ad5b9afbbab2-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-7v9tv\" (UID: \"3e004a11-dc0e-4c29-ade3-ad5b9afbbab2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7v9tv"
Sep 16 00:52:59.315421 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:59.315259    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e004a11-dc0e-4c29-ade3-ad5b9afbbab2-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-7v9tv\" (UID: \"3e004a11-dc0e-4c29-ade3-ad5b9afbbab2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7v9tv"
Sep 16 00:52:59.315421 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:59.315335    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3e004a11-dc0e-4c29-ade3-ad5b9afbbab2-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-7v9tv\" (UID: \"3e004a11-dc0e-4c29-ade3-ad5b9afbbab2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7v9tv"
Sep 16 00:52:59.315421 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:59.315378    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3e004a11-dc0e-4c29-ade3-ad5b9afbbab2-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-7v9tv\" (UID: \"3e004a11-dc0e-4c29-ade3-ad5b9afbbab2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7v9tv"
Sep 16 00:52:59.315660 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:59.315425    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3e004a11-dc0e-4c29-ade3-ad5b9afbbab2-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-7v9tv\" (UID: \"3e004a11-dc0e-4c29-ade3-ad5b9afbbab2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7v9tv"
Sep 16 00:52:59.416634 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:59.416605    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3e004a11-dc0e-4c29-ade3-ad5b9afbbab2-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-7v9tv\" (UID: \"3e004a11-dc0e-4c29-ade3-ad5b9afbbab2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7v9tv"
Sep 16 00:52:59.416753 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:59.416653    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3e004a11-dc0e-4c29-ade3-ad5b9afbbab2-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-7v9tv\" (UID: \"3e004a11-dc0e-4c29-ade3-ad5b9afbbab2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7v9tv"
Sep 16 00:52:59.417116 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:59.417077    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mstqt\" (UniqueName: \"kubernetes.io/projected/3e004a11-dc0e-4c29-ade3-ad5b9afbbab2-kube-api-access-mstqt\") pod \"istiod-openshift-gateway-5677ccb49b-7v9tv\" (UID: \"3e004a11-dc0e-4c29-ade3-ad5b9afbbab2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7v9tv"
Sep 16 00:52:59.417186 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:59.417151    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3e004a11-dc0e-4c29-ade3-ad5b9afbbab2-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-7v9tv\" (UID: \"3e004a11-dc0e-4c29-ade3-ad5b9afbbab2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7v9tv"
Sep 16 00:52:59.417238 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:59.417191    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e004a11-dc0e-4c29-ade3-ad5b9afbbab2-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-7v9tv\" (UID: \"3e004a11-dc0e-4c29-ade3-ad5b9afbbab2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7v9tv"
Sep 16 00:52:59.417238 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:59.417232    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e004a11-dc0e-4c29-ade3-ad5b9afbbab2-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-7v9tv\" (UID: \"3e004a11-dc0e-4c29-ade3-ad5b9afbbab2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7v9tv"
Sep 16 00:52:59.417333 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:59.417279    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3e004a11-dc0e-4c29-ade3-ad5b9afbbab2-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-7v9tv\" (UID: \"3e004a11-dc0e-4c29-ade3-ad5b9afbbab2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7v9tv"
Sep 16 00:52:59.418437 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:59.418409    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3e004a11-dc0e-4c29-ade3-ad5b9afbbab2-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-7v9tv\" (UID: \"3e004a11-dc0e-4c29-ade3-ad5b9afbbab2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7v9tv"
Sep 16 00:52:59.420424 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:59.420391    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3e004a11-dc0e-4c29-ade3-ad5b9afbbab2-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-7v9tv\" (UID: \"3e004a11-dc0e-4c29-ade3-ad5b9afbbab2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7v9tv"
Sep 16 00:52:59.420552 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:59.420399    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3e004a11-dc0e-4c29-ade3-ad5b9afbbab2-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-7v9tv\" (UID: \"3e004a11-dc0e-4c29-ade3-ad5b9afbbab2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7v9tv"
Sep 16 00:52:59.422921 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:59.421161    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e004a11-dc0e-4c29-ade3-ad5b9afbbab2-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-7v9tv\" (UID: \"3e004a11-dc0e-4c29-ade3-ad5b9afbbab2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7v9tv"
Sep 16 00:52:59.423833 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:59.423812    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3e004a11-dc0e-4c29-ade3-ad5b9afbbab2-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-7v9tv\" (UID: \"3e004a11-dc0e-4c29-ade3-ad5b9afbbab2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7v9tv"
Sep 16 00:52:59.440715 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:59.440688    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e004a11-dc0e-4c29-ade3-ad5b9afbbab2-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-7v9tv\" (UID: \"3e004a11-dc0e-4c29-ade3-ad5b9afbbab2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7v9tv"
Sep 16 00:52:59.443327 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:59.443299    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mstqt\" (UniqueName: \"kubernetes.io/projected/3e004a11-dc0e-4c29-ade3-ad5b9afbbab2-kube-api-access-mstqt\") pod \"istiod-openshift-gateway-5677ccb49b-7v9tv\" (UID: \"3e004a11-dc0e-4c29-ade3-ad5b9afbbab2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7v9tv"
Sep 16 00:52:59.517709 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:59.517643    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7v9tv"
Sep 16 00:52:59.806854 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:59.806829    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-7v9tv"]
Sep 16 00:52:59.808088 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:52:59.808057    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e004a11_dc0e_4c29_ade3_ad5b9afbbab2.slice/crio-9f2f9df6562e9207d1aff63a2eeef5ebddd0454b2ef9e9a97bc8829aea404f1c WatchSource:0}: Error finding container 9f2f9df6562e9207d1aff63a2eeef5ebddd0454b2ef9e9a97bc8829aea404f1c: Status 404 returned error can't find the container with id 9f2f9df6562e9207d1aff63a2eeef5ebddd0454b2ef9e9a97bc8829aea404f1c
Sep 16 00:52:59.811206 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:59.811172    2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 00:52:59.811305 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:59.811231    2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 00:52:59.880698 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:59.880666    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7v9tv" event={"ID":"3e004a11-dc0e-4c29-ade3-ad5b9afbbab2","Type":"ContainerStarted","Data":"4effbad3f1019699825c1552acbc4bb734b0615bb7f93870d7473596adb16711"}
Sep 16 00:52:59.880800 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:59.880709    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7v9tv" event={"ID":"3e004a11-dc0e-4c29-ade3-ad5b9afbbab2","Type":"ContainerStarted","Data":"9f2f9df6562e9207d1aff63a2eeef5ebddd0454b2ef9e9a97bc8829aea404f1c"}
Sep 16 00:52:59.880861 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:59.880807    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7v9tv"
Sep 16 00:52:59.904394 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:52:59.904353    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7v9tv" podStartSLOduration=0.90433777 podStartE2EDuration="904.33777ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 00:52:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 00:52:59.903263817 +0000 UTC m=+594.684765411" watchObservedRunningTime="2026-09-16 00:52:59.90433777 +0000 UTC m=+594.685839363"
Sep 16 00:53:00.886491 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:00.886437    2556 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-7v9tv container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 16 00:53:00.886856 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:00.886519    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7v9tv" podUID="3e004a11-dc0e-4c29-ade3-ad5b9afbbab2" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 00:53:03.885792 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:03.885762    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7v9tv"
Sep 16 00:53:03.935852 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:03.935823    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sng4t"]
Sep 16 00:53:03.936025 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:03.936002    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sng4t" podUID="52113c58-dc00-4270-bc9d-58d2cd30b9b6" containerName="discovery" containerID="cri-o://c6f9a6c85ed6f58b1bf79ae31fb0bcf766c728f6cb7d166cba3b1731368aa604" gracePeriod=30
Sep 16 00:53:04.173220 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:04.173197    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sng4t"
Sep 16 00:53:04.356086 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:04.356045    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/52113c58-dc00-4270-bc9d-58d2cd30b9b6-istio-csr-ca-configmap\") pod \"52113c58-dc00-4270-bc9d-58d2cd30b9b6\" (UID: \"52113c58-dc00-4270-bc9d-58d2cd30b9b6\") "
Sep 16 00:53:04.356275 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:04.356101    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kxvfr\" (UniqueName: \"kubernetes.io/projected/52113c58-dc00-4270-bc9d-58d2cd30b9b6-kube-api-access-kxvfr\") pod \"52113c58-dc00-4270-bc9d-58d2cd30b9b6\" (UID: \"52113c58-dc00-4270-bc9d-58d2cd30b9b6\") "
Sep 16 00:53:04.356275 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:04.356134    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52113c58-dc00-4270-bc9d-58d2cd30b9b6-istio-token\") pod \"52113c58-dc00-4270-bc9d-58d2cd30b9b6\" (UID: \"52113c58-dc00-4270-bc9d-58d2cd30b9b6\") "
Sep 16 00:53:04.356275 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:04.356158    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/52113c58-dc00-4270-bc9d-58d2cd30b9b6-cacerts\") pod \"52113c58-dc00-4270-bc9d-58d2cd30b9b6\" (UID: \"52113c58-dc00-4270-bc9d-58d2cd30b9b6\") "
Sep 16 00:53:04.356275 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:04.356200    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/52113c58-dc00-4270-bc9d-58d2cd30b9b6-istio-csr-dns-cert\") pod \"52113c58-dc00-4270-bc9d-58d2cd30b9b6\" (UID: \"52113c58-dc00-4270-bc9d-58d2cd30b9b6\") "
Sep 16 00:53:04.356275 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:04.356247    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/52113c58-dc00-4270-bc9d-58d2cd30b9b6-local-certs\") pod \"52113c58-dc00-4270-bc9d-58d2cd30b9b6\" (UID: \"52113c58-dc00-4270-bc9d-58d2cd30b9b6\") "
Sep 16 00:53:04.356275 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:04.356272    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/52113c58-dc00-4270-bc9d-58d2cd30b9b6-istio-kubeconfig\") pod \"52113c58-dc00-4270-bc9d-58d2cd30b9b6\" (UID: \"52113c58-dc00-4270-bc9d-58d2cd30b9b6\") "
Sep 16 00:53:04.356810 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:04.356773    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52113c58-dc00-4270-bc9d-58d2cd30b9b6-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "52113c58-dc00-4270-bc9d-58d2cd30b9b6" (UID: "52113c58-dc00-4270-bc9d-58d2cd30b9b6"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 00:53:04.359191 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:04.359165    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/52113c58-dc00-4270-bc9d-58d2cd30b9b6-cacerts" (OuterVolumeSpecName: "cacerts") pod "52113c58-dc00-4270-bc9d-58d2cd30b9b6" (UID: "52113c58-dc00-4270-bc9d-58d2cd30b9b6"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 00:53:04.359191 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:04.359175    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52113c58-dc00-4270-bc9d-58d2cd30b9b6-local-certs" (OuterVolumeSpecName: "local-certs") pod "52113c58-dc00-4270-bc9d-58d2cd30b9b6" (UID: "52113c58-dc00-4270-bc9d-58d2cd30b9b6"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 00:53:04.359311 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:04.359186    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/52113c58-dc00-4270-bc9d-58d2cd30b9b6-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "52113c58-dc00-4270-bc9d-58d2cd30b9b6" (UID: "52113c58-dc00-4270-bc9d-58d2cd30b9b6"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 00:53:04.359311 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:04.359230    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52113c58-dc00-4270-bc9d-58d2cd30b9b6-istio-token" (OuterVolumeSpecName: "istio-token") pod "52113c58-dc00-4270-bc9d-58d2cd30b9b6" (UID: "52113c58-dc00-4270-bc9d-58d2cd30b9b6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 00:53:04.359311 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:04.359251    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52113c58-dc00-4270-bc9d-58d2cd30b9b6-kube-api-access-kxvfr" (OuterVolumeSpecName: "kube-api-access-kxvfr") pod "52113c58-dc00-4270-bc9d-58d2cd30b9b6" (UID: "52113c58-dc00-4270-bc9d-58d2cd30b9b6"). InnerVolumeSpecName "kube-api-access-kxvfr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 00:53:04.359311 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:04.359261    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/52113c58-dc00-4270-bc9d-58d2cd30b9b6-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "52113c58-dc00-4270-bc9d-58d2cd30b9b6" (UID: "52113c58-dc00-4270-bc9d-58d2cd30b9b6"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 00:53:04.456855 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:04.456789    2556 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/52113c58-dc00-4270-bc9d-58d2cd30b9b6-istio-csr-dns-cert\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 00:53:04.456855 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:04.456815    2556 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/52113c58-dc00-4270-bc9d-58d2cd30b9b6-local-certs\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 00:53:04.456855 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:04.456824    2556 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/52113c58-dc00-4270-bc9d-58d2cd30b9b6-istio-kubeconfig\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 00:53:04.456855 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:04.456833    2556 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/52113c58-dc00-4270-bc9d-58d2cd30b9b6-istio-csr-ca-configmap\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 00:53:04.456855 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:04.456842    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kxvfr\" (UniqueName: \"kubernetes.io/projected/52113c58-dc00-4270-bc9d-58d2cd30b9b6-kube-api-access-kxvfr\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 00:53:04.456855 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:04.456850    2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52113c58-dc00-4270-bc9d-58d2cd30b9b6-istio-token\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 00:53:04.456855 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:04.456858    2556 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/52113c58-dc00-4270-bc9d-58d2cd30b9b6-cacerts\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 00:53:04.902237 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:04.902199    2556 generic.go:356] "Generic (PLEG): container finished" podID="52113c58-dc00-4270-bc9d-58d2cd30b9b6" containerID="c6f9a6c85ed6f58b1bf79ae31fb0bcf766c728f6cb7d166cba3b1731368aa604" exitCode=0
Sep 16 00:53:04.902608 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:04.902244    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sng4t"
Sep 16 00:53:04.902608 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:04.902290    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sng4t" event={"ID":"52113c58-dc00-4270-bc9d-58d2cd30b9b6","Type":"ContainerDied","Data":"c6f9a6c85ed6f58b1bf79ae31fb0bcf766c728f6cb7d166cba3b1731368aa604"}
Sep 16 00:53:04.902608 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:04.902337    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sng4t" event={"ID":"52113c58-dc00-4270-bc9d-58d2cd30b9b6","Type":"ContainerDied","Data":"37963abcc7c63c7ac18a1170e12907b2d9c48c4d2993d531570d0f4cde4100f8"}
Sep 16 00:53:04.902608 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:04.902356    2556 scope.go:117] "RemoveContainer" containerID="c6f9a6c85ed6f58b1bf79ae31fb0bcf766c728f6cb7d166cba3b1731368aa604"
Sep 16 00:53:04.915530 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:04.915506    2556 scope.go:117] "RemoveContainer" containerID="c6f9a6c85ed6f58b1bf79ae31fb0bcf766c728f6cb7d166cba3b1731368aa604"
Sep 16 00:53:04.915798 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:53:04.915770    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6f9a6c85ed6f58b1bf79ae31fb0bcf766c728f6cb7d166cba3b1731368aa604\": container with ID starting with c6f9a6c85ed6f58b1bf79ae31fb0bcf766c728f6cb7d166cba3b1731368aa604 not found: ID does not exist" containerID="c6f9a6c85ed6f58b1bf79ae31fb0bcf766c728f6cb7d166cba3b1731368aa604"
Sep 16 00:53:04.915841 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:04.915807    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6f9a6c85ed6f58b1bf79ae31fb0bcf766c728f6cb7d166cba3b1731368aa604"} err="failed to get container status \"c6f9a6c85ed6f58b1bf79ae31fb0bcf766c728f6cb7d166cba3b1731368aa604\": rpc error: code = NotFound desc = could not find container \"c6f9a6c85ed6f58b1bf79ae31fb0bcf766c728f6cb7d166cba3b1731368aa604\": container with ID starting with c6f9a6c85ed6f58b1bf79ae31fb0bcf766c728f6cb7d166cba3b1731368aa604 not found: ID does not exist"
Sep 16 00:53:04.939280 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:04.939256    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sng4t"]
Sep 16 00:53:04.948234 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:04.948208    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-sng4t"]
Sep 16 00:53:05.734655 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:05.734631    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52113c58-dc00-4270-bc9d-58d2cd30b9b6" path="/var/lib/kubelet/pods/52113c58-dc00-4270-bc9d-58d2cd30b9b6/volumes"
Sep 16 00:53:20.494809 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:20.494747    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7cd4c5986d-zsddx"]
Sep 16 00:53:20.579991 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:20.495049    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52113c58-dc00-4270-bc9d-58d2cd30b9b6" containerName="discovery"
Sep 16 00:53:20.579991 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:20.495061    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="52113c58-dc00-4270-bc9d-58d2cd30b9b6" containerName="discovery"
Sep 16 00:53:20.579991 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:20.495100    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="52113c58-dc00-4270-bc9d-58d2cd30b9b6" containerName="discovery"
Sep 16 00:53:20.586430 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:20.586406    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7cd4c5986d-zsddx"]
Sep 16 00:53:20.586537 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:20.586525    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cd4c5986d-zsddx"
Sep 16 00:53:20.589134 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:20.589105    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 16 00:53:20.589341 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:20.589329    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 16 00:53:20.589668 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:20.589649    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 16 00:53:20.589744 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:20.589651    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 16 00:53:20.589975 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:20.589959    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 16 00:53:20.590285 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:20.590271    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 16 00:53:20.590692 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:20.590676    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-m679f\""
Sep 16 00:53:20.590779 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:20.590696    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 16 00:53:20.594093 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:20.594075    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 16 00:53:20.664139 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:20.664103    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c02cd606-1a1a-4789-aeee-4c831943babe-oauth-serving-cert\") pod \"console-7cd4c5986d-zsddx\" (UID: \"c02cd606-1a1a-4789-aeee-4c831943babe\") " pod="openshift-console/console-7cd4c5986d-zsddx"
Sep 16 00:53:20.664139 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:20.664141    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c02cd606-1a1a-4789-aeee-4c831943babe-console-config\") pod \"console-7cd4c5986d-zsddx\" (UID: \"c02cd606-1a1a-4789-aeee-4c831943babe\") " pod="openshift-console/console-7cd4c5986d-zsddx"
Sep 16 00:53:20.664311 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:20.664165    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qx4dz\" (UniqueName: \"kubernetes.io/projected/c02cd606-1a1a-4789-aeee-4c831943babe-kube-api-access-qx4dz\") pod \"console-7cd4c5986d-zsddx\" (UID: \"c02cd606-1a1a-4789-aeee-4c831943babe\") " pod="openshift-console/console-7cd4c5986d-zsddx"
Sep 16 00:53:20.664311 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:20.664207    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c02cd606-1a1a-4789-aeee-4c831943babe-console-oauth-config\") pod \"console-7cd4c5986d-zsddx\" (UID: \"c02cd606-1a1a-4789-aeee-4c831943babe\") " pod="openshift-console/console-7cd4c5986d-zsddx"
Sep 16 00:53:20.664311 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:20.664254    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c02cd606-1a1a-4789-aeee-4c831943babe-trusted-ca-bundle\") pod \"console-7cd4c5986d-zsddx\" (UID: \"c02cd606-1a1a-4789-aeee-4c831943babe\") " pod="openshift-console/console-7cd4c5986d-zsddx"
Sep 16 00:53:20.664311 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:20.664281    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c02cd606-1a1a-4789-aeee-4c831943babe-console-serving-cert\") pod \"console-7cd4c5986d-zsddx\" (UID: \"c02cd606-1a1a-4789-aeee-4c831943babe\") " pod="openshift-console/console-7cd4c5986d-zsddx"
Sep 16 00:53:20.664311 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:20.664295    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c02cd606-1a1a-4789-aeee-4c831943babe-service-ca\") pod \"console-7cd4c5986d-zsddx\" (UID: \"c02cd606-1a1a-4789-aeee-4c831943babe\") " pod="openshift-console/console-7cd4c5986d-zsddx"
Sep 16 00:53:20.765581 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:20.765513    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c02cd606-1a1a-4789-aeee-4c831943babe-console-serving-cert\") pod \"console-7cd4c5986d-zsddx\" (UID: \"c02cd606-1a1a-4789-aeee-4c831943babe\") " pod="openshift-console/console-7cd4c5986d-zsddx"
Sep 16 00:53:20.765581 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:20.765548    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c02cd606-1a1a-4789-aeee-4c831943babe-service-ca\") pod \"console-7cd4c5986d-zsddx\" (UID: \"c02cd606-1a1a-4789-aeee-4c831943babe\") " pod="openshift-console/console-7cd4c5986d-zsddx"
Sep 16 00:53:20.765745 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:20.765599    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c02cd606-1a1a-4789-aeee-4c831943babe-oauth-serving-cert\") pod \"console-7cd4c5986d-zsddx\" (UID: \"c02cd606-1a1a-4789-aeee-4c831943babe\") " pod="openshift-console/console-7cd4c5986d-zsddx"
Sep 16 00:53:20.765745 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:20.765621    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c02cd606-1a1a-4789-aeee-4c831943babe-console-config\") pod \"console-7cd4c5986d-zsddx\" (UID: \"c02cd606-1a1a-4789-aeee-4c831943babe\") " pod="openshift-console/console-7cd4c5986d-zsddx"
Sep 16 00:53:20.765745 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:20.765645    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qx4dz\" (UniqueName: \"kubernetes.io/projected/c02cd606-1a1a-4789-aeee-4c831943babe-kube-api-access-qx4dz\") pod \"console-7cd4c5986d-zsddx\" (UID: \"c02cd606-1a1a-4789-aeee-4c831943babe\") " pod="openshift-console/console-7cd4c5986d-zsddx"
Sep 16 00:53:20.765886 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:20.765796    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c02cd606-1a1a-4789-aeee-4c831943babe-console-oauth-config\") pod \"console-7cd4c5986d-zsddx\" (UID: \"c02cd606-1a1a-4789-aeee-4c831943babe\") " pod="openshift-console/console-7cd4c5986d-zsddx"
Sep 16 00:53:20.765938 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:20.765894    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c02cd606-1a1a-4789-aeee-4c831943babe-trusted-ca-bundle\") pod \"console-7cd4c5986d-zsddx\" (UID: \"c02cd606-1a1a-4789-aeee-4c831943babe\") " pod="openshift-console/console-7cd4c5986d-zsddx"
Sep 16 00:53:20.766895 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:20.766808    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c02cd606-1a1a-4789-aeee-4c831943babe-service-ca\") pod \"console-7cd4c5986d-zsddx\" (UID: \"c02cd606-1a1a-4789-aeee-4c831943babe\") " pod="openshift-console/console-7cd4c5986d-zsddx"
Sep 16 00:53:20.767019 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:20.766935    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c02cd606-1a1a-4789-aeee-4c831943babe-console-config\") pod \"console-7cd4c5986d-zsddx\" (UID: \"c02cd606-1a1a-4789-aeee-4c831943babe\") " pod="openshift-console/console-7cd4c5986d-zsddx"
Sep 16 00:53:20.767140 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:20.767117    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c02cd606-1a1a-4789-aeee-4c831943babe-trusted-ca-bundle\") pod \"console-7cd4c5986d-zsddx\" (UID: \"c02cd606-1a1a-4789-aeee-4c831943babe\") " pod="openshift-console/console-7cd4c5986d-zsddx"
Sep 16 00:53:20.767624 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:20.767601    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c02cd606-1a1a-4789-aeee-4c831943babe-oauth-serving-cert\") pod \"console-7cd4c5986d-zsddx\" (UID: \"c02cd606-1a1a-4789-aeee-4c831943babe\") " pod="openshift-console/console-7cd4c5986d-zsddx"
Sep 16 00:53:20.769341 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:20.769313    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c02cd606-1a1a-4789-aeee-4c831943babe-console-serving-cert\") pod \"console-7cd4c5986d-zsddx\" (UID: \"c02cd606-1a1a-4789-aeee-4c831943babe\") " pod="openshift-console/console-7cd4c5986d-zsddx"
Sep 16 00:53:20.773356 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:20.773328    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c02cd606-1a1a-4789-aeee-4c831943babe-console-oauth-config\") pod \"console-7cd4c5986d-zsddx\" (UID: \"c02cd606-1a1a-4789-aeee-4c831943babe\") " pod="openshift-console/console-7cd4c5986d-zsddx"
Sep 16 00:53:20.773608 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:20.773587    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qx4dz\" (UniqueName: \"kubernetes.io/projected/c02cd606-1a1a-4789-aeee-4c831943babe-kube-api-access-qx4dz\") pod \"console-7cd4c5986d-zsddx\" (UID: \"c02cd606-1a1a-4789-aeee-4c831943babe\") " pod="openshift-console/console-7cd4c5986d-zsddx"
Sep 16 00:53:20.895621 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:20.895591    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cd4c5986d-zsddx"
Sep 16 00:53:21.192717 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:21.192695    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7cd4c5986d-zsddx"]
Sep 16 00:53:21.194649 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:53:21.194623    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc02cd606_1a1a_4789_aeee_4c831943babe.slice/crio-3f653c59e67548a6866444d297f169c50b82a4b51aa1636cb4d58a5620175c68 WatchSource:0}: Error finding container 3f653c59e67548a6866444d297f169c50b82a4b51aa1636cb4d58a5620175c68: Status 404 returned error can't find the container with id 3f653c59e67548a6866444d297f169c50b82a4b51aa1636cb4d58a5620175c68
Sep 16 00:53:21.970202 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:21.970165    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cd4c5986d-zsddx" event={"ID":"c02cd606-1a1a-4789-aeee-4c831943babe","Type":"ContainerStarted","Data":"f9c10a42e93c0dad79c1716db7dceece5bf61c656d692e7dbf767342cd790175"}
Sep 16 00:53:21.970604 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:21.970225    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cd4c5986d-zsddx" event={"ID":"c02cd606-1a1a-4789-aeee-4c831943babe","Type":"ContainerStarted","Data":"3f653c59e67548a6866444d297f169c50b82a4b51aa1636cb4d58a5620175c68"}
Sep 16 00:53:21.986641 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:21.986596    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7cd4c5986d-zsddx" podStartSLOduration=1.986580573 podStartE2EDuration="1.986580573s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 00:53:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 00:53:21.985344967 +0000 UTC m=+616.766846558" watchObservedRunningTime="2026-09-16 00:53:21.986580573 +0000 UTC m=+616.768082164"
Sep 16 00:53:30.895983 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:30.895935    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7cd4c5986d-zsddx"
Sep 16 00:53:30.896423 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:30.896009    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7cd4c5986d-zsddx"
Sep 16 00:53:30.900630 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:30.900606    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7cd4c5986d-zsddx"
Sep 16 00:53:31.006979 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:31.006950    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7cd4c5986d-zsddx"
Sep 16 00:53:57.355761 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:57.355724    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7r6fl7"]
Sep 16 00:53:57.359290 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:57.359274    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7r6fl7"
Sep 16 00:53:57.361554 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:57.361527    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 16 00:53:57.361554 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:57.361547    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 16 00:53:57.362328 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:57.362313    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-t4f99\""
Sep 16 00:53:57.366311 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:57.366273    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7r6fl7"]
Sep 16 00:53:57.412420 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:57.412389    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6kd47\" (UniqueName: \"kubernetes.io/projected/3e4a8031-9cc8-423a-b85c-9f0b982bc57e-kube-api-access-6kd47\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7r6fl7\" (UID: \"3e4a8031-9cc8-423a-b85c-9f0b982bc57e\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7r6fl7"
Sep 16 00:53:57.412549 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:57.412436    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3e4a8031-9cc8-423a-b85c-9f0b982bc57e-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7r6fl7\" (UID: \"3e4a8031-9cc8-423a-b85c-9f0b982bc57e\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7r6fl7"
Sep 16 00:53:57.412549 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:57.412481    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3e4a8031-9cc8-423a-b85c-9f0b982bc57e-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7r6fl7\" (UID: \"3e4a8031-9cc8-423a-b85c-9f0b982bc57e\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7r6fl7"
Sep 16 00:53:57.457098 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:57.457073    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6kzk7l"]
Sep 16 00:53:57.462370 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:57.462335    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6kzk7l"
Sep 16 00:53:57.468390 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:57.468370    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6kzk7l"]
Sep 16 00:53:57.513672 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:57.513642    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3e4a8031-9cc8-423a-b85c-9f0b982bc57e-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7r6fl7\" (UID: \"3e4a8031-9cc8-423a-b85c-9f0b982bc57e\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7r6fl7"
Sep 16 00:53:57.513803 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:57.513686    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3e4a8031-9cc8-423a-b85c-9f0b982bc57e-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7r6fl7\" (UID: \"3e4a8031-9cc8-423a-b85c-9f0b982bc57e\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7r6fl7"
Sep 16 00:53:57.513803 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:57.513712    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a97e27e1-2ee0-4e9a-b0a0-e9846c47aca3-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6kzk7l\" (UID: \"a97e27e1-2ee0-4e9a-b0a0-e9846c47aca3\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6kzk7l"
Sep 16 00:53:57.513803 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:57.513729    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a97e27e1-2ee0-4e9a-b0a0-e9846c47aca3-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6kzk7l\" (UID: \"a97e27e1-2ee0-4e9a-b0a0-e9846c47aca3\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6kzk7l"
Sep 16 00:53:57.513803 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:57.513758    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6kd47\" (UniqueName: \"kubernetes.io/projected/3e4a8031-9cc8-423a-b85c-9f0b982bc57e-kube-api-access-6kd47\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7r6fl7\" (UID: \"3e4a8031-9cc8-423a-b85c-9f0b982bc57e\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7r6fl7"
Sep 16 00:53:57.513803 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:57.513785    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p757x\" (UniqueName: \"kubernetes.io/projected/a97e27e1-2ee0-4e9a-b0a0-e9846c47aca3-kube-api-access-p757x\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6kzk7l\" (UID: \"a97e27e1-2ee0-4e9a-b0a0-e9846c47aca3\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6kzk7l"
Sep 16 00:53:57.514091 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:57.514071    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3e4a8031-9cc8-423a-b85c-9f0b982bc57e-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7r6fl7\" (UID: \"3e4a8031-9cc8-423a-b85c-9f0b982bc57e\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7r6fl7"
Sep 16 00:53:57.514144 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:57.514086    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3e4a8031-9cc8-423a-b85c-9f0b982bc57e-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7r6fl7\" (UID: \"3e4a8031-9cc8-423a-b85c-9f0b982bc57e\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7r6fl7"
Sep 16 00:53:57.521527 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:57.521508    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6kd47\" (UniqueName: \"kubernetes.io/projected/3e4a8031-9cc8-423a-b85c-9f0b982bc57e-kube-api-access-6kd47\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7r6fl7\" (UID: \"3e4a8031-9cc8-423a-b85c-9f0b982bc57e\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7r6fl7"
Sep 16 00:53:57.554505 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:57.554483    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqx9ps"]
Sep 16 00:53:57.557667 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:57.557652    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqx9ps"
Sep 16 00:53:57.566372 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:57.566350    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqx9ps"]
Sep 16 00:53:57.614865 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:57.614820    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2xb6\" (UniqueName: \"kubernetes.io/projected/ea985665-c2f8-4e5f-a899-f3ffff95f5e1-kube-api-access-v2xb6\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqx9ps\" (UID: \"ea985665-c2f8-4e5f-a899-f3ffff95f5e1\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqx9ps"
Sep 16 00:53:57.614865 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:57.614853    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ea985665-c2f8-4e5f-a899-f3ffff95f5e1-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqx9ps\" (UID: \"ea985665-c2f8-4e5f-a899-f3ffff95f5e1\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqx9ps"
Sep 16 00:53:57.614987 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:57.614886    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p757x\" (UniqueName: \"kubernetes.io/projected/a97e27e1-2ee0-4e9a-b0a0-e9846c47aca3-kube-api-access-p757x\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6kzk7l\" (UID: \"a97e27e1-2ee0-4e9a-b0a0-e9846c47aca3\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6kzk7l"
Sep 16 00:53:57.614987 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:57.614941    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ea985665-c2f8-4e5f-a899-f3ffff95f5e1-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqx9ps\" (UID: \"ea985665-c2f8-4e5f-a899-f3ffff95f5e1\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqx9ps"
Sep 16 00:53:57.614987 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:57.614975    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a97e27e1-2ee0-4e9a-b0a0-e9846c47aca3-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6kzk7l\" (UID: \"a97e27e1-2ee0-4e9a-b0a0-e9846c47aca3\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6kzk7l"
Sep 16 00:53:57.615082 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:57.614992    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a97e27e1-2ee0-4e9a-b0a0-e9846c47aca3-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6kzk7l\" (UID: \"a97e27e1-2ee0-4e9a-b0a0-e9846c47aca3\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6kzk7l"
Sep 16 00:53:57.615359 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:57.615339    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a97e27e1-2ee0-4e9a-b0a0-e9846c47aca3-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6kzk7l\" (UID: \"a97e27e1-2ee0-4e9a-b0a0-e9846c47aca3\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6kzk7l"
Sep 16 00:53:57.615412 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:57.615379    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a97e27e1-2ee0-4e9a-b0a0-e9846c47aca3-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6kzk7l\" (UID: \"a97e27e1-2ee0-4e9a-b0a0-e9846c47aca3\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6kzk7l"
Sep 16 00:53:57.622673 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:57.622653    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p757x\" (UniqueName: \"kubernetes.io/projected/a97e27e1-2ee0-4e9a-b0a0-e9846c47aca3-kube-api-access-p757x\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6kzk7l\" (UID: \"a97e27e1-2ee0-4e9a-b0a0-e9846c47aca3\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6kzk7l"
Sep 16 00:53:57.656085 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:57.656060    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a9fmcb"]
Sep 16 00:53:57.659284 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:57.659270    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a9fmcb"
Sep 16 00:53:57.665839 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:57.665818    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a9fmcb"]
Sep 16 00:53:57.669690 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:57.669672    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7r6fl7"
Sep 16 00:53:57.716327 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:57.715746    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ea985665-c2f8-4e5f-a899-f3ffff95f5e1-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqx9ps\" (UID: \"ea985665-c2f8-4e5f-a899-f3ffff95f5e1\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqx9ps"
Sep 16 00:53:57.716327 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:57.715839    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/64ee0f9c-d6bb-4a6b-a50b-231e6e1c106d-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a9fmcb\" (UID: \"64ee0f9c-d6bb-4a6b-a50b-231e6e1c106d\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a9fmcb"
Sep 16 00:53:57.716327 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:57.715886    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ea985665-c2f8-4e5f-a899-f3ffff95f5e1-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqx9ps\" (UID: \"ea985665-c2f8-4e5f-a899-f3ffff95f5e1\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqx9ps"
Sep 16 00:53:57.716327 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:57.715948    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/64ee0f9c-d6bb-4a6b-a50b-231e6e1c106d-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a9fmcb\" (UID: \"64ee0f9c-d6bb-4a6b-a50b-231e6e1c106d\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a9fmcb"
Sep 16 00:53:57.716327 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:57.715981    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2xb6\" (UniqueName: \"kubernetes.io/projected/ea985665-c2f8-4e5f-a899-f3ffff95f5e1-kube-api-access-v2xb6\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqx9ps\" (UID: \"ea985665-c2f8-4e5f-a899-f3ffff95f5e1\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqx9ps"
Sep 16 00:53:57.716327 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:57.716011    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxhbf\" (UniqueName: \"kubernetes.io/projected/64ee0f9c-d6bb-4a6b-a50b-231e6e1c106d-kube-api-access-hxhbf\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a9fmcb\" (UID: \"64ee0f9c-d6bb-4a6b-a50b-231e6e1c106d\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a9fmcb"
Sep 16 00:53:57.716831 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:57.716456    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ea985665-c2f8-4e5f-a899-f3ffff95f5e1-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqx9ps\" (UID: \"ea985665-c2f8-4e5f-a899-f3ffff95f5e1\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqx9ps"
Sep 16 00:53:57.716831 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:57.716579    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ea985665-c2f8-4e5f-a899-f3ffff95f5e1-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqx9ps\" (UID: \"ea985665-c2f8-4e5f-a899-f3ffff95f5e1\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqx9ps"
Sep 16 00:53:57.725609 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:57.725583    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2xb6\" (UniqueName: \"kubernetes.io/projected/ea985665-c2f8-4e5f-a899-f3ffff95f5e1-kube-api-access-v2xb6\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqx9ps\" (UID: \"ea985665-c2f8-4e5f-a899-f3ffff95f5e1\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqx9ps"
Sep 16 00:53:57.771095 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:57.771073    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6kzk7l"
Sep 16 00:53:57.817538 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:57.817400    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/64ee0f9c-d6bb-4a6b-a50b-231e6e1c106d-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a9fmcb\" (UID: \"64ee0f9c-d6bb-4a6b-a50b-231e6e1c106d\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a9fmcb"
Sep 16 00:53:57.817538 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:57.817509    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/64ee0f9c-d6bb-4a6b-a50b-231e6e1c106d-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a9fmcb\" (UID: \"64ee0f9c-d6bb-4a6b-a50b-231e6e1c106d\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a9fmcb"
Sep 16 00:53:57.817665 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:57.817549    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxhbf\" (UniqueName: \"kubernetes.io/projected/64ee0f9c-d6bb-4a6b-a50b-231e6e1c106d-kube-api-access-hxhbf\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a9fmcb\" (UID: \"64ee0f9c-d6bb-4a6b-a50b-231e6e1c106d\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a9fmcb"
Sep 16 00:53:57.817964 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:57.817941    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/64ee0f9c-d6bb-4a6b-a50b-231e6e1c106d-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a9fmcb\" (UID: \"64ee0f9c-d6bb-4a6b-a50b-231e6e1c106d\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a9fmcb"
Sep 16 00:53:57.818156 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:57.818135    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/64ee0f9c-d6bb-4a6b-a50b-231e6e1c106d-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a9fmcb\" (UID: \"64ee0f9c-d6bb-4a6b-a50b-231e6e1c106d\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a9fmcb"
Sep 16 00:53:57.827037 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:57.827009    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxhbf\" (UniqueName: \"kubernetes.io/projected/64ee0f9c-d6bb-4a6b-a50b-231e6e1c106d-kube-api-access-hxhbf\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a9fmcb\" (UID: \"64ee0f9c-d6bb-4a6b-a50b-231e6e1c106d\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a9fmcb"
Sep 16 00:53:57.867678 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:57.867622    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqx9ps"
Sep 16 00:53:57.939377 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:57.939352    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7r6fl7"]
Sep 16 00:53:57.941506 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:53:57.941481    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e4a8031_9cc8_423a_b85c_9f0b982bc57e.slice/crio-811c3825c8f954d8d85f89dac335076d17f7656a728d68084deaecd44b151ee3 WatchSource:0}: Error finding container 811c3825c8f954d8d85f89dac335076d17f7656a728d68084deaecd44b151ee3: Status 404 returned error can't find the container with id 811c3825c8f954d8d85f89dac335076d17f7656a728d68084deaecd44b151ee3
Sep 16 00:53:57.968797 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:57.968686    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a9fmcb"
Sep 16 00:53:58.041116 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:58.041092    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6kzk7l"]
Sep 16 00:53:58.084193 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:53:58.084169    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda97e27e1_2ee0_4e9a_b0a0_e9846c47aca3.slice/crio-79be575ee6c3d604c7add42d18bca48eafc88e7f4c1ffc41753378cb0b42f66b WatchSource:0}: Error finding container 79be575ee6c3d604c7add42d18bca48eafc88e7f4c1ffc41753378cb0b42f66b: Status 404 returned error can't find the container with id 79be575ee6c3d604c7add42d18bca48eafc88e7f4c1ffc41753378cb0b42f66b
Sep 16 00:53:58.111715 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:58.111691    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6kzk7l" event={"ID":"a97e27e1-2ee0-4e9a-b0a0-e9846c47aca3","Type":"ContainerStarted","Data":"79be575ee6c3d604c7add42d18bca48eafc88e7f4c1ffc41753378cb0b42f66b"}
Sep 16 00:53:58.113033 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:58.113011    2556 generic.go:356] "Generic (PLEG): container finished" podID="3e4a8031-9cc8-423a-b85c-9f0b982bc57e" containerID="624c0f4079a550e6cf1dedd5ebfca36e4b07271f99e4fdd935c0523b8795e00d" exitCode=0
Sep 16 00:53:58.113109 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:58.113049    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7r6fl7" event={"ID":"3e4a8031-9cc8-423a-b85c-9f0b982bc57e","Type":"ContainerDied","Data":"624c0f4079a550e6cf1dedd5ebfca36e4b07271f99e4fdd935c0523b8795e00d"}
Sep 16 00:53:58.113109 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:58.113080    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7r6fl7" event={"ID":"3e4a8031-9cc8-423a-b85c-9f0b982bc57e","Type":"ContainerStarted","Data":"811c3825c8f954d8d85f89dac335076d17f7656a728d68084deaecd44b151ee3"}
Sep 16 00:53:58.113954 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:58.113929    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 00:53:58.134491 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:58.134403    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqx9ps"]
Sep 16 00:53:58.136609 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:53:58.136588    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea985665_c2f8_4e5f_a899_f3ffff95f5e1.slice/crio-4b3d1b4180496657b9b96b8ad9f09389a3d12289c8d2ea7f250bf85392158d56 WatchSource:0}: Error finding container 4b3d1b4180496657b9b96b8ad9f09389a3d12289c8d2ea7f250bf85392158d56: Status 404 returned error can't find the container with id 4b3d1b4180496657b9b96b8ad9f09389a3d12289c8d2ea7f250bf85392158d56
Sep 16 00:53:58.236418 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:58.236394    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a9fmcb"]
Sep 16 00:53:58.284286 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:53:58.284256    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64ee0f9c_d6bb_4a6b_a50b_231e6e1c106d.slice/crio-bba5275a76e07847997dbf522c51622ddb79e0acd1934da52443675808bf1316 WatchSource:0}: Error finding container bba5275a76e07847997dbf522c51622ddb79e0acd1934da52443675808bf1316: Status 404 returned error can't find the container with id bba5275a76e07847997dbf522c51622ddb79e0acd1934da52443675808bf1316
Sep 16 00:53:59.118720 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:59.118688    2556 generic.go:356] "Generic (PLEG): container finished" podID="a97e27e1-2ee0-4e9a-b0a0-e9846c47aca3" containerID="56e697642f3b8f893abcf72d65629de01fdba82e0f1c1472288004c575999e18" exitCode=0
Sep 16 00:53:59.118997 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:59.118766    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6kzk7l" event={"ID":"a97e27e1-2ee0-4e9a-b0a0-e9846c47aca3","Type":"ContainerDied","Data":"56e697642f3b8f893abcf72d65629de01fdba82e0f1c1472288004c575999e18"}
Sep 16 00:53:59.120763 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:59.120740    2556 generic.go:356] "Generic (PLEG): container finished" podID="3e4a8031-9cc8-423a-b85c-9f0b982bc57e" containerID="6f17e39eafaca20d42faa11a255aa1cecfbaeb4446b1b07c4e18341f8bc8a404" exitCode=0
Sep 16 00:53:59.120878 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:59.120852    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7r6fl7" event={"ID":"3e4a8031-9cc8-423a-b85c-9f0b982bc57e","Type":"ContainerDied","Data":"6f17e39eafaca20d42faa11a255aa1cecfbaeb4446b1b07c4e18341f8bc8a404"}
Sep 16 00:53:59.122357 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:59.122335    2556 generic.go:356] "Generic (PLEG): container finished" podID="64ee0f9c-d6bb-4a6b-a50b-231e6e1c106d" containerID="531b6ddc7b524b8d32b8abbd046a9d861849e3e3f65f0c6141fe806d96f269dd" exitCode=0
Sep 16 00:53:59.122482 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:59.122409    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a9fmcb" event={"ID":"64ee0f9c-d6bb-4a6b-a50b-231e6e1c106d","Type":"ContainerDied","Data":"531b6ddc7b524b8d32b8abbd046a9d861849e3e3f65f0c6141fe806d96f269dd"}
Sep 16 00:53:59.122482 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:59.122454    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a9fmcb" event={"ID":"64ee0f9c-d6bb-4a6b-a50b-231e6e1c106d","Type":"ContainerStarted","Data":"bba5275a76e07847997dbf522c51622ddb79e0acd1934da52443675808bf1316"}
Sep 16 00:53:59.124049 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:59.124030    2556 generic.go:356] "Generic (PLEG): container finished" podID="ea985665-c2f8-4e5f-a899-f3ffff95f5e1" containerID="5dc922613e09ad27be01c25d5ba20b55c802c464ac31cc2531a1a33afb7adf89" exitCode=0
Sep 16 00:53:59.124155 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:59.124083    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqx9ps" event={"ID":"ea985665-c2f8-4e5f-a899-f3ffff95f5e1","Type":"ContainerDied","Data":"5dc922613e09ad27be01c25d5ba20b55c802c464ac31cc2531a1a33afb7adf89"}
Sep 16 00:53:59.124155 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:53:59.124125    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqx9ps" event={"ID":"ea985665-c2f8-4e5f-a899-f3ffff95f5e1","Type":"ContainerStarted","Data":"4b3d1b4180496657b9b96b8ad9f09389a3d12289c8d2ea7f250bf85392158d56"}
Sep 16 00:54:00.130493 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:00.130449    2556 generic.go:356] "Generic (PLEG): container finished" podID="3e4a8031-9cc8-423a-b85c-9f0b982bc57e" containerID="b6b57733d7089aa12a851803f152cb2ad358d693eb2c6364341b3993757782c4" exitCode=0
Sep 16 00:54:00.130914 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:00.130545    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7r6fl7" event={"ID":"3e4a8031-9cc8-423a-b85c-9f0b982bc57e","Type":"ContainerDied","Data":"b6b57733d7089aa12a851803f152cb2ad358d693eb2c6364341b3993757782c4"}
Sep 16 00:54:00.132327 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:00.132306    2556 generic.go:356] "Generic (PLEG): container finished" podID="64ee0f9c-d6bb-4a6b-a50b-231e6e1c106d" containerID="ee2eb779e6abec898c3bc97b5068c73fd387ae7e9ea91d5f61cf7d583123f01f" exitCode=0
Sep 16 00:54:00.132443 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:00.132369    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a9fmcb" event={"ID":"64ee0f9c-d6bb-4a6b-a50b-231e6e1c106d","Type":"ContainerDied","Data":"ee2eb779e6abec898c3bc97b5068c73fd387ae7e9ea91d5f61cf7d583123f01f"}
Sep 16 00:54:01.139019 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:01.138987    2556 generic.go:356] "Generic (PLEG): container finished" podID="64ee0f9c-d6bb-4a6b-a50b-231e6e1c106d" containerID="973b6d07b5ac41de9d335ec92abb46f6885be72b4228f2cf9811d68413b830e5" exitCode=0
Sep 16 00:54:01.139387 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:01.139077    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a9fmcb" event={"ID":"64ee0f9c-d6bb-4a6b-a50b-231e6e1c106d","Type":"ContainerDied","Data":"973b6d07b5ac41de9d335ec92abb46f6885be72b4228f2cf9811d68413b830e5"}
Sep 16 00:54:01.140880 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:01.140855    2556 generic.go:356] "Generic (PLEG): container finished" podID="ea985665-c2f8-4e5f-a899-f3ffff95f5e1" containerID="1b65ea52b04d0b5d8eb6e9fc19a844491869c04197d941a31f039d551063339a" exitCode=0
Sep 16 00:54:01.140995 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:01.140938    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqx9ps" event={"ID":"ea985665-c2f8-4e5f-a899-f3ffff95f5e1","Type":"ContainerDied","Data":"1b65ea52b04d0b5d8eb6e9fc19a844491869c04197d941a31f039d551063339a"}
Sep 16 00:54:01.142932 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:01.142907    2556 generic.go:356] "Generic (PLEG): container finished" podID="a97e27e1-2ee0-4e9a-b0a0-e9846c47aca3" containerID="38a795a0f335f0b214450398780bc1fa507b6cdb3c5d8f3ca8a6dd83c1bb1c81" exitCode=0
Sep 16 00:54:01.143010 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:01.142991    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6kzk7l" event={"ID":"a97e27e1-2ee0-4e9a-b0a0-e9846c47aca3","Type":"ContainerDied","Data":"38a795a0f335f0b214450398780bc1fa507b6cdb3c5d8f3ca8a6dd83c1bb1c81"}
Sep 16 00:54:01.272556 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:01.272280    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7r6fl7"
Sep 16 00:54:01.346758 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:01.346731    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3e4a8031-9cc8-423a-b85c-9f0b982bc57e-bundle\") pod \"3e4a8031-9cc8-423a-b85c-9f0b982bc57e\" (UID: \"3e4a8031-9cc8-423a-b85c-9f0b982bc57e\") "
Sep 16 00:54:01.346875 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:01.346808    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6kd47\" (UniqueName: \"kubernetes.io/projected/3e4a8031-9cc8-423a-b85c-9f0b982bc57e-kube-api-access-6kd47\") pod \"3e4a8031-9cc8-423a-b85c-9f0b982bc57e\" (UID: \"3e4a8031-9cc8-423a-b85c-9f0b982bc57e\") "
Sep 16 00:54:01.346875 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:01.346850    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3e4a8031-9cc8-423a-b85c-9f0b982bc57e-util\") pod \"3e4a8031-9cc8-423a-b85c-9f0b982bc57e\" (UID: \"3e4a8031-9cc8-423a-b85c-9f0b982bc57e\") "
Sep 16 00:54:01.347212 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:01.347186    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e4a8031-9cc8-423a-b85c-9f0b982bc57e-bundle" (OuterVolumeSpecName: "bundle") pod "3e4a8031-9cc8-423a-b85c-9f0b982bc57e" (UID: "3e4a8031-9cc8-423a-b85c-9f0b982bc57e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 00:54:01.348902 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:01.348883    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e4a8031-9cc8-423a-b85c-9f0b982bc57e-kube-api-access-6kd47" (OuterVolumeSpecName: "kube-api-access-6kd47") pod "3e4a8031-9cc8-423a-b85c-9f0b982bc57e" (UID: "3e4a8031-9cc8-423a-b85c-9f0b982bc57e"). InnerVolumeSpecName "kube-api-access-6kd47". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 00:54:01.351987 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:01.351966    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e4a8031-9cc8-423a-b85c-9f0b982bc57e-util" (OuterVolumeSpecName: "util") pod "3e4a8031-9cc8-423a-b85c-9f0b982bc57e" (UID: "3e4a8031-9cc8-423a-b85c-9f0b982bc57e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 00:54:01.447342 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:01.447291    2556 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3e4a8031-9cc8-423a-b85c-9f0b982bc57e-bundle\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 00:54:01.447342 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:01.447314    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6kd47\" (UniqueName: \"kubernetes.io/projected/3e4a8031-9cc8-423a-b85c-9f0b982bc57e-kube-api-access-6kd47\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 00:54:01.447342 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:01.447324    2556 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3e4a8031-9cc8-423a-b85c-9f0b982bc57e-util\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 00:54:02.149280 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:02.149243    2556 generic.go:356] "Generic (PLEG): container finished" podID="a97e27e1-2ee0-4e9a-b0a0-e9846c47aca3" containerID="57a818268feb904f449353bd08e646a404077f7592e145627ff4ff8c44fbcd92" exitCode=0
Sep 16 00:54:02.149689 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:02.149329    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6kzk7l" event={"ID":"a97e27e1-2ee0-4e9a-b0a0-e9846c47aca3","Type":"ContainerDied","Data":"57a818268feb904f449353bd08e646a404077f7592e145627ff4ff8c44fbcd92"}
Sep 16 00:54:02.151067 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:02.151052    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7r6fl7"
Sep 16 00:54:02.151173 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:02.151066    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7r6fl7" event={"ID":"3e4a8031-9cc8-423a-b85c-9f0b982bc57e","Type":"ContainerDied","Data":"811c3825c8f954d8d85f89dac335076d17f7656a728d68084deaecd44b151ee3"}
Sep 16 00:54:02.151173 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:02.151097    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="811c3825c8f954d8d85f89dac335076d17f7656a728d68084deaecd44b151ee3"
Sep 16 00:54:02.153415 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:02.153389    2556 generic.go:356] "Generic (PLEG): container finished" podID="ea985665-c2f8-4e5f-a899-f3ffff95f5e1" containerID="28835665b6bacef570f69b4e97f53a42193959e62f144ec22033de353c728abb" exitCode=0
Sep 16 00:54:02.153535 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:02.153428    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqx9ps" event={"ID":"ea985665-c2f8-4e5f-a899-f3ffff95f5e1","Type":"ContainerDied","Data":"28835665b6bacef570f69b4e97f53a42193959e62f144ec22033de353c728abb"}
Sep 16 00:54:02.286273 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:02.286251    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a9fmcb"
Sep 16 00:54:02.355104 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:02.355078    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hxhbf\" (UniqueName: \"kubernetes.io/projected/64ee0f9c-d6bb-4a6b-a50b-231e6e1c106d-kube-api-access-hxhbf\") pod \"64ee0f9c-d6bb-4a6b-a50b-231e6e1c106d\" (UID: \"64ee0f9c-d6bb-4a6b-a50b-231e6e1c106d\") "
Sep 16 00:54:02.355258 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:02.355145    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/64ee0f9c-d6bb-4a6b-a50b-231e6e1c106d-bundle\") pod \"64ee0f9c-d6bb-4a6b-a50b-231e6e1c106d\" (UID: \"64ee0f9c-d6bb-4a6b-a50b-231e6e1c106d\") "
Sep 16 00:54:02.355258 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:02.355169    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/64ee0f9c-d6bb-4a6b-a50b-231e6e1c106d-util\") pod \"64ee0f9c-d6bb-4a6b-a50b-231e6e1c106d\" (UID: \"64ee0f9c-d6bb-4a6b-a50b-231e6e1c106d\") "
Sep 16 00:54:02.355776 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:02.355746    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64ee0f9c-d6bb-4a6b-a50b-231e6e1c106d-bundle" (OuterVolumeSpecName: "bundle") pod "64ee0f9c-d6bb-4a6b-a50b-231e6e1c106d" (UID: "64ee0f9c-d6bb-4a6b-a50b-231e6e1c106d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 00:54:02.357143 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:02.357111    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64ee0f9c-d6bb-4a6b-a50b-231e6e1c106d-kube-api-access-hxhbf" (OuterVolumeSpecName: "kube-api-access-hxhbf") pod "64ee0f9c-d6bb-4a6b-a50b-231e6e1c106d" (UID: "64ee0f9c-d6bb-4a6b-a50b-231e6e1c106d"). InnerVolumeSpecName "kube-api-access-hxhbf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 00:54:02.360778 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:02.360755    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64ee0f9c-d6bb-4a6b-a50b-231e6e1c106d-util" (OuterVolumeSpecName: "util") pod "64ee0f9c-d6bb-4a6b-a50b-231e6e1c106d" (UID: "64ee0f9c-d6bb-4a6b-a50b-231e6e1c106d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 00:54:02.456257 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:02.456179    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hxhbf\" (UniqueName: \"kubernetes.io/projected/64ee0f9c-d6bb-4a6b-a50b-231e6e1c106d-kube-api-access-hxhbf\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 00:54:02.456257 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:02.456204    2556 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/64ee0f9c-d6bb-4a6b-a50b-231e6e1c106d-bundle\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 00:54:02.456257 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:02.456213    2556 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/64ee0f9c-d6bb-4a6b-a50b-231e6e1c106d-util\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 00:54:03.159943 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:03.159910    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a9fmcb"
Sep 16 00:54:03.159943 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:03.159940    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a9fmcb" event={"ID":"64ee0f9c-d6bb-4a6b-a50b-231e6e1c106d","Type":"ContainerDied","Data":"bba5275a76e07847997dbf522c51622ddb79e0acd1934da52443675808bf1316"}
Sep 16 00:54:03.160350 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:03.159983    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bba5275a76e07847997dbf522c51622ddb79e0acd1934da52443675808bf1316"
Sep 16 00:54:03.283769 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:03.283742    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6kzk7l"
Sep 16 00:54:03.316682 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:03.316658    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqx9ps"
Sep 16 00:54:03.361967 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:03.361938    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ea985665-c2f8-4e5f-a899-f3ffff95f5e1-bundle\") pod \"ea985665-c2f8-4e5f-a899-f3ffff95f5e1\" (UID: \"ea985665-c2f8-4e5f-a899-f3ffff95f5e1\") "
Sep 16 00:54:03.362111 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:03.361992    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a97e27e1-2ee0-4e9a-b0a0-e9846c47aca3-util\") pod \"a97e27e1-2ee0-4e9a-b0a0-e9846c47aca3\" (UID: \"a97e27e1-2ee0-4e9a-b0a0-e9846c47aca3\") "
Sep 16 00:54:03.362111 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:03.362018    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a97e27e1-2ee0-4e9a-b0a0-e9846c47aca3-bundle\") pod \"a97e27e1-2ee0-4e9a-b0a0-e9846c47aca3\" (UID: \"a97e27e1-2ee0-4e9a-b0a0-e9846c47aca3\") "
Sep 16 00:54:03.362111 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:03.362048    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ea985665-c2f8-4e5f-a899-f3ffff95f5e1-util\") pod \"ea985665-c2f8-4e5f-a899-f3ffff95f5e1\" (UID: \"ea985665-c2f8-4e5f-a899-f3ffff95f5e1\") "
Sep 16 00:54:03.362111 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:03.362093    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v2xb6\" (UniqueName: \"kubernetes.io/projected/ea985665-c2f8-4e5f-a899-f3ffff95f5e1-kube-api-access-v2xb6\") pod \"ea985665-c2f8-4e5f-a899-f3ffff95f5e1\" (UID: \"ea985665-c2f8-4e5f-a899-f3ffff95f5e1\") "
Sep 16 00:54:03.362344 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:03.362144    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p757x\" (UniqueName: \"kubernetes.io/projected/a97e27e1-2ee0-4e9a-b0a0-e9846c47aca3-kube-api-access-p757x\") pod \"a97e27e1-2ee0-4e9a-b0a0-e9846c47aca3\" (UID: \"a97e27e1-2ee0-4e9a-b0a0-e9846c47aca3\") "
Sep 16 00:54:03.362578 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:03.362547    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a97e27e1-2ee0-4e9a-b0a0-e9846c47aca3-bundle" (OuterVolumeSpecName: "bundle") pod "a97e27e1-2ee0-4e9a-b0a0-e9846c47aca3" (UID: "a97e27e1-2ee0-4e9a-b0a0-e9846c47aca3"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 00:54:03.362700 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:03.362613    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea985665-c2f8-4e5f-a899-f3ffff95f5e1-bundle" (OuterVolumeSpecName: "bundle") pod "ea985665-c2f8-4e5f-a899-f3ffff95f5e1" (UID: "ea985665-c2f8-4e5f-a899-f3ffff95f5e1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 00:54:03.364203 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:03.364178    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a97e27e1-2ee0-4e9a-b0a0-e9846c47aca3-kube-api-access-p757x" (OuterVolumeSpecName: "kube-api-access-p757x") pod "a97e27e1-2ee0-4e9a-b0a0-e9846c47aca3" (UID: "a97e27e1-2ee0-4e9a-b0a0-e9846c47aca3"). InnerVolumeSpecName "kube-api-access-p757x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 00:54:03.364203 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:03.364195    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea985665-c2f8-4e5f-a899-f3ffff95f5e1-kube-api-access-v2xb6" (OuterVolumeSpecName: "kube-api-access-v2xb6") pod "ea985665-c2f8-4e5f-a899-f3ffff95f5e1" (UID: "ea985665-c2f8-4e5f-a899-f3ffff95f5e1"). InnerVolumeSpecName "kube-api-access-v2xb6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 00:54:03.463444 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:03.463380    2556 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a97e27e1-2ee0-4e9a-b0a0-e9846c47aca3-bundle\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 00:54:03.463444 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:03.463404    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v2xb6\" (UniqueName: \"kubernetes.io/projected/ea985665-c2f8-4e5f-a899-f3ffff95f5e1-kube-api-access-v2xb6\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 00:54:03.463444 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:03.463416    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p757x\" (UniqueName: \"kubernetes.io/projected/a97e27e1-2ee0-4e9a-b0a0-e9846c47aca3-kube-api-access-p757x\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 00:54:03.463444 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:03.463425    2556 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ea985665-c2f8-4e5f-a899-f3ffff95f5e1-bundle\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 00:54:03.539231 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:03.539193    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea985665-c2f8-4e5f-a899-f3ffff95f5e1-util" (OuterVolumeSpecName: "util") pod "ea985665-c2f8-4e5f-a899-f3ffff95f5e1" (UID: "ea985665-c2f8-4e5f-a899-f3ffff95f5e1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 00:54:03.540608 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:03.540582    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a97e27e1-2ee0-4e9a-b0a0-e9846c47aca3-util" (OuterVolumeSpecName: "util") pod "a97e27e1-2ee0-4e9a-b0a0-e9846c47aca3" (UID: "a97e27e1-2ee0-4e9a-b0a0-e9846c47aca3"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 00:54:03.563805 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:03.563779    2556 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a97e27e1-2ee0-4e9a-b0a0-e9846c47aca3-util\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 00:54:03.563805 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:03.563805    2556 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ea985665-c2f8-4e5f-a899-f3ffff95f5e1-util\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 00:54:04.166073 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:04.166046    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqx9ps"
Sep 16 00:54:04.166485 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:04.166074    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqx9ps" event={"ID":"ea985665-c2f8-4e5f-a899-f3ffff95f5e1","Type":"ContainerDied","Data":"4b3d1b4180496657b9b96b8ad9f09389a3d12289c8d2ea7f250bf85392158d56"}
Sep 16 00:54:04.166485 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:04.166120    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4b3d1b4180496657b9b96b8ad9f09389a3d12289c8d2ea7f250bf85392158d56"
Sep 16 00:54:04.168179 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:04.168152    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6kzk7l" event={"ID":"a97e27e1-2ee0-4e9a-b0a0-e9846c47aca3","Type":"ContainerDied","Data":"79be575ee6c3d604c7add42d18bca48eafc88e7f4c1ffc41753378cb0b42f66b"}
Sep 16 00:54:04.168298 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:04.168196    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6kzk7l"
Sep 16 00:54:04.168298 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:04.168195    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="79be575ee6c3d604c7add42d18bca48eafc88e7f4c1ffc41753378cb0b42f66b"
Sep 16 00:54:10.033939 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:10.033907    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-jfnkb"]
Sep 16 00:54:10.034375 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:10.034147    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea985665-c2f8-4e5f-a899-f3ffff95f5e1" containerName="util"
Sep 16 00:54:10.034375 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:10.034158    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea985665-c2f8-4e5f-a899-f3ffff95f5e1" containerName="util"
Sep 16 00:54:10.034375 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:10.034166    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64ee0f9c-d6bb-4a6b-a50b-231e6e1c106d" containerName="util"
Sep 16 00:54:10.034375 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:10.034172    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="64ee0f9c-d6bb-4a6b-a50b-231e6e1c106d" containerName="util"
Sep 16 00:54:10.034375 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:10.034181    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a97e27e1-2ee0-4e9a-b0a0-e9846c47aca3" containerName="extract"
Sep 16 00:54:10.034375 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:10.034187    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="a97e27e1-2ee0-4e9a-b0a0-e9846c47aca3" containerName="extract"
Sep 16 00:54:10.034375 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:10.034194    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea985665-c2f8-4e5f-a899-f3ffff95f5e1" containerName="extract"
Sep 16 00:54:10.034375 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:10.034199    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea985665-c2f8-4e5f-a899-f3ffff95f5e1" containerName="extract"
Sep 16 00:54:10.034375 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:10.034207    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e4a8031-9cc8-423a-b85c-9f0b982bc57e" containerName="extract"
Sep 16 00:54:10.034375 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:10.034212    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e4a8031-9cc8-423a-b85c-9f0b982bc57e" containerName="extract"
Sep 16 00:54:10.034375 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:10.034218    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e4a8031-9cc8-423a-b85c-9f0b982bc57e" containerName="util"
Sep 16 00:54:10.034375 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:10.034223    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e4a8031-9cc8-423a-b85c-9f0b982bc57e" containerName="util"
Sep 16 00:54:10.034375 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:10.034228    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e4a8031-9cc8-423a-b85c-9f0b982bc57e" containerName="pull"
Sep 16 00:54:10.034375 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:10.034233    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e4a8031-9cc8-423a-b85c-9f0b982bc57e" containerName="pull"
Sep 16 00:54:10.034375 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:10.034237    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea985665-c2f8-4e5f-a899-f3ffff95f5e1" containerName="pull"
Sep 16 00:54:10.034375 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:10.034243    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea985665-c2f8-4e5f-a899-f3ffff95f5e1" containerName="pull"
Sep 16 00:54:10.034375 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:10.034249    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a97e27e1-2ee0-4e9a-b0a0-e9846c47aca3" containerName="util"
Sep 16 00:54:10.034375 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:10.034254    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="a97e27e1-2ee0-4e9a-b0a0-e9846c47aca3" containerName="util"
Sep 16 00:54:10.034375 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:10.034260    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a97e27e1-2ee0-4e9a-b0a0-e9846c47aca3" containerName="pull"
Sep 16 00:54:10.034375 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:10.034264    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="a97e27e1-2ee0-4e9a-b0a0-e9846c47aca3" containerName="pull"
Sep 16 00:54:10.034375 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:10.034270    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64ee0f9c-d6bb-4a6b-a50b-231e6e1c106d" containerName="pull"
Sep 16 00:54:10.034375 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:10.034275    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="64ee0f9c-d6bb-4a6b-a50b-231e6e1c106d" containerName="pull"
Sep 16 00:54:10.034375 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:10.034284    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64ee0f9c-d6bb-4a6b-a50b-231e6e1c106d" containerName="extract"
Sep 16 00:54:10.034375 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:10.034289    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="64ee0f9c-d6bb-4a6b-a50b-231e6e1c106d" containerName="extract"
Sep 16 00:54:10.034375 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:10.034333    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="64ee0f9c-d6bb-4a6b-a50b-231e6e1c106d" containerName="extract"
Sep 16 00:54:10.034375 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:10.034340    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea985665-c2f8-4e5f-a899-f3ffff95f5e1" containerName="extract"
Sep 16 00:54:10.034375 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:10.034345    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e4a8031-9cc8-423a-b85c-9f0b982bc57e" containerName="extract"
Sep 16 00:54:10.034375 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:10.034353    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="a97e27e1-2ee0-4e9a-b0a0-e9846c47aca3" containerName="extract"
Sep 16 00:54:10.038457 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:10.038441    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-jfnkb"
Sep 16 00:54:10.040748 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:10.040723    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-bn2qg\""
Sep 16 00:54:10.040748 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:10.040732    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 16 00:54:10.041006 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:10.040992    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 16 00:54:10.048109 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:10.048084    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-jfnkb"]
Sep 16 00:54:10.113404 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:10.113368    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lzgq\" (UniqueName: \"kubernetes.io/projected/6aa9394a-32fe-4a8f-8d27-baa2b6eeb98f-kube-api-access-5lzgq\") pod \"authorino-operator-76b478bd78-jfnkb\" (UID: \"6aa9394a-32fe-4a8f-8d27-baa2b6eeb98f\") " pod="kuadrant-system/authorino-operator-76b478bd78-jfnkb"
Sep 16 00:54:10.214288 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:10.214259    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lzgq\" (UniqueName: \"kubernetes.io/projected/6aa9394a-32fe-4a8f-8d27-baa2b6eeb98f-kube-api-access-5lzgq\") pod \"authorino-operator-76b478bd78-jfnkb\" (UID: \"6aa9394a-32fe-4a8f-8d27-baa2b6eeb98f\") " pod="kuadrant-system/authorino-operator-76b478bd78-jfnkb"
Sep 16 00:54:10.227665 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:10.227639    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lzgq\" (UniqueName: \"kubernetes.io/projected/6aa9394a-32fe-4a8f-8d27-baa2b6eeb98f-kube-api-access-5lzgq\") pod \"authorino-operator-76b478bd78-jfnkb\" (UID: \"6aa9394a-32fe-4a8f-8d27-baa2b6eeb98f\") " pod="kuadrant-system/authorino-operator-76b478bd78-jfnkb"
Sep 16 00:54:10.348563 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:10.348533    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-jfnkb"
Sep 16 00:54:10.619729 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:10.619705    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-jfnkb"]
Sep 16 00:54:10.621622 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:54:10.621594    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6aa9394a_32fe_4a8f_8d27_baa2b6eeb98f.slice/crio-a7fd6b6b28aa9f48e9b1a0c78ddc67626e36988b398c6f97f86933334de1fbf1 WatchSource:0}: Error finding container a7fd6b6b28aa9f48e9b1a0c78ddc67626e36988b398c6f97f86933334de1fbf1: Status 404 returned error can't find the container with id a7fd6b6b28aa9f48e9b1a0c78ddc67626e36988b398c6f97f86933334de1fbf1
Sep 16 00:54:11.198714 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:11.198682    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-jfnkb" event={"ID":"6aa9394a-32fe-4a8f-8d27-baa2b6eeb98f","Type":"ContainerStarted","Data":"a7fd6b6b28aa9f48e9b1a0c78ddc67626e36988b398c6f97f86933334de1fbf1"}
Sep 16 00:54:12.245052 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:12.245013    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-zrmwp"]
Sep 16 00:54:12.248304 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:12.248279    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-zrmwp"
Sep 16 00:54:12.250837 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:12.250815    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 16 00:54:12.250959 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:12.250836    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-tczh9\""
Sep 16 00:54:12.260857 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:12.260834    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-zrmwp"]
Sep 16 00:54:12.330811 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:12.330776    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rj96p\" (UniqueName: \"kubernetes.io/projected/5a57e258-33e9-4fb5-b808-41021dfb50d6-kube-api-access-rj96p\") pod \"dns-operator-controller-manager-7b84cd7c9c-zrmwp\" (UID: \"5a57e258-33e9-4fb5-b808-41021dfb50d6\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-zrmwp"
Sep 16 00:54:12.431832 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:12.431799    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rj96p\" (UniqueName: \"kubernetes.io/projected/5a57e258-33e9-4fb5-b808-41021dfb50d6-kube-api-access-rj96p\") pod \"dns-operator-controller-manager-7b84cd7c9c-zrmwp\" (UID: \"5a57e258-33e9-4fb5-b808-41021dfb50d6\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-zrmwp"
Sep 16 00:54:12.441910 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:12.441883    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rj96p\" (UniqueName: \"kubernetes.io/projected/5a57e258-33e9-4fb5-b808-41021dfb50d6-kube-api-access-rj96p\") pod \"dns-operator-controller-manager-7b84cd7c9c-zrmwp\" (UID: \"5a57e258-33e9-4fb5-b808-41021dfb50d6\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-zrmwp"
Sep 16 00:54:12.561029 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:12.560950    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-zrmwp"
Sep 16 00:54:13.615419 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:13.615397    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-zrmwp"]
Sep 16 00:54:13.617553 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:54:13.617519    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a57e258_33e9_4fb5_b808_41021dfb50d6.slice/crio-68ccea072cdb5f941166312d22f035560c76aff9b6b8329b8b55e6eea023ea76 WatchSource:0}: Error finding container 68ccea072cdb5f941166312d22f035560c76aff9b6b8329b8b55e6eea023ea76: Status 404 returned error can't find the container with id 68ccea072cdb5f941166312d22f035560c76aff9b6b8329b8b55e6eea023ea76
Sep 16 00:54:14.214986 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:14.214941    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-zrmwp" event={"ID":"5a57e258-33e9-4fb5-b808-41021dfb50d6","Type":"ContainerStarted","Data":"68ccea072cdb5f941166312d22f035560c76aff9b6b8329b8b55e6eea023ea76"}
Sep 16 00:54:14.216372 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:14.216352    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-jfnkb" event={"ID":"6aa9394a-32fe-4a8f-8d27-baa2b6eeb98f","Type":"ContainerStarted","Data":"22b2d35f934f794c2d863077efb048658a3944d36fa05a06f0648d8dcd6a125d"}
Sep 16 00:54:14.216525 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:14.216511    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-76b478bd78-jfnkb"
Sep 16 00:54:14.233058 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:14.233012    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-76b478bd78-jfnkb" podStartSLOduration=1.440730035 podStartE2EDuration="4.232998151s" totalImagesPullingTime="2.792268116s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 00:54:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 00:54:14.231271615 +0000 UTC m=+669.012773208" watchObservedRunningTime="2026-09-16 00:54:14.232998151 +0000 UTC m=+669.014499753"
Sep 16 00:54:14.806998 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:14.806963    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-gbk5w"]
Sep 16 00:54:14.810126 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:14.810109    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-gbk5w"
Sep 16 00:54:14.813855 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:14.813836    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-5lfzl\""
Sep 16 00:54:14.839008 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:14.838977    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-gbk5w"]
Sep 16 00:54:14.957663 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:14.957628    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txpnv\" (UniqueName: \"kubernetes.io/projected/dca5b7f6-e944-44d5-8562-0c8130778217-kube-api-access-txpnv\") pod \"limitador-operator-controller-manager-74495c69f8-gbk5w\" (UID: \"dca5b7f6-e944-44d5-8562-0c8130778217\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-gbk5w"
Sep 16 00:54:15.058737 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:15.058669    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txpnv\" (UniqueName: \"kubernetes.io/projected/dca5b7f6-e944-44d5-8562-0c8130778217-kube-api-access-txpnv\") pod \"limitador-operator-controller-manager-74495c69f8-gbk5w\" (UID: \"dca5b7f6-e944-44d5-8562-0c8130778217\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-gbk5w"
Sep 16 00:54:15.067061 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:15.067034    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-txpnv\" (UniqueName: \"kubernetes.io/projected/dca5b7f6-e944-44d5-8562-0c8130778217-kube-api-access-txpnv\") pod \"limitador-operator-controller-manager-74495c69f8-gbk5w\" (UID: \"dca5b7f6-e944-44d5-8562-0c8130778217\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-gbk5w"
Sep 16 00:54:15.120631 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:15.120601    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-gbk5w"
Sep 16 00:54:15.398926 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:15.398809    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-gbk5w"]
Sep 16 00:54:15.401046 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:54:15.401017    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddca5b7f6_e944_44d5_8562_0c8130778217.slice/crio-443853a6ef47d05e5f3f3214862d44b75695de82cbb0473e03a231fad48e774d WatchSource:0}: Error finding container 443853a6ef47d05e5f3f3214862d44b75695de82cbb0473e03a231fad48e774d: Status 404 returned error can't find the container with id 443853a6ef47d05e5f3f3214862d44b75695de82cbb0473e03a231fad48e774d
Sep 16 00:54:16.227038 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:16.226955    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-zrmwp" event={"ID":"5a57e258-33e9-4fb5-b808-41021dfb50d6","Type":"ContainerStarted","Data":"1d3d78c5872edcb202ee2c0139cb04e7cba4a7045305a9a6512c4b4e9dd31cd3"}
Sep 16 00:54:16.227038 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:16.227025    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-zrmwp"
Sep 16 00:54:16.228309 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:16.228282    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-gbk5w" event={"ID":"dca5b7f6-e944-44d5-8562-0c8130778217","Type":"ContainerStarted","Data":"443853a6ef47d05e5f3f3214862d44b75695de82cbb0473e03a231fad48e774d"}
Sep 16 00:54:16.274326 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:16.274279    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-zrmwp" podStartSLOduration=1.97312015 podStartE2EDuration="4.274264489s" totalImagesPullingTime="2.301144339s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 00:54:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 00:54:16.273179448 +0000 UTC m=+671.054681040" watchObservedRunningTime="2026-09-16 00:54:16.274264489 +0000 UTC m=+671.055766143"
Sep 16 00:54:18.241004 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:18.240966    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-gbk5w" event={"ID":"dca5b7f6-e944-44d5-8562-0c8130778217","Type":"ContainerStarted","Data":"27b638e7977acec88404e7590fb169f158a884ab5526d2767ca7653e9029a5c2"}
Sep 16 00:54:18.241497 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:18.241092    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-gbk5w"
Sep 16 00:54:18.266455 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:18.266410    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-gbk5w" podStartSLOduration=2.359428491 podStartE2EDuration="4.266397233s" totalImagesPullingTime="1.906968742s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 00:54:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 00:54:18.265801584 +0000 UTC m=+673.047303188" watchObservedRunningTime="2026-09-16 00:54:18.266397233 +0000 UTC m=+673.047898825"
Sep 16 00:54:25.223185 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:25.223154    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-76b478bd78-jfnkb"
Sep 16 00:54:27.236982 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:27.236954    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-zrmwp"
Sep 16 00:54:29.247807 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:29.247774    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-gbk5w"
Sep 16 00:54:36.326033 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:36.325997    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-rq96w"]
Sep 16 00:54:36.329999 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:36.329318    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-rq96w"
Sep 16 00:54:36.332825 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:36.332801    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-wsn7n\""
Sep 16 00:54:36.347042 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:36.347021    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-rq96w"]
Sep 16 00:54:36.505091 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:36.505063    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/925338ae-da90-405b-9a02-c1d61ecfb8e9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-rq96w\" (UID: \"925338ae-da90-405b-9a02-c1d61ecfb8e9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-rq96w"
Sep 16 00:54:36.505241 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:36.505105    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-775j9\" (UniqueName: \"kubernetes.io/projected/925338ae-da90-405b-9a02-c1d61ecfb8e9-kube-api-access-775j9\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-rq96w\" (UID: \"925338ae-da90-405b-9a02-c1d61ecfb8e9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-rq96w"
Sep 16 00:54:36.605667 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:36.605595    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/925338ae-da90-405b-9a02-c1d61ecfb8e9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-rq96w\" (UID: \"925338ae-da90-405b-9a02-c1d61ecfb8e9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-rq96w"
Sep 16 00:54:36.605667 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:36.605642    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-775j9\" (UniqueName: \"kubernetes.io/projected/925338ae-da90-405b-9a02-c1d61ecfb8e9-kube-api-access-775j9\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-rq96w\" (UID: \"925338ae-da90-405b-9a02-c1d61ecfb8e9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-rq96w"
Sep 16 00:54:36.605976 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:36.605956    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/925338ae-da90-405b-9a02-c1d61ecfb8e9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-rq96w\" (UID: \"925338ae-da90-405b-9a02-c1d61ecfb8e9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-rq96w"
Sep 16 00:54:36.613451 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:36.613422    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-775j9\" (UniqueName: \"kubernetes.io/projected/925338ae-da90-405b-9a02-c1d61ecfb8e9-kube-api-access-775j9\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-rq96w\" (UID: \"925338ae-da90-405b-9a02-c1d61ecfb8e9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-rq96w"
Sep 16 00:54:36.640438 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:36.640415    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-rq96w"
Sep 16 00:54:36.913288 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:36.913257    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-rq96w"]
Sep 16 00:54:36.914818 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:54:36.914786    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod925338ae_da90_405b_9a02_c1d61ecfb8e9.slice/crio-0009a4cff1c3b10581093af1fb56da2db7b31bd9da4df59b5481750ed2500e24 WatchSource:0}: Error finding container 0009a4cff1c3b10581093af1fb56da2db7b31bd9da4df59b5481750ed2500e24: Status 404 returned error can't find the container with id 0009a4cff1c3b10581093af1fb56da2db7b31bd9da4df59b5481750ed2500e24
Sep 16 00:54:37.337669 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:37.337639    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-rq96w" event={"ID":"925338ae-da90-405b-9a02-c1d61ecfb8e9","Type":"ContainerStarted","Data":"0009a4cff1c3b10581093af1fb56da2db7b31bd9da4df59b5481750ed2500e24"}
Sep 16 00:54:37.553569 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:37.553533    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-rq96w"]
Sep 16 00:54:37.558808 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:37.558782    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-rq96w"]
Sep 16 00:54:37.600022 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:37.599956    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-gbk5w"]
Sep 16 00:54:37.600167 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:37.600146    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-gbk5w" podUID="dca5b7f6-e944-44d5-8562-0c8130778217" containerName="manager" containerID="cri-o://27b638e7977acec88404e7590fb169f158a884ab5526d2767ca7653e9029a5c2" gracePeriod=2
Sep 16 00:54:37.615441 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:37.615415    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-gbk5w"]
Sep 16 00:54:37.620882 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:54:37.620844    2556 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"dca5b7f6-e944-44d5-8562-0c8130778217\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-74495c69f8-gbk5w\": pods \"limitador-operator-controller-manager-74495c69f8-gbk5w\" not found" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-gbk5w"
Sep 16 00:54:37.623160 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:37.623136    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-cssg6"]
Sep 16 00:54:37.623396 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:37.623385    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dca5b7f6-e944-44d5-8562-0c8130778217" containerName="manager"
Sep 16 00:54:37.623437 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:37.623398    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="dca5b7f6-e944-44d5-8562-0c8130778217" containerName="manager"
Sep 16 00:54:37.623484 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:37.623448    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="dca5b7f6-e944-44d5-8562-0c8130778217" containerName="manager"
Sep 16 00:54:37.626217 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:37.626197    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-cssg6"
Sep 16 00:54:37.628038 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:54:37.628010    2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-gbk5w\" is forbidden: User \"system:node:ip-10-0-130-207.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-207.ec2.internal' and this object" podUID="dca5b7f6-e944-44d5-8562-0c8130778217" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-gbk5w"
Sep 16 00:54:37.639350 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:37.639329    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-cssg6"]
Sep 16 00:54:37.715282 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:37.715259    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7wwm\" (UniqueName: \"kubernetes.io/projected/24200bd2-1f1f-48e9-a8c4-2bca32931b5c-kube-api-access-p7wwm\") pod \"limitador-operator-controller-manager-74495c69f8-cssg6\" (UID: \"24200bd2-1f1f-48e9-a8c4-2bca32931b5c\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-cssg6"
Sep 16 00:54:37.815986 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:37.815952    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7wwm\" (UniqueName: \"kubernetes.io/projected/24200bd2-1f1f-48e9-a8c4-2bca32931b5c-kube-api-access-p7wwm\") pod \"limitador-operator-controller-manager-74495c69f8-cssg6\" (UID: \"24200bd2-1f1f-48e9-a8c4-2bca32931b5c\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-cssg6"
Sep 16 00:54:37.824339 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:37.824311    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7wwm\" (UniqueName: \"kubernetes.io/projected/24200bd2-1f1f-48e9-a8c4-2bca32931b5c-kube-api-access-p7wwm\") pod \"limitador-operator-controller-manager-74495c69f8-cssg6\" (UID: \"24200bd2-1f1f-48e9-a8c4-2bca32931b5c\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-cssg6"
Sep 16 00:54:37.839768 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:37.839746    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-gbk5w"
Sep 16 00:54:37.972002 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:37.971934    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-cssg6"
Sep 16 00:54:38.018054 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:38.018018    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-txpnv\" (UniqueName: \"kubernetes.io/projected/dca5b7f6-e944-44d5-8562-0c8130778217-kube-api-access-txpnv\") pod \"dca5b7f6-e944-44d5-8562-0c8130778217\" (UID: \"dca5b7f6-e944-44d5-8562-0c8130778217\") "
Sep 16 00:54:38.020521 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:38.020481    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dca5b7f6-e944-44d5-8562-0c8130778217-kube-api-access-txpnv" (OuterVolumeSpecName: "kube-api-access-txpnv") pod "dca5b7f6-e944-44d5-8562-0c8130778217" (UID: "dca5b7f6-e944-44d5-8562-0c8130778217"). InnerVolumeSpecName "kube-api-access-txpnv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 00:54:38.118972 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:38.118941    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-txpnv\" (UniqueName: \"kubernetes.io/projected/dca5b7f6-e944-44d5-8562-0c8130778217-kube-api-access-txpnv\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 00:54:38.263309 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:38.263280    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-cssg6"]
Sep 16 00:54:38.264387 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:54:38.264362    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24200bd2_1f1f_48e9_a8c4_2bca32931b5c.slice/crio-f01a677eac569ae1f5f2c2f2286f895d0a3e03102bba37a2809e3d3c252cc600 WatchSource:0}: Error finding container f01a677eac569ae1f5f2c2f2286f895d0a3e03102bba37a2809e3d3c252cc600: Status 404 returned error can't find the container with id f01a677eac569ae1f5f2c2f2286f895d0a3e03102bba37a2809e3d3c252cc600
Sep 16 00:54:38.345220 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:38.345183    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-cssg6" event={"ID":"24200bd2-1f1f-48e9-a8c4-2bca32931b5c","Type":"ContainerStarted","Data":"963d7b0979d3b0cf66479f08c3d41475376048a27292c72636c70f6a721a0659"}
Sep 16 00:54:38.345613 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:38.345252    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-cssg6" event={"ID":"24200bd2-1f1f-48e9-a8c4-2bca32931b5c","Type":"ContainerStarted","Data":"f01a677eac569ae1f5f2c2f2286f895d0a3e03102bba37a2809e3d3c252cc600"}
Sep 16 00:54:38.345613 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:38.345284    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-cssg6"
Sep 16 00:54:38.346861 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:38.346829    2556 generic.go:356] "Generic (PLEG): container finished" podID="dca5b7f6-e944-44d5-8562-0c8130778217" containerID="27b638e7977acec88404e7590fb169f158a884ab5526d2767ca7653e9029a5c2" exitCode=0
Sep 16 00:54:38.346965 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:38.346879    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-gbk5w"
Sep 16 00:54:38.346965 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:38.346891    2556 scope.go:117] "RemoveContainer" containerID="27b638e7977acec88404e7590fb169f158a884ab5526d2767ca7653e9029a5c2"
Sep 16 00:54:38.357025 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:38.357005    2556 scope.go:117] "RemoveContainer" containerID="27b638e7977acec88404e7590fb169f158a884ab5526d2767ca7653e9029a5c2"
Sep 16 00:54:38.357308 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:54:38.357280    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27b638e7977acec88404e7590fb169f158a884ab5526d2767ca7653e9029a5c2\": container with ID starting with 27b638e7977acec88404e7590fb169f158a884ab5526d2767ca7653e9029a5c2 not found: ID does not exist" containerID="27b638e7977acec88404e7590fb169f158a884ab5526d2767ca7653e9029a5c2"
Sep 16 00:54:38.357417 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:38.357314    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27b638e7977acec88404e7590fb169f158a884ab5526d2767ca7653e9029a5c2"} err="failed to get container status \"27b638e7977acec88404e7590fb169f158a884ab5526d2767ca7653e9029a5c2\": rpc error: code = NotFound desc = could not find container \"27b638e7977acec88404e7590fb169f158a884ab5526d2767ca7653e9029a5c2\": container with ID starting with 27b638e7977acec88404e7590fb169f158a884ab5526d2767ca7653e9029a5c2 not found: ID does not exist"
Sep 16 00:54:38.363583 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:38.363513    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-cssg6" podStartSLOduration=1.363497233 podStartE2EDuration="1.363497233s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 00:54:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 00:54:38.362846265 +0000 UTC m=+693.144347862" watchObservedRunningTime="2026-09-16 00:54:38.363497233 +0000 UTC m=+693.144998823"
Sep 16 00:54:39.735720 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:39.735682    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dca5b7f6-e944-44d5-8562-0c8130778217" path="/var/lib/kubelet/pods/dca5b7f6-e944-44d5-8562-0c8130778217/volumes"
Sep 16 00:54:41.367029 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:41.366990    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-rq96w" podUID="925338ae-da90-405b-9a02-c1d61ecfb8e9" containerName="manager" containerID="cri-o://344048d89d4c3e049c8469925b1aaf720d204442bbe6cafc61ace4738b73baf3" gracePeriod=2
Sep 16 00:54:41.595400 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:41.595371    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-rq96w"
Sep 16 00:54:41.597412 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:54:41.597382    2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-67c5dcb48c-rq96w\" is forbidden: User \"system:node:ip-10-0-130-207.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-207.ec2.internal' and this object" podUID="925338ae-da90-405b-9a02-c1d61ecfb8e9" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-rq96w"
Sep 16 00:54:41.747379 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:41.747316    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/925338ae-da90-405b-9a02-c1d61ecfb8e9-extensions-socket-volume\") pod \"925338ae-da90-405b-9a02-c1d61ecfb8e9\" (UID: \"925338ae-da90-405b-9a02-c1d61ecfb8e9\") "
Sep 16 00:54:41.747379 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:41.747356    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-775j9\" (UniqueName: \"kubernetes.io/projected/925338ae-da90-405b-9a02-c1d61ecfb8e9-kube-api-access-775j9\") pod \"925338ae-da90-405b-9a02-c1d61ecfb8e9\" (UID: \"925338ae-da90-405b-9a02-c1d61ecfb8e9\") "
Sep 16 00:54:41.747578 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:41.747545    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/925338ae-da90-405b-9a02-c1d61ecfb8e9-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "925338ae-da90-405b-9a02-c1d61ecfb8e9" (UID: "925338ae-da90-405b-9a02-c1d61ecfb8e9"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 00:54:41.749357 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:41.749331    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/925338ae-da90-405b-9a02-c1d61ecfb8e9-kube-api-access-775j9" (OuterVolumeSpecName: "kube-api-access-775j9") pod "925338ae-da90-405b-9a02-c1d61ecfb8e9" (UID: "925338ae-da90-405b-9a02-c1d61ecfb8e9"). InnerVolumeSpecName "kube-api-access-775j9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 00:54:41.848339 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:41.848317    2556 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/925338ae-da90-405b-9a02-c1d61ecfb8e9-extensions-socket-volume\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 00:54:41.848339 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:41.848337    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-775j9\" (UniqueName: \"kubernetes.io/projected/925338ae-da90-405b-9a02-c1d61ecfb8e9-kube-api-access-775j9\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 00:54:42.372998 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:42.372963    2556 generic.go:356] "Generic (PLEG): container finished" podID="925338ae-da90-405b-9a02-c1d61ecfb8e9" containerID="344048d89d4c3e049c8469925b1aaf720d204442bbe6cafc61ace4738b73baf3" exitCode=2
Sep 16 00:54:42.373420 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:42.373006    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-rq96w"
Sep 16 00:54:42.373420 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:42.373032    2556 scope.go:117] "RemoveContainer" containerID="344048d89d4c3e049c8469925b1aaf720d204442bbe6cafc61ace4738b73baf3"
Sep 16 00:54:42.381925 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:42.381910    2556 scope.go:117] "RemoveContainer" containerID="344048d89d4c3e049c8469925b1aaf720d204442bbe6cafc61ace4738b73baf3"
Sep 16 00:54:42.382177 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:54:42.382159    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"344048d89d4c3e049c8469925b1aaf720d204442bbe6cafc61ace4738b73baf3\": container with ID starting with 344048d89d4c3e049c8469925b1aaf720d204442bbe6cafc61ace4738b73baf3 not found: ID does not exist" containerID="344048d89d4c3e049c8469925b1aaf720d204442bbe6cafc61ace4738b73baf3"
Sep 16 00:54:42.382221 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:42.382188    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"344048d89d4c3e049c8469925b1aaf720d204442bbe6cafc61ace4738b73baf3"} err="failed to get container status \"344048d89d4c3e049c8469925b1aaf720d204442bbe6cafc61ace4738b73baf3\": rpc error: code = NotFound desc = could not find container \"344048d89d4c3e049c8469925b1aaf720d204442bbe6cafc61ace4738b73baf3\": container with ID starting with 344048d89d4c3e049c8469925b1aaf720d204442bbe6cafc61ace4738b73baf3 not found: ID does not exist"
Sep 16 00:54:42.383141 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:54:42.383119    2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-67c5dcb48c-rq96w\" is forbidden: User \"system:node:ip-10-0-130-207.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-207.ec2.internal' and this object" podUID="925338ae-da90-405b-9a02-c1d61ecfb8e9" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-rq96w"
Sep 16 00:54:43.735981 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:43.735935    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="925338ae-da90-405b-9a02-c1d61ecfb8e9" path="/var/lib/kubelet/pods/925338ae-da90-405b-9a02-c1d61ecfb8e9/volumes"
Sep 16 00:54:49.356120 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:54:49.356086    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-cssg6"
Sep 16 00:55:20.587557 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:20.587525    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-cm7c9"]
Sep 16 00:55:20.587983 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:20.587811    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="925338ae-da90-405b-9a02-c1d61ecfb8e9" containerName="manager"
Sep 16 00:55:20.587983 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:20.587822    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="925338ae-da90-405b-9a02-c1d61ecfb8e9" containerName="manager"
Sep 16 00:55:20.587983 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:20.587866    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="925338ae-da90-405b-9a02-c1d61ecfb8e9" containerName="manager"
Sep 16 00:55:20.590522 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:20.590503    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-cm7c9"
Sep 16 00:55:20.592624 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:20.592604    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-mkv6l\""
Sep 16 00:55:20.592864 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:20.592849    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 16 00:55:20.598459 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:20.598438    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-cm7c9"]
Sep 16 00:55:20.614538 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:20.614509    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/79c17afa-08d4-44b9-9392-c7eab5c591b2-config-file\") pod \"limitador-limitador-8588749977-cm7c9\" (UID: \"79c17afa-08d4-44b9-9392-c7eab5c591b2\") " pod="kuadrant-system/limitador-limitador-8588749977-cm7c9"
Sep 16 00:55:20.614645 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:20.614572    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jqnf\" (UniqueName: \"kubernetes.io/projected/79c17afa-08d4-44b9-9392-c7eab5c591b2-kube-api-access-8jqnf\") pod \"limitador-limitador-8588749977-cm7c9\" (UID: \"79c17afa-08d4-44b9-9392-c7eab5c591b2\") " pod="kuadrant-system/limitador-limitador-8588749977-cm7c9"
Sep 16 00:55:20.683753 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:20.683727    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-cm7c9"]
Sep 16 00:55:20.715153 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:20.715127    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jqnf\" (UniqueName: \"kubernetes.io/projected/79c17afa-08d4-44b9-9392-c7eab5c591b2-kube-api-access-8jqnf\") pod \"limitador-limitador-8588749977-cm7c9\" (UID: \"79c17afa-08d4-44b9-9392-c7eab5c591b2\") " pod="kuadrant-system/limitador-limitador-8588749977-cm7c9"
Sep 16 00:55:20.715293 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:20.715180    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/79c17afa-08d4-44b9-9392-c7eab5c591b2-config-file\") pod \"limitador-limitador-8588749977-cm7c9\" (UID: \"79c17afa-08d4-44b9-9392-c7eab5c591b2\") " pod="kuadrant-system/limitador-limitador-8588749977-cm7c9"
Sep 16 00:55:20.715750 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:20.715733    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/79c17afa-08d4-44b9-9392-c7eab5c591b2-config-file\") pod \"limitador-limitador-8588749977-cm7c9\" (UID: \"79c17afa-08d4-44b9-9392-c7eab5c591b2\") " pod="kuadrant-system/limitador-limitador-8588749977-cm7c9"
Sep 16 00:55:20.722832 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:20.722808    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jqnf\" (UniqueName: \"kubernetes.io/projected/79c17afa-08d4-44b9-9392-c7eab5c591b2-kube-api-access-8jqnf\") pod \"limitador-limitador-8588749977-cm7c9\" (UID: \"79c17afa-08d4-44b9-9392-c7eab5c591b2\") " pod="kuadrant-system/limitador-limitador-8588749977-cm7c9"
Sep 16 00:55:20.900604 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:20.900552    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-cm7c9"
Sep 16 00:55:21.167946 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:21.167837    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-cm7c9"]
Sep 16 00:55:21.170003 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:55:21.169975    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79c17afa_08d4_44b9_9392_c7eab5c591b2.slice/crio-796a1943795238256cc9f22e709235c587768e1794af26689de9bba8d3ba638b WatchSource:0}: Error finding container 796a1943795238256cc9f22e709235c587768e1794af26689de9bba8d3ba638b: Status 404 returned error can't find the container with id 796a1943795238256cc9f22e709235c587768e1794af26689de9bba8d3ba638b
Sep 16 00:55:21.558452 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:21.558377    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-cm7c9" event={"ID":"79c17afa-08d4-44b9-9392-c7eab5c591b2","Type":"ContainerStarted","Data":"796a1943795238256cc9f22e709235c587768e1794af26689de9bba8d3ba638b"}
Sep 16 00:55:22.564231 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:22.564201    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-cm7c9" event={"ID":"79c17afa-08d4-44b9-9392-c7eab5c591b2","Type":"ContainerStarted","Data":"6fbd190bfebee3dbf94d1b6e48145d54a00d65e61b57f3006236c5052fec8934"}
Sep 16 00:55:22.564556 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:22.564295    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-cm7c9"
Sep 16 00:55:22.579170 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:22.579128    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-cm7c9" podStartSLOduration=1.359565316 podStartE2EDuration="2.579114689s" totalImagesPullingTime="1.219549373s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 00:55:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 00:55:22.577944742 +0000 UTC m=+737.359446337" watchObservedRunningTime="2026-09-16 00:55:22.579114689 +0000 UTC m=+737.360616281"
Sep 16 00:55:33.570568 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:33.570533    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-cm7c9"
Sep 16 00:55:35.146680 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:35.146640    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-cm7c9"]
Sep 16 00:55:35.147161 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:35.146852    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-cm7c9" podUID="79c17afa-08d4-44b9-9392-c7eab5c591b2" containerName="limitador" containerID="cri-o://6fbd190bfebee3dbf94d1b6e48145d54a00d65e61b57f3006236c5052fec8934" gracePeriod=30
Sep 16 00:55:35.629306 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:35.629273    2556 generic.go:356] "Generic (PLEG): container finished" podID="79c17afa-08d4-44b9-9392-c7eab5c591b2" containerID="6fbd190bfebee3dbf94d1b6e48145d54a00d65e61b57f3006236c5052fec8934" exitCode=0
Sep 16 00:55:35.629443 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:35.629356    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-cm7c9" event={"ID":"79c17afa-08d4-44b9-9392-c7eab5c591b2","Type":"ContainerDied","Data":"6fbd190bfebee3dbf94d1b6e48145d54a00d65e61b57f3006236c5052fec8934"}
Sep 16 00:55:35.700940 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:35.700916    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-cm7c9"
Sep 16 00:55:35.822217 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:35.822147    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8jqnf\" (UniqueName: \"kubernetes.io/projected/79c17afa-08d4-44b9-9392-c7eab5c591b2-kube-api-access-8jqnf\") pod \"79c17afa-08d4-44b9-9392-c7eab5c591b2\" (UID: \"79c17afa-08d4-44b9-9392-c7eab5c591b2\") "
Sep 16 00:55:35.822217 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:35.822214    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/79c17afa-08d4-44b9-9392-c7eab5c591b2-config-file\") pod \"79c17afa-08d4-44b9-9392-c7eab5c591b2\" (UID: \"79c17afa-08d4-44b9-9392-c7eab5c591b2\") "
Sep 16 00:55:35.822632 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:35.822601    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79c17afa-08d4-44b9-9392-c7eab5c591b2-config-file" (OuterVolumeSpecName: "config-file") pod "79c17afa-08d4-44b9-9392-c7eab5c591b2" (UID: "79c17afa-08d4-44b9-9392-c7eab5c591b2"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 00:55:35.824582 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:35.824551    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79c17afa-08d4-44b9-9392-c7eab5c591b2-kube-api-access-8jqnf" (OuterVolumeSpecName: "kube-api-access-8jqnf") pod "79c17afa-08d4-44b9-9392-c7eab5c591b2" (UID: "79c17afa-08d4-44b9-9392-c7eab5c591b2"). InnerVolumeSpecName "kube-api-access-8jqnf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 00:55:35.922891 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:35.922866    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8jqnf\" (UniqueName: \"kubernetes.io/projected/79c17afa-08d4-44b9-9392-c7eab5c591b2-kube-api-access-8jqnf\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 00:55:35.922891 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:35.922887    2556 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/79c17afa-08d4-44b9-9392-c7eab5c591b2-config-file\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 00:55:36.635090 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:36.635062    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-cm7c9"
Sep 16 00:55:36.635625 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:36.635052    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-cm7c9" event={"ID":"79c17afa-08d4-44b9-9392-c7eab5c591b2","Type":"ContainerDied","Data":"796a1943795238256cc9f22e709235c587768e1794af26689de9bba8d3ba638b"}
Sep 16 00:55:36.635625 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:36.635202    2556 scope.go:117] "RemoveContainer" containerID="6fbd190bfebee3dbf94d1b6e48145d54a00d65e61b57f3006236c5052fec8934"
Sep 16 00:55:36.655715 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:36.655692    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-cm7c9"]
Sep 16 00:55:36.659420 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:36.659401    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-cm7c9"]
Sep 16 00:55:37.738629 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:37.736997    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79c17afa-08d4-44b9-9392-c7eab5c591b2" path="/var/lib/kubelet/pods/79c17afa-08d4-44b9-9392-c7eab5c591b2/volumes"
Sep 16 00:55:40.652729 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:40.652688    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-vcblt"]
Sep 16 00:55:40.653129 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:40.653038    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79c17afa-08d4-44b9-9392-c7eab5c591b2" containerName="limitador"
Sep 16 00:55:40.653129 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:40.653057    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="79c17afa-08d4-44b9-9392-c7eab5c591b2" containerName="limitador"
Sep 16 00:55:40.653129 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:40.653123    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="79c17afa-08d4-44b9-9392-c7eab5c591b2" containerName="limitador"
Sep 16 00:55:40.657520 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:40.657494    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-vcblt"
Sep 16 00:55:40.659824 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:40.659802    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 16 00:55:40.660608 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:40.660577    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-nbbd5\""
Sep 16 00:55:40.660608 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:40.660584    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 16 00:55:40.660774 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:40.660644    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 00:55:40.663442 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:40.663421    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-vcblt"]
Sep 16 00:55:40.753565 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:40.753539    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-477hb\" (UniqueName: \"kubernetes.io/projected/8108b9d2-6d25-47f8-a23c-963c5f115bc9-kube-api-access-477hb\") pod \"postgres-69fb4594cf-vcblt\" (UID: \"8108b9d2-6d25-47f8-a23c-963c5f115bc9\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-vcblt"
Sep 16 00:55:40.753695 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:40.753577    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/8108b9d2-6d25-47f8-a23c-963c5f115bc9-data\") pod \"postgres-69fb4594cf-vcblt\" (UID: \"8108b9d2-6d25-47f8-a23c-963c5f115bc9\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-vcblt"
Sep 16 00:55:40.854878 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:40.854848    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-477hb\" (UniqueName: \"kubernetes.io/projected/8108b9d2-6d25-47f8-a23c-963c5f115bc9-kube-api-access-477hb\") pod \"postgres-69fb4594cf-vcblt\" (UID: \"8108b9d2-6d25-47f8-a23c-963c5f115bc9\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-vcblt"
Sep 16 00:55:40.854989 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:40.854883    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/8108b9d2-6d25-47f8-a23c-963c5f115bc9-data\") pod \"postgres-69fb4594cf-vcblt\" (UID: \"8108b9d2-6d25-47f8-a23c-963c5f115bc9\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-vcblt"
Sep 16 00:55:40.855225 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:40.855205    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/8108b9d2-6d25-47f8-a23c-963c5f115bc9-data\") pod \"postgres-69fb4594cf-vcblt\" (UID: \"8108b9d2-6d25-47f8-a23c-963c5f115bc9\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-vcblt"
Sep 16 00:55:40.862563 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:40.862537    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-477hb\" (UniqueName: \"kubernetes.io/projected/8108b9d2-6d25-47f8-a23c-963c5f115bc9-kube-api-access-477hb\") pod \"postgres-69fb4594cf-vcblt\" (UID: \"8108b9d2-6d25-47f8-a23c-963c5f115bc9\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-vcblt"
Sep 16 00:55:40.967592 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:40.967542    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-vcblt"
Sep 16 00:55:41.236964 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:41.236939    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-vcblt"]
Sep 16 00:55:41.238349 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:55:41.238321    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8108b9d2_6d25_47f8_a23c_963c5f115bc9.slice/crio-f7ca62293fd1d7e2b4a5f28d7e988a77ff9248db07bc963ae91f6ab652bac566 WatchSource:0}: Error finding container f7ca62293fd1d7e2b4a5f28d7e988a77ff9248db07bc963ae91f6ab652bac566: Status 404 returned error can't find the container with id f7ca62293fd1d7e2b4a5f28d7e988a77ff9248db07bc963ae91f6ab652bac566
Sep 16 00:55:41.664478 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:41.664439    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-vcblt" event={"ID":"8108b9d2-6d25-47f8-a23c-963c5f115bc9","Type":"ContainerStarted","Data":"f7ca62293fd1d7e2b4a5f28d7e988a77ff9248db07bc963ae91f6ab652bac566"}
Sep 16 00:55:46.699953 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:46.699907    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-vcblt" event={"ID":"8108b9d2-6d25-47f8-a23c-963c5f115bc9","Type":"ContainerStarted","Data":"e848c88f1a175d3e6e3f1b5cca1508f4296331424fc10851def85e36d033059e"}
Sep 16 00:55:46.700309 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:46.700033    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-vcblt"
Sep 16 00:55:46.715169 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:46.715127    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-vcblt" podStartSLOduration=1.952588763 podStartE2EDuration="6.715114356s" totalImagesPullingTime="4.762525593s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 00:55:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 00:55:46.714070682 +0000 UTC m=+761.495572275" watchObservedRunningTime="2026-09-16 00:55:46.715114356 +0000 UTC m=+761.496615947"
Sep 16 00:55:52.733504 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:52.733449    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-vcblt"
Sep 16 00:55:59.613171 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:59.613139    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-75dc6cbdf-qxsm6"]
Sep 16 00:55:59.632267 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:59.632239    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-75dc6cbdf-qxsm6"]
Sep 16 00:55:59.632399 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:59.632336    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75dc6cbdf-qxsm6"
Sep 16 00:55:59.634555 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:59.634531    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 16 00:55:59.634680 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:59.634663    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 16 00:55:59.634741 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:59.634712    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-q5q6q\""
Sep 16 00:55:59.634786 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:59.634701    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 16 00:55:59.699699 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:59.699674    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a0e35a21-fd8c-4d03-9d2c-5e46b731d82e-metrics-certs\") pod \"maas-controller-75dc6cbdf-qxsm6\" (UID: \"a0e35a21-fd8c-4d03-9d2c-5e46b731d82e\") " pod="opendatahub/maas-controller-75dc6cbdf-qxsm6"
Sep 16 00:55:59.699821 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:59.699700    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7zrd\" (UniqueName: \"kubernetes.io/projected/a0e35a21-fd8c-4d03-9d2c-5e46b731d82e-kube-api-access-f7zrd\") pod \"maas-controller-75dc6cbdf-qxsm6\" (UID: \"a0e35a21-fd8c-4d03-9d2c-5e46b731d82e\") " pod="opendatahub/maas-controller-75dc6cbdf-qxsm6"
Sep 16 00:55:59.699821 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:59.699741    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a0e35a21-fd8c-4d03-9d2c-5e46b731d82e-webhook-cert\") pod \"maas-controller-75dc6cbdf-qxsm6\" (UID: \"a0e35a21-fd8c-4d03-9d2c-5e46b731d82e\") " pod="opendatahub/maas-controller-75dc6cbdf-qxsm6"
Sep 16 00:55:59.776752 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:59.776727    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6b69bf8998-fhxjr"]
Sep 16 00:55:59.779938 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:59.779909    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b69bf8998-fhxjr"
Sep 16 00:55:59.788211 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:59.788189    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6b69bf8998-fhxjr"]
Sep 16 00:55:59.800501 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:59.800463    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a0e35a21-fd8c-4d03-9d2c-5e46b731d82e-metrics-certs\") pod \"maas-controller-75dc6cbdf-qxsm6\" (UID: \"a0e35a21-fd8c-4d03-9d2c-5e46b731d82e\") " pod="opendatahub/maas-controller-75dc6cbdf-qxsm6"
Sep 16 00:55:59.800589 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:59.800520    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7zrd\" (UniqueName: \"kubernetes.io/projected/a0e35a21-fd8c-4d03-9d2c-5e46b731d82e-kube-api-access-f7zrd\") pod \"maas-controller-75dc6cbdf-qxsm6\" (UID: \"a0e35a21-fd8c-4d03-9d2c-5e46b731d82e\") " pod="opendatahub/maas-controller-75dc6cbdf-qxsm6"
Sep 16 00:55:59.800589 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:59.800578    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a0e35a21-fd8c-4d03-9d2c-5e46b731d82e-webhook-cert\") pod \"maas-controller-75dc6cbdf-qxsm6\" (UID: \"a0e35a21-fd8c-4d03-9d2c-5e46b731d82e\") " pod="opendatahub/maas-controller-75dc6cbdf-qxsm6"
Sep 16 00:55:59.802940 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:59.802914    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a0e35a21-fd8c-4d03-9d2c-5e46b731d82e-metrics-certs\") pod \"maas-controller-75dc6cbdf-qxsm6\" (UID: \"a0e35a21-fd8c-4d03-9d2c-5e46b731d82e\") " pod="opendatahub/maas-controller-75dc6cbdf-qxsm6"
Sep 16 00:55:59.803025 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:59.803010    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a0e35a21-fd8c-4d03-9d2c-5e46b731d82e-webhook-cert\") pod \"maas-controller-75dc6cbdf-qxsm6\" (UID: \"a0e35a21-fd8c-4d03-9d2c-5e46b731d82e\") " pod="opendatahub/maas-controller-75dc6cbdf-qxsm6"
Sep 16 00:55:59.807778 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:59.807761    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7zrd\" (UniqueName: \"kubernetes.io/projected/a0e35a21-fd8c-4d03-9d2c-5e46b731d82e-kube-api-access-f7zrd\") pod \"maas-controller-75dc6cbdf-qxsm6\" (UID: \"a0e35a21-fd8c-4d03-9d2c-5e46b731d82e\") " pod="opendatahub/maas-controller-75dc6cbdf-qxsm6"
Sep 16 00:55:59.901094 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:59.901027    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nscmf\" (UniqueName: \"kubernetes.io/projected/d6303b5a-7625-4fd4-877a-8091ee23df43-kube-api-access-nscmf\") pod \"maas-controller-6b69bf8998-fhxjr\" (UID: \"d6303b5a-7625-4fd4-877a-8091ee23df43\") " pod="opendatahub/maas-controller-6b69bf8998-fhxjr"
Sep 16 00:55:59.901094 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:59.901067    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6303b5a-7625-4fd4-877a-8091ee23df43-metrics-certs\") pod \"maas-controller-6b69bf8998-fhxjr\" (UID: \"d6303b5a-7625-4fd4-877a-8091ee23df43\") " pod="opendatahub/maas-controller-6b69bf8998-fhxjr"
Sep 16 00:55:59.901246 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:59.901139    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d6303b5a-7625-4fd4-877a-8091ee23df43-webhook-cert\") pod \"maas-controller-6b69bf8998-fhxjr\" (UID: \"d6303b5a-7625-4fd4-877a-8091ee23df43\") " pod="opendatahub/maas-controller-6b69bf8998-fhxjr"
Sep 16 00:55:59.945362 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:55:59.945340    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75dc6cbdf-qxsm6"
Sep 16 00:56:00.002098 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:00.002064    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d6303b5a-7625-4fd4-877a-8091ee23df43-webhook-cert\") pod \"maas-controller-6b69bf8998-fhxjr\" (UID: \"d6303b5a-7625-4fd4-877a-8091ee23df43\") " pod="opendatahub/maas-controller-6b69bf8998-fhxjr"
Sep 16 00:56:00.002224 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:00.002130    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nscmf\" (UniqueName: \"kubernetes.io/projected/d6303b5a-7625-4fd4-877a-8091ee23df43-kube-api-access-nscmf\") pod \"maas-controller-6b69bf8998-fhxjr\" (UID: \"d6303b5a-7625-4fd4-877a-8091ee23df43\") " pod="opendatahub/maas-controller-6b69bf8998-fhxjr"
Sep 16 00:56:00.002224 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:00.002180    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6303b5a-7625-4fd4-877a-8091ee23df43-metrics-certs\") pod \"maas-controller-6b69bf8998-fhxjr\" (UID: \"d6303b5a-7625-4fd4-877a-8091ee23df43\") " pod="opendatahub/maas-controller-6b69bf8998-fhxjr"
Sep 16 00:56:00.005156 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:00.005126    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6303b5a-7625-4fd4-877a-8091ee23df43-metrics-certs\") pod \"maas-controller-6b69bf8998-fhxjr\" (UID: \"d6303b5a-7625-4fd4-877a-8091ee23df43\") " pod="opendatahub/maas-controller-6b69bf8998-fhxjr"
Sep 16 00:56:00.005599 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:00.005580    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d6303b5a-7625-4fd4-877a-8091ee23df43-webhook-cert\") pod \"maas-controller-6b69bf8998-fhxjr\" (UID: \"d6303b5a-7625-4fd4-877a-8091ee23df43\") " pod="opendatahub/maas-controller-6b69bf8998-fhxjr"
Sep 16 00:56:00.010699 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:00.010680    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nscmf\" (UniqueName: \"kubernetes.io/projected/d6303b5a-7625-4fd4-877a-8091ee23df43-kube-api-access-nscmf\") pod \"maas-controller-6b69bf8998-fhxjr\" (UID: \"d6303b5a-7625-4fd4-877a-8091ee23df43\") " pod="opendatahub/maas-controller-6b69bf8998-fhxjr"
Sep 16 00:56:00.090199 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:00.090163    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b69bf8998-fhxjr"
Sep 16 00:56:00.212964 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:00.212940    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-75dc6cbdf-qxsm6"]
Sep 16 00:56:00.214906 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:56:00.214880    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0e35a21_fd8c_4d03_9d2c_5e46b731d82e.slice/crio-f43b511e4cb01e32f5d5375d8c144286e100b4d170eb8684148edd55e421007f WatchSource:0}: Error finding container f43b511e4cb01e32f5d5375d8c144286e100b4d170eb8684148edd55e421007f: Status 404 returned error can't find the container with id f43b511e4cb01e32f5d5375d8c144286e100b4d170eb8684148edd55e421007f
Sep 16 00:56:00.358350 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:00.358328    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6b69bf8998-fhxjr"]
Sep 16 00:56:00.360145 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:56:00.360113    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6303b5a_7625_4fd4_877a_8091ee23df43.slice/crio-ffb92ea43e36c6fa6461ecc9569739f09e5fd41cd18b71723d0fdecb57da851e WatchSource:0}: Error finding container ffb92ea43e36c6fa6461ecc9569739f09e5fd41cd18b71723d0fdecb57da851e: Status 404 returned error can't find the container with id ffb92ea43e36c6fa6461ecc9569739f09e5fd41cd18b71723d0fdecb57da851e
Sep 16 00:56:00.769778 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:00.769689    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b69bf8998-fhxjr" event={"ID":"d6303b5a-7625-4fd4-877a-8091ee23df43","Type":"ContainerStarted","Data":"ffb92ea43e36c6fa6461ecc9569739f09e5fd41cd18b71723d0fdecb57da851e"}
Sep 16 00:56:00.771821 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:00.771751    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75dc6cbdf-qxsm6" event={"ID":"a0e35a21-fd8c-4d03-9d2c-5e46b731d82e","Type":"ContainerStarted","Data":"f43b511e4cb01e32f5d5375d8c144286e100b4d170eb8684148edd55e421007f"}
Sep 16 00:56:02.785228 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:02.785188    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75dc6cbdf-qxsm6" event={"ID":"a0e35a21-fd8c-4d03-9d2c-5e46b731d82e","Type":"ContainerStarted","Data":"20619ebdb0f44e3596cbf222d062d858902082c4a2741b802b8337da95c36b2d"}
Sep 16 00:56:02.785702 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:02.785259    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-75dc6cbdf-qxsm6"
Sep 16 00:56:02.787388 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:02.787355    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b69bf8998-fhxjr" event={"ID":"d6303b5a-7625-4fd4-877a-8091ee23df43","Type":"ContainerStarted","Data":"62c564f11f9af0ed9b21943d8a381a2b6ec3e6f0085d4416623653d4fe80154c"}
Sep 16 00:56:02.787523 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:02.787498    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6b69bf8998-fhxjr"
Sep 16 00:56:02.807532 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:02.807460    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-75dc6cbdf-qxsm6" podStartSLOduration=1.394388243 podStartE2EDuration="3.8074401s" totalImagesPullingTime="2.413051857s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 00:55:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 00:56:02.806976655 +0000 UTC m=+777.588478248" watchObservedRunningTime="2026-09-16 00:56:02.8074401 +0000 UTC m=+777.588941692"
Sep 16 00:56:02.829266 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:02.826014    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6b69bf8998-fhxjr" podStartSLOduration=1.547557622 podStartE2EDuration="3.825989756s" totalImagesPullingTime="2.278432134s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 00:55:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 00:56:02.823278811 +0000 UTC m=+777.604780403" watchObservedRunningTime="2026-09-16 00:56:02.825989756 +0000 UTC m=+777.607491351"
Sep 16 00:56:13.793971 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:13.793936    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6b69bf8998-fhxjr"
Sep 16 00:56:13.794352 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:13.794019    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-75dc6cbdf-qxsm6"
Sep 16 00:56:13.853332 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:13.853289    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-75dc6cbdf-qxsm6"]
Sep 16 00:56:13.853537 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:13.853519    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-75dc6cbdf-qxsm6" podUID="a0e35a21-fd8c-4d03-9d2c-5e46b731d82e" containerName="manager" containerID="cri-o://20619ebdb0f44e3596cbf222d062d858902082c4a2741b802b8337da95c36b2d" gracePeriod=30
Sep 16 00:56:14.096245 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:14.096220    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75dc6cbdf-qxsm6"
Sep 16 00:56:14.225047 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:14.225013    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a0e35a21-fd8c-4d03-9d2c-5e46b731d82e-metrics-certs\") pod \"a0e35a21-fd8c-4d03-9d2c-5e46b731d82e\" (UID: \"a0e35a21-fd8c-4d03-9d2c-5e46b731d82e\") "
Sep 16 00:56:14.225221 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:14.225092    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f7zrd\" (UniqueName: \"kubernetes.io/projected/a0e35a21-fd8c-4d03-9d2c-5e46b731d82e-kube-api-access-f7zrd\") pod \"a0e35a21-fd8c-4d03-9d2c-5e46b731d82e\" (UID: \"a0e35a21-fd8c-4d03-9d2c-5e46b731d82e\") "
Sep 16 00:56:14.225221 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:14.225128    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a0e35a21-fd8c-4d03-9d2c-5e46b731d82e-webhook-cert\") pod \"a0e35a21-fd8c-4d03-9d2c-5e46b731d82e\" (UID: \"a0e35a21-fd8c-4d03-9d2c-5e46b731d82e\") "
Sep 16 00:56:14.227304 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:14.227270    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0e35a21-fd8c-4d03-9d2c-5e46b731d82e-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "a0e35a21-fd8c-4d03-9d2c-5e46b731d82e" (UID: "a0e35a21-fd8c-4d03-9d2c-5e46b731d82e"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 00:56:14.227418 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:14.227350    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0e35a21-fd8c-4d03-9d2c-5e46b731d82e-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a0e35a21-fd8c-4d03-9d2c-5e46b731d82e" (UID: "a0e35a21-fd8c-4d03-9d2c-5e46b731d82e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 00:56:14.227418 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:14.227364    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0e35a21-fd8c-4d03-9d2c-5e46b731d82e-kube-api-access-f7zrd" (OuterVolumeSpecName: "kube-api-access-f7zrd") pod "a0e35a21-fd8c-4d03-9d2c-5e46b731d82e" (UID: "a0e35a21-fd8c-4d03-9d2c-5e46b731d82e"). InnerVolumeSpecName "kube-api-access-f7zrd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 00:56:14.325924 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:14.325888    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f7zrd\" (UniqueName: \"kubernetes.io/projected/a0e35a21-fd8c-4d03-9d2c-5e46b731d82e-kube-api-access-f7zrd\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 00:56:14.325924 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:14.325920    2556 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a0e35a21-fd8c-4d03-9d2c-5e46b731d82e-webhook-cert\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 00:56:14.325924 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:14.325930    2556 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a0e35a21-fd8c-4d03-9d2c-5e46b731d82e-metrics-certs\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 00:56:14.848747 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:14.848716    2556 generic.go:356] "Generic (PLEG): container finished" podID="a0e35a21-fd8c-4d03-9d2c-5e46b731d82e" containerID="20619ebdb0f44e3596cbf222d062d858902082c4a2741b802b8337da95c36b2d" exitCode=0
Sep 16 00:56:14.849147 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:14.848759    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75dc6cbdf-qxsm6"
Sep 16 00:56:14.849147 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:14.848809    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75dc6cbdf-qxsm6" event={"ID":"a0e35a21-fd8c-4d03-9d2c-5e46b731d82e","Type":"ContainerDied","Data":"20619ebdb0f44e3596cbf222d062d858902082c4a2741b802b8337da95c36b2d"}
Sep 16 00:56:14.849147 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:14.848859    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75dc6cbdf-qxsm6" event={"ID":"a0e35a21-fd8c-4d03-9d2c-5e46b731d82e","Type":"ContainerDied","Data":"f43b511e4cb01e32f5d5375d8c144286e100b4d170eb8684148edd55e421007f"}
Sep 16 00:56:14.849147 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:14.848884    2556 scope.go:117] "RemoveContainer" containerID="20619ebdb0f44e3596cbf222d062d858902082c4a2741b802b8337da95c36b2d"
Sep 16 00:56:14.858060 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:14.857880    2556 scope.go:117] "RemoveContainer" containerID="20619ebdb0f44e3596cbf222d062d858902082c4a2741b802b8337da95c36b2d"
Sep 16 00:56:14.858179 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:56:14.858160    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20619ebdb0f44e3596cbf222d062d858902082c4a2741b802b8337da95c36b2d\": container with ID starting with 20619ebdb0f44e3596cbf222d062d858902082c4a2741b802b8337da95c36b2d not found: ID does not exist" containerID="20619ebdb0f44e3596cbf222d062d858902082c4a2741b802b8337da95c36b2d"
Sep 16 00:56:14.858229 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:14.858186    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20619ebdb0f44e3596cbf222d062d858902082c4a2741b802b8337da95c36b2d"} err="failed to get container status \"20619ebdb0f44e3596cbf222d062d858902082c4a2741b802b8337da95c36b2d\": rpc error: code = NotFound desc = could not find container \"20619ebdb0f44e3596cbf222d062d858902082c4a2741b802b8337da95c36b2d\": container with ID starting with 20619ebdb0f44e3596cbf222d062d858902082c4a2741b802b8337da95c36b2d not found: ID does not exist"
Sep 16 00:56:14.869849 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:14.869825    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-75dc6cbdf-qxsm6"]
Sep 16 00:56:14.873303 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:14.873283    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-75dc6cbdf-qxsm6"]
Sep 16 00:56:15.735968 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:15.735934    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0e35a21-fd8c-4d03-9d2c-5e46b731d82e" path="/var/lib/kubelet/pods/a0e35a21-fd8c-4d03-9d2c-5e46b731d82e/volumes"
Sep 16 00:56:36.249399 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:36.249362    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-59f6566bbc-7cjw8"]
Sep 16 00:56:36.249813 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:36.249652    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0e35a21-fd8c-4d03-9d2c-5e46b731d82e" containerName="manager"
Sep 16 00:56:36.249813 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:36.249664    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0e35a21-fd8c-4d03-9d2c-5e46b731d82e" containerName="manager"
Sep 16 00:56:36.249813 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:36.249739    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0e35a21-fd8c-4d03-9d2c-5e46b731d82e" containerName="manager"
Sep 16 00:56:36.255678 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:36.255656    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-7cjw8"
Sep 16 00:56:36.258059 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:36.258028    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 16 00:56:36.258163 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:36.258062    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 16 00:56:36.258163 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:36.258120    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-7n79q\""
Sep 16 00:56:36.262520 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:36.262496    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-59f6566bbc-7cjw8"]
Sep 16 00:56:36.280878 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:36.280856    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b3419ce2-609a-402d-8f48-4dcef653d0a0-maas-api-tls\") pod \"maas-api-59f6566bbc-7cjw8\" (UID: \"b3419ce2-609a-402d-8f48-4dcef653d0a0\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-7cjw8"
Sep 16 00:56:36.280970 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:36.280892    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b3419ce2-609a-402d-8f48-4dcef653d0a0-metrics-certs\") pod \"maas-api-59f6566bbc-7cjw8\" (UID: \"b3419ce2-609a-402d-8f48-4dcef653d0a0\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-7cjw8"
Sep 16 00:56:36.280970 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:36.280944    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3419ce2-609a-402d-8f48-4dcef653d0a0-openshift-service-ca\") pod \"maas-api-59f6566bbc-7cjw8\" (UID: \"b3419ce2-609a-402d-8f48-4dcef653d0a0\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-7cjw8"
Sep 16 00:56:36.281045 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:36.280976    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7r9m\" (UniqueName: \"kubernetes.io/projected/b3419ce2-609a-402d-8f48-4dcef653d0a0-kube-api-access-f7r9m\") pod \"maas-api-59f6566bbc-7cjw8\" (UID: \"b3419ce2-609a-402d-8f48-4dcef653d0a0\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-7cjw8"
Sep 16 00:56:36.381616 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:36.381593    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b3419ce2-609a-402d-8f48-4dcef653d0a0-maas-api-tls\") pod \"maas-api-59f6566bbc-7cjw8\" (UID: \"b3419ce2-609a-402d-8f48-4dcef653d0a0\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-7cjw8"
Sep 16 00:56:36.381728 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:36.381631    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b3419ce2-609a-402d-8f48-4dcef653d0a0-metrics-certs\") pod \"maas-api-59f6566bbc-7cjw8\" (UID: \"b3419ce2-609a-402d-8f48-4dcef653d0a0\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-7cjw8"
Sep 16 00:56:36.381728 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:36.381670    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3419ce2-609a-402d-8f48-4dcef653d0a0-openshift-service-ca\") pod \"maas-api-59f6566bbc-7cjw8\" (UID: \"b3419ce2-609a-402d-8f48-4dcef653d0a0\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-7cjw8"
Sep 16 00:56:36.381728 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:36.381696    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7r9m\" (UniqueName: \"kubernetes.io/projected/b3419ce2-609a-402d-8f48-4dcef653d0a0-kube-api-access-f7r9m\") pod \"maas-api-59f6566bbc-7cjw8\" (UID: \"b3419ce2-609a-402d-8f48-4dcef653d0a0\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-7cjw8"
Sep 16 00:56:36.381887 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:56:36.381783    2556 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-metrics-tls: secret "maas-api-metrics-tls" not found
Sep 16 00:56:36.381887 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:56:36.381856    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3419ce2-609a-402d-8f48-4dcef653d0a0-metrics-certs podName:b3419ce2-609a-402d-8f48-4dcef653d0a0 nodeName:}" failed. No retries permitted until 2026-09-16 00:56:36.881835259 +0000 UTC m=+811.663336831 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b3419ce2-609a-402d-8f48-4dcef653d0a0-metrics-certs") pod "maas-api-59f6566bbc-7cjw8" (UID: "b3419ce2-609a-402d-8f48-4dcef653d0a0") : secret "maas-api-metrics-tls" not found
Sep 16 00:56:36.382301 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:36.382280    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3419ce2-609a-402d-8f48-4dcef653d0a0-openshift-service-ca\") pod \"maas-api-59f6566bbc-7cjw8\" (UID: \"b3419ce2-609a-402d-8f48-4dcef653d0a0\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-7cjw8"
Sep 16 00:56:36.384170 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:36.384149    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b3419ce2-609a-402d-8f48-4dcef653d0a0-maas-api-tls\") pod \"maas-api-59f6566bbc-7cjw8\" (UID: \"b3419ce2-609a-402d-8f48-4dcef653d0a0\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-7cjw8"
Sep 16 00:56:36.389963 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:36.389938    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7r9m\" (UniqueName: \"kubernetes.io/projected/b3419ce2-609a-402d-8f48-4dcef653d0a0-kube-api-access-f7r9m\") pod \"maas-api-59f6566bbc-7cjw8\" (UID: \"b3419ce2-609a-402d-8f48-4dcef653d0a0\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-7cjw8"
Sep 16 00:56:36.816695 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:36.816660    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-8j925"]
Sep 16 00:56:36.820696 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:36.820676    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-8j925"
Sep 16 00:56:36.822847 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:36.822825    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-2xnr2\""
Sep 16 00:56:36.822947 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:36.822934    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 16 00:56:36.827678 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:36.827656    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-8j925"]
Sep 16 00:56:36.886627 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:36.886600    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12f8e3b0-5262-47ac-9247-856ad9327cb6-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-8j925\" (UID: \"12f8e3b0-5262-47ac-9247-856ad9327cb6\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-8j925"
Sep 16 00:56:36.886753 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:36.886642    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b3419ce2-609a-402d-8f48-4dcef653d0a0-metrics-certs\") pod \"maas-api-59f6566bbc-7cjw8\" (UID: \"b3419ce2-609a-402d-8f48-4dcef653d0a0\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-7cjw8"
Sep 16 00:56:36.886753 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:36.886664    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ddq2\" (UniqueName: \"kubernetes.io/projected/12f8e3b0-5262-47ac-9247-856ad9327cb6-kube-api-access-5ddq2\") pod \"payload-processing-5b54b7fbb7-8j925\" (UID: \"12f8e3b0-5262-47ac-9247-856ad9327cb6\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-8j925"
Sep 16 00:56:36.888859 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:36.888838    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b3419ce2-609a-402d-8f48-4dcef653d0a0-metrics-certs\") pod \"maas-api-59f6566bbc-7cjw8\" (UID: \"b3419ce2-609a-402d-8f48-4dcef653d0a0\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-7cjw8"
Sep 16 00:56:36.987063 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:36.987033    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ddq2\" (UniqueName: \"kubernetes.io/projected/12f8e3b0-5262-47ac-9247-856ad9327cb6-kube-api-access-5ddq2\") pod \"payload-processing-5b54b7fbb7-8j925\" (UID: \"12f8e3b0-5262-47ac-9247-856ad9327cb6\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-8j925"
Sep 16 00:56:36.987220 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:36.987095    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12f8e3b0-5262-47ac-9247-856ad9327cb6-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-8j925\" (UID: \"12f8e3b0-5262-47ac-9247-856ad9327cb6\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-8j925"
Sep 16 00:56:36.987772 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:36.987752    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12f8e3b0-5262-47ac-9247-856ad9327cb6-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-8j925\" (UID: \"12f8e3b0-5262-47ac-9247-856ad9327cb6\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-8j925"
Sep 16 00:56:36.994216 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:36.994188    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ddq2\" (UniqueName: \"kubernetes.io/projected/12f8e3b0-5262-47ac-9247-856ad9327cb6-kube-api-access-5ddq2\") pod \"payload-processing-5b54b7fbb7-8j925\" (UID: \"12f8e3b0-5262-47ac-9247-856ad9327cb6\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-8j925"
Sep 16 00:56:37.131485 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:37.131422    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-8j925"
Sep 16 00:56:37.166638 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:37.166611    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-7cjw8"
Sep 16 00:56:37.403242 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:37.403216    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-8j925"]
Sep 16 00:56:37.404547 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:56:37.404517    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12f8e3b0_5262_47ac_9247_856ad9327cb6.slice/crio-9869dcd5dfdae4fd6e381df7a5f92d33145b0b7e5c9f66cfd3fb8825c17f6fc3 WatchSource:0}: Error finding container 9869dcd5dfdae4fd6e381df7a5f92d33145b0b7e5c9f66cfd3fb8825c17f6fc3: Status 404 returned error can't find the container with id 9869dcd5dfdae4fd6e381df7a5f92d33145b0b7e5c9f66cfd3fb8825c17f6fc3
Sep 16 00:56:37.442206 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:37.442184    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-59f6566bbc-7cjw8"]
Sep 16 00:56:37.444006 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:56:37.443983    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3419ce2_609a_402d_8f48_4dcef653d0a0.slice/crio-409ff294753662ffcadbb6ed88ac1d8e48322703fa1894ee3d6bcadd97e9e475 WatchSource:0}: Error finding container 409ff294753662ffcadbb6ed88ac1d8e48322703fa1894ee3d6bcadd97e9e475: Status 404 returned error can't find the container with id 409ff294753662ffcadbb6ed88ac1d8e48322703fa1894ee3d6bcadd97e9e475
Sep 16 00:56:37.951875 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:37.951827    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-7cjw8" event={"ID":"b3419ce2-609a-402d-8f48-4dcef653d0a0","Type":"ContainerStarted","Data":"409ff294753662ffcadbb6ed88ac1d8e48322703fa1894ee3d6bcadd97e9e475"}
Sep 16 00:56:37.953195 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:37.953162    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-8j925" event={"ID":"12f8e3b0-5262-47ac-9247-856ad9327cb6","Type":"ContainerStarted","Data":"9869dcd5dfdae4fd6e381df7a5f92d33145b0b7e5c9f66cfd3fb8825c17f6fc3"}
Sep 16 00:56:40.974632 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:40.974589    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-7cjw8" event={"ID":"b3419ce2-609a-402d-8f48-4dcef653d0a0","Type":"ContainerStarted","Data":"94f81deb9b67b56028e29af2a3a8289ee3604681e89f29f64b57fe3e2e88642d"}
Sep 16 00:56:40.975104 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:40.974688    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-7cjw8"
Sep 16 00:56:40.976205 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:40.976169    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-8j925" event={"ID":"12f8e3b0-5262-47ac-9247-856ad9327cb6","Type":"ContainerStarted","Data":"1122e67bdd61e72f79d7eef2aa882b2f30f67bf25849732e4ff9dadccb35293e"}
Sep 16 00:56:40.976329 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:40.976223    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-8j925"
Sep 16 00:56:40.990871 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:40.990828    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-7cjw8" podStartSLOduration=2.462523291 podStartE2EDuration="4.990812941s" totalImagesPullingTime="2.52828965s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 00:56:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 00:56:40.989632795 +0000 UTC m=+815.771134387" watchObservedRunningTime="2026-09-16 00:56:40.990812941 +0000 UTC m=+815.772314530"
Sep 16 00:56:46.985551 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:46.985519    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-7cjw8"
Sep 16 00:56:47.001083 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:47.001010    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-8j925" podStartSLOduration=8.388895182 podStartE2EDuration="11.000995287s" totalImagesPullingTime="2.612100105s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 00:56:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 00:56:41.003310264 +0000 UTC m=+815.784811856" watchObservedRunningTime="2026-09-16 00:56:47.000995287 +0000 UTC m=+821.782496879"
Sep 16 00:56:51.982363 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:56:51.982330    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-8j925"
Sep 16 00:57:16.577894 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:57:16.577863    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-bnkct"]
Sep 16 00:57:16.583261 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:57:16.583228    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bnkct"
Sep 16 00:57:16.586194 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:57:16.586167    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-spxkl\""
Sep 16 00:57:16.586332 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:57:16.586208    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 16 00:57:16.586332 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:57:16.586167    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 16 00:57:16.586332 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:57:16.586171    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 16 00:57:16.592273 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:57:16.592251    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-bnkct"]
Sep 16 00:57:16.670666 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:57:16.670641    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6376395-6632-4fb4-917d-3b5c9cd3b34e-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bnkct\" (UID: \"f6376395-6632-4fb4-917d-3b5c9cd3b34e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bnkct"
Sep 16 00:57:16.670767 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:57:16.670670    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6376395-6632-4fb4-917d-3b5c9cd3b34e-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bnkct\" (UID: \"f6376395-6632-4fb4-917d-3b5c9cd3b34e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bnkct"
Sep 16 00:57:16.670767 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:57:16.670697    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6376395-6632-4fb4-917d-3b5c9cd3b34e-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bnkct\" (UID: \"f6376395-6632-4fb4-917d-3b5c9cd3b34e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bnkct"
Sep 16 00:57:16.670767 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:57:16.670749    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6376395-6632-4fb4-917d-3b5c9cd3b34e-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bnkct\" (UID: \"f6376395-6632-4fb4-917d-3b5c9cd3b34e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bnkct"
Sep 16 00:57:16.670881 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:57:16.670796    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vv52s\" (UniqueName: \"kubernetes.io/projected/f6376395-6632-4fb4-917d-3b5c9cd3b34e-kube-api-access-vv52s\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bnkct\" (UID: \"f6376395-6632-4fb4-917d-3b5c9cd3b34e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bnkct"
Sep 16 00:57:16.670881 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:57:16.670854    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6376395-6632-4fb4-917d-3b5c9cd3b34e-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bnkct\" (UID: \"f6376395-6632-4fb4-917d-3b5c9cd3b34e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bnkct"
Sep 16 00:57:16.771672 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:57:16.771645    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6376395-6632-4fb4-917d-3b5c9cd3b34e-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bnkct\" (UID: \"f6376395-6632-4fb4-917d-3b5c9cd3b34e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bnkct"
Sep 16 00:57:16.771771 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:57:16.771682    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6376395-6632-4fb4-917d-3b5c9cd3b34e-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bnkct\" (UID: \"f6376395-6632-4fb4-917d-3b5c9cd3b34e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bnkct"
Sep 16 00:57:16.771771 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:57:16.771700    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6376395-6632-4fb4-917d-3b5c9cd3b34e-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bnkct\" (UID: \"f6376395-6632-4fb4-917d-3b5c9cd3b34e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bnkct"
Sep 16 00:57:16.771842 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:57:16.771823    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6376395-6632-4fb4-917d-3b5c9cd3b34e-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bnkct\" (UID: \"f6376395-6632-4fb4-917d-3b5c9cd3b34e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bnkct"
Sep 16 00:57:16.771872 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:57:16.771857    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6376395-6632-4fb4-917d-3b5c9cd3b34e-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bnkct\" (UID: \"f6376395-6632-4fb4-917d-3b5c9cd3b34e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bnkct"
Sep 16 00:57:16.771931 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:57:16.771889    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vv52s\" (UniqueName: \"kubernetes.io/projected/f6376395-6632-4fb4-917d-3b5c9cd3b34e-kube-api-access-vv52s\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bnkct\" (UID: \"f6376395-6632-4fb4-917d-3b5c9cd3b34e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bnkct"
Sep 16 00:57:16.772104 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:57:16.772081    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6376395-6632-4fb4-917d-3b5c9cd3b34e-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bnkct\" (UID: \"f6376395-6632-4fb4-917d-3b5c9cd3b34e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bnkct"
Sep 16 00:57:16.772171 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:57:16.772140    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6376395-6632-4fb4-917d-3b5c9cd3b34e-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bnkct\" (UID: \"f6376395-6632-4fb4-917d-3b5c9cd3b34e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bnkct"
Sep 16 00:57:16.772248 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:57:16.772227    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6376395-6632-4fb4-917d-3b5c9cd3b34e-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bnkct\" (UID: \"f6376395-6632-4fb4-917d-3b5c9cd3b34e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bnkct"
Sep 16 00:57:16.773896 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:57:16.773877    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6376395-6632-4fb4-917d-3b5c9cd3b34e-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bnkct\" (UID: \"f6376395-6632-4fb4-917d-3b5c9cd3b34e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bnkct"
Sep 16 00:57:16.774140 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:57:16.774124    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6376395-6632-4fb4-917d-3b5c9cd3b34e-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bnkct\" (UID: \"f6376395-6632-4fb4-917d-3b5c9cd3b34e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bnkct"
Sep 16 00:57:16.779009 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:57:16.778986    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vv52s\" (UniqueName: \"kubernetes.io/projected/f6376395-6632-4fb4-917d-3b5c9cd3b34e-kube-api-access-vv52s\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bnkct\" (UID: \"f6376395-6632-4fb4-917d-3b5c9cd3b34e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bnkct"
Sep 16 00:57:16.894722 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:57:16.894670    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bnkct"
Sep 16 00:57:17.161832 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:57:17.161809    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-bnkct"]
Sep 16 00:57:17.163431 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:57:17.163405    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6376395_6632_4fb4_917d_3b5c9cd3b34e.slice/crio-bd26109941cc4c87df501974424501bed034e4053f6909c2122df37526747b03 WatchSource:0}: Error finding container bd26109941cc4c87df501974424501bed034e4053f6909c2122df37526747b03: Status 404 returned error can't find the container with id bd26109941cc4c87df501974424501bed034e4053f6909c2122df37526747b03
Sep 16 00:57:18.154240 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:57:18.154215    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bnkct" event={"ID":"f6376395-6632-4fb4-917d-3b5c9cd3b34e","Type":"ContainerStarted","Data":"bd26109941cc4c87df501974424501bed034e4053f6909c2122df37526747b03"}
Sep 16 00:57:19.160956 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:57:19.160919    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bnkct" event={"ID":"f6376395-6632-4fb4-917d-3b5c9cd3b34e","Type":"ContainerStarted","Data":"9dfe4ccca3e9c51c8e709dee4db43f39ea7fe7c0583c2c1153d97a13dc73a01a"}
Sep 16 00:57:19.178308 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:57:19.178259    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bnkct" podStartSLOduration=2.232342203 podStartE2EDuration="3.178244977s" totalImagesPullingTime="945.902774ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 00:57:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 00:57:19.176313208 +0000 UTC m=+853.957814801" watchObservedRunningTime="2026-09-16 00:57:19.178244977 +0000 UTC m=+853.959746604"
Sep 16 00:57:26.895362 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:57:26.895320    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bnkct"
Sep 16 00:57:26.895758 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:57:26.895380    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bnkct"
Sep 16 00:57:26.907825 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:57:26.907800    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bnkct"
Sep 16 00:57:27.208381 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:57:27.208310    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bnkct"
Sep 16 00:57:32.882533 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:57:32.882499    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-l66sh"]
Sep 16 00:57:32.885672 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:57:32.885653    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l66sh"
Sep 16 00:57:32.887948 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:57:32.887922    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Sep 16 00:57:32.895676 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:57:32.895654    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-l66sh"]
Sep 16 00:57:32.991194 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:57:32.991170    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9e126893-103b-474e-8a23-42ce6d7e853e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-l66sh\" (UID: \"9e126893-103b-474e-8a23-42ce6d7e853e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l66sh"
Sep 16 00:57:32.991308 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:57:32.991235    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9e126893-103b-474e-8a23-42ce6d7e853e-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-l66sh\" (UID: \"9e126893-103b-474e-8a23-42ce6d7e853e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l66sh"
Sep 16 00:57:32.991357 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:57:32.991323    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9e126893-103b-474e-8a23-42ce6d7e853e-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-l66sh\" (UID: \"9e126893-103b-474e-8a23-42ce6d7e853e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l66sh"
Sep 16 00:57:32.991401 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:57:32.991359    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9e126893-103b-474e-8a23-42ce6d7e853e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-l66sh\" (UID: \"9e126893-103b-474e-8a23-42ce6d7e853e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l66sh"
Sep 16 00:57:32.991401 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:57:32.991392    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e126893-103b-474e-8a23-42ce6d7e853e-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-l66sh\" (UID: \"9e126893-103b-474e-8a23-42ce6d7e853e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l66sh"
Sep 16 00:57:32.991498 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:57:32.991428    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2bcz\" (UniqueName: \"kubernetes.io/projected/9e126893-103b-474e-8a23-42ce6d7e853e-kube-api-access-r2bcz\") pod \"e2e-trlp-test-simulated-kserve-598466548d-l66sh\" (UID: \"9e126893-103b-474e-8a23-42ce6d7e853e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l66sh"
Sep 16 00:57:33.092384 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:57:33.092347    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9e126893-103b-474e-8a23-42ce6d7e853e-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-l66sh\" (UID: \"9e126893-103b-474e-8a23-42ce6d7e853e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l66sh"
Sep 16 00:57:33.092384 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:57:33.092383    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9e126893-103b-474e-8a23-42ce6d7e853e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-l66sh\" (UID: \"9e126893-103b-474e-8a23-42ce6d7e853e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l66sh"
Sep 16 00:57:33.092635 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:57:33.092402    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e126893-103b-474e-8a23-42ce6d7e853e-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-l66sh\" (UID: \"9e126893-103b-474e-8a23-42ce6d7e853e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l66sh"
Sep 16 00:57:33.092635 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:57:33.092419    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2bcz\" (UniqueName: \"kubernetes.io/projected/9e126893-103b-474e-8a23-42ce6d7e853e-kube-api-access-r2bcz\") pod \"e2e-trlp-test-simulated-kserve-598466548d-l66sh\" (UID: \"9e126893-103b-474e-8a23-42ce6d7e853e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l66sh"
Sep 16 00:57:33.092635 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:57:33.092460    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9e126893-103b-474e-8a23-42ce6d7e853e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-l66sh\" (UID: \"9e126893-103b-474e-8a23-42ce6d7e853e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l66sh"
Sep 16 00:57:33.092798 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:57:33.092640    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9e126893-103b-474e-8a23-42ce6d7e853e-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-l66sh\" (UID: \"9e126893-103b-474e-8a23-42ce6d7e853e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l66sh"
Sep 16 00:57:33.092798 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:57:33.092768    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9e126893-103b-474e-8a23-42ce6d7e853e-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-l66sh\" (UID: \"9e126893-103b-474e-8a23-42ce6d7e853e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l66sh"
Sep 16 00:57:33.092868 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:57:33.092830    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9e126893-103b-474e-8a23-42ce6d7e853e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-l66sh\" (UID: \"9e126893-103b-474e-8a23-42ce6d7e853e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l66sh"
Sep 16 00:57:33.092905 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:57:33.092872    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e126893-103b-474e-8a23-42ce6d7e853e-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-l66sh\" (UID: \"9e126893-103b-474e-8a23-42ce6d7e853e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l66sh"
Sep 16 00:57:33.095242 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:57:33.095181    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9e126893-103b-474e-8a23-42ce6d7e853e-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-l66sh\" (UID: \"9e126893-103b-474e-8a23-42ce6d7e853e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l66sh"
Sep 16 00:57:33.095585 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:57:33.095562    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9e126893-103b-474e-8a23-42ce6d7e853e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-l66sh\" (UID: \"9e126893-103b-474e-8a23-42ce6d7e853e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l66sh"
Sep 16 00:57:33.099720 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:57:33.099695    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2bcz\" (UniqueName: \"kubernetes.io/projected/9e126893-103b-474e-8a23-42ce6d7e853e-kube-api-access-r2bcz\") pod \"e2e-trlp-test-simulated-kserve-598466548d-l66sh\" (UID: \"9e126893-103b-474e-8a23-42ce6d7e853e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l66sh"
Sep 16 00:57:33.195825 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:57:33.195771    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l66sh"
Sep 16 00:57:33.468592 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:57:33.468566    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-l66sh"]
Sep 16 00:57:33.470413 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:57:33.470387    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e126893_103b_474e_8a23_42ce6d7e853e.slice/crio-ca2b1cbc8726a37949ad133a75cdf5902d4937a7819336b41c75658fcebb83fc WatchSource:0}: Error finding container ca2b1cbc8726a37949ad133a75cdf5902d4937a7819336b41c75658fcebb83fc: Status 404 returned error can't find the container with id ca2b1cbc8726a37949ad133a75cdf5902d4937a7819336b41c75658fcebb83fc
Sep 16 00:57:34.233525 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:57:34.233489    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l66sh" event={"ID":"9e126893-103b-474e-8a23-42ce6d7e853e","Type":"ContainerStarted","Data":"f4de7d8da75a85191872b37914b8c80ec8fdd1eb1841e35e6bded109343a819a"}
Sep 16 00:57:34.233943 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:57:34.233551    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l66sh" event={"ID":"9e126893-103b-474e-8a23-42ce6d7e853e","Type":"ContainerStarted","Data":"ca2b1cbc8726a37949ad133a75cdf5902d4937a7819336b41c75658fcebb83fc"}
Sep 16 00:57:34.250717 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:57:34.250674    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l66sh" podStartSLOduration=2.058998926 podStartE2EDuration="2.25065839s" totalImagesPullingTime="191.659464ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 00:57:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 00:57:34.249324057 +0000 UTC m=+869.030825650" watchObservedRunningTime="2026-09-16 00:57:34.25065839 +0000 UTC m=+869.032159994"
Sep 16 00:57:43.195943 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:57:43.195902    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l66sh"
Sep 16 00:57:43.196519 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:57:43.195978    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l66sh"
Sep 16 00:57:43.208152 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:57:43.208129    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l66sh"
Sep 16 00:57:43.286245 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:57:43.286220    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l66sh"
Sep 16 00:58:15.984252 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:58:15.984222    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-8jjjr"]
Sep 16 00:58:15.987630 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:58:15.987609    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8jjjr"
Sep 16 00:58:15.989907 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:58:15.989887    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Sep 16 00:58:15.996896 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:58:15.996875    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-8jjjr"]
Sep 16 00:58:16.009680 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:58:16.009655    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dc0b1ef3-3415-4c47-b20e-07a2e1b5f1dc-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8jjjr\" (UID: \"dc0b1ef3-3415-4c47-b20e-07a2e1b5f1dc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8jjjr"
Sep 16 00:58:16.009791 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:58:16.009721    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dc0b1ef3-3415-4c47-b20e-07a2e1b5f1dc-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8jjjr\" (UID: \"dc0b1ef3-3415-4c47-b20e-07a2e1b5f1dc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8jjjr"
Sep 16 00:58:16.009791 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:58:16.009757    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc0b1ef3-3415-4c47-b20e-07a2e1b5f1dc-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8jjjr\" (UID: \"dc0b1ef3-3415-4c47-b20e-07a2e1b5f1dc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8jjjr"
Sep 16 00:58:16.009791 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:58:16.009777    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djbch\" (UniqueName: \"kubernetes.io/projected/dc0b1ef3-3415-4c47-b20e-07a2e1b5f1dc-kube-api-access-djbch\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8jjjr\" (UID: \"dc0b1ef3-3415-4c47-b20e-07a2e1b5f1dc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8jjjr"
Sep 16 00:58:16.009949 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:58:16.009848    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dc0b1ef3-3415-4c47-b20e-07a2e1b5f1dc-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8jjjr\" (UID: \"dc0b1ef3-3415-4c47-b20e-07a2e1b5f1dc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8jjjr"
Sep 16 00:58:16.009949 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:58:16.009900    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dc0b1ef3-3415-4c47-b20e-07a2e1b5f1dc-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8jjjr\" (UID: \"dc0b1ef3-3415-4c47-b20e-07a2e1b5f1dc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8jjjr"
Sep 16 00:58:16.111094 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:58:16.111066    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dc0b1ef3-3415-4c47-b20e-07a2e1b5f1dc-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8jjjr\" (UID: \"dc0b1ef3-3415-4c47-b20e-07a2e1b5f1dc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8jjjr"
Sep 16 00:58:16.111225 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:58:16.111121    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dc0b1ef3-3415-4c47-b20e-07a2e1b5f1dc-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8jjjr\" (UID: \"dc0b1ef3-3415-4c47-b20e-07a2e1b5f1dc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8jjjr"
Sep 16 00:58:16.111225 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:58:16.111152    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc0b1ef3-3415-4c47-b20e-07a2e1b5f1dc-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8jjjr\" (UID: \"dc0b1ef3-3415-4c47-b20e-07a2e1b5f1dc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8jjjr"
Sep 16 00:58:16.111225 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:58:16.111186    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djbch\" (UniqueName: \"kubernetes.io/projected/dc0b1ef3-3415-4c47-b20e-07a2e1b5f1dc-kube-api-access-djbch\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8jjjr\" (UID: \"dc0b1ef3-3415-4c47-b20e-07a2e1b5f1dc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8jjjr"
Sep 16 00:58:16.111225 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:58:16.111218    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dc0b1ef3-3415-4c47-b20e-07a2e1b5f1dc-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8jjjr\" (UID: \"dc0b1ef3-3415-4c47-b20e-07a2e1b5f1dc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8jjjr"
Sep 16 00:58:16.111449 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:58:16.111263    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dc0b1ef3-3415-4c47-b20e-07a2e1b5f1dc-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8jjjr\" (UID: \"dc0b1ef3-3415-4c47-b20e-07a2e1b5f1dc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8jjjr"
Sep 16 00:58:16.111734 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:58:16.111709    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc0b1ef3-3415-4c47-b20e-07a2e1b5f1dc-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8jjjr\" (UID: \"dc0b1ef3-3415-4c47-b20e-07a2e1b5f1dc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8jjjr"
Sep 16 00:58:16.111863 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:58:16.111758    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dc0b1ef3-3415-4c47-b20e-07a2e1b5f1dc-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8jjjr\" (UID: \"dc0b1ef3-3415-4c47-b20e-07a2e1b5f1dc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8jjjr"
Sep 16 00:58:16.111863 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:58:16.111774    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dc0b1ef3-3415-4c47-b20e-07a2e1b5f1dc-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8jjjr\" (UID: \"dc0b1ef3-3415-4c47-b20e-07a2e1b5f1dc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8jjjr"
Sep 16 00:58:16.113408 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:58:16.113382    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dc0b1ef3-3415-4c47-b20e-07a2e1b5f1dc-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8jjjr\" (UID: \"dc0b1ef3-3415-4c47-b20e-07a2e1b5f1dc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8jjjr"
Sep 16 00:58:16.113892 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:58:16.113873    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dc0b1ef3-3415-4c47-b20e-07a2e1b5f1dc-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8jjjr\" (UID: \"dc0b1ef3-3415-4c47-b20e-07a2e1b5f1dc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8jjjr"
Sep 16 00:58:16.118109 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:58:16.118085    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djbch\" (UniqueName: \"kubernetes.io/projected/dc0b1ef3-3415-4c47-b20e-07a2e1b5f1dc-kube-api-access-djbch\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8jjjr\" (UID: \"dc0b1ef3-3415-4c47-b20e-07a2e1b5f1dc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8jjjr"
Sep 16 00:58:16.299157 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:58:16.299078    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8jjjr"
Sep 16 00:58:16.567184 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:58:16.567160    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-8jjjr"]
Sep 16 00:58:16.568987 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:58:16.568958    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc0b1ef3_3415_4c47_b20e_07a2e1b5f1dc.slice/crio-f4fe315c964865c0977ff5fad611166539502a6ba999a69cce16aeb5ff6b3452 WatchSource:0}: Error finding container f4fe315c964865c0977ff5fad611166539502a6ba999a69cce16aeb5ff6b3452: Status 404 returned error can't find the container with id f4fe315c964865c0977ff5fad611166539502a6ba999a69cce16aeb5ff6b3452
Sep 16 00:58:17.440845 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:58:17.440802    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8jjjr" event={"ID":"dc0b1ef3-3415-4c47-b20e-07a2e1b5f1dc","Type":"ContainerStarted","Data":"ed2d169d1dd70205c164ee4383b75447eab25bbb5bfc453496ce19a735bb02ec"}
Sep 16 00:58:17.441218 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:58:17.440870    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8jjjr" event={"ID":"dc0b1ef3-3415-4c47-b20e-07a2e1b5f1dc","Type":"ContainerStarted","Data":"f4fe315c964865c0977ff5fad611166539502a6ba999a69cce16aeb5ff6b3452"}
Sep 16 00:58:17.458163 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:58:17.458111    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8jjjr" podStartSLOduration=2.284281105 podStartE2EDuration="2.458094512s" totalImagesPullingTime="173.813407ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 00:58:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 00:58:17.45566698 +0000 UTC m=+912.237168572" watchObservedRunningTime="2026-09-16 00:58:17.458094512 +0000 UTC m=+912.239596104"
Sep 16 00:58:26.299420 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:58:26.299383    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8jjjr"
Sep 16 00:58:26.301816 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:58:26.299448    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8jjjr"
Sep 16 00:58:26.311951 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:58:26.311926    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8jjjr"
Sep 16 00:58:26.496160 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:58:26.496137    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8jjjr"
Sep 16 00:59:40.878110 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:40.878074    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-8j925"]
Sep 16 00:59:40.878784 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:40.878304    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-5b54b7fbb7-8j925" podUID="12f8e3b0-5262-47ac-9247-856ad9327cb6" containerName="payload-processing" containerID="cri-o://1122e67bdd61e72f79d7eef2aa882b2f30f67bf25849732e4ff9dadccb35293e" gracePeriod=30
Sep 16 00:59:41.073801 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:41.073731    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6b69bf8998-fhxjr"]
Sep 16 00:59:41.074072 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:41.074001    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6b69bf8998-fhxjr" podUID="d6303b5a-7625-4fd4-877a-8091ee23df43" containerName="manager" containerID="cri-o://62c564f11f9af0ed9b21943d8a381a2b6ec3e6f0085d4416623653d4fe80154c" gracePeriod=30
Sep 16 00:59:41.198459 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:41.198430    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-8j925"
Sep 16 00:59:41.245977 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:41.245948    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5ddq2\" (UniqueName: \"kubernetes.io/projected/12f8e3b0-5262-47ac-9247-856ad9327cb6-kube-api-access-5ddq2\") pod \"12f8e3b0-5262-47ac-9247-856ad9327cb6\" (UID: \"12f8e3b0-5262-47ac-9247-856ad9327cb6\") "
Sep 16 00:59:41.246101 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:41.246018    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12f8e3b0-5262-47ac-9247-856ad9327cb6-plugins-config-volume\") pod \"12f8e3b0-5262-47ac-9247-856ad9327cb6\" (UID: \"12f8e3b0-5262-47ac-9247-856ad9327cb6\") "
Sep 16 00:59:41.246358 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:41.246335    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12f8e3b0-5262-47ac-9247-856ad9327cb6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "12f8e3b0-5262-47ac-9247-856ad9327cb6" (UID: "12f8e3b0-5262-47ac-9247-856ad9327cb6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 00:59:41.249972 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:41.249943    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12f8e3b0-5262-47ac-9247-856ad9327cb6-kube-api-access-5ddq2" (OuterVolumeSpecName: "kube-api-access-5ddq2") pod "12f8e3b0-5262-47ac-9247-856ad9327cb6" (UID: "12f8e3b0-5262-47ac-9247-856ad9327cb6"). InnerVolumeSpecName "kube-api-access-5ddq2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 00:59:41.327296 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:41.327274    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b69bf8998-fhxjr"
Sep 16 00:59:41.346489 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:41.346439    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6303b5a-7625-4fd4-877a-8091ee23df43-metrics-certs\") pod \"d6303b5a-7625-4fd4-877a-8091ee23df43\" (UID: \"d6303b5a-7625-4fd4-877a-8091ee23df43\") "
Sep 16 00:59:41.346599 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:41.346531    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d6303b5a-7625-4fd4-877a-8091ee23df43-webhook-cert\") pod \"d6303b5a-7625-4fd4-877a-8091ee23df43\" (UID: \"d6303b5a-7625-4fd4-877a-8091ee23df43\") "
Sep 16 00:59:41.346599 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:41.346583    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nscmf\" (UniqueName: \"kubernetes.io/projected/d6303b5a-7625-4fd4-877a-8091ee23df43-kube-api-access-nscmf\") pod \"d6303b5a-7625-4fd4-877a-8091ee23df43\" (UID: \"d6303b5a-7625-4fd4-877a-8091ee23df43\") "
Sep 16 00:59:41.346821 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:41.346802    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12f8e3b0-5262-47ac-9247-856ad9327cb6-plugins-config-volume\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 00:59:41.346896 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:41.346824    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5ddq2\" (UniqueName: \"kubernetes.io/projected/12f8e3b0-5262-47ac-9247-856ad9327cb6-kube-api-access-5ddq2\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 00:59:41.348486 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:41.348444    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d6303b5a-7625-4fd4-877a-8091ee23df43-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "d6303b5a-7625-4fd4-877a-8091ee23df43" (UID: "d6303b5a-7625-4fd4-877a-8091ee23df43"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 00:59:41.348933 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:41.348909    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d6303b5a-7625-4fd4-877a-8091ee23df43-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "d6303b5a-7625-4fd4-877a-8091ee23df43" (UID: "d6303b5a-7625-4fd4-877a-8091ee23df43"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 00:59:41.348933 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:41.348922    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6303b5a-7625-4fd4-877a-8091ee23df43-kube-api-access-nscmf" (OuterVolumeSpecName: "kube-api-access-nscmf") pod "d6303b5a-7625-4fd4-877a-8091ee23df43" (UID: "d6303b5a-7625-4fd4-877a-8091ee23df43"). InnerVolumeSpecName "kube-api-access-nscmf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 00:59:41.447417 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:41.447346    2556 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6303b5a-7625-4fd4-877a-8091ee23df43-metrics-certs\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 00:59:41.447417 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:41.447373    2556 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d6303b5a-7625-4fd4-877a-8091ee23df43-webhook-cert\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 00:59:41.447417 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:41.447384    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nscmf\" (UniqueName: \"kubernetes.io/projected/d6303b5a-7625-4fd4-877a-8091ee23df43-kube-api-access-nscmf\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 00:59:41.854027 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:41.853993    2556 generic.go:356] "Generic (PLEG): container finished" podID="d6303b5a-7625-4fd4-877a-8091ee23df43" containerID="62c564f11f9af0ed9b21943d8a381a2b6ec3e6f0085d4416623653d4fe80154c" exitCode=0
Sep 16 00:59:41.854158 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:41.854037    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b69bf8998-fhxjr"
Sep 16 00:59:41.854158 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:41.854081    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b69bf8998-fhxjr" event={"ID":"d6303b5a-7625-4fd4-877a-8091ee23df43","Type":"ContainerDied","Data":"62c564f11f9af0ed9b21943d8a381a2b6ec3e6f0085d4416623653d4fe80154c"}
Sep 16 00:59:41.854158 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:41.854132    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b69bf8998-fhxjr" event={"ID":"d6303b5a-7625-4fd4-877a-8091ee23df43","Type":"ContainerDied","Data":"ffb92ea43e36c6fa6461ecc9569739f09e5fd41cd18b71723d0fdecb57da851e"}
Sep 16 00:59:41.854158 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:41.854159    2556 scope.go:117] "RemoveContainer" containerID="62c564f11f9af0ed9b21943d8a381a2b6ec3e6f0085d4416623653d4fe80154c"
Sep 16 00:59:41.855462 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:41.855442    2556 generic.go:356] "Generic (PLEG): container finished" podID="12f8e3b0-5262-47ac-9247-856ad9327cb6" containerID="1122e67bdd61e72f79d7eef2aa882b2f30f67bf25849732e4ff9dadccb35293e" exitCode=0
Sep 16 00:59:41.855628 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:41.855493    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-8j925" event={"ID":"12f8e3b0-5262-47ac-9247-856ad9327cb6","Type":"ContainerDied","Data":"1122e67bdd61e72f79d7eef2aa882b2f30f67bf25849732e4ff9dadccb35293e"}
Sep 16 00:59:41.855628 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:41.855513    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-8j925"
Sep 16 00:59:41.855628 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:41.855544    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-8j925" event={"ID":"12f8e3b0-5262-47ac-9247-856ad9327cb6","Type":"ContainerDied","Data":"9869dcd5dfdae4fd6e381df7a5f92d33145b0b7e5c9f66cfd3fb8825c17f6fc3"}
Sep 16 00:59:41.862993 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:41.862977    2556 scope.go:117] "RemoveContainer" containerID="62c564f11f9af0ed9b21943d8a381a2b6ec3e6f0085d4416623653d4fe80154c"
Sep 16 00:59:41.863269 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:59:41.863251    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62c564f11f9af0ed9b21943d8a381a2b6ec3e6f0085d4416623653d4fe80154c\": container with ID starting with 62c564f11f9af0ed9b21943d8a381a2b6ec3e6f0085d4416623653d4fe80154c not found: ID does not exist" containerID="62c564f11f9af0ed9b21943d8a381a2b6ec3e6f0085d4416623653d4fe80154c"
Sep 16 00:59:41.863346 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:41.863279    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62c564f11f9af0ed9b21943d8a381a2b6ec3e6f0085d4416623653d4fe80154c"} err="failed to get container status \"62c564f11f9af0ed9b21943d8a381a2b6ec3e6f0085d4416623653d4fe80154c\": rpc error: code = NotFound desc = could not find container \"62c564f11f9af0ed9b21943d8a381a2b6ec3e6f0085d4416623653d4fe80154c\": container with ID starting with 62c564f11f9af0ed9b21943d8a381a2b6ec3e6f0085d4416623653d4fe80154c not found: ID does not exist"
Sep 16 00:59:41.863346 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:41.863296    2556 scope.go:117] "RemoveContainer" containerID="1122e67bdd61e72f79d7eef2aa882b2f30f67bf25849732e4ff9dadccb35293e"
Sep 16 00:59:41.871189 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:41.871167    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6b69bf8998-fhxjr"]
Sep 16 00:59:41.871864 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:41.871848    2556 scope.go:117] "RemoveContainer" containerID="1122e67bdd61e72f79d7eef2aa882b2f30f67bf25849732e4ff9dadccb35293e"
Sep 16 00:59:41.872114 ip-10-0-130-207 kubenswrapper[2556]: E0916 00:59:41.872096    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1122e67bdd61e72f79d7eef2aa882b2f30f67bf25849732e4ff9dadccb35293e\": container with ID starting with 1122e67bdd61e72f79d7eef2aa882b2f30f67bf25849732e4ff9dadccb35293e not found: ID does not exist" containerID="1122e67bdd61e72f79d7eef2aa882b2f30f67bf25849732e4ff9dadccb35293e"
Sep 16 00:59:41.872172 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:41.872120    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1122e67bdd61e72f79d7eef2aa882b2f30f67bf25849732e4ff9dadccb35293e"} err="failed to get container status \"1122e67bdd61e72f79d7eef2aa882b2f30f67bf25849732e4ff9dadccb35293e\": rpc error: code = NotFound desc = could not find container \"1122e67bdd61e72f79d7eef2aa882b2f30f67bf25849732e4ff9dadccb35293e\": container with ID starting with 1122e67bdd61e72f79d7eef2aa882b2f30f67bf25849732e4ff9dadccb35293e not found: ID does not exist"
Sep 16 00:59:41.876265 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:41.876244    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6b69bf8998-fhxjr"]
Sep 16 00:59:41.886090 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:41.886069    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-8j925"]
Sep 16 00:59:41.891659 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:41.891641    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-8j925"]
Sep 16 00:59:42.261779 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:42.261744    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6b69bf8998-k7cr8"]
Sep 16 00:59:42.262095 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:42.262082    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6303b5a-7625-4fd4-877a-8091ee23df43" containerName="manager"
Sep 16 00:59:42.262139 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:42.262097    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6303b5a-7625-4fd4-877a-8091ee23df43" containerName="manager"
Sep 16 00:59:42.262139 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:42.262112    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12f8e3b0-5262-47ac-9247-856ad9327cb6" containerName="payload-processing"
Sep 16 00:59:42.262139 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:42.262118    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="12f8e3b0-5262-47ac-9247-856ad9327cb6" containerName="payload-processing"
Sep 16 00:59:42.262235 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:42.262174    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6303b5a-7625-4fd4-877a-8091ee23df43" containerName="manager"
Sep 16 00:59:42.262235 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:42.262181    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="12f8e3b0-5262-47ac-9247-856ad9327cb6" containerName="payload-processing"
Sep 16 00:59:42.266257 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:42.266241    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b69bf8998-k7cr8"
Sep 16 00:59:42.268537 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:42.268514    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 16 00:59:42.268656 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:42.268565    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 16 00:59:42.269309 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:42.269293    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 16 00:59:42.269352 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:42.269306    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-q5q6q\""
Sep 16 00:59:42.272816 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:42.272793    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6b69bf8998-k7cr8"]
Sep 16 00:59:42.353151 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:42.353127    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nk7wc\" (UniqueName: \"kubernetes.io/projected/11cd5078-9a7c-4208-8805-f9c7b6711521-kube-api-access-nk7wc\") pod \"maas-controller-6b69bf8998-k7cr8\" (UID: \"11cd5078-9a7c-4208-8805-f9c7b6711521\") " pod="opendatahub/maas-controller-6b69bf8998-k7cr8"
Sep 16 00:59:42.353260 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:42.353168    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/11cd5078-9a7c-4208-8805-f9c7b6711521-webhook-cert\") pod \"maas-controller-6b69bf8998-k7cr8\" (UID: \"11cd5078-9a7c-4208-8805-f9c7b6711521\") " pod="opendatahub/maas-controller-6b69bf8998-k7cr8"
Sep 16 00:59:42.353260 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:42.353188    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/11cd5078-9a7c-4208-8805-f9c7b6711521-metrics-certs\") pod \"maas-controller-6b69bf8998-k7cr8\" (UID: \"11cd5078-9a7c-4208-8805-f9c7b6711521\") " pod="opendatahub/maas-controller-6b69bf8998-k7cr8"
Sep 16 00:59:42.453858 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:42.453834    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nk7wc\" (UniqueName: \"kubernetes.io/projected/11cd5078-9a7c-4208-8805-f9c7b6711521-kube-api-access-nk7wc\") pod \"maas-controller-6b69bf8998-k7cr8\" (UID: \"11cd5078-9a7c-4208-8805-f9c7b6711521\") " pod="opendatahub/maas-controller-6b69bf8998-k7cr8"
Sep 16 00:59:42.453917 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:42.453882    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/11cd5078-9a7c-4208-8805-f9c7b6711521-webhook-cert\") pod \"maas-controller-6b69bf8998-k7cr8\" (UID: \"11cd5078-9a7c-4208-8805-f9c7b6711521\") " pod="opendatahub/maas-controller-6b69bf8998-k7cr8"
Sep 16 00:59:42.453917 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:42.453901    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/11cd5078-9a7c-4208-8805-f9c7b6711521-metrics-certs\") pod \"maas-controller-6b69bf8998-k7cr8\" (UID: \"11cd5078-9a7c-4208-8805-f9c7b6711521\") " pod="opendatahub/maas-controller-6b69bf8998-k7cr8"
Sep 16 00:59:42.456296 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:42.456270    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/11cd5078-9a7c-4208-8805-f9c7b6711521-metrics-certs\") pod \"maas-controller-6b69bf8998-k7cr8\" (UID: \"11cd5078-9a7c-4208-8805-f9c7b6711521\") " pod="opendatahub/maas-controller-6b69bf8998-k7cr8"
Sep 16 00:59:42.456379 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:42.456333    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/11cd5078-9a7c-4208-8805-f9c7b6711521-webhook-cert\") pod \"maas-controller-6b69bf8998-k7cr8\" (UID: \"11cd5078-9a7c-4208-8805-f9c7b6711521\") " pod="opendatahub/maas-controller-6b69bf8998-k7cr8"
Sep 16 00:59:42.461185 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:42.461168    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nk7wc\" (UniqueName: \"kubernetes.io/projected/11cd5078-9a7c-4208-8805-f9c7b6711521-kube-api-access-nk7wc\") pod \"maas-controller-6b69bf8998-k7cr8\" (UID: \"11cd5078-9a7c-4208-8805-f9c7b6711521\") " pod="opendatahub/maas-controller-6b69bf8998-k7cr8"
Sep 16 00:59:42.578091 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:42.578066    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b69bf8998-k7cr8"
Sep 16 00:59:42.846740 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:42.846713    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6b69bf8998-k7cr8"]
Sep 16 00:59:42.848669 ip-10-0-130-207 kubenswrapper[2556]: W0916 00:59:42.848634    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11cd5078_9a7c_4208_8805_f9c7b6711521.slice/crio-d2fc46660d1dfc759f865d32f0bc152283c282ab186fce21b20c77db91520f48 WatchSource:0}: Error finding container d2fc46660d1dfc759f865d32f0bc152283c282ab186fce21b20c77db91520f48: Status 404 returned error can't find the container with id d2fc46660d1dfc759f865d32f0bc152283c282ab186fce21b20c77db91520f48
Sep 16 00:59:42.849918 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:42.849901    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 00:59:42.863189 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:42.863151    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b69bf8998-k7cr8" event={"ID":"11cd5078-9a7c-4208-8805-f9c7b6711521","Type":"ContainerStarted","Data":"d2fc46660d1dfc759f865d32f0bc152283c282ab186fce21b20c77db91520f48"}
Sep 16 00:59:43.736872 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:43.736836    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12f8e3b0-5262-47ac-9247-856ad9327cb6" path="/var/lib/kubelet/pods/12f8e3b0-5262-47ac-9247-856ad9327cb6/volumes"
Sep 16 00:59:43.737228 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:43.737185    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6303b5a-7625-4fd4-877a-8091ee23df43" path="/var/lib/kubelet/pods/d6303b5a-7625-4fd4-877a-8091ee23df43/volumes"
Sep 16 00:59:43.870262 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:43.870227    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b69bf8998-k7cr8" event={"ID":"11cd5078-9a7c-4208-8805-f9c7b6711521","Type":"ContainerStarted","Data":"fc204e3dabbfbeba1aff559eb359912da35c35910a9285cb3c9fa8f4d8346986"}
Sep 16 00:59:43.870416 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:43.870283    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6b69bf8998-k7cr8"
Sep 16 00:59:43.887677 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:43.887636    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6b69bf8998-k7cr8" podStartSLOduration=1.4206200500000001 podStartE2EDuration="1.887620914s" totalImagesPullingTime="467.000864ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 00:59:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 00:59:43.885566292 +0000 UTC m=+998.667067885" watchObservedRunningTime="2026-09-16 00:59:43.887620914 +0000 UTC m=+998.669122506"
Sep 16 00:59:54.877192 ip-10-0-130-207 kubenswrapper[2556]: I0916 00:59:54.877159    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6b69bf8998-k7cr8"
Sep 16 01:00:46.448661 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:46.448626    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-96c840-5bdd89c7f8-2c9h9"]
Sep 16 01:00:46.452113 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:46.452089    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-96c840-5bdd89c7f8-2c9h9"
Sep 16 01:00:46.454575 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:46.454545    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w0-96c840-serving-cert\""
Sep 16 01:00:46.462506 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:46.462459    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-96c840-5bdd89c7f8-2c9h9"]
Sep 16 01:00:46.548309 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:46.548273    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e636703-6f9e-4043-961c-27b191e47d79-metrics-certs\") pod \"maas-api-e2e-worker-w0-96c840-5bdd89c7f8-2c9h9\" (UID: \"2e636703-6f9e-4043-961c-27b191e47d79\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-96c840-5bdd89c7f8-2c9h9"
Sep 16 01:00:46.548309 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:46.548309    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gsd65\" (UniqueName: \"kubernetes.io/projected/2e636703-6f9e-4043-961c-27b191e47d79-kube-api-access-gsd65\") pod \"maas-api-e2e-worker-w0-96c840-5bdd89c7f8-2c9h9\" (UID: \"2e636703-6f9e-4043-961c-27b191e47d79\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-96c840-5bdd89c7f8-2c9h9"
Sep 16 01:00:46.548554 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:46.548368    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2e636703-6f9e-4043-961c-27b191e47d79-maas-api-tls\") pod \"maas-api-e2e-worker-w0-96c840-5bdd89c7f8-2c9h9\" (UID: \"2e636703-6f9e-4043-961c-27b191e47d79\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-96c840-5bdd89c7f8-2c9h9"
Sep 16 01:00:46.548554 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:46.548384    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e636703-6f9e-4043-961c-27b191e47d79-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-96c840-5bdd89c7f8-2c9h9\" (UID: \"2e636703-6f9e-4043-961c-27b191e47d79\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-96c840-5bdd89c7f8-2c9h9"
Sep 16 01:00:46.649372 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:46.649337    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsd65\" (UniqueName: \"kubernetes.io/projected/2e636703-6f9e-4043-961c-27b191e47d79-kube-api-access-gsd65\") pod \"maas-api-e2e-worker-w0-96c840-5bdd89c7f8-2c9h9\" (UID: \"2e636703-6f9e-4043-961c-27b191e47d79\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-96c840-5bdd89c7f8-2c9h9"
Sep 16 01:00:46.649544 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:46.649427    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2e636703-6f9e-4043-961c-27b191e47d79-maas-api-tls\") pod \"maas-api-e2e-worker-w0-96c840-5bdd89c7f8-2c9h9\" (UID: \"2e636703-6f9e-4043-961c-27b191e47d79\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-96c840-5bdd89c7f8-2c9h9"
Sep 16 01:00:46.649544 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:46.649455    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e636703-6f9e-4043-961c-27b191e47d79-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-96c840-5bdd89c7f8-2c9h9\" (UID: \"2e636703-6f9e-4043-961c-27b191e47d79\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-96c840-5bdd89c7f8-2c9h9"
Sep 16 01:00:46.649631 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:46.649553    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e636703-6f9e-4043-961c-27b191e47d79-metrics-certs\") pod \"maas-api-e2e-worker-w0-96c840-5bdd89c7f8-2c9h9\" (UID: \"2e636703-6f9e-4043-961c-27b191e47d79\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-96c840-5bdd89c7f8-2c9h9"
Sep 16 01:00:46.649631 ip-10-0-130-207 kubenswrapper[2556]: E0916 01:00:46.649560    2556 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w0-96c840-serving-cert: secret "maas-api-e2e-worker-w0-96c840-serving-cert" not found
Sep 16 01:00:46.649631 ip-10-0-130-207 kubenswrapper[2556]: E0916 01:00:46.649620    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e636703-6f9e-4043-961c-27b191e47d79-maas-api-tls podName:2e636703-6f9e-4043-961c-27b191e47d79 nodeName:}" failed. No retries permitted until 2026-09-16 01:00:47.149604134 +0000 UTC m=+1061.931105704 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2e636703-6f9e-4043-961c-27b191e47d79-maas-api-tls") pod "maas-api-e2e-worker-w0-96c840-5bdd89c7f8-2c9h9" (UID: "2e636703-6f9e-4043-961c-27b191e47d79") : secret "maas-api-e2e-worker-w0-96c840-serving-cert" not found
Sep 16 01:00:46.650098 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:46.650081    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e636703-6f9e-4043-961c-27b191e47d79-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-96c840-5bdd89c7f8-2c9h9\" (UID: \"2e636703-6f9e-4043-961c-27b191e47d79\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-96c840-5bdd89c7f8-2c9h9"
Sep 16 01:00:46.651918 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:46.651889    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e636703-6f9e-4043-961c-27b191e47d79-metrics-certs\") pod \"maas-api-e2e-worker-w0-96c840-5bdd89c7f8-2c9h9\" (UID: \"2e636703-6f9e-4043-961c-27b191e47d79\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-96c840-5bdd89c7f8-2c9h9"
Sep 16 01:00:46.657251 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:46.657232    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gsd65\" (UniqueName: \"kubernetes.io/projected/2e636703-6f9e-4043-961c-27b191e47d79-kube-api-access-gsd65\") pod \"maas-api-e2e-worker-w0-96c840-5bdd89c7f8-2c9h9\" (UID: \"2e636703-6f9e-4043-961c-27b191e47d79\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-96c840-5bdd89c7f8-2c9h9"
Sep 16 01:00:47.152926 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:47.152889    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2e636703-6f9e-4043-961c-27b191e47d79-maas-api-tls\") pod \"maas-api-e2e-worker-w0-96c840-5bdd89c7f8-2c9h9\" (UID: \"2e636703-6f9e-4043-961c-27b191e47d79\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-96c840-5bdd89c7f8-2c9h9"
Sep 16 01:00:47.155245 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:47.155216    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2e636703-6f9e-4043-961c-27b191e47d79-maas-api-tls\") pod \"maas-api-e2e-worker-w0-96c840-5bdd89c7f8-2c9h9\" (UID: \"2e636703-6f9e-4043-961c-27b191e47d79\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-96c840-5bdd89c7f8-2c9h9"
Sep 16 01:00:47.194480 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:47.194445    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-96c840-56b9dd4654-5rp7j"]
Sep 16 01:00:47.198008 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:47.197990    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-96c840-56b9dd4654-5rp7j"
Sep 16 01:00:47.200440 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:47.200256    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-96c840\""
Sep 16 01:00:47.200667 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:47.200646    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-96c840-dockercfg-mdrxg\""
Sep 16 01:00:47.207145 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:47.207124    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-96c840-56b9dd4654-5rp7j"]
Sep 16 01:00:47.219175 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:47.219149    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-96c840-76f5c8849f-ks9pz"]
Sep 16 01:00:47.222415 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:47.222398    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-96c840-76f5c8849f-ks9pz"
Sep 16 01:00:47.236526 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:47.236501    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-96c840-76f5c8849f-ks9pz"]
Sep 16 01:00:47.354173 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:47.354141    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2583e2cb-79b4-4b90-931f-99de7835d6e4-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-96c840-56b9dd4654-5rp7j\" (UID: \"2583e2cb-79b4-4b90-931f-99de7835d6e4\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-96c840-56b9dd4654-5rp7j"
Sep 16 01:00:47.354336 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:47.354195    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkwnp\" (UniqueName: \"kubernetes.io/projected/2583e2cb-79b4-4b90-931f-99de7835d6e4-kube-api-access-tkwnp\") pod \"payload-pre-processing-e2e-worker-w0-96c840-56b9dd4654-5rp7j\" (UID: \"2583e2cb-79b4-4b90-931f-99de7835d6e4\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-96c840-56b9dd4654-5rp7j"
Sep 16 01:00:47.354336 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:47.354271    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7frt\" (UniqueName: \"kubernetes.io/projected/fa9f4c4b-4e60-4638-aa94-b03b1877df9b-kube-api-access-b7frt\") pod \"payload-processing-e2e-worker-w0-96c840-76f5c8849f-ks9pz\" (UID: \"fa9f4c4b-4e60-4638-aa94-b03b1877df9b\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-96c840-76f5c8849f-ks9pz"
Sep 16 01:00:47.354433 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:47.354354    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa9f4c4b-4e60-4638-aa94-b03b1877df9b-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-96c840-76f5c8849f-ks9pz\" (UID: \"fa9f4c4b-4e60-4638-aa94-b03b1877df9b\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-96c840-76f5c8849f-ks9pz"
Sep 16 01:00:47.363064 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:47.363039    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-96c840-5bdd89c7f8-2c9h9"
Sep 16 01:00:47.455303 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:47.455273    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2583e2cb-79b4-4b90-931f-99de7835d6e4-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-96c840-56b9dd4654-5rp7j\" (UID: \"2583e2cb-79b4-4b90-931f-99de7835d6e4\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-96c840-56b9dd4654-5rp7j"
Sep 16 01:00:47.455711 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:47.455329    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkwnp\" (UniqueName: \"kubernetes.io/projected/2583e2cb-79b4-4b90-931f-99de7835d6e4-kube-api-access-tkwnp\") pod \"payload-pre-processing-e2e-worker-w0-96c840-56b9dd4654-5rp7j\" (UID: \"2583e2cb-79b4-4b90-931f-99de7835d6e4\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-96c840-56b9dd4654-5rp7j"
Sep 16 01:00:47.455711 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:47.455347    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7frt\" (UniqueName: \"kubernetes.io/projected/fa9f4c4b-4e60-4638-aa94-b03b1877df9b-kube-api-access-b7frt\") pod \"payload-processing-e2e-worker-w0-96c840-76f5c8849f-ks9pz\" (UID: \"fa9f4c4b-4e60-4638-aa94-b03b1877df9b\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-96c840-76f5c8849f-ks9pz"
Sep 16 01:00:47.455711 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:47.455384    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa9f4c4b-4e60-4638-aa94-b03b1877df9b-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-96c840-76f5c8849f-ks9pz\" (UID: \"fa9f4c4b-4e60-4638-aa94-b03b1877df9b\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-96c840-76f5c8849f-ks9pz"
Sep 16 01:00:47.455999 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:47.455979    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2583e2cb-79b4-4b90-931f-99de7835d6e4-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-96c840-56b9dd4654-5rp7j\" (UID: \"2583e2cb-79b4-4b90-931f-99de7835d6e4\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-96c840-56b9dd4654-5rp7j"
Sep 16 01:00:47.456451 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:47.456425    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa9f4c4b-4e60-4638-aa94-b03b1877df9b-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-96c840-76f5c8849f-ks9pz\" (UID: \"fa9f4c4b-4e60-4638-aa94-b03b1877df9b\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-96c840-76f5c8849f-ks9pz"
Sep 16 01:00:47.462665 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:47.462643    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkwnp\" (UniqueName: \"kubernetes.io/projected/2583e2cb-79b4-4b90-931f-99de7835d6e4-kube-api-access-tkwnp\") pod \"payload-pre-processing-e2e-worker-w0-96c840-56b9dd4654-5rp7j\" (UID: \"2583e2cb-79b4-4b90-931f-99de7835d6e4\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-96c840-56b9dd4654-5rp7j"
Sep 16 01:00:47.462782 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:47.462680    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7frt\" (UniqueName: \"kubernetes.io/projected/fa9f4c4b-4e60-4638-aa94-b03b1877df9b-kube-api-access-b7frt\") pod \"payload-processing-e2e-worker-w0-96c840-76f5c8849f-ks9pz\" (UID: \"fa9f4c4b-4e60-4638-aa94-b03b1877df9b\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-96c840-76f5c8849f-ks9pz"
Sep 16 01:00:47.509239 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:47.509211    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-96c840-56b9dd4654-5rp7j"
Sep 16 01:00:47.535952 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:47.535925    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-96c840-76f5c8849f-ks9pz"
Sep 16 01:00:47.633598 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:47.633577    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-96c840-5bdd89c7f8-2c9h9"]
Sep 16 01:00:47.636065 ip-10-0-130-207 kubenswrapper[2556]: W0916 01:00:47.636038    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e636703_6f9e_4043_961c_27b191e47d79.slice/crio-9911d072888bf6c7a67938ea9a2faa808a89b225584a897164924748466d487e WatchSource:0}: Error finding container 9911d072888bf6c7a67938ea9a2faa808a89b225584a897164924748466d487e: Status 404 returned error can't find the container with id 9911d072888bf6c7a67938ea9a2faa808a89b225584a897164924748466d487e
Sep 16 01:00:47.782146 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:47.782119    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-96c840-56b9dd4654-5rp7j"]
Sep 16 01:00:47.783279 ip-10-0-130-207 kubenswrapper[2556]: W0916 01:00:47.783254    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2583e2cb_79b4_4b90_931f_99de7835d6e4.slice/crio-057da483159e7b7b6df189096ac008770d7d149694c7dfc0698343285239fe52 WatchSource:0}: Error finding container 057da483159e7b7b6df189096ac008770d7d149694c7dfc0698343285239fe52: Status 404 returned error can't find the container with id 057da483159e7b7b6df189096ac008770d7d149694c7dfc0698343285239fe52
Sep 16 01:00:47.816102 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:47.816082    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-96c840-76f5c8849f-ks9pz"]
Sep 16 01:00:47.818019 ip-10-0-130-207 kubenswrapper[2556]: W0916 01:00:47.817942    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa9f4c4b_4e60_4638_aa94_b03b1877df9b.slice/crio-fc131c6f22313126cd419ec4416c1e89e205d49415806fa91adfe0399122f159 WatchSource:0}: Error finding container fc131c6f22313126cd419ec4416c1e89e205d49415806fa91adfe0399122f159: Status 404 returned error can't find the container with id fc131c6f22313126cd419ec4416c1e89e205d49415806fa91adfe0399122f159
Sep 16 01:00:48.185157 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:48.185124    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-96c840-5bdd89c7f8-2c9h9" event={"ID":"2e636703-6f9e-4043-961c-27b191e47d79","Type":"ContainerStarted","Data":"9911d072888bf6c7a67938ea9a2faa808a89b225584a897164924748466d487e"}
Sep 16 01:00:48.186961 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:48.186930    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-96c840-76f5c8849f-ks9pz" event={"ID":"fa9f4c4b-4e60-4638-aa94-b03b1877df9b","Type":"ContainerStarted","Data":"bfd75d69293cde13b23f4266a6d88577b96760ad802c925934c5727e441a18e8"}
Sep 16 01:00:48.187139 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:48.186984    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-96c840-76f5c8849f-ks9pz" event={"ID":"fa9f4c4b-4e60-4638-aa94-b03b1877df9b","Type":"ContainerStarted","Data":"fc131c6f22313126cd419ec4416c1e89e205d49415806fa91adfe0399122f159"}
Sep 16 01:00:48.187139 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:48.187017    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-96c840-76f5c8849f-ks9pz"
Sep 16 01:00:48.188613 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:48.188590    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-96c840-56b9dd4654-5rp7j" event={"ID":"2583e2cb-79b4-4b90-931f-99de7835d6e4","Type":"ContainerStarted","Data":"6389de8e731342a8ce369b868fb1aea0a2638d9431130a93f5c0aabfec575c23"}
Sep 16 01:00:48.188717 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:48.188630    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-96c840-56b9dd4654-5rp7j" event={"ID":"2583e2cb-79b4-4b90-931f-99de7835d6e4","Type":"ContainerStarted","Data":"057da483159e7b7b6df189096ac008770d7d149694c7dfc0698343285239fe52"}
Sep 16 01:00:48.188717 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:48.188661    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-96c840-56b9dd4654-5rp7j"
Sep 16 01:00:48.203240 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:48.203198    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w0-96c840-76f5c8849f-ks9pz" podStartSLOduration=1.203184659 podStartE2EDuration="1.203184659s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 01:00:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 01:00:48.201643127 +0000 UTC m=+1062.983144719" watchObservedRunningTime="2026-09-16 01:00:48.203184659 +0000 UTC m=+1062.984686253"
Sep 16 01:00:48.216812 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:48.216754    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-96c840-56b9dd4654-5rp7j" podStartSLOduration=1.216736447 podStartE2EDuration="1.216736447s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 01:00:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 01:00:48.216035098 +0000 UTC m=+1062.997536693" watchObservedRunningTime="2026-09-16 01:00:48.216736447 +0000 UTC m=+1062.998238044"
Sep 16 01:00:48.233254 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:48.233229    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-2534e3-5bcb57cff4-tz8rf"]
Sep 16 01:00:48.236359 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:48.236342    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2534e3-5bcb57cff4-tz8rf"
Sep 16 01:00:48.238581 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:48.238560    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w3-2534e3\""
Sep 16 01:00:48.238701 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:48.238616    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w3-2534e3-dockercfg-6blkl\""
Sep 16 01:00:48.247292 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:48.247271    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-2534e3-5bcb57cff4-tz8rf"]
Sep 16 01:00:48.363310 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:48.363256    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eca4a3b2-0c02-4e9e-a86d-0be2ec3518c2-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-2534e3-5bcb57cff4-tz8rf\" (UID: \"eca4a3b2-0c02-4e9e-a86d-0be2ec3518c2\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2534e3-5bcb57cff4-tz8rf"
Sep 16 01:00:48.363553 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:48.363389    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7bv8\" (UniqueName: \"kubernetes.io/projected/eca4a3b2-0c02-4e9e-a86d-0be2ec3518c2-kube-api-access-t7bv8\") pod \"payload-pre-processing-e2e-worker-w3-2534e3-5bcb57cff4-tz8rf\" (UID: \"eca4a3b2-0c02-4e9e-a86d-0be2ec3518c2\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2534e3-5bcb57cff4-tz8rf"
Sep 16 01:00:48.464189 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:48.464111    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eca4a3b2-0c02-4e9e-a86d-0be2ec3518c2-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-2534e3-5bcb57cff4-tz8rf\" (UID: \"eca4a3b2-0c02-4e9e-a86d-0be2ec3518c2\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2534e3-5bcb57cff4-tz8rf"
Sep 16 01:00:48.464618 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:48.464202    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7bv8\" (UniqueName: \"kubernetes.io/projected/eca4a3b2-0c02-4e9e-a86d-0be2ec3518c2-kube-api-access-t7bv8\") pod \"payload-pre-processing-e2e-worker-w3-2534e3-5bcb57cff4-tz8rf\" (UID: \"eca4a3b2-0c02-4e9e-a86d-0be2ec3518c2\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2534e3-5bcb57cff4-tz8rf"
Sep 16 01:00:48.464999 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:48.464974    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eca4a3b2-0c02-4e9e-a86d-0be2ec3518c2-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-2534e3-5bcb57cff4-tz8rf\" (UID: \"eca4a3b2-0c02-4e9e-a86d-0be2ec3518c2\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2534e3-5bcb57cff4-tz8rf"
Sep 16 01:00:48.471790 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:48.471760    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7bv8\" (UniqueName: \"kubernetes.io/projected/eca4a3b2-0c02-4e9e-a86d-0be2ec3518c2-kube-api-access-t7bv8\") pod \"payload-pre-processing-e2e-worker-w3-2534e3-5bcb57cff4-tz8rf\" (UID: \"eca4a3b2-0c02-4e9e-a86d-0be2ec3518c2\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2534e3-5bcb57cff4-tz8rf"
Sep 16 01:00:48.494582 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:48.494556    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-d58169-bbb6d87f4-hhc2r"]
Sep 16 01:00:48.498066 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:48.498049    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d58169-bbb6d87f4-hhc2r"
Sep 16 01:00:48.500045 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:48.500025    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-d58169-serving-cert\""
Sep 16 01:00:48.507865 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:48.507847    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-d58169-bbb6d87f4-hhc2r"]
Sep 16 01:00:48.547723 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:48.547702    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2534e3-5bcb57cff4-tz8rf"
Sep 16 01:00:48.666310 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:48.666077    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c51bbd7-e903-4350-956b-25c5c56d9804-openshift-service-ca\") pod \"maas-api-e2e-inf-a-d58169-bbb6d87f4-hhc2r\" (UID: \"2c51bbd7-e903-4350-956b-25c5c56d9804\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d58169-bbb6d87f4-hhc2r"
Sep 16 01:00:48.666310 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:48.666132    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c51bbd7-e903-4350-956b-25c5c56d9804-metrics-certs\") pod \"maas-api-e2e-inf-a-d58169-bbb6d87f4-hhc2r\" (UID: \"2c51bbd7-e903-4350-956b-25c5c56d9804\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d58169-bbb6d87f4-hhc2r"
Sep 16 01:00:48.666310 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:48.666168    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c51bbd7-e903-4350-956b-25c5c56d9804-maas-api-tls\") pod \"maas-api-e2e-inf-a-d58169-bbb6d87f4-hhc2r\" (UID: \"2c51bbd7-e903-4350-956b-25c5c56d9804\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d58169-bbb6d87f4-hhc2r"
Sep 16 01:00:48.666310 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:48.666200    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xp5m\" (UniqueName: \"kubernetes.io/projected/2c51bbd7-e903-4350-956b-25c5c56d9804-kube-api-access-5xp5m\") pod \"maas-api-e2e-inf-a-d58169-bbb6d87f4-hhc2r\" (UID: \"2c51bbd7-e903-4350-956b-25c5c56d9804\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d58169-bbb6d87f4-hhc2r"
Sep 16 01:00:48.767104 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:48.767021    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c51bbd7-e903-4350-956b-25c5c56d9804-openshift-service-ca\") pod \"maas-api-e2e-inf-a-d58169-bbb6d87f4-hhc2r\" (UID: \"2c51bbd7-e903-4350-956b-25c5c56d9804\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d58169-bbb6d87f4-hhc2r"
Sep 16 01:00:48.767104 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:48.767059    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c51bbd7-e903-4350-956b-25c5c56d9804-metrics-certs\") pod \"maas-api-e2e-inf-a-d58169-bbb6d87f4-hhc2r\" (UID: \"2c51bbd7-e903-4350-956b-25c5c56d9804\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d58169-bbb6d87f4-hhc2r"
Sep 16 01:00:48.767104 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:48.767079    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c51bbd7-e903-4350-956b-25c5c56d9804-maas-api-tls\") pod \"maas-api-e2e-inf-a-d58169-bbb6d87f4-hhc2r\" (UID: \"2c51bbd7-e903-4350-956b-25c5c56d9804\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d58169-bbb6d87f4-hhc2r"
Sep 16 01:00:48.767356 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:48.767228    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xp5m\" (UniqueName: \"kubernetes.io/projected/2c51bbd7-e903-4350-956b-25c5c56d9804-kube-api-access-5xp5m\") pod \"maas-api-e2e-inf-a-d58169-bbb6d87f4-hhc2r\" (UID: \"2c51bbd7-e903-4350-956b-25c5c56d9804\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d58169-bbb6d87f4-hhc2r"
Sep 16 01:00:48.767769 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:48.767741    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c51bbd7-e903-4350-956b-25c5c56d9804-openshift-service-ca\") pod \"maas-api-e2e-inf-a-d58169-bbb6d87f4-hhc2r\" (UID: \"2c51bbd7-e903-4350-956b-25c5c56d9804\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d58169-bbb6d87f4-hhc2r"
Sep 16 01:00:48.769604 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:48.769581    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c51bbd7-e903-4350-956b-25c5c56d9804-metrics-certs\") pod \"maas-api-e2e-inf-a-d58169-bbb6d87f4-hhc2r\" (UID: \"2c51bbd7-e903-4350-956b-25c5c56d9804\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d58169-bbb6d87f4-hhc2r"
Sep 16 01:00:48.769741 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:48.769668    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c51bbd7-e903-4350-956b-25c5c56d9804-maas-api-tls\") pod \"maas-api-e2e-inf-a-d58169-bbb6d87f4-hhc2r\" (UID: \"2c51bbd7-e903-4350-956b-25c5c56d9804\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d58169-bbb6d87f4-hhc2r"
Sep 16 01:00:48.774580 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:48.774561    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xp5m\" (UniqueName: \"kubernetes.io/projected/2c51bbd7-e903-4350-956b-25c5c56d9804-kube-api-access-5xp5m\") pod \"maas-api-e2e-inf-a-d58169-bbb6d87f4-hhc2r\" (UID: \"2c51bbd7-e903-4350-956b-25c5c56d9804\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d58169-bbb6d87f4-hhc2r"
Sep 16 01:00:48.810588 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:48.810566    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d58169-bbb6d87f4-hhc2r"
Sep 16 01:00:48.817333 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:48.817311    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-2534e3-5bcb57cff4-tz8rf"]
Sep 16 01:00:48.821975 ip-10-0-130-207 kubenswrapper[2556]: W0916 01:00:48.821948    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeca4a3b2_0c02_4e9e_a86d_0be2ec3518c2.slice/crio-f0356ca0d75a00d7ad2403a9b478ea49a5e0596890484decf0dc7f849ad8cef8 WatchSource:0}: Error finding container f0356ca0d75a00d7ad2403a9b478ea49a5e0596890484decf0dc7f849ad8cef8: Status 404 returned error can't find the container with id f0356ca0d75a00d7ad2403a9b478ea49a5e0596890484decf0dc7f849ad8cef8
Sep 16 01:00:49.087180 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:49.087153    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-d58169-bbb6d87f4-hhc2r"]
Sep 16 01:00:49.088514 ip-10-0-130-207 kubenswrapper[2556]: W0916 01:00:49.088489    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c51bbd7_e903_4350_956b_25c5c56d9804.slice/crio-9e743fa8a97ffc72f0c455df863782dabed832ba319031603b7d56767f95b5d6 WatchSource:0}: Error finding container 9e743fa8a97ffc72f0c455df863782dabed832ba319031603b7d56767f95b5d6: Status 404 returned error can't find the container with id 9e743fa8a97ffc72f0c455df863782dabed832ba319031603b7d56767f95b5d6
Sep 16 01:00:49.201440 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:49.201401    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d58169-bbb6d87f4-hhc2r" event={"ID":"2c51bbd7-e903-4350-956b-25c5c56d9804","Type":"ContainerStarted","Data":"9e743fa8a97ffc72f0c455df863782dabed832ba319031603b7d56767f95b5d6"}
Sep 16 01:00:49.202899 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:49.202867    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-96c840-5bdd89c7f8-2c9h9" event={"ID":"2e636703-6f9e-4043-961c-27b191e47d79","Type":"ContainerStarted","Data":"c42301cb3b3e10f006e30b3916c9a108fe28a9344800f6b9ec4c5b8330b83581"}
Sep 16 01:00:49.203115 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:49.203097    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-96c840-5bdd89c7f8-2c9h9"
Sep 16 01:00:49.204447 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:49.204420    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2534e3-5bcb57cff4-tz8rf" event={"ID":"eca4a3b2-0c02-4e9e-a86d-0be2ec3518c2","Type":"ContainerStarted","Data":"405b74649a3707672b00fb0988b6f58f114c1cacb8c4bfb38c1cf8f49d16908c"}
Sep 16 01:00:49.204590 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:49.204506    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2534e3-5bcb57cff4-tz8rf"
Sep 16 01:00:49.204590 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:49.204540    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2534e3-5bcb57cff4-tz8rf" event={"ID":"eca4a3b2-0c02-4e9e-a86d-0be2ec3518c2","Type":"ContainerStarted","Data":"f0356ca0d75a00d7ad2403a9b478ea49a5e0596890484decf0dc7f849ad8cef8"}
Sep 16 01:00:49.235541 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:49.235490    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2534e3-5bcb57cff4-tz8rf" podStartSLOduration=1.235455639 podStartE2EDuration="1.235455639s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 01:00:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 01:00:49.234045258 +0000 UTC m=+1064.015546849" watchObservedRunningTime="2026-09-16 01:00:49.235455639 +0000 UTC m=+1064.016957243"
Sep 16 01:00:49.236362 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:49.236332    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-96c840-5bdd89c7f8-2c9h9" podStartSLOduration=1.836757214 podStartE2EDuration="3.236323616s" totalImagesPullingTime="1.399566402s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 01:00:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 01:00:49.219106782 +0000 UTC m=+1064.000608375" watchObservedRunningTime="2026-09-16 01:00:49.236323616 +0000 UTC m=+1064.017825208"
Sep 16 01:00:51.217948 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:51.217910    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d58169-bbb6d87f4-hhc2r" event={"ID":"2c51bbd7-e903-4350-956b-25c5c56d9804","Type":"ContainerStarted","Data":"696513912d766453bd62e6ba7a9dc461c95b9df6acdf5aa3e1718426288dd2ae"}
Sep 16 01:00:51.218405 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:51.218016    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d58169-bbb6d87f4-hhc2r"
Sep 16 01:00:51.235278 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:51.235226    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d58169-bbb6d87f4-hhc2r" podStartSLOduration=1.530730323 podStartE2EDuration="3.235211884s" totalImagesPullingTime="1.704481561s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 01:00:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 01:00:51.233020058 +0000 UTC m=+1066.014521650" watchObservedRunningTime="2026-09-16 01:00:51.235211884 +0000 UTC m=+1066.016713508"
Sep 16 01:00:55.215134 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:55.215101    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-96c840-5bdd89c7f8-2c9h9"
Sep 16 01:00:57.227438 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:57.227408    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d58169-bbb6d87f4-hhc2r"
Sep 16 01:00:59.206922 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:59.206838    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-96c840-56b9dd4654-5rp7j"
Sep 16 01:00:59.206922 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:00:59.206920    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-96c840-76f5c8849f-ks9pz"
Sep 16 01:01:00.213018 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:01:00.212985    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2534e3-5bcb57cff4-tz8rf"
Sep 16 01:01:24.924262 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:01:24.924232    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-6d0d72-5875fff75d-htxqr"]
Sep 16 01:01:24.928928 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:01:24.928910    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6d0d72-5875fff75d-htxqr"
Sep 16 01:01:24.931272 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:01:24.931253    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-6d0d72\""
Sep 16 01:01:24.937486 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:01:24.937451    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-6d0d72-5875fff75d-htxqr"]
Sep 16 01:01:24.967727 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:01:24.967701    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-6d0d72-66b74965bf-x4dl6"]
Sep 16 01:01:24.971161 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:01:24.971146    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-6d0d72-66b74965bf-x4dl6"
Sep 16 01:01:24.974069 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:01:24.974050    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-6d0d72-dockercfg-9qwd6\""
Sep 16 01:01:24.980728 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:01:24.980707    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-6d0d72-66b74965bf-x4dl6"]
Sep 16 01:01:25.061814 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:01:25.061749    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4fb25d7a-dcc9-45eb-9f75-d9db5e74d228-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-6d0d72-66b74965bf-x4dl6\" (UID: \"4fb25d7a-dcc9-45eb-9f75-d9db5e74d228\") " pod="openshift-ingress/payload-processing-e2e-inf-b-6d0d72-66b74965bf-x4dl6"
Sep 16 01:01:25.061814 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:01:25.061796    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kln5\" (UniqueName: \"kubernetes.io/projected/35acb09e-33d9-46bf-8c4f-ea8d6ec02785-kube-api-access-7kln5\") pod \"payload-pre-processing-e2e-inf-b-6d0d72-5875fff75d-htxqr\" (UID: \"35acb09e-33d9-46bf-8c4f-ea8d6ec02785\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6d0d72-5875fff75d-htxqr"
Sep 16 01:01:25.062019 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:01:25.061944    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35acb09e-33d9-46bf-8c4f-ea8d6ec02785-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-6d0d72-5875fff75d-htxqr\" (UID: \"35acb09e-33d9-46bf-8c4f-ea8d6ec02785\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6d0d72-5875fff75d-htxqr"
Sep 16 01:01:25.062019 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:01:25.061997    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cx86g\" (UniqueName: \"kubernetes.io/projected/4fb25d7a-dcc9-45eb-9f75-d9db5e74d228-kube-api-access-cx86g\") pod \"payload-processing-e2e-inf-b-6d0d72-66b74965bf-x4dl6\" (UID: \"4fb25d7a-dcc9-45eb-9f75-d9db5e74d228\") " pod="openshift-ingress/payload-processing-e2e-inf-b-6d0d72-66b74965bf-x4dl6"
Sep 16 01:01:25.163455 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:01:25.163422    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kln5\" (UniqueName: \"kubernetes.io/projected/35acb09e-33d9-46bf-8c4f-ea8d6ec02785-kube-api-access-7kln5\") pod \"payload-pre-processing-e2e-inf-b-6d0d72-5875fff75d-htxqr\" (UID: \"35acb09e-33d9-46bf-8c4f-ea8d6ec02785\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6d0d72-5875fff75d-htxqr"
Sep 16 01:01:25.163627 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:01:25.163503    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35acb09e-33d9-46bf-8c4f-ea8d6ec02785-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-6d0d72-5875fff75d-htxqr\" (UID: \"35acb09e-33d9-46bf-8c4f-ea8d6ec02785\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6d0d72-5875fff75d-htxqr"
Sep 16 01:01:25.163627 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:01:25.163526    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cx86g\" (UniqueName: \"kubernetes.io/projected/4fb25d7a-dcc9-45eb-9f75-d9db5e74d228-kube-api-access-cx86g\") pod \"payload-processing-e2e-inf-b-6d0d72-66b74965bf-x4dl6\" (UID: \"4fb25d7a-dcc9-45eb-9f75-d9db5e74d228\") " pod="openshift-ingress/payload-processing-e2e-inf-b-6d0d72-66b74965bf-x4dl6"
Sep 16 01:01:25.163627 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:01:25.163562    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4fb25d7a-dcc9-45eb-9f75-d9db5e74d228-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-6d0d72-66b74965bf-x4dl6\" (UID: \"4fb25d7a-dcc9-45eb-9f75-d9db5e74d228\") " pod="openshift-ingress/payload-processing-e2e-inf-b-6d0d72-66b74965bf-x4dl6"
Sep 16 01:01:25.164215 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:01:25.164194    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35acb09e-33d9-46bf-8c4f-ea8d6ec02785-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-6d0d72-5875fff75d-htxqr\" (UID: \"35acb09e-33d9-46bf-8c4f-ea8d6ec02785\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6d0d72-5875fff75d-htxqr"
Sep 16 01:01:25.164304 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:01:25.164230    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4fb25d7a-dcc9-45eb-9f75-d9db5e74d228-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-6d0d72-66b74965bf-x4dl6\" (UID: \"4fb25d7a-dcc9-45eb-9f75-d9db5e74d228\") " pod="openshift-ingress/payload-processing-e2e-inf-b-6d0d72-66b74965bf-x4dl6"
Sep 16 01:01:25.172828 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:01:25.172803    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kln5\" (UniqueName: \"kubernetes.io/projected/35acb09e-33d9-46bf-8c4f-ea8d6ec02785-kube-api-access-7kln5\") pod \"payload-pre-processing-e2e-inf-b-6d0d72-5875fff75d-htxqr\" (UID: \"35acb09e-33d9-46bf-8c4f-ea8d6ec02785\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6d0d72-5875fff75d-htxqr"
Sep 16 01:01:25.172932 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:01:25.172887    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cx86g\" (UniqueName: \"kubernetes.io/projected/4fb25d7a-dcc9-45eb-9f75-d9db5e74d228-kube-api-access-cx86g\") pod \"payload-processing-e2e-inf-b-6d0d72-66b74965bf-x4dl6\" (UID: \"4fb25d7a-dcc9-45eb-9f75-d9db5e74d228\") " pod="openshift-ingress/payload-processing-e2e-inf-b-6d0d72-66b74965bf-x4dl6"
Sep 16 01:01:25.241427 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:01:25.241374    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6d0d72-5875fff75d-htxqr"
Sep 16 01:01:25.280923 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:01:25.280672    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-6d0d72-66b74965bf-x4dl6"
Sep 16 01:01:25.518960 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:01:25.518937    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-6d0d72-5875fff75d-htxqr"]
Sep 16 01:01:25.520304 ip-10-0-130-207 kubenswrapper[2556]: W0916 01:01:25.520278    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35acb09e_33d9_46bf_8c4f_ea8d6ec02785.slice/crio-9aceace069912aa27ddab01c9e045f7d1fc3a2e54632df8fc3f6393cbf04741b WatchSource:0}: Error finding container 9aceace069912aa27ddab01c9e045f7d1fc3a2e54632df8fc3f6393cbf04741b: Status 404 returned error can't find the container with id 9aceace069912aa27ddab01c9e045f7d1fc3a2e54632df8fc3f6393cbf04741b
Sep 16 01:01:25.557782 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:01:25.557763    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-6d0d72-66b74965bf-x4dl6"]
Sep 16 01:01:25.559443 ip-10-0-130-207 kubenswrapper[2556]: W0916 01:01:25.559413    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fb25d7a_dcc9_45eb_9f75_d9db5e74d228.slice/crio-a69612837509056ac7eb7fbcd5fd79db6e7ce7f6ab7342ea4744f7c4ef0a78f7 WatchSource:0}: Error finding container a69612837509056ac7eb7fbcd5fd79db6e7ce7f6ab7342ea4744f7c4ef0a78f7: Status 404 returned error can't find the container with id a69612837509056ac7eb7fbcd5fd79db6e7ce7f6ab7342ea4744f7c4ef0a78f7
Sep 16 01:01:26.403096 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:01:26.403059    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-6d0d72-66b74965bf-x4dl6" event={"ID":"4fb25d7a-dcc9-45eb-9f75-d9db5e74d228","Type":"ContainerStarted","Data":"089876491b0778e2eae205ece3a9f25dab340f3e0c805d095249bfc5458b9281"}
Sep 16 01:01:26.403526 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:01:26.403119    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-6d0d72-66b74965bf-x4dl6" event={"ID":"4fb25d7a-dcc9-45eb-9f75-d9db5e74d228","Type":"ContainerStarted","Data":"a69612837509056ac7eb7fbcd5fd79db6e7ce7f6ab7342ea4744f7c4ef0a78f7"}
Sep 16 01:01:26.403526 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:01:26.403150    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-6d0d72-66b74965bf-x4dl6"
Sep 16 01:01:26.404754 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:01:26.404729    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6d0d72-5875fff75d-htxqr" event={"ID":"35acb09e-33d9-46bf-8c4f-ea8d6ec02785","Type":"ContainerStarted","Data":"b1fca5045a6831b94b70abfd671e25ef55fc8d69a342644f2793dd093755c3f0"}
Sep 16 01:01:26.404883 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:01:26.404775    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6d0d72-5875fff75d-htxqr" event={"ID":"35acb09e-33d9-46bf-8c4f-ea8d6ec02785","Type":"ContainerStarted","Data":"9aceace069912aa27ddab01c9e045f7d1fc3a2e54632df8fc3f6393cbf04741b"}
Sep 16 01:01:26.404883 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:01:26.404803    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6d0d72-5875fff75d-htxqr"
Sep 16 01:01:26.421163 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:01:26.421118    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-6d0d72-66b74965bf-x4dl6" podStartSLOduration=2.421104052 podStartE2EDuration="2.421104052s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 01:01:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 01:01:26.418324489 +0000 UTC m=+1101.199826080" watchObservedRunningTime="2026-09-16 01:01:26.421104052 +0000 UTC m=+1101.202605644"
Sep 16 01:01:26.434126 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:01:26.434085    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6d0d72-5875fff75d-htxqr" podStartSLOduration=2.434070739 podStartE2EDuration="2.434070739s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 01:01:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 01:01:26.432841868 +0000 UTC m=+1101.214343461" watchObservedRunningTime="2026-09-16 01:01:26.434070739 +0000 UTC m=+1101.215572342"
Sep 16 01:01:37.411997 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:01:37.411965    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6d0d72-5875fff75d-htxqr"
Sep 16 01:01:37.412494 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:01:37.412036    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-6d0d72-66b74965bf-x4dl6"
Sep 16 01:02:29.988133 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:02:29.988100    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-96c840/premium-w0-96c840-kserve-6d9c67476-t9n4q"]
Sep 16 01:02:29.991584 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:02:29.991567    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-96c840/premium-w0-96c840-kserve-6d9c67476-t9n4q"
Sep 16 01:02:29.994551 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:02:29.994528    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-96c840\"/\"openshift-service-ca.crt\""
Sep 16 01:02:29.994551 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:02:29.994541    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-96c840\"/\"kube-root-ca.crt\""
Sep 16 01:02:29.994753 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:02:29.994542    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-96c840\"/\"default-dockercfg-67hvz\""
Sep 16 01:02:29.994753 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:02:29.994549    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-96c840\"/\"premium-w0-96c840-kserve-self-signed-certs\""
Sep 16 01:02:30.000223 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:02:30.000201    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-96c840/premium-w0-96c840-kserve-6d9c67476-t9n4q"]
Sep 16 01:02:30.091536 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:02:30.091506    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75077e26-5c4a-45dd-855d-8cc1a44cafd6-tmp-dir\") pod \"premium-w0-96c840-kserve-6d9c67476-t9n4q\" (UID: \"75077e26-5c4a-45dd-855d-8cc1a44cafd6\") " pod="e2e-models-e2e-worker-w0-96c840/premium-w0-96c840-kserve-6d9c67476-t9n4q"
Sep 16 01:02:30.091690 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:02:30.091559    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/75077e26-5c4a-45dd-855d-8cc1a44cafd6-model-cache\") pod \"premium-w0-96c840-kserve-6d9c67476-t9n4q\" (UID: \"75077e26-5c4a-45dd-855d-8cc1a44cafd6\") " pod="e2e-models-e2e-worker-w0-96c840/premium-w0-96c840-kserve-6d9c67476-t9n4q"
Sep 16 01:02:30.091690 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:02:30.091653    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2mg8\" (UniqueName: \"kubernetes.io/projected/75077e26-5c4a-45dd-855d-8cc1a44cafd6-kube-api-access-k2mg8\") pod \"premium-w0-96c840-kserve-6d9c67476-t9n4q\" (UID: \"75077e26-5c4a-45dd-855d-8cc1a44cafd6\") " pod="e2e-models-e2e-worker-w0-96c840/premium-w0-96c840-kserve-6d9c67476-t9n4q"
Sep 16 01:02:30.091766 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:02:30.091694    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75077e26-5c4a-45dd-855d-8cc1a44cafd6-tls-certs\") pod \"premium-w0-96c840-kserve-6d9c67476-t9n4q\" (UID: \"75077e26-5c4a-45dd-855d-8cc1a44cafd6\") " pod="e2e-models-e2e-worker-w0-96c840/premium-w0-96c840-kserve-6d9c67476-t9n4q"
Sep 16 01:02:30.091766 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:02:30.091718    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/75077e26-5c4a-45dd-855d-8cc1a44cafd6-dshm\") pod \"premium-w0-96c840-kserve-6d9c67476-t9n4q\" (UID: \"75077e26-5c4a-45dd-855d-8cc1a44cafd6\") " pod="e2e-models-e2e-worker-w0-96c840/premium-w0-96c840-kserve-6d9c67476-t9n4q"
Sep 16 01:02:30.091833 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:02:30.091764    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/75077e26-5c4a-45dd-855d-8cc1a44cafd6-home\") pod \"premium-w0-96c840-kserve-6d9c67476-t9n4q\" (UID: \"75077e26-5c4a-45dd-855d-8cc1a44cafd6\") " pod="e2e-models-e2e-worker-w0-96c840/premium-w0-96c840-kserve-6d9c67476-t9n4q"
Sep 16 01:02:30.192161 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:02:30.192124    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75077e26-5c4a-45dd-855d-8cc1a44cafd6-tls-certs\") pod \"premium-w0-96c840-kserve-6d9c67476-t9n4q\" (UID: \"75077e26-5c4a-45dd-855d-8cc1a44cafd6\") " pod="e2e-models-e2e-worker-w0-96c840/premium-w0-96c840-kserve-6d9c67476-t9n4q"
Sep 16 01:02:30.192161 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:02:30.192161    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/75077e26-5c4a-45dd-855d-8cc1a44cafd6-dshm\") pod \"premium-w0-96c840-kserve-6d9c67476-t9n4q\" (UID: \"75077e26-5c4a-45dd-855d-8cc1a44cafd6\") " pod="e2e-models-e2e-worker-w0-96c840/premium-w0-96c840-kserve-6d9c67476-t9n4q"
Sep 16 01:02:30.192381 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:02:30.192202    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/75077e26-5c4a-45dd-855d-8cc1a44cafd6-home\") pod \"premium-w0-96c840-kserve-6d9c67476-t9n4q\" (UID: \"75077e26-5c4a-45dd-855d-8cc1a44cafd6\") " pod="e2e-models-e2e-worker-w0-96c840/premium-w0-96c840-kserve-6d9c67476-t9n4q"
Sep 16 01:02:30.192381 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:02:30.192250    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75077e26-5c4a-45dd-855d-8cc1a44cafd6-tmp-dir\") pod \"premium-w0-96c840-kserve-6d9c67476-t9n4q\" (UID: \"75077e26-5c4a-45dd-855d-8cc1a44cafd6\") " pod="e2e-models-e2e-worker-w0-96c840/premium-w0-96c840-kserve-6d9c67476-t9n4q"
Sep 16 01:02:30.192381 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:02:30.192299    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/75077e26-5c4a-45dd-855d-8cc1a44cafd6-model-cache\") pod \"premium-w0-96c840-kserve-6d9c67476-t9n4q\" (UID: \"75077e26-5c4a-45dd-855d-8cc1a44cafd6\") " pod="e2e-models-e2e-worker-w0-96c840/premium-w0-96c840-kserve-6d9c67476-t9n4q"
Sep 16 01:02:30.192381 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:02:30.192354    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2mg8\" (UniqueName: \"kubernetes.io/projected/75077e26-5c4a-45dd-855d-8cc1a44cafd6-kube-api-access-k2mg8\") pod \"premium-w0-96c840-kserve-6d9c67476-t9n4q\" (UID: \"75077e26-5c4a-45dd-855d-8cc1a44cafd6\") " pod="e2e-models-e2e-worker-w0-96c840/premium-w0-96c840-kserve-6d9c67476-t9n4q"
Sep 16 01:02:30.192709 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:02:30.192684    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/75077e26-5c4a-45dd-855d-8cc1a44cafd6-home\") pod \"premium-w0-96c840-kserve-6d9c67476-t9n4q\" (UID: \"75077e26-5c4a-45dd-855d-8cc1a44cafd6\") " pod="e2e-models-e2e-worker-w0-96c840/premium-w0-96c840-kserve-6d9c67476-t9n4q"
Sep 16 01:02:30.192787 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:02:30.192741    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75077e26-5c4a-45dd-855d-8cc1a44cafd6-tmp-dir\") pod \"premium-w0-96c840-kserve-6d9c67476-t9n4q\" (UID: \"75077e26-5c4a-45dd-855d-8cc1a44cafd6\") " pod="e2e-models-e2e-worker-w0-96c840/premium-w0-96c840-kserve-6d9c67476-t9n4q"
Sep 16 01:02:30.192891 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:02:30.192873    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/75077e26-5c4a-45dd-855d-8cc1a44cafd6-model-cache\") pod \"premium-w0-96c840-kserve-6d9c67476-t9n4q\" (UID: \"75077e26-5c4a-45dd-855d-8cc1a44cafd6\") " pod="e2e-models-e2e-worker-w0-96c840/premium-w0-96c840-kserve-6d9c67476-t9n4q"
Sep 16 01:02:30.194405 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:02:30.194384    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/75077e26-5c4a-45dd-855d-8cc1a44cafd6-dshm\") pod \"premium-w0-96c840-kserve-6d9c67476-t9n4q\" (UID: \"75077e26-5c4a-45dd-855d-8cc1a44cafd6\") " pod="e2e-models-e2e-worker-w0-96c840/premium-w0-96c840-kserve-6d9c67476-t9n4q"
Sep 16 01:02:30.194775 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:02:30.194753    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75077e26-5c4a-45dd-855d-8cc1a44cafd6-tls-certs\") pod \"premium-w0-96c840-kserve-6d9c67476-t9n4q\" (UID: \"75077e26-5c4a-45dd-855d-8cc1a44cafd6\") " pod="e2e-models-e2e-worker-w0-96c840/premium-w0-96c840-kserve-6d9c67476-t9n4q"
Sep 16 01:02:30.199703 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:02:30.199683    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2mg8\" (UniqueName: \"kubernetes.io/projected/75077e26-5c4a-45dd-855d-8cc1a44cafd6-kube-api-access-k2mg8\") pod \"premium-w0-96c840-kserve-6d9c67476-t9n4q\" (UID: \"75077e26-5c4a-45dd-855d-8cc1a44cafd6\") " pod="e2e-models-e2e-worker-w0-96c840/premium-w0-96c840-kserve-6d9c67476-t9n4q"
Sep 16 01:02:30.302285 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:02:30.302226    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-96c840/premium-w0-96c840-kserve-6d9c67476-t9n4q"
Sep 16 01:02:30.575260 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:02:30.575229    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-96c840/premium-w0-96c840-kserve-6d9c67476-t9n4q"]
Sep 16 01:02:30.577030 ip-10-0-130-207 kubenswrapper[2556]: W0916 01:02:30.576995    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod75077e26_5c4a_45dd_855d_8cc1a44cafd6.slice/crio-8225bae385255574925697f47de6274477e3a695e85da45120c8b73204e27889 WatchSource:0}: Error finding container 8225bae385255574925697f47de6274477e3a695e85da45120c8b73204e27889: Status 404 returned error can't find the container with id 8225bae385255574925697f47de6274477e3a695e85da45120c8b73204e27889
Sep 16 01:02:30.746399 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:02:30.746368    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-96c840/premium-w0-96c840-kserve-6d9c67476-t9n4q" event={"ID":"75077e26-5c4a-45dd-855d-8cc1a44cafd6","Type":"ContainerStarted","Data":"216dc23fc2695e7389a25435c644623774f998849bdb75f2a69722f989bbdc3b"}
Sep 16 01:02:30.746399 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:02:30.746406    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-96c840/premium-w0-96c840-kserve-6d9c67476-t9n4q" event={"ID":"75077e26-5c4a-45dd-855d-8cc1a44cafd6","Type":"ContainerStarted","Data":"8225bae385255574925697f47de6274477e3a695e85da45120c8b73204e27889"}
Sep 16 01:02:30.762289 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:02:30.762241    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-96c840/premium-w0-96c840-kserve-6d9c67476-t9n4q" podStartSLOduration=1.762225855 podStartE2EDuration="1.762225855s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 01:02:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 01:02:30.760389907 +0000 UTC m=+1165.541891501" watchObservedRunningTime="2026-09-16 01:02:30.762225855 +0000 UTC m=+1165.543727497"
Sep 16 01:02:40.302945 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:02:40.302912    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-96c840/premium-w0-96c840-kserve-6d9c67476-t9n4q"
Sep 16 01:02:40.303319 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:02:40.302957    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-96c840/premium-w0-96c840-kserve-6d9c67476-t9n4q"
Sep 16 01:02:40.315343 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:02:40.315323    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-96c840/premium-w0-96c840-kserve-6d9c67476-t9n4q"
Sep 16 01:02:40.815891 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:02:40.815852    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-96c840/premium-w0-96c840-kserve-6d9c67476-t9n4q"
Sep 16 01:02:49.070940 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:02:49.070900    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-2534e3/premium-w3-2534e3-kserve-679669b7-48q62"]
Sep 16 01:02:49.074480 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:02:49.074441    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-2534e3/premium-w3-2534e3-kserve-679669b7-48q62"
Sep 16 01:02:49.077376 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:02:49.077353    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-2534e3\"/\"default-dockercfg-t7hw2\""
Sep 16 01:02:49.077376 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:02:49.077369    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-2534e3\"/\"kube-root-ca.crt\""
Sep 16 01:02:49.077587 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:02:49.077373    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-2534e3\"/\"premium-w3-2534e3-kserve-self-signed-certs\""
Sep 16 01:02:49.077587 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:02:49.077371    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-2534e3\"/\"openshift-service-ca.crt\""
Sep 16 01:02:49.084804 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:02:49.084784    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-2534e3/premium-w3-2534e3-kserve-679669b7-48q62"]
Sep 16 01:02:49.243199 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:02:49.243171    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/68b17a9b-109b-4fd5-8f97-920e1cb237d0-dshm\") pod \"premium-w3-2534e3-kserve-679669b7-48q62\" (UID: \"68b17a9b-109b-4fd5-8f97-920e1cb237d0\") " pod="e2e-models-e2e-worker-w3-2534e3/premium-w3-2534e3-kserve-679669b7-48q62"
Sep 16 01:02:49.243346 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:02:49.243215    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/68b17a9b-109b-4fd5-8f97-920e1cb237d0-tls-certs\") pod \"premium-w3-2534e3-kserve-679669b7-48q62\" (UID: \"68b17a9b-109b-4fd5-8f97-920e1cb237d0\") " pod="e2e-models-e2e-worker-w3-2534e3/premium-w3-2534e3-kserve-679669b7-48q62"
Sep 16 01:02:49.243346 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:02:49.243246    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2hqr\" (UniqueName: \"kubernetes.io/projected/68b17a9b-109b-4fd5-8f97-920e1cb237d0-kube-api-access-p2hqr\") pod \"premium-w3-2534e3-kserve-679669b7-48q62\" (UID: \"68b17a9b-109b-4fd5-8f97-920e1cb237d0\") " pod="e2e-models-e2e-worker-w3-2534e3/premium-w3-2534e3-kserve-679669b7-48q62"
Sep 16 01:02:49.243346 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:02:49.243309    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/68b17a9b-109b-4fd5-8f97-920e1cb237d0-model-cache\") pod \"premium-w3-2534e3-kserve-679669b7-48q62\" (UID: \"68b17a9b-109b-4fd5-8f97-920e1cb237d0\") " pod="e2e-models-e2e-worker-w3-2534e3/premium-w3-2534e3-kserve-679669b7-48q62"
Sep 16 01:02:49.243346 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:02:49.243324    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/68b17a9b-109b-4fd5-8f97-920e1cb237d0-tmp-dir\") pod \"premium-w3-2534e3-kserve-679669b7-48q62\" (UID: \"68b17a9b-109b-4fd5-8f97-920e1cb237d0\") " pod="e2e-models-e2e-worker-w3-2534e3/premium-w3-2534e3-kserve-679669b7-48q62"
Sep 16 01:02:49.243516 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:02:49.243351    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/68b17a9b-109b-4fd5-8f97-920e1cb237d0-home\") pod \"premium-w3-2534e3-kserve-679669b7-48q62\" (UID: \"68b17a9b-109b-4fd5-8f97-920e1cb237d0\") " pod="e2e-models-e2e-worker-w3-2534e3/premium-w3-2534e3-kserve-679669b7-48q62"
Sep 16 01:02:49.344693 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:02:49.344663    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/68b17a9b-109b-4fd5-8f97-920e1cb237d0-dshm\") pod \"premium-w3-2534e3-kserve-679669b7-48q62\" (UID: \"68b17a9b-109b-4fd5-8f97-920e1cb237d0\") " pod="e2e-models-e2e-worker-w3-2534e3/premium-w3-2534e3-kserve-679669b7-48q62"
Sep 16 01:02:49.344817 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:02:49.344713    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/68b17a9b-109b-4fd5-8f97-920e1cb237d0-tls-certs\") pod \"premium-w3-2534e3-kserve-679669b7-48q62\" (UID: \"68b17a9b-109b-4fd5-8f97-920e1cb237d0\") " pod="e2e-models-e2e-worker-w3-2534e3/premium-w3-2534e3-kserve-679669b7-48q62"
Sep 16 01:02:49.344817 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:02:49.344742    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2hqr\" (UniqueName: \"kubernetes.io/projected/68b17a9b-109b-4fd5-8f97-920e1cb237d0-kube-api-access-p2hqr\") pod \"premium-w3-2534e3-kserve-679669b7-48q62\" (UID: \"68b17a9b-109b-4fd5-8f97-920e1cb237d0\") " pod="e2e-models-e2e-worker-w3-2534e3/premium-w3-2534e3-kserve-679669b7-48q62"
Sep 16 01:02:49.344817 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:02:49.344778    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/68b17a9b-109b-4fd5-8f97-920e1cb237d0-model-cache\") pod \"premium-w3-2534e3-kserve-679669b7-48q62\" (UID: \"68b17a9b-109b-4fd5-8f97-920e1cb237d0\") " pod="e2e-models-e2e-worker-w3-2534e3/premium-w3-2534e3-kserve-679669b7-48q62"
Sep 16 01:02:49.344817 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:02:49.344793    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/68b17a9b-109b-4fd5-8f97-920e1cb237d0-tmp-dir\") pod \"premium-w3-2534e3-kserve-679669b7-48q62\" (UID: \"68b17a9b-109b-4fd5-8f97-920e1cb237d0\") " pod="e2e-models-e2e-worker-w3-2534e3/premium-w3-2534e3-kserve-679669b7-48q62"
Sep 16 01:02:49.345023 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:02:49.344820    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/68b17a9b-109b-4fd5-8f97-920e1cb237d0-home\") pod \"premium-w3-2534e3-kserve-679669b7-48q62\" (UID: \"68b17a9b-109b-4fd5-8f97-920e1cb237d0\") " pod="e2e-models-e2e-worker-w3-2534e3/premium-w3-2534e3-kserve-679669b7-48q62"
Sep 16 01:02:49.345172 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:02:49.345150    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/68b17a9b-109b-4fd5-8f97-920e1cb237d0-model-cache\") pod \"premium-w3-2534e3-kserve-679669b7-48q62\" (UID: \"68b17a9b-109b-4fd5-8f97-920e1cb237d0\") " pod="e2e-models-e2e-worker-w3-2534e3/premium-w3-2534e3-kserve-679669b7-48q62"
Sep 16 01:02:49.345223 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:02:49.345186    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/68b17a9b-109b-4fd5-8f97-920e1cb237d0-home\") pod \"premium-w3-2534e3-kserve-679669b7-48q62\" (UID: \"68b17a9b-109b-4fd5-8f97-920e1cb237d0\") " pod="e2e-models-e2e-worker-w3-2534e3/premium-w3-2534e3-kserve-679669b7-48q62"
Sep 16 01:02:49.345263 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:02:49.345220    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/68b17a9b-109b-4fd5-8f97-920e1cb237d0-tmp-dir\") pod \"premium-w3-2534e3-kserve-679669b7-48q62\" (UID: \"68b17a9b-109b-4fd5-8f97-920e1cb237d0\") " pod="e2e-models-e2e-worker-w3-2534e3/premium-w3-2534e3-kserve-679669b7-48q62"
Sep 16 01:02:49.347000 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:02:49.346981    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/68b17a9b-109b-4fd5-8f97-920e1cb237d0-dshm\") pod \"premium-w3-2534e3-kserve-679669b7-48q62\" (UID: \"68b17a9b-109b-4fd5-8f97-920e1cb237d0\") " pod="e2e-models-e2e-worker-w3-2534e3/premium-w3-2534e3-kserve-679669b7-48q62"
Sep 16 01:02:49.347294 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:02:49.347275    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/68b17a9b-109b-4fd5-8f97-920e1cb237d0-tls-certs\") pod \"premium-w3-2534e3-kserve-679669b7-48q62\" (UID: \"68b17a9b-109b-4fd5-8f97-920e1cb237d0\") " pod="e2e-models-e2e-worker-w3-2534e3/premium-w3-2534e3-kserve-679669b7-48q62"
Sep 16 01:02:49.351957 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:02:49.351935    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2hqr\" (UniqueName: \"kubernetes.io/projected/68b17a9b-109b-4fd5-8f97-920e1cb237d0-kube-api-access-p2hqr\") pod \"premium-w3-2534e3-kserve-679669b7-48q62\" (UID: \"68b17a9b-109b-4fd5-8f97-920e1cb237d0\") " pod="e2e-models-e2e-worker-w3-2534e3/premium-w3-2534e3-kserve-679669b7-48q62"
Sep 16 01:02:49.385917 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:02:49.385897    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-2534e3/premium-w3-2534e3-kserve-679669b7-48q62"
Sep 16 01:02:49.654918 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:02:49.654892    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-2534e3/premium-w3-2534e3-kserve-679669b7-48q62"]
Sep 16 01:02:49.656696 ip-10-0-130-207 kubenswrapper[2556]: W0916 01:02:49.656672    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod68b17a9b_109b_4fd5_8f97_920e1cb237d0.slice/crio-0ad7737e40540d71c64e3ea8d84f838c5ec3807e5e47686020d70f3c87032b5d WatchSource:0}: Error finding container 0ad7737e40540d71c64e3ea8d84f838c5ec3807e5e47686020d70f3c87032b5d: Status 404 returned error can't find the container with id 0ad7737e40540d71c64e3ea8d84f838c5ec3807e5e47686020d70f3c87032b5d
Sep 16 01:02:49.850392 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:02:49.850350    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-2534e3/premium-w3-2534e3-kserve-679669b7-48q62" event={"ID":"68b17a9b-109b-4fd5-8f97-920e1cb237d0","Type":"ContainerStarted","Data":"614f77ea21fb42ae64f31d0c0a95d93c5ff61a30c917dda404ebdc01959c9300"}
Sep 16 01:02:49.850392 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:02:49.850395    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-2534e3/premium-w3-2534e3-kserve-679669b7-48q62" event={"ID":"68b17a9b-109b-4fd5-8f97-920e1cb237d0","Type":"ContainerStarted","Data":"0ad7737e40540d71c64e3ea8d84f838c5ec3807e5e47686020d70f3c87032b5d"}
Sep 16 01:02:49.867785 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:02:49.867736    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-2534e3/premium-w3-2534e3-kserve-679669b7-48q62" podStartSLOduration=0.867721463 podStartE2EDuration="867.721463ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 01:02:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 01:02:49.864939706 +0000 UTC m=+1184.646441310" watchObservedRunningTime="2026-09-16 01:02:49.867721463 +0000 UTC m=+1184.649223054"
Sep 16 01:02:59.386305 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:02:59.386270    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-2534e3/premium-w3-2534e3-kserve-679669b7-48q62"
Sep 16 01:02:59.386305 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:02:59.386312    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-2534e3/premium-w3-2534e3-kserve-679669b7-48q62"
Sep 16 01:02:59.399321 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:02:59.399294    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-2534e3/premium-w3-2534e3-kserve-679669b7-48q62"
Sep 16 01:02:59.912978 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:02:59.912952    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-2534e3/premium-w3-2534e3-kserve-679669b7-48q62"
Sep 16 01:04:17.984341 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:04:17.984256    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-96c840/distinct-w0-96c840-kserve-c8989f89f-qwjjl"]
Sep 16 01:04:17.987800 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:04:17.987782    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-96c840/distinct-w0-96c840-kserve-c8989f89f-qwjjl"
Sep 16 01:04:17.990039 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:04:17.990011    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-96c840\"/\"distinct-w0-96c840-kserve-self-signed-certs\""
Sep 16 01:04:17.996221 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:04:17.996199    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-96c840/distinct-w0-96c840-kserve-c8989f89f-qwjjl"]
Sep 16 01:04:18.076612 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:04:18.076577    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fckd\" (UniqueName: \"kubernetes.io/projected/e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3-kube-api-access-5fckd\") pod \"distinct-w0-96c840-kserve-c8989f89f-qwjjl\" (UID: \"e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3\") " pod="e2e-models-e2e-worker-w0-96c840/distinct-w0-96c840-kserve-c8989f89f-qwjjl"
Sep 16 01:04:18.076791 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:04:18.076700    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3-dshm\") pod \"distinct-w0-96c840-kserve-c8989f89f-qwjjl\" (UID: \"e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3\") " pod="e2e-models-e2e-worker-w0-96c840/distinct-w0-96c840-kserve-c8989f89f-qwjjl"
Sep 16 01:04:18.076791 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:04:18.076752    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3-model-cache\") pod \"distinct-w0-96c840-kserve-c8989f89f-qwjjl\" (UID: \"e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3\") " pod="e2e-models-e2e-worker-w0-96c840/distinct-w0-96c840-kserve-c8989f89f-qwjjl"
Sep 16 01:04:18.076906 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:04:18.076796    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3-tls-certs\") pod \"distinct-w0-96c840-kserve-c8989f89f-qwjjl\" (UID: \"e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3\") " pod="e2e-models-e2e-worker-w0-96c840/distinct-w0-96c840-kserve-c8989f89f-qwjjl"
Sep 16 01:04:18.076906 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:04:18.076880    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3-home\") pod \"distinct-w0-96c840-kserve-c8989f89f-qwjjl\" (UID: \"e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3\") " pod="e2e-models-e2e-worker-w0-96c840/distinct-w0-96c840-kserve-c8989f89f-qwjjl"
Sep 16 01:04:18.076983 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:04:18.076966    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3-tmp-dir\") pod \"distinct-w0-96c840-kserve-c8989f89f-qwjjl\" (UID: \"e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3\") " pod="e2e-models-e2e-worker-w0-96c840/distinct-w0-96c840-kserve-c8989f89f-qwjjl"
Sep 16 01:04:18.177533 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:04:18.177487    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3-dshm\") pod \"distinct-w0-96c840-kserve-c8989f89f-qwjjl\" (UID: \"e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3\") " pod="e2e-models-e2e-worker-w0-96c840/distinct-w0-96c840-kserve-c8989f89f-qwjjl"
Sep 16 01:04:18.177702 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:04:18.177562    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3-model-cache\") pod \"distinct-w0-96c840-kserve-c8989f89f-qwjjl\" (UID: \"e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3\") " pod="e2e-models-e2e-worker-w0-96c840/distinct-w0-96c840-kserve-c8989f89f-qwjjl"
Sep 16 01:04:18.177702 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:04:18.177580    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3-tls-certs\") pod \"distinct-w0-96c840-kserve-c8989f89f-qwjjl\" (UID: \"e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3\") " pod="e2e-models-e2e-worker-w0-96c840/distinct-w0-96c840-kserve-c8989f89f-qwjjl"
Sep 16 01:04:18.177702 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:04:18.177618    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3-home\") pod \"distinct-w0-96c840-kserve-c8989f89f-qwjjl\" (UID: \"e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3\") " pod="e2e-models-e2e-worker-w0-96c840/distinct-w0-96c840-kserve-c8989f89f-qwjjl"
Sep 16 01:04:18.177702 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:04:18.177663    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3-tmp-dir\") pod \"distinct-w0-96c840-kserve-c8989f89f-qwjjl\" (UID: \"e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3\") " pod="e2e-models-e2e-worker-w0-96c840/distinct-w0-96c840-kserve-c8989f89f-qwjjl"
Sep 16 01:04:18.177931 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:04:18.177737    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fckd\" (UniqueName: \"kubernetes.io/projected/e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3-kube-api-access-5fckd\") pod \"distinct-w0-96c840-kserve-c8989f89f-qwjjl\" (UID: \"e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3\") " pod="e2e-models-e2e-worker-w0-96c840/distinct-w0-96c840-kserve-c8989f89f-qwjjl"
Sep 16 01:04:18.178113 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:04:18.178051    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3-home\") pod \"distinct-w0-96c840-kserve-c8989f89f-qwjjl\" (UID: \"e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3\") " pod="e2e-models-e2e-worker-w0-96c840/distinct-w0-96c840-kserve-c8989f89f-qwjjl"
Sep 16 01:04:18.178113 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:04:18.178079    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3-tmp-dir\") pod \"distinct-w0-96c840-kserve-c8989f89f-qwjjl\" (UID: \"e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3\") " pod="e2e-models-e2e-worker-w0-96c840/distinct-w0-96c840-kserve-c8989f89f-qwjjl"
Sep 16 01:04:18.178325 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:04:18.178305    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3-model-cache\") pod \"distinct-w0-96c840-kserve-c8989f89f-qwjjl\" (UID: \"e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3\") " pod="e2e-models-e2e-worker-w0-96c840/distinct-w0-96c840-kserve-c8989f89f-qwjjl"
Sep 16 01:04:18.180593 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:04:18.180565    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3-dshm\") pod \"distinct-w0-96c840-kserve-c8989f89f-qwjjl\" (UID: \"e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3\") " pod="e2e-models-e2e-worker-w0-96c840/distinct-w0-96c840-kserve-c8989f89f-qwjjl"
Sep 16 01:04:18.181087 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:04:18.181065    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3-tls-certs\") pod \"distinct-w0-96c840-kserve-c8989f89f-qwjjl\" (UID: \"e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3\") " pod="e2e-models-e2e-worker-w0-96c840/distinct-w0-96c840-kserve-c8989f89f-qwjjl"
Sep 16 01:04:18.186746 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:04:18.185911    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fckd\" (UniqueName: \"kubernetes.io/projected/e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3-kube-api-access-5fckd\") pod \"distinct-w0-96c840-kserve-c8989f89f-qwjjl\" (UID: \"e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3\") " pod="e2e-models-e2e-worker-w0-96c840/distinct-w0-96c840-kserve-c8989f89f-qwjjl"
Sep 16 01:04:18.297730 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:04:18.297669    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-96c840/distinct-w0-96c840-kserve-c8989f89f-qwjjl"
Sep 16 01:04:18.569684 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:04:18.569662    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-96c840/distinct-w0-96c840-kserve-c8989f89f-qwjjl"]
Sep 16 01:04:18.571174 ip-10-0-130-207 kubenswrapper[2556]: W0916 01:04:18.571144    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode7087dd0_00aa_46a9_ba1e_7f2f56c6d4c3.slice/crio-ec72219fd212bd1f44718bdb4532ff65521ed7c2b27d95c9fcaf7ea917864254 WatchSource:0}: Error finding container ec72219fd212bd1f44718bdb4532ff65521ed7c2b27d95c9fcaf7ea917864254: Status 404 returned error can't find the container with id ec72219fd212bd1f44718bdb4532ff65521ed7c2b27d95c9fcaf7ea917864254
Sep 16 01:04:19.309240 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:04:19.309195    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-96c840/distinct-w0-96c840-kserve-c8989f89f-qwjjl" event={"ID":"e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3","Type":"ContainerStarted","Data":"cb3b5f4e2c0a7195331a0fbf79ec5a67fc7b349ee4380344e7dde40726e1f19e"}
Sep 16 01:04:19.309697 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:04:19.309252    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-96c840/distinct-w0-96c840-kserve-c8989f89f-qwjjl" event={"ID":"e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3","Type":"ContainerStarted","Data":"ec72219fd212bd1f44718bdb4532ff65521ed7c2b27d95c9fcaf7ea917864254"}
Sep 16 01:04:19.325394 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:04:19.325345    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-96c840/distinct-w0-96c840-kserve-c8989f89f-qwjjl" podStartSLOduration=2.32532597 podStartE2EDuration="2.32532597s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 01:04:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 01:04:19.32337125 +0000 UTC m=+1274.104872842" watchObservedRunningTime="2026-09-16 01:04:19.32532597 +0000 UTC m=+1274.106827561"
Sep 16 01:04:28.298457 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:04:28.298419    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-96c840/distinct-w0-96c840-kserve-c8989f89f-qwjjl"
Sep 16 01:04:28.298883 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:04:28.298485    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-96c840/distinct-w0-96c840-kserve-c8989f89f-qwjjl"
Sep 16 01:04:28.311277 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:04:28.311248    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-96c840/distinct-w0-96c840-kserve-c8989f89f-qwjjl"
Sep 16 01:04:28.369731 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:04:28.369711    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-96c840/distinct-w0-96c840-kserve-c8989f89f-qwjjl"
Sep 16 01:04:54.919245 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:04:54.919208    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-f097ba39-847548d9c7-fw67p"]
Sep 16 01:04:54.923037 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:04:54.923014    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-f097ba39-847548d9c7-fw67p"
Sep 16 01:04:54.925130 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:04:54.925107    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-f097ba39\""
Sep 16 01:04:54.925244 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:04:54.925223    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-f097ba39-dockercfg-chshg\""
Sep 16 01:04:54.933031 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:04:54.933002    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-f097ba39-847548d9c7-fw67p"]
Sep 16 01:04:54.975711 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:04:54.975690    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/afb4b227-10c8-43d7-8f70-20bcf0460fe2-plugins-config-volume\") pod \"payload-processing-e2e-ait-f097ba39-847548d9c7-fw67p\" (UID: \"afb4b227-10c8-43d7-8f70-20bcf0460fe2\") " pod="openshift-ingress/payload-processing-e2e-ait-f097ba39-847548d9c7-fw67p"
Sep 16 01:04:54.975807 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:04:54.975777    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8g62m\" (UniqueName: \"kubernetes.io/projected/afb4b227-10c8-43d7-8f70-20bcf0460fe2-kube-api-access-8g62m\") pod \"payload-processing-e2e-ait-f097ba39-847548d9c7-fw67p\" (UID: \"afb4b227-10c8-43d7-8f70-20bcf0460fe2\") " pod="openshift-ingress/payload-processing-e2e-ait-f097ba39-847548d9c7-fw67p"
Sep 16 01:04:55.076684 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:04:55.076646    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8g62m\" (UniqueName: \"kubernetes.io/projected/afb4b227-10c8-43d7-8f70-20bcf0460fe2-kube-api-access-8g62m\") pod \"payload-processing-e2e-ait-f097ba39-847548d9c7-fw67p\" (UID: \"afb4b227-10c8-43d7-8f70-20bcf0460fe2\") " pod="openshift-ingress/payload-processing-e2e-ait-f097ba39-847548d9c7-fw67p"
Sep 16 01:04:55.076842 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:04:55.076705    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/afb4b227-10c8-43d7-8f70-20bcf0460fe2-plugins-config-volume\") pod \"payload-processing-e2e-ait-f097ba39-847548d9c7-fw67p\" (UID: \"afb4b227-10c8-43d7-8f70-20bcf0460fe2\") " pod="openshift-ingress/payload-processing-e2e-ait-f097ba39-847548d9c7-fw67p"
Sep 16 01:04:55.077449 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:04:55.077430    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/afb4b227-10c8-43d7-8f70-20bcf0460fe2-plugins-config-volume\") pod \"payload-processing-e2e-ait-f097ba39-847548d9c7-fw67p\" (UID: \"afb4b227-10c8-43d7-8f70-20bcf0460fe2\") " pod="openshift-ingress/payload-processing-e2e-ait-f097ba39-847548d9c7-fw67p"
Sep 16 01:04:55.083572 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:04:55.083549    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8g62m\" (UniqueName: \"kubernetes.io/projected/afb4b227-10c8-43d7-8f70-20bcf0460fe2-kube-api-access-8g62m\") pod \"payload-processing-e2e-ait-f097ba39-847548d9c7-fw67p\" (UID: \"afb4b227-10c8-43d7-8f70-20bcf0460fe2\") " pod="openshift-ingress/payload-processing-e2e-ait-f097ba39-847548d9c7-fw67p"
Sep 16 01:04:55.235039 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:04:55.234957    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-f097ba39-847548d9c7-fw67p"
Sep 16 01:04:55.505080 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:04:55.505052    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-f097ba39-847548d9c7-fw67p"]
Sep 16 01:04:55.507418 ip-10-0-130-207 kubenswrapper[2556]: W0916 01:04:55.507381    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafb4b227_10c8_43d7_8f70_20bcf0460fe2.slice/crio-e02450286b661bf06732832ffbe60d114fd844df0d09600ac1191c7aa8c4ea71 WatchSource:0}: Error finding container e02450286b661bf06732832ffbe60d114fd844df0d09600ac1191c7aa8c4ea71: Status 404 returned error can't find the container with id e02450286b661bf06732832ffbe60d114fd844df0d09600ac1191c7aa8c4ea71
Sep 16 01:04:55.520749 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:04:55.520728    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-f097ba39-847548d9c7-fw67p" event={"ID":"afb4b227-10c8-43d7-8f70-20bcf0460fe2","Type":"ContainerStarted","Data":"e02450286b661bf06732832ffbe60d114fd844df0d09600ac1191c7aa8c4ea71"}
Sep 16 01:04:56.527036 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:04:56.526999    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-f097ba39-847548d9c7-fw67p" event={"ID":"afb4b227-10c8-43d7-8f70-20bcf0460fe2","Type":"ContainerStarted","Data":"0babb04d402ddf10d195a52f9d69716139dbe8cd96e65da31dc28fccfd27bf7c"}
Sep 16 01:04:56.527413 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:04:56.527078    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-f097ba39-847548d9c7-fw67p"
Sep 16 01:04:56.545493 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:04:56.545434    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-f097ba39-847548d9c7-fw67p" podStartSLOduration=2.545418787 podStartE2EDuration="2.545418787s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 01:04:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 01:04:56.542962134 +0000 UTC m=+1311.324463727" watchObservedRunningTime="2026-09-16 01:04:56.545418787 +0000 UTC m=+1311.326920409"
Sep 16 01:05:07.535154 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:07.535123    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-f097ba39-847548d9c7-fw67p"
Sep 16 01:05:33.747783 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:33.747750    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-f097ba39-847548d9c7-fw67p"]
Sep 16 01:05:33.748222 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:33.747983    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-f097ba39-847548d9c7-fw67p" podUID="afb4b227-10c8-43d7-8f70-20bcf0460fe2" containerName="payload-processing" containerID="cri-o://0babb04d402ddf10d195a52f9d69716139dbe8cd96e65da31dc28fccfd27bf7c" gracePeriod=30
Sep 16 01:05:34.007206 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:34.007151    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-f097ba39-847548d9c7-fw67p"
Sep 16 01:05:34.114963 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:34.114933    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8g62m\" (UniqueName: \"kubernetes.io/projected/afb4b227-10c8-43d7-8f70-20bcf0460fe2-kube-api-access-8g62m\") pod \"afb4b227-10c8-43d7-8f70-20bcf0460fe2\" (UID: \"afb4b227-10c8-43d7-8f70-20bcf0460fe2\") "
Sep 16 01:05:34.115113 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:34.115069    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/afb4b227-10c8-43d7-8f70-20bcf0460fe2-plugins-config-volume\") pod \"afb4b227-10c8-43d7-8f70-20bcf0460fe2\" (UID: \"afb4b227-10c8-43d7-8f70-20bcf0460fe2\") "
Sep 16 01:05:34.115459 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:34.115432    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/afb4b227-10c8-43d7-8f70-20bcf0460fe2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "afb4b227-10c8-43d7-8f70-20bcf0460fe2" (UID: "afb4b227-10c8-43d7-8f70-20bcf0460fe2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 01:05:34.117968 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:34.117937    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/afb4b227-10c8-43d7-8f70-20bcf0460fe2-kube-api-access-8g62m" (OuterVolumeSpecName: "kube-api-access-8g62m") pod "afb4b227-10c8-43d7-8f70-20bcf0460fe2" (UID: "afb4b227-10c8-43d7-8f70-20bcf0460fe2"). InnerVolumeSpecName "kube-api-access-8g62m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 01:05:34.216616 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:34.216565    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/afb4b227-10c8-43d7-8f70-20bcf0460fe2-plugins-config-volume\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:05:34.216616 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:34.216607    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8g62m\" (UniqueName: \"kubernetes.io/projected/afb4b227-10c8-43d7-8f70-20bcf0460fe2-kube-api-access-8g62m\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:05:34.749899 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:34.749864    2556 generic.go:356] "Generic (PLEG): container finished" podID="afb4b227-10c8-43d7-8f70-20bcf0460fe2" containerID="0babb04d402ddf10d195a52f9d69716139dbe8cd96e65da31dc28fccfd27bf7c" exitCode=0
Sep 16 01:05:34.750327 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:34.749908    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-f097ba39-847548d9c7-fw67p"
Sep 16 01:05:34.750327 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:34.749935    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-f097ba39-847548d9c7-fw67p" event={"ID":"afb4b227-10c8-43d7-8f70-20bcf0460fe2","Type":"ContainerDied","Data":"0babb04d402ddf10d195a52f9d69716139dbe8cd96e65da31dc28fccfd27bf7c"}
Sep 16 01:05:34.750327 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:34.749973    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-f097ba39-847548d9c7-fw67p" event={"ID":"afb4b227-10c8-43d7-8f70-20bcf0460fe2","Type":"ContainerDied","Data":"e02450286b661bf06732832ffbe60d114fd844df0d09600ac1191c7aa8c4ea71"}
Sep 16 01:05:34.750327 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:34.749994    2556 scope.go:117] "RemoveContainer" containerID="0babb04d402ddf10d195a52f9d69716139dbe8cd96e65da31dc28fccfd27bf7c"
Sep 16 01:05:34.759612 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:34.759595    2556 scope.go:117] "RemoveContainer" containerID="0babb04d402ddf10d195a52f9d69716139dbe8cd96e65da31dc28fccfd27bf7c"
Sep 16 01:05:34.759860 ip-10-0-130-207 kubenswrapper[2556]: E0916 01:05:34.759844    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0babb04d402ddf10d195a52f9d69716139dbe8cd96e65da31dc28fccfd27bf7c\": container with ID starting with 0babb04d402ddf10d195a52f9d69716139dbe8cd96e65da31dc28fccfd27bf7c not found: ID does not exist" containerID="0babb04d402ddf10d195a52f9d69716139dbe8cd96e65da31dc28fccfd27bf7c"
Sep 16 01:05:34.759911 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:34.759868    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0babb04d402ddf10d195a52f9d69716139dbe8cd96e65da31dc28fccfd27bf7c"} err="failed to get container status \"0babb04d402ddf10d195a52f9d69716139dbe8cd96e65da31dc28fccfd27bf7c\": rpc error: code = NotFound desc = could not find container \"0babb04d402ddf10d195a52f9d69716139dbe8cd96e65da31dc28fccfd27bf7c\": container with ID starting with 0babb04d402ddf10d195a52f9d69716139dbe8cd96e65da31dc28fccfd27bf7c not found: ID does not exist"
Sep 16 01:05:34.770270 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:34.770247    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-f097ba39-847548d9c7-fw67p"]
Sep 16 01:05:34.777062 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:34.777043    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-f097ba39-847548d9c7-fw67p"]
Sep 16 01:05:35.736297 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:35.736267    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="afb4b227-10c8-43d7-8f70-20bcf0460fe2" path="/var/lib/kubelet/pods/afb4b227-10c8-43d7-8f70-20bcf0460fe2/volumes"
Sep 16 01:05:35.983041 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:35.983011    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-d58169-bbb6d87f4-hhc2r"]
Sep 16 01:05:35.983497 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:35.983410    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d58169-bbb6d87f4-hhc2r" podUID="2c51bbd7-e903-4350-956b-25c5c56d9804" containerName="maas-api" containerID="cri-o://696513912d766453bd62e6ba7a9dc461c95b9df6acdf5aa3e1718426288dd2ae" gracePeriod=30
Sep 16 01:05:36.228536 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:36.228513    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d58169-bbb6d87f4-hhc2r"
Sep 16 01:05:36.337169 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:36.337137    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c51bbd7-e903-4350-956b-25c5c56d9804-maas-api-tls\") pod \"2c51bbd7-e903-4350-956b-25c5c56d9804\" (UID: \"2c51bbd7-e903-4350-956b-25c5c56d9804\") "
Sep 16 01:05:36.337340 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:36.337177    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c51bbd7-e903-4350-956b-25c5c56d9804-metrics-certs\") pod \"2c51bbd7-e903-4350-956b-25c5c56d9804\" (UID: \"2c51bbd7-e903-4350-956b-25c5c56d9804\") "
Sep 16 01:05:36.337340 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:36.337261    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5xp5m\" (UniqueName: \"kubernetes.io/projected/2c51bbd7-e903-4350-956b-25c5c56d9804-kube-api-access-5xp5m\") pod \"2c51bbd7-e903-4350-956b-25c5c56d9804\" (UID: \"2c51bbd7-e903-4350-956b-25c5c56d9804\") "
Sep 16 01:05:36.337340 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:36.337323    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c51bbd7-e903-4350-956b-25c5c56d9804-openshift-service-ca\") pod \"2c51bbd7-e903-4350-956b-25c5c56d9804\" (UID: \"2c51bbd7-e903-4350-956b-25c5c56d9804\") "
Sep 16 01:05:36.337728 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:36.337701    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c51bbd7-e903-4350-956b-25c5c56d9804-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2c51bbd7-e903-4350-956b-25c5c56d9804" (UID: "2c51bbd7-e903-4350-956b-25c5c56d9804"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 01:05:36.339721 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:36.339683    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c51bbd7-e903-4350-956b-25c5c56d9804-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2c51bbd7-e903-4350-956b-25c5c56d9804" (UID: "2c51bbd7-e903-4350-956b-25c5c56d9804"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 01:05:36.339842 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:36.339772    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c51bbd7-e903-4350-956b-25c5c56d9804-kube-api-access-5xp5m" (OuterVolumeSpecName: "kube-api-access-5xp5m") pod "2c51bbd7-e903-4350-956b-25c5c56d9804" (UID: "2c51bbd7-e903-4350-956b-25c5c56d9804"). InnerVolumeSpecName "kube-api-access-5xp5m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 01:05:36.339930 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:36.339905    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c51bbd7-e903-4350-956b-25c5c56d9804-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "2c51bbd7-e903-4350-956b-25c5c56d9804" (UID: "2c51bbd7-e903-4350-956b-25c5c56d9804"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 01:05:36.438974 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:36.438934    2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c51bbd7-e903-4350-956b-25c5c56d9804-openshift-service-ca\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:05:36.438974 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:36.438978    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c51bbd7-e903-4350-956b-25c5c56d9804-maas-api-tls\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:05:36.439168 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:36.438992    2556 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c51bbd7-e903-4350-956b-25c5c56d9804-metrics-certs\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:05:36.439168 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:36.439005    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5xp5m\" (UniqueName: \"kubernetes.io/projected/2c51bbd7-e903-4350-956b-25c5c56d9804-kube-api-access-5xp5m\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:05:36.763072 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:36.762985    2556 generic.go:356] "Generic (PLEG): container finished" podID="2c51bbd7-e903-4350-956b-25c5c56d9804" containerID="696513912d766453bd62e6ba7a9dc461c95b9df6acdf5aa3e1718426288dd2ae" exitCode=0
Sep 16 01:05:36.763072 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:36.763029    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d58169-bbb6d87f4-hhc2r"
Sep 16 01:05:36.763295 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:36.763077    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d58169-bbb6d87f4-hhc2r" event={"ID":"2c51bbd7-e903-4350-956b-25c5c56d9804","Type":"ContainerDied","Data":"696513912d766453bd62e6ba7a9dc461c95b9df6acdf5aa3e1718426288dd2ae"}
Sep 16 01:05:36.763295 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:36.763139    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d58169-bbb6d87f4-hhc2r" event={"ID":"2c51bbd7-e903-4350-956b-25c5c56d9804","Type":"ContainerDied","Data":"9e743fa8a97ffc72f0c455df863782dabed832ba319031603b7d56767f95b5d6"}
Sep 16 01:05:36.763295 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:36.763167    2556 scope.go:117] "RemoveContainer" containerID="696513912d766453bd62e6ba7a9dc461c95b9df6acdf5aa3e1718426288dd2ae"
Sep 16 01:05:36.772896 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:36.772877    2556 scope.go:117] "RemoveContainer" containerID="696513912d766453bd62e6ba7a9dc461c95b9df6acdf5aa3e1718426288dd2ae"
Sep 16 01:05:36.773181 ip-10-0-130-207 kubenswrapper[2556]: E0916 01:05:36.773162    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"696513912d766453bd62e6ba7a9dc461c95b9df6acdf5aa3e1718426288dd2ae\": container with ID starting with 696513912d766453bd62e6ba7a9dc461c95b9df6acdf5aa3e1718426288dd2ae not found: ID does not exist" containerID="696513912d766453bd62e6ba7a9dc461c95b9df6acdf5aa3e1718426288dd2ae"
Sep 16 01:05:36.773230 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:36.773189    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"696513912d766453bd62e6ba7a9dc461c95b9df6acdf5aa3e1718426288dd2ae"} err="failed to get container status \"696513912d766453bd62e6ba7a9dc461c95b9df6acdf5aa3e1718426288dd2ae\": rpc error: code = NotFound desc = could not find container \"696513912d766453bd62e6ba7a9dc461c95b9df6acdf5aa3e1718426288dd2ae\": container with ID starting with 696513912d766453bd62e6ba7a9dc461c95b9df6acdf5aa3e1718426288dd2ae not found: ID does not exist"
Sep 16 01:05:36.785141 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:36.785119    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-d58169-bbb6d87f4-hhc2r"]
Sep 16 01:05:36.788854 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:36.788832    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-d58169-bbb6d87f4-hhc2r"]
Sep 16 01:05:37.736171 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:37.736141    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c51bbd7-e903-4350-956b-25c5c56d9804" path="/var/lib/kubelet/pods/2c51bbd7-e903-4350-956b-25c5c56d9804/volumes"
Sep 16 01:05:39.620380 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:39.620343    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-a58c3d9e-dd4f9d4b-mfjzp"]
Sep 16 01:05:39.620816 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:39.620801    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c51bbd7-e903-4350-956b-25c5c56d9804" containerName="maas-api"
Sep 16 01:05:39.620816 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:39.620818    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c51bbd7-e903-4350-956b-25c5c56d9804" containerName="maas-api"
Sep 16 01:05:39.620890 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:39.620840    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="afb4b227-10c8-43d7-8f70-20bcf0460fe2" containerName="payload-processing"
Sep 16 01:05:39.620890 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:39.620851    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="afb4b227-10c8-43d7-8f70-20bcf0460fe2" containerName="payload-processing"
Sep 16 01:05:39.620954 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:39.620908    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="afb4b227-10c8-43d7-8f70-20bcf0460fe2" containerName="payload-processing"
Sep 16 01:05:39.620954 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:39.620922    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c51bbd7-e903-4350-956b-25c5c56d9804" containerName="maas-api"
Sep 16 01:05:39.624749 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:39.624731    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-a58c3d9e-dd4f9d4b-mfjzp"
Sep 16 01:05:39.626878 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:39.626853    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-a58c3d9e-dockercfg-mp872\""
Sep 16 01:05:39.627047 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:39.627026    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-a58c3d9e\""
Sep 16 01:05:39.631303 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:39.631243    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-a58c3d9e-dd4f9d4b-mfjzp"]
Sep 16 01:05:39.768926 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:39.768895    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdkbm\" (UniqueName: \"kubernetes.io/projected/a8087b04-4a35-49f4-8d17-a399ad061d45-kube-api-access-hdkbm\") pod \"payload-processing-e2e-migrate-a58c3d9e-dd4f9d4b-mfjzp\" (UID: \"a8087b04-4a35-49f4-8d17-a399ad061d45\") " pod="openshift-ingress/payload-processing-e2e-migrate-a58c3d9e-dd4f9d4b-mfjzp"
Sep 16 01:05:39.769107 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:39.768981    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8087b04-4a35-49f4-8d17-a399ad061d45-plugins-config-volume\") pod \"payload-processing-e2e-migrate-a58c3d9e-dd4f9d4b-mfjzp\" (UID: \"a8087b04-4a35-49f4-8d17-a399ad061d45\") " pod="openshift-ingress/payload-processing-e2e-migrate-a58c3d9e-dd4f9d4b-mfjzp"
Sep 16 01:05:39.870167 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:39.870129    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdkbm\" (UniqueName: \"kubernetes.io/projected/a8087b04-4a35-49f4-8d17-a399ad061d45-kube-api-access-hdkbm\") pod \"payload-processing-e2e-migrate-a58c3d9e-dd4f9d4b-mfjzp\" (UID: \"a8087b04-4a35-49f4-8d17-a399ad061d45\") " pod="openshift-ingress/payload-processing-e2e-migrate-a58c3d9e-dd4f9d4b-mfjzp"
Sep 16 01:05:39.870339 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:39.870274    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8087b04-4a35-49f4-8d17-a399ad061d45-plugins-config-volume\") pod \"payload-processing-e2e-migrate-a58c3d9e-dd4f9d4b-mfjzp\" (UID: \"a8087b04-4a35-49f4-8d17-a399ad061d45\") " pod="openshift-ingress/payload-processing-e2e-migrate-a58c3d9e-dd4f9d4b-mfjzp"
Sep 16 01:05:39.871044 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:39.870996    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8087b04-4a35-49f4-8d17-a399ad061d45-plugins-config-volume\") pod \"payload-processing-e2e-migrate-a58c3d9e-dd4f9d4b-mfjzp\" (UID: \"a8087b04-4a35-49f4-8d17-a399ad061d45\") " pod="openshift-ingress/payload-processing-e2e-migrate-a58c3d9e-dd4f9d4b-mfjzp"
Sep 16 01:05:39.878438 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:39.878404    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdkbm\" (UniqueName: \"kubernetes.io/projected/a8087b04-4a35-49f4-8d17-a399ad061d45-kube-api-access-hdkbm\") pod \"payload-processing-e2e-migrate-a58c3d9e-dd4f9d4b-mfjzp\" (UID: \"a8087b04-4a35-49f4-8d17-a399ad061d45\") " pod="openshift-ingress/payload-processing-e2e-migrate-a58c3d9e-dd4f9d4b-mfjzp"
Sep 16 01:05:39.936273 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:39.936241    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-a58c3d9e-dd4f9d4b-mfjzp"
Sep 16 01:05:40.207596 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:40.207520    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-a58c3d9e-dd4f9d4b-mfjzp"]
Sep 16 01:05:40.210964 ip-10-0-130-207 kubenswrapper[2556]: W0916 01:05:40.210937    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8087b04_4a35_49f4_8d17_a399ad061d45.slice/crio-b005b57e5758b0601b7a4ed020455bf3c5a453e6bd9b9ab7d5ac464c6fd4a7e5 WatchSource:0}: Error finding container b005b57e5758b0601b7a4ed020455bf3c5a453e6bd9b9ab7d5ac464c6fd4a7e5: Status 404 returned error can't find the container with id b005b57e5758b0601b7a4ed020455bf3c5a453e6bd9b9ab7d5ac464c6fd4a7e5
Sep 16 01:05:40.787214 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:40.787182    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-a58c3d9e-dd4f9d4b-mfjzp" event={"ID":"a8087b04-4a35-49f4-8d17-a399ad061d45","Type":"ContainerStarted","Data":"e4bd9479168c606b538255cd2b562e6eef502532af5af36edaa3d359d11e719b"}
Sep 16 01:05:40.787700 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:40.787229    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-a58c3d9e-dd4f9d4b-mfjzp" event={"ID":"a8087b04-4a35-49f4-8d17-a399ad061d45","Type":"ContainerStarted","Data":"b005b57e5758b0601b7a4ed020455bf3c5a453e6bd9b9ab7d5ac464c6fd4a7e5"}
Sep 16 01:05:40.787700 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:40.787265    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-a58c3d9e-dd4f9d4b-mfjzp"
Sep 16 01:05:40.803219 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:40.803172    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-a58c3d9e-dd4f9d4b-mfjzp" podStartSLOduration=1.803157495 podStartE2EDuration="1.803157495s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 01:05:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 01:05:40.80092352 +0000 UTC m=+1355.582425115" watchObservedRunningTime="2026-09-16 01:05:40.803157495 +0000 UTC m=+1355.584659097"
Sep 16 01:05:51.794981 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:51.794953    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-a58c3d9e-dd4f9d4b-mfjzp"
Sep 16 01:05:53.798284 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:53.798253    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-6d0d72-66b74965bf-x4dl6"]
Sep 16 01:05:53.798757 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:53.798480    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-6d0d72-66b74965bf-x4dl6" podUID="4fb25d7a-dcc9-45eb-9f75-d9db5e74d228" containerName="payload-processing" containerID="cri-o://089876491b0778e2eae205ece3a9f25dab340f3e0c805d095249bfc5458b9281" gracePeriod=30
Sep 16 01:05:53.807316 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:53.807286    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-6d0d72-5875fff75d-htxqr"]
Sep 16 01:05:53.807628 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:53.807549    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6d0d72-5875fff75d-htxqr" podUID="35acb09e-33d9-46bf-8c4f-ea8d6ec02785" containerName="payload-pre-processing" containerID="cri-o://b1fca5045a6831b94b70abfd671e25ef55fc8d69a342644f2793dd093755c3f0" gracePeriod=30
Sep 16 01:05:54.081250 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:54.081227    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-6d0d72-66b74965bf-x4dl6"
Sep 16 01:05:54.084022 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:54.084002    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6d0d72-5875fff75d-htxqr"
Sep 16 01:05:54.195285 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:54.195251    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35acb09e-33d9-46bf-8c4f-ea8d6ec02785-plugins-config-volume\") pod \"35acb09e-33d9-46bf-8c4f-ea8d6ec02785\" (UID: \"35acb09e-33d9-46bf-8c4f-ea8d6ec02785\") "
Sep 16 01:05:54.195447 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:54.195328    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4fb25d7a-dcc9-45eb-9f75-d9db5e74d228-plugins-config-volume\") pod \"4fb25d7a-dcc9-45eb-9f75-d9db5e74d228\" (UID: \"4fb25d7a-dcc9-45eb-9f75-d9db5e74d228\") "
Sep 16 01:05:54.195447 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:54.195344    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cx86g\" (UniqueName: \"kubernetes.io/projected/4fb25d7a-dcc9-45eb-9f75-d9db5e74d228-kube-api-access-cx86g\") pod \"4fb25d7a-dcc9-45eb-9f75-d9db5e74d228\" (UID: \"4fb25d7a-dcc9-45eb-9f75-d9db5e74d228\") "
Sep 16 01:05:54.195447 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:54.195385    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7kln5\" (UniqueName: \"kubernetes.io/projected/35acb09e-33d9-46bf-8c4f-ea8d6ec02785-kube-api-access-7kln5\") pod \"35acb09e-33d9-46bf-8c4f-ea8d6ec02785\" (UID: \"35acb09e-33d9-46bf-8c4f-ea8d6ec02785\") "
Sep 16 01:05:54.195717 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:54.195687    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4fb25d7a-dcc9-45eb-9f75-d9db5e74d228-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4fb25d7a-dcc9-45eb-9f75-d9db5e74d228" (UID: "4fb25d7a-dcc9-45eb-9f75-d9db5e74d228"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 01:05:54.195782 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:54.195689    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35acb09e-33d9-46bf-8c4f-ea8d6ec02785-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "35acb09e-33d9-46bf-8c4f-ea8d6ec02785" (UID: "35acb09e-33d9-46bf-8c4f-ea8d6ec02785"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 01:05:54.197502 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:54.197453    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35acb09e-33d9-46bf-8c4f-ea8d6ec02785-kube-api-access-7kln5" (OuterVolumeSpecName: "kube-api-access-7kln5") pod "35acb09e-33d9-46bf-8c4f-ea8d6ec02785" (UID: "35acb09e-33d9-46bf-8c4f-ea8d6ec02785"). InnerVolumeSpecName "kube-api-access-7kln5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 01:05:54.197624 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:54.197506    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4fb25d7a-dcc9-45eb-9f75-d9db5e74d228-kube-api-access-cx86g" (OuterVolumeSpecName: "kube-api-access-cx86g") pod "4fb25d7a-dcc9-45eb-9f75-d9db5e74d228" (UID: "4fb25d7a-dcc9-45eb-9f75-d9db5e74d228"). InnerVolumeSpecName "kube-api-access-cx86g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 01:05:54.296317 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:54.296292    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4fb25d7a-dcc9-45eb-9f75-d9db5e74d228-plugins-config-volume\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:05:54.296317 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:54.296315    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cx86g\" (UniqueName: \"kubernetes.io/projected/4fb25d7a-dcc9-45eb-9f75-d9db5e74d228-kube-api-access-cx86g\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:05:54.296501 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:54.296326    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7kln5\" (UniqueName: \"kubernetes.io/projected/35acb09e-33d9-46bf-8c4f-ea8d6ec02785-kube-api-access-7kln5\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:05:54.296501 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:54.296334    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35acb09e-33d9-46bf-8c4f-ea8d6ec02785-plugins-config-volume\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:05:54.861358 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:54.861324    2556 generic.go:356] "Generic (PLEG): container finished" podID="4fb25d7a-dcc9-45eb-9f75-d9db5e74d228" containerID="089876491b0778e2eae205ece3a9f25dab340f3e0c805d095249bfc5458b9281" exitCode=0
Sep 16 01:05:54.861802 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:54.861366    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-6d0d72-66b74965bf-x4dl6"
Sep 16 01:05:54.861802 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:54.861417    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-6d0d72-66b74965bf-x4dl6" event={"ID":"4fb25d7a-dcc9-45eb-9f75-d9db5e74d228","Type":"ContainerDied","Data":"089876491b0778e2eae205ece3a9f25dab340f3e0c805d095249bfc5458b9281"}
Sep 16 01:05:54.861802 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:54.861487    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-6d0d72-66b74965bf-x4dl6" event={"ID":"4fb25d7a-dcc9-45eb-9f75-d9db5e74d228","Type":"ContainerDied","Data":"a69612837509056ac7eb7fbcd5fd79db6e7ce7f6ab7342ea4744f7c4ef0a78f7"}
Sep 16 01:05:54.861802 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:54.861517    2556 scope.go:117] "RemoveContainer" containerID="089876491b0778e2eae205ece3a9f25dab340f3e0c805d095249bfc5458b9281"
Sep 16 01:05:54.862968 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:54.862934    2556 generic.go:356] "Generic (PLEG): container finished" podID="35acb09e-33d9-46bf-8c4f-ea8d6ec02785" containerID="b1fca5045a6831b94b70abfd671e25ef55fc8d69a342644f2793dd093755c3f0" exitCode=0
Sep 16 01:05:54.863083 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:54.862974    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6d0d72-5875fff75d-htxqr" event={"ID":"35acb09e-33d9-46bf-8c4f-ea8d6ec02785","Type":"ContainerDied","Data":"b1fca5045a6831b94b70abfd671e25ef55fc8d69a342644f2793dd093755c3f0"}
Sep 16 01:05:54.863083 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:54.863026    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6d0d72-5875fff75d-htxqr" event={"ID":"35acb09e-33d9-46bf-8c4f-ea8d6ec02785","Type":"ContainerDied","Data":"9aceace069912aa27ddab01c9e045f7d1fc3a2e54632df8fc3f6393cbf04741b"}
Sep 16 01:05:54.863083 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:54.862984    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6d0d72-5875fff75d-htxqr"
Sep 16 01:05:54.871275 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:54.871115    2556 scope.go:117] "RemoveContainer" containerID="089876491b0778e2eae205ece3a9f25dab340f3e0c805d095249bfc5458b9281"
Sep 16 01:05:54.871460 ip-10-0-130-207 kubenswrapper[2556]: E0916 01:05:54.871442    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"089876491b0778e2eae205ece3a9f25dab340f3e0c805d095249bfc5458b9281\": container with ID starting with 089876491b0778e2eae205ece3a9f25dab340f3e0c805d095249bfc5458b9281 not found: ID does not exist" containerID="089876491b0778e2eae205ece3a9f25dab340f3e0c805d095249bfc5458b9281"
Sep 16 01:05:54.871545 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:54.871484    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"089876491b0778e2eae205ece3a9f25dab340f3e0c805d095249bfc5458b9281"} err="failed to get container status \"089876491b0778e2eae205ece3a9f25dab340f3e0c805d095249bfc5458b9281\": rpc error: code = NotFound desc = could not find container \"089876491b0778e2eae205ece3a9f25dab340f3e0c805d095249bfc5458b9281\": container with ID starting with 089876491b0778e2eae205ece3a9f25dab340f3e0c805d095249bfc5458b9281 not found: ID does not exist"
Sep 16 01:05:54.871545 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:54.871504    2556 scope.go:117] "RemoveContainer" containerID="b1fca5045a6831b94b70abfd671e25ef55fc8d69a342644f2793dd093755c3f0"
Sep 16 01:05:54.880271 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:54.880255    2556 scope.go:117] "RemoveContainer" containerID="b1fca5045a6831b94b70abfd671e25ef55fc8d69a342644f2793dd093755c3f0"
Sep 16 01:05:54.880585 ip-10-0-130-207 kubenswrapper[2556]: E0916 01:05:54.880563    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1fca5045a6831b94b70abfd671e25ef55fc8d69a342644f2793dd093755c3f0\": container with ID starting with b1fca5045a6831b94b70abfd671e25ef55fc8d69a342644f2793dd093755c3f0 not found: ID does not exist" containerID="b1fca5045a6831b94b70abfd671e25ef55fc8d69a342644f2793dd093755c3f0"
Sep 16 01:05:54.880658 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:54.880596    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1fca5045a6831b94b70abfd671e25ef55fc8d69a342644f2793dd093755c3f0"} err="failed to get container status \"b1fca5045a6831b94b70abfd671e25ef55fc8d69a342644f2793dd093755c3f0\": rpc error: code = NotFound desc = could not find container \"b1fca5045a6831b94b70abfd671e25ef55fc8d69a342644f2793dd093755c3f0\": container with ID starting with b1fca5045a6831b94b70abfd671e25ef55fc8d69a342644f2793dd093755c3f0 not found: ID does not exist"
Sep 16 01:05:54.889881 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:54.889861    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-6d0d72-5875fff75d-htxqr"]
Sep 16 01:05:54.894924 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:54.894897    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-6d0d72-5875fff75d-htxqr"]
Sep 16 01:05:54.902598 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:54.902577    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-6d0d72-66b74965bf-x4dl6"]
Sep 16 01:05:54.906054 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:54.906030    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-6d0d72-66b74965bf-x4dl6"]
Sep 16 01:05:55.736746 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:55.736712    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35acb09e-33d9-46bf-8c4f-ea8d6ec02785" path="/var/lib/kubelet/pods/35acb09e-33d9-46bf-8c4f-ea8d6ec02785/volumes"
Sep 16 01:05:55.737069 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:05:55.737056    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4fb25d7a-dcc9-45eb-9f75-d9db5e74d228" path="/var/lib/kubelet/pods/4fb25d7a-dcc9-45eb-9f75-d9db5e74d228/volumes"
Sep 16 01:06:07.276918 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:07.276883    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f831c5-7fcf9bb6f8-86vjn"]
Sep 16 01:06:07.277327 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:07.277237    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4fb25d7a-dcc9-45eb-9f75-d9db5e74d228" containerName="payload-processing"
Sep 16 01:06:07.277327 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:07.277248    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="4fb25d7a-dcc9-45eb-9f75-d9db5e74d228" containerName="payload-processing"
Sep 16 01:06:07.277327 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:07.277269    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35acb09e-33d9-46bf-8c4f-ea8d6ec02785" containerName="payload-pre-processing"
Sep 16 01:06:07.277327 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:07.277275    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="35acb09e-33d9-46bf-8c4f-ea8d6ec02785" containerName="payload-pre-processing"
Sep 16 01:06:07.277327 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:07.277328    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="4fb25d7a-dcc9-45eb-9f75-d9db5e74d228" containerName="payload-processing"
Sep 16 01:06:07.277608 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:07.277339    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="35acb09e-33d9-46bf-8c4f-ea8d6ec02785" containerName="payload-pre-processing"
Sep 16 01:06:07.281635 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:07.281618    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f831c5-7fcf9bb6f8-86vjn"
Sep 16 01:06:07.283913 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:07.283891    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-f831c5-dockercfg-4sw4k\""
Sep 16 01:06:07.284022 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:07.283938    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-f831c5\""
Sep 16 01:06:07.294340 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:07.294306    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f831c5-7fcf9bb6f8-86vjn"]
Sep 16 01:06:07.420015 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:07.419980    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fdd181b6-f35b-4862-8988-23d546bd0c36-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-f831c5-7fcf9bb6f8-86vjn\" (UID: \"fdd181b6-f35b-4862-8988-23d546bd0c36\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f831c5-7fcf9bb6f8-86vjn"
Sep 16 01:06:07.420195 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:07.420115    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pfjm\" (UniqueName: \"kubernetes.io/projected/fdd181b6-f35b-4862-8988-23d546bd0c36-kube-api-access-9pfjm\") pod \"payload-pre-processing-e2e-shared-a-w2-f831c5-7fcf9bb6f8-86vjn\" (UID: \"fdd181b6-f35b-4862-8988-23d546bd0c36\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f831c5-7fcf9bb6f8-86vjn"
Sep 16 01:06:07.521178 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:07.521140    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pfjm\" (UniqueName: \"kubernetes.io/projected/fdd181b6-f35b-4862-8988-23d546bd0c36-kube-api-access-9pfjm\") pod \"payload-pre-processing-e2e-shared-a-w2-f831c5-7fcf9bb6f8-86vjn\" (UID: \"fdd181b6-f35b-4862-8988-23d546bd0c36\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f831c5-7fcf9bb6f8-86vjn"
Sep 16 01:06:07.521336 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:07.521206    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fdd181b6-f35b-4862-8988-23d546bd0c36-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-f831c5-7fcf9bb6f8-86vjn\" (UID: \"fdd181b6-f35b-4862-8988-23d546bd0c36\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f831c5-7fcf9bb6f8-86vjn"
Sep 16 01:06:07.521865 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:07.521845    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fdd181b6-f35b-4862-8988-23d546bd0c36-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-f831c5-7fcf9bb6f8-86vjn\" (UID: \"fdd181b6-f35b-4862-8988-23d546bd0c36\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f831c5-7fcf9bb6f8-86vjn"
Sep 16 01:06:07.528703 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:07.528628    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pfjm\" (UniqueName: \"kubernetes.io/projected/fdd181b6-f35b-4862-8988-23d546bd0c36-kube-api-access-9pfjm\") pod \"payload-pre-processing-e2e-shared-a-w2-f831c5-7fcf9bb6f8-86vjn\" (UID: \"fdd181b6-f35b-4862-8988-23d546bd0c36\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f831c5-7fcf9bb6f8-86vjn"
Sep 16 01:06:07.591444 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:07.591414    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f831c5-7fcf9bb6f8-86vjn"
Sep 16 01:06:07.860785 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:07.860758    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f831c5-7fcf9bb6f8-86vjn"]
Sep 16 01:06:07.862164 ip-10-0-130-207 kubenswrapper[2556]: W0916 01:06:07.862137    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdd181b6_f35b_4862_8988_23d546bd0c36.slice/crio-514150212c6fc7476777f6efb709d0c1648426fd5643a6a05471b44370fa892e WatchSource:0}: Error finding container 514150212c6fc7476777f6efb709d0c1648426fd5643a6a05471b44370fa892e: Status 404 returned error can't find the container with id 514150212c6fc7476777f6efb709d0c1648426fd5643a6a05471b44370fa892e
Sep 16 01:06:07.936938 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:07.936879    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f831c5-7fcf9bb6f8-86vjn" event={"ID":"fdd181b6-f35b-4862-8988-23d546bd0c36","Type":"ContainerStarted","Data":"f7f38adadee82603606abfe93bb96fc40bf039eb0dd1560cd94f08bb6f044809"}
Sep 16 01:06:07.937069 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:07.936952    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f831c5-7fcf9bb6f8-86vjn" event={"ID":"fdd181b6-f35b-4862-8988-23d546bd0c36","Type":"ContainerStarted","Data":"514150212c6fc7476777f6efb709d0c1648426fd5643a6a05471b44370fa892e"}
Sep 16 01:06:07.937161 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:07.937141    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f831c5-7fcf9bb6f8-86vjn"
Sep 16 01:06:07.952181 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:07.952129    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f831c5-7fcf9bb6f8-86vjn" podStartSLOduration=0.952113344 podStartE2EDuration="952.113344ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 01:06:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 01:06:07.950671096 +0000 UTC m=+1382.732172707" watchObservedRunningTime="2026-09-16 01:06:07.952113344 +0000 UTC m=+1382.733614963"
Sep 16 01:06:09.170613 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:09.170581    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-a58c3d9e-dd4f9d4b-mfjzp"]
Sep 16 01:06:09.171043 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:09.170776    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-a58c3d9e-dd4f9d4b-mfjzp" podUID="a8087b04-4a35-49f4-8d17-a399ad061d45" containerName="payload-processing" containerID="cri-o://e4bd9479168c606b538255cd2b562e6eef502532af5af36edaa3d359d11e719b" gracePeriod=30
Sep 16 01:06:09.420186 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:09.420162    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-a58c3d9e-dd4f9d4b-mfjzp"
Sep 16 01:06:09.542398 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:09.542302    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hdkbm\" (UniqueName: \"kubernetes.io/projected/a8087b04-4a35-49f4-8d17-a399ad061d45-kube-api-access-hdkbm\") pod \"a8087b04-4a35-49f4-8d17-a399ad061d45\" (UID: \"a8087b04-4a35-49f4-8d17-a399ad061d45\") "
Sep 16 01:06:09.542595 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:09.542441    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8087b04-4a35-49f4-8d17-a399ad061d45-plugins-config-volume\") pod \"a8087b04-4a35-49f4-8d17-a399ad061d45\" (UID: \"a8087b04-4a35-49f4-8d17-a399ad061d45\") "
Sep 16 01:06:09.542783 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:09.542752    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8087b04-4a35-49f4-8d17-a399ad061d45-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a8087b04-4a35-49f4-8d17-a399ad061d45" (UID: "a8087b04-4a35-49f4-8d17-a399ad061d45"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 01:06:09.544423 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:09.544401    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8087b04-4a35-49f4-8d17-a399ad061d45-kube-api-access-hdkbm" (OuterVolumeSpecName: "kube-api-access-hdkbm") pod "a8087b04-4a35-49f4-8d17-a399ad061d45" (UID: "a8087b04-4a35-49f4-8d17-a399ad061d45"). InnerVolumeSpecName "kube-api-access-hdkbm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 01:06:09.643367 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:09.643337    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8087b04-4a35-49f4-8d17-a399ad061d45-plugins-config-volume\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:06:09.643367 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:09.643364    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hdkbm\" (UniqueName: \"kubernetes.io/projected/a8087b04-4a35-49f4-8d17-a399ad061d45-kube-api-access-hdkbm\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:06:09.948211 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:09.948175    2556 generic.go:356] "Generic (PLEG): container finished" podID="a8087b04-4a35-49f4-8d17-a399ad061d45" containerID="e4bd9479168c606b538255cd2b562e6eef502532af5af36edaa3d359d11e719b" exitCode=0
Sep 16 01:06:09.948382 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:09.948226    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-a58c3d9e-dd4f9d4b-mfjzp"
Sep 16 01:06:09.948382 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:09.948237    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-a58c3d9e-dd4f9d4b-mfjzp" event={"ID":"a8087b04-4a35-49f4-8d17-a399ad061d45","Type":"ContainerDied","Data":"e4bd9479168c606b538255cd2b562e6eef502532af5af36edaa3d359d11e719b"}
Sep 16 01:06:09.948382 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:09.948283    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-a58c3d9e-dd4f9d4b-mfjzp" event={"ID":"a8087b04-4a35-49f4-8d17-a399ad061d45","Type":"ContainerDied","Data":"b005b57e5758b0601b7a4ed020455bf3c5a453e6bd9b9ab7d5ac464c6fd4a7e5"}
Sep 16 01:06:09.948382 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:09.948304    2556 scope.go:117] "RemoveContainer" containerID="e4bd9479168c606b538255cd2b562e6eef502532af5af36edaa3d359d11e719b"
Sep 16 01:06:09.956879 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:09.956863    2556 scope.go:117] "RemoveContainer" containerID="e4bd9479168c606b538255cd2b562e6eef502532af5af36edaa3d359d11e719b"
Sep 16 01:06:09.957140 ip-10-0-130-207 kubenswrapper[2556]: E0916 01:06:09.957114    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4bd9479168c606b538255cd2b562e6eef502532af5af36edaa3d359d11e719b\": container with ID starting with e4bd9479168c606b538255cd2b562e6eef502532af5af36edaa3d359d11e719b not found: ID does not exist" containerID="e4bd9479168c606b538255cd2b562e6eef502532af5af36edaa3d359d11e719b"
Sep 16 01:06:09.957202 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:09.957148    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4bd9479168c606b538255cd2b562e6eef502532af5af36edaa3d359d11e719b"} err="failed to get container status \"e4bd9479168c606b538255cd2b562e6eef502532af5af36edaa3d359d11e719b\": rpc error: code = NotFound desc = could not find container \"e4bd9479168c606b538255cd2b562e6eef502532af5af36edaa3d359d11e719b\": container with ID starting with e4bd9479168c606b538255cd2b562e6eef502532af5af36edaa3d359d11e719b not found: ID does not exist"
Sep 16 01:06:09.971216 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:09.971191    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-a58c3d9e-dd4f9d4b-mfjzp"]
Sep 16 01:06:09.976977 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:09.976954    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-a58c3d9e-dd4f9d4b-mfjzp"]
Sep 16 01:06:11.738865 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:11.738827    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8087b04-4a35-49f4-8d17-a399ad061d45" path="/var/lib/kubelet/pods/a8087b04-4a35-49f4-8d17-a399ad061d45/volumes"
Sep 16 01:06:16.895646 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:16.895613    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-30fb702c-68f994b98f-wq2lg"]
Sep 16 01:06:16.896035 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:16.896018    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8087b04-4a35-49f4-8d17-a399ad061d45" containerName="payload-processing"
Sep 16 01:06:16.896099 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:16.896037    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8087b04-4a35-49f4-8d17-a399ad061d45" containerName="payload-processing"
Sep 16 01:06:16.896134 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:16.896102    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8087b04-4a35-49f4-8d17-a399ad061d45" containerName="payload-processing"
Sep 16 01:06:16.900196 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:16.900179    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-30fb702c-68f994b98f-wq2lg"
Sep 16 01:06:16.902451 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:16.902422    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-30fb702c-dockercfg-xqlkm\""
Sep 16 01:06:16.902596 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:16.902451    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-30fb702c\""
Sep 16 01:06:16.908718 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:16.908693    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-30fb702c-68f994b98f-wq2lg"]
Sep 16 01:06:16.930911 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:16.930886    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-30fb702c-849ff5678b-bj9fz"]
Sep 16 01:06:16.934169 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:16.934152    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-30fb702c-849ff5678b-bj9fz"
Sep 16 01:06:16.944001 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:16.943979    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-30fb702c-849ff5678b-bj9fz"]
Sep 16 01:06:17.009292 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:17.009263    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef8ad993-f627-491d-87be-dbfef99d431c-plugins-config-volume\") pod \"payload-processing-e2e-ait-30fb702c-68f994b98f-wq2lg\" (UID: \"ef8ad993-f627-491d-87be-dbfef99d431c\") " pod="openshift-ingress/payload-processing-e2e-ait-30fb702c-68f994b98f-wq2lg"
Sep 16 01:06:17.009482 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:17.009307    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4cd5caf-7b82-4831-92a5-c00bd6b55333-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-30fb702c-849ff5678b-bj9fz\" (UID: \"f4cd5caf-7b82-4831-92a5-c00bd6b55333\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-30fb702c-849ff5678b-bj9fz"
Sep 16 01:06:17.009482 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:17.009327    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7df6x\" (UniqueName: \"kubernetes.io/projected/f4cd5caf-7b82-4831-92a5-c00bd6b55333-kube-api-access-7df6x\") pod \"payload-pre-processing-e2e-ait-30fb702c-849ff5678b-bj9fz\" (UID: \"f4cd5caf-7b82-4831-92a5-c00bd6b55333\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-30fb702c-849ff5678b-bj9fz"
Sep 16 01:06:17.009482 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:17.009361    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nb6gr\" (UniqueName: \"kubernetes.io/projected/ef8ad993-f627-491d-87be-dbfef99d431c-kube-api-access-nb6gr\") pod \"payload-processing-e2e-ait-30fb702c-68f994b98f-wq2lg\" (UID: \"ef8ad993-f627-491d-87be-dbfef99d431c\") " pod="openshift-ingress/payload-processing-e2e-ait-30fb702c-68f994b98f-wq2lg"
Sep 16 01:06:17.110017 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:17.109987    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef8ad993-f627-491d-87be-dbfef99d431c-plugins-config-volume\") pod \"payload-processing-e2e-ait-30fb702c-68f994b98f-wq2lg\" (UID: \"ef8ad993-f627-491d-87be-dbfef99d431c\") " pod="openshift-ingress/payload-processing-e2e-ait-30fb702c-68f994b98f-wq2lg"
Sep 16 01:06:17.110153 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:17.110032    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4cd5caf-7b82-4831-92a5-c00bd6b55333-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-30fb702c-849ff5678b-bj9fz\" (UID: \"f4cd5caf-7b82-4831-92a5-c00bd6b55333\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-30fb702c-849ff5678b-bj9fz"
Sep 16 01:06:17.110153 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:17.110051    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7df6x\" (UniqueName: \"kubernetes.io/projected/f4cd5caf-7b82-4831-92a5-c00bd6b55333-kube-api-access-7df6x\") pod \"payload-pre-processing-e2e-ait-30fb702c-849ff5678b-bj9fz\" (UID: \"f4cd5caf-7b82-4831-92a5-c00bd6b55333\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-30fb702c-849ff5678b-bj9fz"
Sep 16 01:06:17.110153 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:17.110083    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nb6gr\" (UniqueName: \"kubernetes.io/projected/ef8ad993-f627-491d-87be-dbfef99d431c-kube-api-access-nb6gr\") pod \"payload-processing-e2e-ait-30fb702c-68f994b98f-wq2lg\" (UID: \"ef8ad993-f627-491d-87be-dbfef99d431c\") " pod="openshift-ingress/payload-processing-e2e-ait-30fb702c-68f994b98f-wq2lg"
Sep 16 01:06:17.110777 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:17.110757    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4cd5caf-7b82-4831-92a5-c00bd6b55333-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-30fb702c-849ff5678b-bj9fz\" (UID: \"f4cd5caf-7b82-4831-92a5-c00bd6b55333\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-30fb702c-849ff5678b-bj9fz"
Sep 16 01:06:17.110821 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:17.110759    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef8ad993-f627-491d-87be-dbfef99d431c-plugins-config-volume\") pod \"payload-processing-e2e-ait-30fb702c-68f994b98f-wq2lg\" (UID: \"ef8ad993-f627-491d-87be-dbfef99d431c\") " pod="openshift-ingress/payload-processing-e2e-ait-30fb702c-68f994b98f-wq2lg"
Sep 16 01:06:17.117596 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:17.117570    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nb6gr\" (UniqueName: \"kubernetes.io/projected/ef8ad993-f627-491d-87be-dbfef99d431c-kube-api-access-nb6gr\") pod \"payload-processing-e2e-ait-30fb702c-68f994b98f-wq2lg\" (UID: \"ef8ad993-f627-491d-87be-dbfef99d431c\") " pod="openshift-ingress/payload-processing-e2e-ait-30fb702c-68f994b98f-wq2lg"
Sep 16 01:06:17.117702 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:17.117617    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7df6x\" (UniqueName: \"kubernetes.io/projected/f4cd5caf-7b82-4831-92a5-c00bd6b55333-kube-api-access-7df6x\") pod \"payload-pre-processing-e2e-ait-30fb702c-849ff5678b-bj9fz\" (UID: \"f4cd5caf-7b82-4831-92a5-c00bd6b55333\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-30fb702c-849ff5678b-bj9fz"
Sep 16 01:06:17.211854 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:17.211790    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-30fb702c-68f994b98f-wq2lg"
Sep 16 01:06:17.244600 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:17.244578    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-30fb702c-849ff5678b-bj9fz"
Sep 16 01:06:17.482407 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:17.482384    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-30fb702c-68f994b98f-wq2lg"]
Sep 16 01:06:17.484219 ip-10-0-130-207 kubenswrapper[2556]: W0916 01:06:17.484196    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef8ad993_f627_491d_87be_dbfef99d431c.slice/crio-a88501d7c359865ac251c370702519fe2096678dcf295ca60c8973a55c58f8ef WatchSource:0}: Error finding container a88501d7c359865ac251c370702519fe2096678dcf295ca60c8973a55c58f8ef: Status 404 returned error can't find the container with id a88501d7c359865ac251c370702519fe2096678dcf295ca60c8973a55c58f8ef
Sep 16 01:06:17.526354 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:17.526332    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-30fb702c-849ff5678b-bj9fz"]
Sep 16 01:06:17.532443 ip-10-0-130-207 kubenswrapper[2556]: W0916 01:06:17.532418    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4cd5caf_7b82_4831_92a5_c00bd6b55333.slice/crio-8422aa84adab4eedd9d8e6e8c29131b0dd698f3c06df680779ff93c862fd16c1 WatchSource:0}: Error finding container 8422aa84adab4eedd9d8e6e8c29131b0dd698f3c06df680779ff93c862fd16c1: Status 404 returned error can't find the container with id 8422aa84adab4eedd9d8e6e8c29131b0dd698f3c06df680779ff93c862fd16c1
Sep 16 01:06:17.997275 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:17.997236    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-30fb702c-68f994b98f-wq2lg" event={"ID":"ef8ad993-f627-491d-87be-dbfef99d431c","Type":"ContainerStarted","Data":"f850895faf1925cbced7073029e90fe91bc35e8039dab0d1310675e6694e7912"}
Sep 16 01:06:17.997745 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:17.997303    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-30fb702c-68f994b98f-wq2lg" event={"ID":"ef8ad993-f627-491d-87be-dbfef99d431c","Type":"ContainerStarted","Data":"a88501d7c359865ac251c370702519fe2096678dcf295ca60c8973a55c58f8ef"}
Sep 16 01:06:17.997745 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:17.997334    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-30fb702c-68f994b98f-wq2lg"
Sep 16 01:06:17.998968 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:17.998942    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-30fb702c-849ff5678b-bj9fz" event={"ID":"f4cd5caf-7b82-4831-92a5-c00bd6b55333","Type":"ContainerStarted","Data":"a41e59108ef12a6e4e29be2c315a42d28f644df3b3ad90b43bb9d090210caa67"}
Sep 16 01:06:17.999103 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:17.998984    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-30fb702c-849ff5678b-bj9fz" event={"ID":"f4cd5caf-7b82-4831-92a5-c00bd6b55333","Type":"ContainerStarted","Data":"8422aa84adab4eedd9d8e6e8c29131b0dd698f3c06df680779ff93c862fd16c1"}
Sep 16 01:06:17.999103 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:17.999003    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-30fb702c-849ff5678b-bj9fz"
Sep 16 01:06:18.013408 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:18.013368    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-30fb702c-68f994b98f-wq2lg" podStartSLOduration=2.013353135 podStartE2EDuration="2.013353135s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 01:06:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 01:06:18.012706054 +0000 UTC m=+1392.794207649" watchObservedRunningTime="2026-09-16 01:06:18.013353135 +0000 UTC m=+1392.794854726"
Sep 16 01:06:18.026726 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:18.026641    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-30fb702c-849ff5678b-bj9fz" podStartSLOduration=2.026623694 podStartE2EDuration="2.026623694s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 01:06:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 01:06:18.026595918 +0000 UTC m=+1392.808097516" watchObservedRunningTime="2026-09-16 01:06:18.026623694 +0000 UTC m=+1392.808125287"
Sep 16 01:06:18.943762 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:18.943730    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f831c5-7fcf9bb6f8-86vjn"
Sep 16 01:06:25.785969 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:25.785938    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0ee1bf-589b478bf4-tmjrm"]
Sep 16 01:06:25.789428 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:25.789408    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0ee1bf-589b478bf4-tmjrm"
Sep 16 01:06:25.791926 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:25.791898    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-0ee1bf-dockercfg-j7hvn\""
Sep 16 01:06:25.792028 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:25.791931    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-0ee1bf\""
Sep 16 01:06:25.798312 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:25.798287    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0ee1bf-589b478bf4-tmjrm"]
Sep 16 01:06:25.812308 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:25.812286    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-0ee1bf-9d8459445-ddpbs"]
Sep 16 01:06:25.815767 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:25.815751    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-0ee1bf-9d8459445-ddpbs"
Sep 16 01:06:25.825082 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:25.825061    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-0ee1bf-9d8459445-ddpbs"]
Sep 16 01:06:25.895205 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:25.895175    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49eeadd0-10a0-4130-b174-0088166e8d60-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-0ee1bf-589b478bf4-tmjrm\" (UID: \"49eeadd0-10a0-4130-b174-0088166e8d60\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0ee1bf-589b478bf4-tmjrm"
Sep 16 01:06:25.895352 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:25.895214    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xpgc4\" (UniqueName: \"kubernetes.io/projected/49eeadd0-10a0-4130-b174-0088166e8d60-kube-api-access-xpgc4\") pod \"payload-pre-processing-e2e-shared-b-w2-0ee1bf-589b478bf4-tmjrm\" (UID: \"49eeadd0-10a0-4130-b174-0088166e8d60\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0ee1bf-589b478bf4-tmjrm"
Sep 16 01:06:25.995729 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:25.995695    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49eeadd0-10a0-4130-b174-0088166e8d60-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-0ee1bf-589b478bf4-tmjrm\" (UID: \"49eeadd0-10a0-4130-b174-0088166e8d60\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0ee1bf-589b478bf4-tmjrm"
Sep 16 01:06:25.995729 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:25.995736    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpgc4\" (UniqueName: \"kubernetes.io/projected/49eeadd0-10a0-4130-b174-0088166e8d60-kube-api-access-xpgc4\") pod \"payload-pre-processing-e2e-shared-b-w2-0ee1bf-589b478bf4-tmjrm\" (UID: \"49eeadd0-10a0-4130-b174-0088166e8d60\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0ee1bf-589b478bf4-tmjrm"
Sep 16 01:06:25.995987 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:25.995762    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26bcde78-012a-4c98-9d68-1d56904af202-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-0ee1bf-9d8459445-ddpbs\" (UID: \"26bcde78-012a-4c98-9d68-1d56904af202\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-0ee1bf-9d8459445-ddpbs"
Sep 16 01:06:25.995987 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:25.995804    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trrrk\" (UniqueName: \"kubernetes.io/projected/26bcde78-012a-4c98-9d68-1d56904af202-kube-api-access-trrrk\") pod \"payload-processing-e2e-shared-b-w2-0ee1bf-9d8459445-ddpbs\" (UID: \"26bcde78-012a-4c98-9d68-1d56904af202\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-0ee1bf-9d8459445-ddpbs"
Sep 16 01:06:25.996516 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:25.996461    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49eeadd0-10a0-4130-b174-0088166e8d60-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-0ee1bf-589b478bf4-tmjrm\" (UID: \"49eeadd0-10a0-4130-b174-0088166e8d60\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0ee1bf-589b478bf4-tmjrm"
Sep 16 01:06:26.003391 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:26.003369    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpgc4\" (UniqueName: \"kubernetes.io/projected/49eeadd0-10a0-4130-b174-0088166e8d60-kube-api-access-xpgc4\") pod \"payload-pre-processing-e2e-shared-b-w2-0ee1bf-589b478bf4-tmjrm\" (UID: \"49eeadd0-10a0-4130-b174-0088166e8d60\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0ee1bf-589b478bf4-tmjrm"
Sep 16 01:06:26.096939 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:26.096912    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26bcde78-012a-4c98-9d68-1d56904af202-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-0ee1bf-9d8459445-ddpbs\" (UID: \"26bcde78-012a-4c98-9d68-1d56904af202\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-0ee1bf-9d8459445-ddpbs"
Sep 16 01:06:26.097084 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:26.096966    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trrrk\" (UniqueName: \"kubernetes.io/projected/26bcde78-012a-4c98-9d68-1d56904af202-kube-api-access-trrrk\") pod \"payload-processing-e2e-shared-b-w2-0ee1bf-9d8459445-ddpbs\" (UID: \"26bcde78-012a-4c98-9d68-1d56904af202\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-0ee1bf-9d8459445-ddpbs"
Sep 16 01:06:26.097632 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:26.097613    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26bcde78-012a-4c98-9d68-1d56904af202-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-0ee1bf-9d8459445-ddpbs\" (UID: \"26bcde78-012a-4c98-9d68-1d56904af202\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-0ee1bf-9d8459445-ddpbs"
Sep 16 01:06:26.101133 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:26.101116    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0ee1bf-589b478bf4-tmjrm"
Sep 16 01:06:26.103953 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:26.103926    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-trrrk\" (UniqueName: \"kubernetes.io/projected/26bcde78-012a-4c98-9d68-1d56904af202-kube-api-access-trrrk\") pod \"payload-processing-e2e-shared-b-w2-0ee1bf-9d8459445-ddpbs\" (UID: \"26bcde78-012a-4c98-9d68-1d56904af202\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-0ee1bf-9d8459445-ddpbs"
Sep 16 01:06:26.126313 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:26.126290    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-0ee1bf-9d8459445-ddpbs"
Sep 16 01:06:26.374871 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:26.374845    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0ee1bf-589b478bf4-tmjrm"]
Sep 16 01:06:26.377066 ip-10-0-130-207 kubenswrapper[2556]: W0916 01:06:26.377036    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49eeadd0_10a0_4130_b174_0088166e8d60.slice/crio-5cca7506b1065543513d177a70c3d8078cb8554b84277568b9683b61045084bc WatchSource:0}: Error finding container 5cca7506b1065543513d177a70c3d8078cb8554b84277568b9683b61045084bc: Status 404 returned error can't find the container with id 5cca7506b1065543513d177a70c3d8078cb8554b84277568b9683b61045084bc
Sep 16 01:06:26.410193 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:26.410173    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-0ee1bf-9d8459445-ddpbs"]
Sep 16 01:06:26.413510 ip-10-0-130-207 kubenswrapper[2556]: W0916 01:06:26.413460    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26bcde78_012a_4c98_9d68_1d56904af202.slice/crio-28955dc954c1939ef0cf7546552f75e675f4adb3e1fa3455b059a3c809934875 WatchSource:0}: Error finding container 28955dc954c1939ef0cf7546552f75e675f4adb3e1fa3455b059a3c809934875: Status 404 returned error can't find the container with id 28955dc954c1939ef0cf7546552f75e675f4adb3e1fa3455b059a3c809934875
Sep 16 01:06:27.048402 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:27.048366    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0ee1bf-589b478bf4-tmjrm" event={"ID":"49eeadd0-10a0-4130-b174-0088166e8d60","Type":"ContainerStarted","Data":"966a7f60535e2a3f43cf2881190ebd4633874812b0494803d9e574f5bc58e6f2"}
Sep 16 01:06:27.048847 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:27.048741    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0ee1bf-589b478bf4-tmjrm"
Sep 16 01:06:27.048847 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:27.048776    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0ee1bf-589b478bf4-tmjrm" event={"ID":"49eeadd0-10a0-4130-b174-0088166e8d60","Type":"ContainerStarted","Data":"5cca7506b1065543513d177a70c3d8078cb8554b84277568b9683b61045084bc"}
Sep 16 01:06:27.050113 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:27.050088    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-0ee1bf-9d8459445-ddpbs" event={"ID":"26bcde78-012a-4c98-9d68-1d56904af202","Type":"ContainerStarted","Data":"6c4bb89dd4efb7f65b15cf05e14afda9895256a175a80935b5ca73108691082a"}
Sep 16 01:06:27.050236 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:27.050126    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-0ee1bf-9d8459445-ddpbs" event={"ID":"26bcde78-012a-4c98-9d68-1d56904af202","Type":"ContainerStarted","Data":"28955dc954c1939ef0cf7546552f75e675f4adb3e1fa3455b059a3c809934875"}
Sep 16 01:06:27.050236 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:27.050167    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-0ee1bf-9d8459445-ddpbs"
Sep 16 01:06:27.064945 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:27.064904    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0ee1bf-589b478bf4-tmjrm" podStartSLOduration=2.064891533 podStartE2EDuration="2.064891533s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 01:06:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 01:06:27.06225723 +0000 UTC m=+1401.843758822" watchObservedRunningTime="2026-09-16 01:06:27.064891533 +0000 UTC m=+1401.846393126"
Sep 16 01:06:27.079282 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:27.079195    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-0ee1bf-9d8459445-ddpbs" podStartSLOduration=2.079178491 podStartE2EDuration="2.079178491s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 01:06:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 01:06:27.076280363 +0000 UTC m=+1401.857781956" watchObservedRunningTime="2026-09-16 01:06:27.079178491 +0000 UTC m=+1401.860680083"
Sep 16 01:06:29.008404 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:29.008328    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-30fb702c-68f994b98f-wq2lg"
Sep 16 01:06:29.008404 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:29.008392    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-30fb702c-849ff5678b-bj9fz"
Sep 16 01:06:38.057594 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:38.057564    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-0ee1bf-9d8459445-ddpbs"
Sep 16 01:06:38.058066 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:38.057735    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0ee1bf-589b478bf4-tmjrm"
Sep 16 01:06:55.309248 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:55.309193    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-30fb702c-68f994b98f-wq2lg"]
Sep 16 01:06:55.309709 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:55.309407    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-30fb702c-68f994b98f-wq2lg" podUID="ef8ad993-f627-491d-87be-dbfef99d431c" containerName="payload-processing" containerID="cri-o://f850895faf1925cbced7073029e90fe91bc35e8039dab0d1310675e6694e7912" gracePeriod=30
Sep 16 01:06:55.317712 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:55.317687    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-30fb702c-849ff5678b-bj9fz"]
Sep 16 01:06:55.317927 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:55.317906    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-30fb702c-849ff5678b-bj9fz" podUID="f4cd5caf-7b82-4831-92a5-c00bd6b55333" containerName="payload-pre-processing" containerID="cri-o://a41e59108ef12a6e4e29be2c315a42d28f644df3b3ad90b43bb9d090210caa67" gracePeriod=30
Sep 16 01:06:55.556497 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:55.556455    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-30fb702c-68f994b98f-wq2lg"
Sep 16 01:06:55.596011 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:55.595985    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-30fb702c-849ff5678b-bj9fz"
Sep 16 01:06:55.652118 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:55.652077    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef8ad993-f627-491d-87be-dbfef99d431c-plugins-config-volume\") pod \"ef8ad993-f627-491d-87be-dbfef99d431c\" (UID: \"ef8ad993-f627-491d-87be-dbfef99d431c\") "
Sep 16 01:06:55.652305 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:55.652175    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4cd5caf-7b82-4831-92a5-c00bd6b55333-plugins-config-volume\") pod \"f4cd5caf-7b82-4831-92a5-c00bd6b55333\" (UID: \"f4cd5caf-7b82-4831-92a5-c00bd6b55333\") "
Sep 16 01:06:55.652305 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:55.652197    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nb6gr\" (UniqueName: \"kubernetes.io/projected/ef8ad993-f627-491d-87be-dbfef99d431c-kube-api-access-nb6gr\") pod \"ef8ad993-f627-491d-87be-dbfef99d431c\" (UID: \"ef8ad993-f627-491d-87be-dbfef99d431c\") "
Sep 16 01:06:55.652305 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:55.652241    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7df6x\" (UniqueName: \"kubernetes.io/projected/f4cd5caf-7b82-4831-92a5-c00bd6b55333-kube-api-access-7df6x\") pod \"f4cd5caf-7b82-4831-92a5-c00bd6b55333\" (UID: \"f4cd5caf-7b82-4831-92a5-c00bd6b55333\") "
Sep 16 01:06:55.652571 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:55.652538    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef8ad993-f627-491d-87be-dbfef99d431c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ef8ad993-f627-491d-87be-dbfef99d431c" (UID: "ef8ad993-f627-491d-87be-dbfef99d431c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 01:06:55.652635 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:55.652609    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4cd5caf-7b82-4831-92a5-c00bd6b55333-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f4cd5caf-7b82-4831-92a5-c00bd6b55333" (UID: "f4cd5caf-7b82-4831-92a5-c00bd6b55333"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 01:06:55.654876 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:55.654845    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4cd5caf-7b82-4831-92a5-c00bd6b55333-kube-api-access-7df6x" (OuterVolumeSpecName: "kube-api-access-7df6x") pod "f4cd5caf-7b82-4831-92a5-c00bd6b55333" (UID: "f4cd5caf-7b82-4831-92a5-c00bd6b55333"). InnerVolumeSpecName "kube-api-access-7df6x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 01:06:55.654985 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:55.654945    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef8ad993-f627-491d-87be-dbfef99d431c-kube-api-access-nb6gr" (OuterVolumeSpecName: "kube-api-access-nb6gr") pod "ef8ad993-f627-491d-87be-dbfef99d431c" (UID: "ef8ad993-f627-491d-87be-dbfef99d431c"). InnerVolumeSpecName "kube-api-access-nb6gr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 01:06:55.753863 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:55.753829    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4cd5caf-7b82-4831-92a5-c00bd6b55333-plugins-config-volume\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:06:55.753863 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:55.753864    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nb6gr\" (UniqueName: \"kubernetes.io/projected/ef8ad993-f627-491d-87be-dbfef99d431c-kube-api-access-nb6gr\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:06:55.753863 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:55.753873    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7df6x\" (UniqueName: \"kubernetes.io/projected/f4cd5caf-7b82-4831-92a5-c00bd6b55333-kube-api-access-7df6x\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:06:55.754051 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:55.753883    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef8ad993-f627-491d-87be-dbfef99d431c-plugins-config-volume\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:06:56.206497 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:56.206450    2556 generic.go:356] "Generic (PLEG): container finished" podID="f4cd5caf-7b82-4831-92a5-c00bd6b55333" containerID="a41e59108ef12a6e4e29be2c315a42d28f644df3b3ad90b43bb9d090210caa67" exitCode=0
Sep 16 01:06:56.206692 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:56.206517    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-30fb702c-849ff5678b-bj9fz"
Sep 16 01:06:56.206692 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:56.206539    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-30fb702c-849ff5678b-bj9fz" event={"ID":"f4cd5caf-7b82-4831-92a5-c00bd6b55333","Type":"ContainerDied","Data":"a41e59108ef12a6e4e29be2c315a42d28f644df3b3ad90b43bb9d090210caa67"}
Sep 16 01:06:56.206692 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:56.206628    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-30fb702c-849ff5678b-bj9fz" event={"ID":"f4cd5caf-7b82-4831-92a5-c00bd6b55333","Type":"ContainerDied","Data":"8422aa84adab4eedd9d8e6e8c29131b0dd698f3c06df680779ff93c862fd16c1"}
Sep 16 01:06:56.206692 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:56.206657    2556 scope.go:117] "RemoveContainer" containerID="a41e59108ef12a6e4e29be2c315a42d28f644df3b3ad90b43bb9d090210caa67"
Sep 16 01:06:56.207936 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:56.207906    2556 generic.go:356] "Generic (PLEG): container finished" podID="ef8ad993-f627-491d-87be-dbfef99d431c" containerID="f850895faf1925cbced7073029e90fe91bc35e8039dab0d1310675e6694e7912" exitCode=0
Sep 16 01:06:56.208030 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:56.207965    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-30fb702c-68f994b98f-wq2lg"
Sep 16 01:06:56.208085 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:56.207968    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-30fb702c-68f994b98f-wq2lg" event={"ID":"ef8ad993-f627-491d-87be-dbfef99d431c","Type":"ContainerDied","Data":"f850895faf1925cbced7073029e90fe91bc35e8039dab0d1310675e6694e7912"}
Sep 16 01:06:56.208085 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:56.208080    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-30fb702c-68f994b98f-wq2lg" event={"ID":"ef8ad993-f627-491d-87be-dbfef99d431c","Type":"ContainerDied","Data":"a88501d7c359865ac251c370702519fe2096678dcf295ca60c8973a55c58f8ef"}
Sep 16 01:06:56.215614 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:56.215596    2556 scope.go:117] "RemoveContainer" containerID="a41e59108ef12a6e4e29be2c315a42d28f644df3b3ad90b43bb9d090210caa67"
Sep 16 01:06:56.215863 ip-10-0-130-207 kubenswrapper[2556]: E0916 01:06:56.215846    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a41e59108ef12a6e4e29be2c315a42d28f644df3b3ad90b43bb9d090210caa67\": container with ID starting with a41e59108ef12a6e4e29be2c315a42d28f644df3b3ad90b43bb9d090210caa67 not found: ID does not exist" containerID="a41e59108ef12a6e4e29be2c315a42d28f644df3b3ad90b43bb9d090210caa67"
Sep 16 01:06:56.215918 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:56.215871    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a41e59108ef12a6e4e29be2c315a42d28f644df3b3ad90b43bb9d090210caa67"} err="failed to get container status \"a41e59108ef12a6e4e29be2c315a42d28f644df3b3ad90b43bb9d090210caa67\": rpc error: code = NotFound desc = could not find container \"a41e59108ef12a6e4e29be2c315a42d28f644df3b3ad90b43bb9d090210caa67\": container with ID starting with a41e59108ef12a6e4e29be2c315a42d28f644df3b3ad90b43bb9d090210caa67 not found: ID does not exist"
Sep 16 01:06:56.215918 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:56.215887    2556 scope.go:117] "RemoveContainer" containerID="f850895faf1925cbced7073029e90fe91bc35e8039dab0d1310675e6694e7912"
Sep 16 01:06:56.223451 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:56.223434    2556 scope.go:117] "RemoveContainer" containerID="f850895faf1925cbced7073029e90fe91bc35e8039dab0d1310675e6694e7912"
Sep 16 01:06:56.223729 ip-10-0-130-207 kubenswrapper[2556]: E0916 01:06:56.223713    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f850895faf1925cbced7073029e90fe91bc35e8039dab0d1310675e6694e7912\": container with ID starting with f850895faf1925cbced7073029e90fe91bc35e8039dab0d1310675e6694e7912 not found: ID does not exist" containerID="f850895faf1925cbced7073029e90fe91bc35e8039dab0d1310675e6694e7912"
Sep 16 01:06:56.223785 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:56.223733    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f850895faf1925cbced7073029e90fe91bc35e8039dab0d1310675e6694e7912"} err="failed to get container status \"f850895faf1925cbced7073029e90fe91bc35e8039dab0d1310675e6694e7912\": rpc error: code = NotFound desc = could not find container \"f850895faf1925cbced7073029e90fe91bc35e8039dab0d1310675e6694e7912\": container with ID starting with f850895faf1925cbced7073029e90fe91bc35e8039dab0d1310675e6694e7912 not found: ID does not exist"
Sep 16 01:06:56.227410 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:56.227390    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-30fb702c-68f994b98f-wq2lg"]
Sep 16 01:06:56.229800 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:56.229780    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-30fb702c-68f994b98f-wq2lg"]
Sep 16 01:06:56.237860 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:56.237837    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-30fb702c-849ff5678b-bj9fz"]
Sep 16 01:06:56.241222 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:56.241197    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-30fb702c-849ff5678b-bj9fz"]
Sep 16 01:06:57.736567 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:57.736540    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef8ad993-f627-491d-87be-dbfef99d431c" path="/var/lib/kubelet/pods/ef8ad993-f627-491d-87be-dbfef99d431c/volumes"
Sep 16 01:06:57.736948 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:06:57.736934    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4cd5caf-7b82-4831-92a5-c00bd6b55333" path="/var/lib/kubelet/pods/f4cd5caf-7b82-4831-92a5-c00bd6b55333/volumes"
Sep 16 01:07:03.306083 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:03.306052    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-6f50fc84-6789f9678-h67n8"]
Sep 16 01:07:03.306438 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:03.306426    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4cd5caf-7b82-4831-92a5-c00bd6b55333" containerName="payload-pre-processing"
Sep 16 01:07:03.306438 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:03.306437    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4cd5caf-7b82-4831-92a5-c00bd6b55333" containerName="payload-pre-processing"
Sep 16 01:07:03.306547 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:03.306447    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef8ad993-f627-491d-87be-dbfef99d431c" containerName="payload-processing"
Sep 16 01:07:03.306547 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:03.306453    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef8ad993-f627-491d-87be-dbfef99d431c" containerName="payload-processing"
Sep 16 01:07:03.306547 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:03.306524    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef8ad993-f627-491d-87be-dbfef99d431c" containerName="payload-processing"
Sep 16 01:07:03.306547 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:03.306535    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4cd5caf-7b82-4831-92a5-c00bd6b55333" containerName="payload-pre-processing"
Sep 16 01:07:03.311005 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:03.310986    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-6f50fc84-6789f9678-h67n8"
Sep 16 01:07:03.313183 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:03.313155    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-6f50fc84\""
Sep 16 01:07:03.313300 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:03.313203    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-6f50fc84-dockercfg-gthmf\""
Sep 16 01:07:03.320480 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:03.320449    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-6f50fc84-6789f9678-h67n8"]
Sep 16 01:07:03.340142 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:03.340117    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-6f50fc84-7bdc466c5c-n27qv"]
Sep 16 01:07:03.343603 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:03.343585    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-6f50fc84-7bdc466c5c-n27qv"
Sep 16 01:07:03.352556 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:03.352536    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-6f50fc84-7bdc466c5c-n27qv"]
Sep 16 01:07:03.421170 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:03.421137    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4ghd\" (UniqueName: \"kubernetes.io/projected/07c6eafe-8e8f-48af-b6e6-574cfe907788-kube-api-access-v4ghd\") pod \"payload-processing-e2e-derive-6f50fc84-7bdc466c5c-n27qv\" (UID: \"07c6eafe-8e8f-48af-b6e6-574cfe907788\") " pod="openshift-ingress/payload-processing-e2e-derive-6f50fc84-7bdc466c5c-n27qv"
Sep 16 01:07:03.421311 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:03.421257    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28ec2671-7da5-44ca-b740-72e32db6e8da-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-6f50fc84-6789f9678-h67n8\" (UID: \"28ec2671-7da5-44ca-b740-72e32db6e8da\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-6f50fc84-6789f9678-h67n8"
Sep 16 01:07:03.421311 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:03.421291    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wpcv\" (UniqueName: \"kubernetes.io/projected/28ec2671-7da5-44ca-b740-72e32db6e8da-kube-api-access-5wpcv\") pod \"payload-pre-processing-e2e-derive-6f50fc84-6789f9678-h67n8\" (UID: \"28ec2671-7da5-44ca-b740-72e32db6e8da\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-6f50fc84-6789f9678-h67n8"
Sep 16 01:07:03.421394 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:03.421334    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/07c6eafe-8e8f-48af-b6e6-574cfe907788-plugins-config-volume\") pod \"payload-processing-e2e-derive-6f50fc84-7bdc466c5c-n27qv\" (UID: \"07c6eafe-8e8f-48af-b6e6-574cfe907788\") " pod="openshift-ingress/payload-processing-e2e-derive-6f50fc84-7bdc466c5c-n27qv"
Sep 16 01:07:03.521889 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:03.521856    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28ec2671-7da5-44ca-b740-72e32db6e8da-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-6f50fc84-6789f9678-h67n8\" (UID: \"28ec2671-7da5-44ca-b740-72e32db6e8da\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-6f50fc84-6789f9678-h67n8"
Sep 16 01:07:03.521889 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:03.521891    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wpcv\" (UniqueName: \"kubernetes.io/projected/28ec2671-7da5-44ca-b740-72e32db6e8da-kube-api-access-5wpcv\") pod \"payload-pre-processing-e2e-derive-6f50fc84-6789f9678-h67n8\" (UID: \"28ec2671-7da5-44ca-b740-72e32db6e8da\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-6f50fc84-6789f9678-h67n8"
Sep 16 01:07:03.522076 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:03.521921    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/07c6eafe-8e8f-48af-b6e6-574cfe907788-plugins-config-volume\") pod \"payload-processing-e2e-derive-6f50fc84-7bdc466c5c-n27qv\" (UID: \"07c6eafe-8e8f-48af-b6e6-574cfe907788\") " pod="openshift-ingress/payload-processing-e2e-derive-6f50fc84-7bdc466c5c-n27qv"
Sep 16 01:07:03.522076 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:03.521966    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4ghd\" (UniqueName: \"kubernetes.io/projected/07c6eafe-8e8f-48af-b6e6-574cfe907788-kube-api-access-v4ghd\") pod \"payload-processing-e2e-derive-6f50fc84-7bdc466c5c-n27qv\" (UID: \"07c6eafe-8e8f-48af-b6e6-574cfe907788\") " pod="openshift-ingress/payload-processing-e2e-derive-6f50fc84-7bdc466c5c-n27qv"
Sep 16 01:07:03.522543 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:03.522523    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28ec2671-7da5-44ca-b740-72e32db6e8da-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-6f50fc84-6789f9678-h67n8\" (UID: \"28ec2671-7da5-44ca-b740-72e32db6e8da\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-6f50fc84-6789f9678-h67n8"
Sep 16 01:07:03.522671 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:03.522655    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/07c6eafe-8e8f-48af-b6e6-574cfe907788-plugins-config-volume\") pod \"payload-processing-e2e-derive-6f50fc84-7bdc466c5c-n27qv\" (UID: \"07c6eafe-8e8f-48af-b6e6-574cfe907788\") " pod="openshift-ingress/payload-processing-e2e-derive-6f50fc84-7bdc466c5c-n27qv"
Sep 16 01:07:03.530331 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:03.530308    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wpcv\" (UniqueName: \"kubernetes.io/projected/28ec2671-7da5-44ca-b740-72e32db6e8da-kube-api-access-5wpcv\") pod \"payload-pre-processing-e2e-derive-6f50fc84-6789f9678-h67n8\" (UID: \"28ec2671-7da5-44ca-b740-72e32db6e8da\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-6f50fc84-6789f9678-h67n8"
Sep 16 01:07:03.530432 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:03.530391    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4ghd\" (UniqueName: \"kubernetes.io/projected/07c6eafe-8e8f-48af-b6e6-574cfe907788-kube-api-access-v4ghd\") pod \"payload-processing-e2e-derive-6f50fc84-7bdc466c5c-n27qv\" (UID: \"07c6eafe-8e8f-48af-b6e6-574cfe907788\") " pod="openshift-ingress/payload-processing-e2e-derive-6f50fc84-7bdc466c5c-n27qv"
Sep 16 01:07:03.621766 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:03.621738    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-6f50fc84-6789f9678-h67n8"
Sep 16 01:07:03.655239 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:03.655215    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-6f50fc84-7bdc466c5c-n27qv"
Sep 16 01:07:03.894099 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:03.894071    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-6f50fc84-6789f9678-h67n8"]
Sep 16 01:07:03.896627 ip-10-0-130-207 kubenswrapper[2556]: W0916 01:07:03.896603    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28ec2671_7da5_44ca_b740_72e32db6e8da.slice/crio-a5bdfeca8259b38958769eb8d168eb526c2b8d7ed9874259ba6d58278ad07edb WatchSource:0}: Error finding container a5bdfeca8259b38958769eb8d168eb526c2b8d7ed9874259ba6d58278ad07edb: Status 404 returned error can't find the container with id a5bdfeca8259b38958769eb8d168eb526c2b8d7ed9874259ba6d58278ad07edb
Sep 16 01:07:03.950638 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:03.950615    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-6f50fc84-7bdc466c5c-n27qv"]
Sep 16 01:07:03.951340 ip-10-0-130-207 kubenswrapper[2556]: W0916 01:07:03.951317    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07c6eafe_8e8f_48af_b6e6_574cfe907788.slice/crio-130e98bc096290d6165ae244ecd71e0463b8956a4d5da4a3ff1c057894cfe58d WatchSource:0}: Error finding container 130e98bc096290d6165ae244ecd71e0463b8956a4d5da4a3ff1c057894cfe58d: Status 404 returned error can't find the container with id 130e98bc096290d6165ae244ecd71e0463b8956a4d5da4a3ff1c057894cfe58d
Sep 16 01:07:04.258036 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:04.257934    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-6f50fc84-7bdc466c5c-n27qv" event={"ID":"07c6eafe-8e8f-48af-b6e6-574cfe907788","Type":"ContainerStarted","Data":"fc9508f4651843674654b9b8d96fb08a778da61fdffcfc54d846b33f46242e22"}
Sep 16 01:07:04.258036 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:04.257988    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-6f50fc84-7bdc466c5c-n27qv" event={"ID":"07c6eafe-8e8f-48af-b6e6-574cfe907788","Type":"ContainerStarted","Data":"130e98bc096290d6165ae244ecd71e0463b8956a4d5da4a3ff1c057894cfe58d"}
Sep 16 01:07:04.258263 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:04.258177    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-6f50fc84-7bdc466c5c-n27qv"
Sep 16 01:07:04.259815 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:04.259788    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-6f50fc84-6789f9678-h67n8" event={"ID":"28ec2671-7da5-44ca-b740-72e32db6e8da","Type":"ContainerStarted","Data":"0ae692fbb984c5e4981a42deebf7aaf5e754b169ebb5c0701824118ce9f4f355"}
Sep 16 01:07:04.259971 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:04.259840    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-6f50fc84-6789f9678-h67n8" event={"ID":"28ec2671-7da5-44ca-b740-72e32db6e8da","Type":"ContainerStarted","Data":"a5bdfeca8259b38958769eb8d168eb526c2b8d7ed9874259ba6d58278ad07edb"}
Sep 16 01:07:04.259971 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:04.259871    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-6f50fc84-6789f9678-h67n8"
Sep 16 01:07:04.275139 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:04.275097    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-6f50fc84-7bdc466c5c-n27qv" podStartSLOduration=1.275080708 podStartE2EDuration="1.275080708s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 01:07:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 01:07:04.273353288 +0000 UTC m=+1439.054854881" watchObservedRunningTime="2026-09-16 01:07:04.275080708 +0000 UTC m=+1439.056582310"
Sep 16 01:07:04.287958 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:04.287919    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-6f50fc84-6789f9678-h67n8" podStartSLOduration=1.287895292 podStartE2EDuration="1.287895292s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 01:07:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 01:07:04.286404708 +0000 UTC m=+1439.067906300" watchObservedRunningTime="2026-09-16 01:07:04.287895292 +0000 UTC m=+1439.069396907"
Sep 16 01:07:15.266664 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:15.266629    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-6f50fc84-7bdc466c5c-n27qv"
Sep 16 01:07:15.267235 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:15.266919    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-6f50fc84-6789f9678-h67n8"
Sep 16 01:07:31.707320 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:31.707287    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-6f50fc84-7bdc466c5c-n27qv"]
Sep 16 01:07:31.707771 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:31.707516    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-6f50fc84-7bdc466c5c-n27qv" podUID="07c6eafe-8e8f-48af-b6e6-574cfe907788" containerName="payload-processing" containerID="cri-o://fc9508f4651843674654b9b8d96fb08a778da61fdffcfc54d846b33f46242e22" gracePeriod=30
Sep 16 01:07:31.714136 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:31.714098    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-6f50fc84-6789f9678-h67n8"]
Sep 16 01:07:31.714341 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:31.714316    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-6f50fc84-6789f9678-h67n8" podUID="28ec2671-7da5-44ca-b740-72e32db6e8da" containerName="payload-pre-processing" containerID="cri-o://0ae692fbb984c5e4981a42deebf7aaf5e754b169ebb5c0701824118ce9f4f355" gracePeriod=30
Sep 16 01:07:31.994979 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:31.994945    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-6f50fc84-7bdc466c5c-n27qv"
Sep 16 01:07:31.998081 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:31.998059    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-6f50fc84-6789f9678-h67n8"
Sep 16 01:07:32.056125 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:32.056098    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5wpcv\" (UniqueName: \"kubernetes.io/projected/28ec2671-7da5-44ca-b740-72e32db6e8da-kube-api-access-5wpcv\") pod \"28ec2671-7da5-44ca-b740-72e32db6e8da\" (UID: \"28ec2671-7da5-44ca-b740-72e32db6e8da\") "
Sep 16 01:07:32.056282 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:32.056142    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28ec2671-7da5-44ca-b740-72e32db6e8da-plugins-config-volume\") pod \"28ec2671-7da5-44ca-b740-72e32db6e8da\" (UID: \"28ec2671-7da5-44ca-b740-72e32db6e8da\") "
Sep 16 01:07:32.056282 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:32.056226    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/07c6eafe-8e8f-48af-b6e6-574cfe907788-plugins-config-volume\") pod \"07c6eafe-8e8f-48af-b6e6-574cfe907788\" (UID: \"07c6eafe-8e8f-48af-b6e6-574cfe907788\") "
Sep 16 01:07:32.056360 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:32.056291    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v4ghd\" (UniqueName: \"kubernetes.io/projected/07c6eafe-8e8f-48af-b6e6-574cfe907788-kube-api-access-v4ghd\") pod \"07c6eafe-8e8f-48af-b6e6-574cfe907788\" (UID: \"07c6eafe-8e8f-48af-b6e6-574cfe907788\") "
Sep 16 01:07:32.056573 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:32.056549    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28ec2671-7da5-44ca-b740-72e32db6e8da-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "28ec2671-7da5-44ca-b740-72e32db6e8da" (UID: "28ec2671-7da5-44ca-b740-72e32db6e8da"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 01:07:32.056643 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:32.056613    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07c6eafe-8e8f-48af-b6e6-574cfe907788-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "07c6eafe-8e8f-48af-b6e6-574cfe907788" (UID: "07c6eafe-8e8f-48af-b6e6-574cfe907788"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 01:07:32.058242 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:32.058224    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28ec2671-7da5-44ca-b740-72e32db6e8da-kube-api-access-5wpcv" (OuterVolumeSpecName: "kube-api-access-5wpcv") pod "28ec2671-7da5-44ca-b740-72e32db6e8da" (UID: "28ec2671-7da5-44ca-b740-72e32db6e8da"). InnerVolumeSpecName "kube-api-access-5wpcv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 01:07:32.058304 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:32.058250    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07c6eafe-8e8f-48af-b6e6-574cfe907788-kube-api-access-v4ghd" (OuterVolumeSpecName: "kube-api-access-v4ghd") pod "07c6eafe-8e8f-48af-b6e6-574cfe907788" (UID: "07c6eafe-8e8f-48af-b6e6-574cfe907788"). InnerVolumeSpecName "kube-api-access-v4ghd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 01:07:32.157372 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:32.157340    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v4ghd\" (UniqueName: \"kubernetes.io/projected/07c6eafe-8e8f-48af-b6e6-574cfe907788-kube-api-access-v4ghd\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:07:32.157372 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:32.157373    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5wpcv\" (UniqueName: \"kubernetes.io/projected/28ec2671-7da5-44ca-b740-72e32db6e8da-kube-api-access-5wpcv\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:07:32.157598 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:32.157388    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28ec2671-7da5-44ca-b740-72e32db6e8da-plugins-config-volume\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:07:32.157598 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:32.157404    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/07c6eafe-8e8f-48af-b6e6-574cfe907788-plugins-config-volume\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:07:32.419159 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:32.419106    2556 generic.go:356] "Generic (PLEG): container finished" podID="07c6eafe-8e8f-48af-b6e6-574cfe907788" containerID="fc9508f4651843674654b9b8d96fb08a778da61fdffcfc54d846b33f46242e22" exitCode=0
Sep 16 01:07:32.419159 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:32.419159    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-6f50fc84-7bdc466c5c-n27qv"
Sep 16 01:07:32.419403 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:32.419199    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-6f50fc84-7bdc466c5c-n27qv" event={"ID":"07c6eafe-8e8f-48af-b6e6-574cfe907788","Type":"ContainerDied","Data":"fc9508f4651843674654b9b8d96fb08a778da61fdffcfc54d846b33f46242e22"}
Sep 16 01:07:32.419403 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:32.419254    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-6f50fc84-7bdc466c5c-n27qv" event={"ID":"07c6eafe-8e8f-48af-b6e6-574cfe907788","Type":"ContainerDied","Data":"130e98bc096290d6165ae244ecd71e0463b8956a4d5da4a3ff1c057894cfe58d"}
Sep 16 01:07:32.419403 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:32.419279    2556 scope.go:117] "RemoveContainer" containerID="fc9508f4651843674654b9b8d96fb08a778da61fdffcfc54d846b33f46242e22"
Sep 16 01:07:32.420767 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:32.420745    2556 generic.go:356] "Generic (PLEG): container finished" podID="28ec2671-7da5-44ca-b740-72e32db6e8da" containerID="0ae692fbb984c5e4981a42deebf7aaf5e754b169ebb5c0701824118ce9f4f355" exitCode=0
Sep 16 01:07:32.420842 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:32.420796    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-6f50fc84-6789f9678-h67n8"
Sep 16 01:07:32.420842 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:32.420819    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-6f50fc84-6789f9678-h67n8" event={"ID":"28ec2671-7da5-44ca-b740-72e32db6e8da","Type":"ContainerDied","Data":"0ae692fbb984c5e4981a42deebf7aaf5e754b169ebb5c0701824118ce9f4f355"}
Sep 16 01:07:32.420928 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:32.420852    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-6f50fc84-6789f9678-h67n8" event={"ID":"28ec2671-7da5-44ca-b740-72e32db6e8da","Type":"ContainerDied","Data":"a5bdfeca8259b38958769eb8d168eb526c2b8d7ed9874259ba6d58278ad07edb"}
Sep 16 01:07:32.428551 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:32.428530    2556 scope.go:117] "RemoveContainer" containerID="fc9508f4651843674654b9b8d96fb08a778da61fdffcfc54d846b33f46242e22"
Sep 16 01:07:32.428828 ip-10-0-130-207 kubenswrapper[2556]: E0916 01:07:32.428805    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc9508f4651843674654b9b8d96fb08a778da61fdffcfc54d846b33f46242e22\": container with ID starting with fc9508f4651843674654b9b8d96fb08a778da61fdffcfc54d846b33f46242e22 not found: ID does not exist" containerID="fc9508f4651843674654b9b8d96fb08a778da61fdffcfc54d846b33f46242e22"
Sep 16 01:07:32.428900 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:32.428842    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc9508f4651843674654b9b8d96fb08a778da61fdffcfc54d846b33f46242e22"} err="failed to get container status \"fc9508f4651843674654b9b8d96fb08a778da61fdffcfc54d846b33f46242e22\": rpc error: code = NotFound desc = could not find container \"fc9508f4651843674654b9b8d96fb08a778da61fdffcfc54d846b33f46242e22\": container with ID starting with fc9508f4651843674654b9b8d96fb08a778da61fdffcfc54d846b33f46242e22 not found: ID does not exist"
Sep 16 01:07:32.428900 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:32.428866    2556 scope.go:117] "RemoveContainer" containerID="0ae692fbb984c5e4981a42deebf7aaf5e754b169ebb5c0701824118ce9f4f355"
Sep 16 01:07:32.437119 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:32.437102    2556 scope.go:117] "RemoveContainer" containerID="0ae692fbb984c5e4981a42deebf7aaf5e754b169ebb5c0701824118ce9f4f355"
Sep 16 01:07:32.437367 ip-10-0-130-207 kubenswrapper[2556]: E0916 01:07:32.437346    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ae692fbb984c5e4981a42deebf7aaf5e754b169ebb5c0701824118ce9f4f355\": container with ID starting with 0ae692fbb984c5e4981a42deebf7aaf5e754b169ebb5c0701824118ce9f4f355 not found: ID does not exist" containerID="0ae692fbb984c5e4981a42deebf7aaf5e754b169ebb5c0701824118ce9f4f355"
Sep 16 01:07:32.437432 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:32.437373    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ae692fbb984c5e4981a42deebf7aaf5e754b169ebb5c0701824118ce9f4f355"} err="failed to get container status \"0ae692fbb984c5e4981a42deebf7aaf5e754b169ebb5c0701824118ce9f4f355\": rpc error: code = NotFound desc = could not find container \"0ae692fbb984c5e4981a42deebf7aaf5e754b169ebb5c0701824118ce9f4f355\": container with ID starting with 0ae692fbb984c5e4981a42deebf7aaf5e754b169ebb5c0701824118ce9f4f355 not found: ID does not exist"
Sep 16 01:07:32.443797 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:32.443776    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-6f50fc84-6789f9678-h67n8"]
Sep 16 01:07:32.447041 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:32.447023    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-6f50fc84-6789f9678-h67n8"]
Sep 16 01:07:32.455988 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:32.455969    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-6f50fc84-7bdc466c5c-n27qv"]
Sep 16 01:07:32.459407 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:32.459385    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-6f50fc84-7bdc466c5c-n27qv"]
Sep 16 01:07:33.736970 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:33.736934    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07c6eafe-8e8f-48af-b6e6-574cfe907788" path="/var/lib/kubelet/pods/07c6eafe-8e8f-48af-b6e6-574cfe907788/volumes"
Sep 16 01:07:33.737597 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:33.737575    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28ec2671-7da5-44ca-b740-72e32db6e8da" path="/var/lib/kubelet/pods/28ec2671-7da5-44ca-b740-72e32db6e8da/volumes"
Sep 16 01:07:35.617668 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:35.617633    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-2534e3/premium-w3-2534e3-kserve-679669b7-48q62"]
Sep 16 01:07:46.658886 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:46.658853    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-2534e3-5bcb57cff4-tz8rf"]
Sep 16 01:07:46.659456 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:46.659105    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2534e3-5bcb57cff4-tz8rf" podUID="eca4a3b2-0c02-4e9e-a86d-0be2ec3518c2" containerName="payload-pre-processing" containerID="cri-o://405b74649a3707672b00fb0988b6f58f114c1cacb8c4bfb38c1cf8f49d16908c" gracePeriod=30
Sep 16 01:07:46.918329 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:46.918270    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2534e3-5bcb57cff4-tz8rf"
Sep 16 01:07:46.981740 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:46.981645    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t7bv8\" (UniqueName: \"kubernetes.io/projected/eca4a3b2-0c02-4e9e-a86d-0be2ec3518c2-kube-api-access-t7bv8\") pod \"eca4a3b2-0c02-4e9e-a86d-0be2ec3518c2\" (UID: \"eca4a3b2-0c02-4e9e-a86d-0be2ec3518c2\") "
Sep 16 01:07:46.981934 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:46.981896    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eca4a3b2-0c02-4e9e-a86d-0be2ec3518c2-plugins-config-volume\") pod \"eca4a3b2-0c02-4e9e-a86d-0be2ec3518c2\" (UID: \"eca4a3b2-0c02-4e9e-a86d-0be2ec3518c2\") "
Sep 16 01:07:46.982639 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:46.982608    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eca4a3b2-0c02-4e9e-a86d-0be2ec3518c2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "eca4a3b2-0c02-4e9e-a86d-0be2ec3518c2" (UID: "eca4a3b2-0c02-4e9e-a86d-0be2ec3518c2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 01:07:46.984324 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:46.984298    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eca4a3b2-0c02-4e9e-a86d-0be2ec3518c2-kube-api-access-t7bv8" (OuterVolumeSpecName: "kube-api-access-t7bv8") pod "eca4a3b2-0c02-4e9e-a86d-0be2ec3518c2" (UID: "eca4a3b2-0c02-4e9e-a86d-0be2ec3518c2"). InnerVolumeSpecName "kube-api-access-t7bv8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 01:07:47.082759 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:47.082729    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eca4a3b2-0c02-4e9e-a86d-0be2ec3518c2-plugins-config-volume\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:07:47.082759 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:47.082754    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t7bv8\" (UniqueName: \"kubernetes.io/projected/eca4a3b2-0c02-4e9e-a86d-0be2ec3518c2-kube-api-access-t7bv8\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:07:47.502530 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:47.502492    2556 generic.go:356] "Generic (PLEG): container finished" podID="eca4a3b2-0c02-4e9e-a86d-0be2ec3518c2" containerID="405b74649a3707672b00fb0988b6f58f114c1cacb8c4bfb38c1cf8f49d16908c" exitCode=0
Sep 16 01:07:47.502702 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:47.502553    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2534e3-5bcb57cff4-tz8rf"
Sep 16 01:07:47.502702 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:47.502579    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2534e3-5bcb57cff4-tz8rf" event={"ID":"eca4a3b2-0c02-4e9e-a86d-0be2ec3518c2","Type":"ContainerDied","Data":"405b74649a3707672b00fb0988b6f58f114c1cacb8c4bfb38c1cf8f49d16908c"}
Sep 16 01:07:47.502702 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:47.502665    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2534e3-5bcb57cff4-tz8rf" event={"ID":"eca4a3b2-0c02-4e9e-a86d-0be2ec3518c2","Type":"ContainerDied","Data":"f0356ca0d75a00d7ad2403a9b478ea49a5e0596890484decf0dc7f849ad8cef8"}
Sep 16 01:07:47.502702 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:47.502693    2556 scope.go:117] "RemoveContainer" containerID="405b74649a3707672b00fb0988b6f58f114c1cacb8c4bfb38c1cf8f49d16908c"
Sep 16 01:07:47.511796 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:47.511780    2556 scope.go:117] "RemoveContainer" containerID="405b74649a3707672b00fb0988b6f58f114c1cacb8c4bfb38c1cf8f49d16908c"
Sep 16 01:07:47.512058 ip-10-0-130-207 kubenswrapper[2556]: E0916 01:07:47.512039    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"405b74649a3707672b00fb0988b6f58f114c1cacb8c4bfb38c1cf8f49d16908c\": container with ID starting with 405b74649a3707672b00fb0988b6f58f114c1cacb8c4bfb38c1cf8f49d16908c not found: ID does not exist" containerID="405b74649a3707672b00fb0988b6f58f114c1cacb8c4bfb38c1cf8f49d16908c"
Sep 16 01:07:47.512107 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:47.512066    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"405b74649a3707672b00fb0988b6f58f114c1cacb8c4bfb38c1cf8f49d16908c"} err="failed to get container status \"405b74649a3707672b00fb0988b6f58f114c1cacb8c4bfb38c1cf8f49d16908c\": rpc error: code = NotFound desc = could not find container \"405b74649a3707672b00fb0988b6f58f114c1cacb8c4bfb38c1cf8f49d16908c\": container with ID starting with 405b74649a3707672b00fb0988b6f58f114c1cacb8c4bfb38c1cf8f49d16908c not found: ID does not exist"
Sep 16 01:07:47.525666 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:47.525620    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-2534e3-5bcb57cff4-tz8rf"]
Sep 16 01:07:47.529742 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:47.529720    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-2534e3-5bcb57cff4-tz8rf"]
Sep 16 01:07:47.736161 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:47.736132    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eca4a3b2-0c02-4e9e-a86d-0be2ec3518c2" path="/var/lib/kubelet/pods/eca4a3b2-0c02-4e9e-a86d-0be2ec3518c2/volumes"
Sep 16 01:07:50.636971 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:50.636937    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-2534e3/premium-w3-2534e3-kserve-679669b7-48q62" podUID="68b17a9b-109b-4fd5-8f97-920e1cb237d0" containerName="main" containerID="cri-o://614f77ea21fb42ae64f31d0c0a95d93c5ff61a30c917dda404ebdc01959c9300" gracePeriod=45
Sep 16 01:07:50.893139 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:50.893089    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-2534e3/premium-w3-2534e3-kserve-679669b7-48q62"
Sep 16 01:07:51.018659 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:51.018630    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p2hqr\" (UniqueName: \"kubernetes.io/projected/68b17a9b-109b-4fd5-8f97-920e1cb237d0-kube-api-access-p2hqr\") pod \"68b17a9b-109b-4fd5-8f97-920e1cb237d0\" (UID: \"68b17a9b-109b-4fd5-8f97-920e1cb237d0\") "
Sep 16 01:07:51.018807 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:51.018694    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/68b17a9b-109b-4fd5-8f97-920e1cb237d0-dshm\") pod \"68b17a9b-109b-4fd5-8f97-920e1cb237d0\" (UID: \"68b17a9b-109b-4fd5-8f97-920e1cb237d0\") "
Sep 16 01:07:51.018807 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:51.018717    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/68b17a9b-109b-4fd5-8f97-920e1cb237d0-tmp-dir\") pod \"68b17a9b-109b-4fd5-8f97-920e1cb237d0\" (UID: \"68b17a9b-109b-4fd5-8f97-920e1cb237d0\") "
Sep 16 01:07:51.018807 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:51.018763    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/68b17a9b-109b-4fd5-8f97-920e1cb237d0-tls-certs\") pod \"68b17a9b-109b-4fd5-8f97-920e1cb237d0\" (UID: \"68b17a9b-109b-4fd5-8f97-920e1cb237d0\") "
Sep 16 01:07:51.018807 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:51.018784    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/68b17a9b-109b-4fd5-8f97-920e1cb237d0-home\") pod \"68b17a9b-109b-4fd5-8f97-920e1cb237d0\" (UID: \"68b17a9b-109b-4fd5-8f97-920e1cb237d0\") "
Sep 16 01:07:51.019029 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:51.018812    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/68b17a9b-109b-4fd5-8f97-920e1cb237d0-model-cache\") pod \"68b17a9b-109b-4fd5-8f97-920e1cb237d0\" (UID: \"68b17a9b-109b-4fd5-8f97-920e1cb237d0\") "
Sep 16 01:07:51.019094 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:51.019062    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68b17a9b-109b-4fd5-8f97-920e1cb237d0-home" (OuterVolumeSpecName: "home") pod "68b17a9b-109b-4fd5-8f97-920e1cb237d0" (UID: "68b17a9b-109b-4fd5-8f97-920e1cb237d0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 01:07:51.019094 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:51.019049    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68b17a9b-109b-4fd5-8f97-920e1cb237d0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "68b17a9b-109b-4fd5-8f97-920e1cb237d0" (UID: "68b17a9b-109b-4fd5-8f97-920e1cb237d0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 01:07:51.019192 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:51.019116    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68b17a9b-109b-4fd5-8f97-920e1cb237d0-model-cache" (OuterVolumeSpecName: "model-cache") pod "68b17a9b-109b-4fd5-8f97-920e1cb237d0" (UID: "68b17a9b-109b-4fd5-8f97-920e1cb237d0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 01:07:51.019247 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:51.019188    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/68b17a9b-109b-4fd5-8f97-920e1cb237d0-tmp-dir\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:07:51.019247 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:51.019201    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/68b17a9b-109b-4fd5-8f97-920e1cb237d0-home\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:07:51.020787 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:51.020759    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68b17a9b-109b-4fd5-8f97-920e1cb237d0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "68b17a9b-109b-4fd5-8f97-920e1cb237d0" (UID: "68b17a9b-109b-4fd5-8f97-920e1cb237d0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 01:07:51.020787 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:51.020770    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68b17a9b-109b-4fd5-8f97-920e1cb237d0-dshm" (OuterVolumeSpecName: "dshm") pod "68b17a9b-109b-4fd5-8f97-920e1cb237d0" (UID: "68b17a9b-109b-4fd5-8f97-920e1cb237d0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 01:07:51.020945 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:51.020844    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68b17a9b-109b-4fd5-8f97-920e1cb237d0-kube-api-access-p2hqr" (OuterVolumeSpecName: "kube-api-access-p2hqr") pod "68b17a9b-109b-4fd5-8f97-920e1cb237d0" (UID: "68b17a9b-109b-4fd5-8f97-920e1cb237d0"). InnerVolumeSpecName "kube-api-access-p2hqr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 01:07:51.120667 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:51.120643    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/68b17a9b-109b-4fd5-8f97-920e1cb237d0-tls-certs\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:07:51.120667 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:51.120664    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/68b17a9b-109b-4fd5-8f97-920e1cb237d0-model-cache\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:07:51.120809 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:51.120673    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p2hqr\" (UniqueName: \"kubernetes.io/projected/68b17a9b-109b-4fd5-8f97-920e1cb237d0-kube-api-access-p2hqr\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:07:51.120809 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:51.120681    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/68b17a9b-109b-4fd5-8f97-920e1cb237d0-dshm\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:07:51.525139 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:51.525103    2556 generic.go:356] "Generic (PLEG): container finished" podID="68b17a9b-109b-4fd5-8f97-920e1cb237d0" containerID="614f77ea21fb42ae64f31d0c0a95d93c5ff61a30c917dda404ebdc01959c9300" exitCode=0
Sep 16 01:07:51.525331 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:51.525163    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-2534e3/premium-w3-2534e3-kserve-679669b7-48q62"
Sep 16 01:07:51.525331 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:51.525187    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-2534e3/premium-w3-2534e3-kserve-679669b7-48q62" event={"ID":"68b17a9b-109b-4fd5-8f97-920e1cb237d0","Type":"ContainerDied","Data":"614f77ea21fb42ae64f31d0c0a95d93c5ff61a30c917dda404ebdc01959c9300"}
Sep 16 01:07:51.525331 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:51.525241    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-2534e3/premium-w3-2534e3-kserve-679669b7-48q62" event={"ID":"68b17a9b-109b-4fd5-8f97-920e1cb237d0","Type":"ContainerDied","Data":"0ad7737e40540d71c64e3ea8d84f838c5ec3807e5e47686020d70f3c87032b5d"}
Sep 16 01:07:51.525331 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:51.525267    2556 scope.go:117] "RemoveContainer" containerID="614f77ea21fb42ae64f31d0c0a95d93c5ff61a30c917dda404ebdc01959c9300"
Sep 16 01:07:51.534690 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:51.534674    2556 scope.go:117] "RemoveContainer" containerID="614f77ea21fb42ae64f31d0c0a95d93c5ff61a30c917dda404ebdc01959c9300"
Sep 16 01:07:51.534952 ip-10-0-130-207 kubenswrapper[2556]: E0916 01:07:51.534936    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"614f77ea21fb42ae64f31d0c0a95d93c5ff61a30c917dda404ebdc01959c9300\": container with ID starting with 614f77ea21fb42ae64f31d0c0a95d93c5ff61a30c917dda404ebdc01959c9300 not found: ID does not exist" containerID="614f77ea21fb42ae64f31d0c0a95d93c5ff61a30c917dda404ebdc01959c9300"
Sep 16 01:07:51.534997 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:51.534959    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"614f77ea21fb42ae64f31d0c0a95d93c5ff61a30c917dda404ebdc01959c9300"} err="failed to get container status \"614f77ea21fb42ae64f31d0c0a95d93c5ff61a30c917dda404ebdc01959c9300\": rpc error: code = NotFound desc = could not find container \"614f77ea21fb42ae64f31d0c0a95d93c5ff61a30c917dda404ebdc01959c9300\": container with ID starting with 614f77ea21fb42ae64f31d0c0a95d93c5ff61a30c917dda404ebdc01959c9300 not found: ID does not exist"
Sep 16 01:07:51.545179 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:51.545158    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-2534e3/premium-w3-2534e3-kserve-679669b7-48q62"]
Sep 16 01:07:51.548342 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:51.548323    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-2534e3/premium-w3-2534e3-kserve-679669b7-48q62"]
Sep 16 01:07:51.736425 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:07:51.736396    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68b17a9b-109b-4fd5-8f97-920e1cb237d0" path="/var/lib/kubelet/pods/68b17a9b-109b-4fd5-8f97-920e1cb237d0/volumes"
Sep 16 01:09:12.606179 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:12.606144    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f29ccae3-6fdb5f549d-2jshw"]
Sep 16 01:09:12.606639 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:12.606510    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eca4a3b2-0c02-4e9e-a86d-0be2ec3518c2" containerName="payload-pre-processing"
Sep 16 01:09:12.606639 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:12.606523    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="eca4a3b2-0c02-4e9e-a86d-0be2ec3518c2" containerName="payload-pre-processing"
Sep 16 01:09:12.606639 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:12.606533    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07c6eafe-8e8f-48af-b6e6-574cfe907788" containerName="payload-processing"
Sep 16 01:09:12.606639 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:12.606539    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="07c6eafe-8e8f-48af-b6e6-574cfe907788" containerName="payload-processing"
Sep 16 01:09:12.606639 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:12.606547    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28ec2671-7da5-44ca-b740-72e32db6e8da" containerName="payload-pre-processing"
Sep 16 01:09:12.606639 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:12.606554    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="28ec2671-7da5-44ca-b740-72e32db6e8da" containerName="payload-pre-processing"
Sep 16 01:09:12.606639 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:12.606564    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68b17a9b-109b-4fd5-8f97-920e1cb237d0" containerName="main"
Sep 16 01:09:12.606639 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:12.606569    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="68b17a9b-109b-4fd5-8f97-920e1cb237d0" containerName="main"
Sep 16 01:09:12.606890 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:12.606645    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="68b17a9b-109b-4fd5-8f97-920e1cb237d0" containerName="main"
Sep 16 01:09:12.606890 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:12.606653    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="28ec2671-7da5-44ca-b740-72e32db6e8da" containerName="payload-pre-processing"
Sep 16 01:09:12.606890 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:12.606659    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="07c6eafe-8e8f-48af-b6e6-574cfe907788" containerName="payload-processing"
Sep 16 01:09:12.606890 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:12.606666    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="eca4a3b2-0c02-4e9e-a86d-0be2ec3518c2" containerName="payload-pre-processing"
Sep 16 01:09:12.609727 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:12.609706    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-f29ccae3-6fdb5f549d-2jshw"
Sep 16 01:09:12.611870 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:12.611840    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-f29ccae3\""
Sep 16 01:09:12.611978 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:12.611922    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-f29ccae3-dockercfg-ncgcq\""
Sep 16 01:09:12.619610 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:12.619584    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f29ccae3-6fdb5f549d-2jshw"]
Sep 16 01:09:12.700960 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:12.700931    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a82d7b06-73f3-49aa-9713-bff9f50ae190-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-f29ccae3-6fdb5f549d-2jshw\" (UID: \"a82d7b06-73f3-49aa-9713-bff9f50ae190\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f29ccae3-6fdb5f549d-2jshw"
Sep 16 01:09:12.701138 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:12.701011    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tf459\" (UniqueName: \"kubernetes.io/projected/a82d7b06-73f3-49aa-9713-bff9f50ae190-kube-api-access-tf459\") pod \"payload-pre-processing-e2e-mt-f29ccae3-6fdb5f549d-2jshw\" (UID: \"a82d7b06-73f3-49aa-9713-bff9f50ae190\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f29ccae3-6fdb5f549d-2jshw"
Sep 16 01:09:12.802258 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:12.802221    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a82d7b06-73f3-49aa-9713-bff9f50ae190-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-f29ccae3-6fdb5f549d-2jshw\" (UID: \"a82d7b06-73f3-49aa-9713-bff9f50ae190\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f29ccae3-6fdb5f549d-2jshw"
Sep 16 01:09:12.802410 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:12.802284    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tf459\" (UniqueName: \"kubernetes.io/projected/a82d7b06-73f3-49aa-9713-bff9f50ae190-kube-api-access-tf459\") pod \"payload-pre-processing-e2e-mt-f29ccae3-6fdb5f549d-2jshw\" (UID: \"a82d7b06-73f3-49aa-9713-bff9f50ae190\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f29ccae3-6fdb5f549d-2jshw"
Sep 16 01:09:12.802897 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:12.802876    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a82d7b06-73f3-49aa-9713-bff9f50ae190-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-f29ccae3-6fdb5f549d-2jshw\" (UID: \"a82d7b06-73f3-49aa-9713-bff9f50ae190\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f29ccae3-6fdb5f549d-2jshw"
Sep 16 01:09:12.809877 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:12.809855    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tf459\" (UniqueName: \"kubernetes.io/projected/a82d7b06-73f3-49aa-9713-bff9f50ae190-kube-api-access-tf459\") pod \"payload-pre-processing-e2e-mt-f29ccae3-6fdb5f549d-2jshw\" (UID: \"a82d7b06-73f3-49aa-9713-bff9f50ae190\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f29ccae3-6fdb5f549d-2jshw"
Sep 16 01:09:12.920949 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:12.920888    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-f29ccae3-6fdb5f549d-2jshw"
Sep 16 01:09:13.192923 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:13.192761    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f29ccae3-6fdb5f549d-2jshw"]
Sep 16 01:09:13.195228 ip-10-0-130-207 kubenswrapper[2556]: W0916 01:09:13.195193    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda82d7b06_73f3_49aa_9713_bff9f50ae190.slice/crio-167d5d70377bab0774e61e6df44c09ea6e3cee77b4ea0248bf51942100810480 WatchSource:0}: Error finding container 167d5d70377bab0774e61e6df44c09ea6e3cee77b4ea0248bf51942100810480: Status 404 returned error can't find the container with id 167d5d70377bab0774e61e6df44c09ea6e3cee77b4ea0248bf51942100810480
Sep 16 01:09:13.962101 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:13.962065    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-f29ccae3-6fdb5f549d-2jshw" event={"ID":"a82d7b06-73f3-49aa-9713-bff9f50ae190","Type":"ContainerStarted","Data":"e8ff32692043f056c208fa119e2e38bc2c80415338f8074ad276be725c8ffe22"}
Sep 16 01:09:13.962559 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:13.962124    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-f29ccae3-6fdb5f549d-2jshw" event={"ID":"a82d7b06-73f3-49aa-9713-bff9f50ae190","Type":"ContainerStarted","Data":"167d5d70377bab0774e61e6df44c09ea6e3cee77b4ea0248bf51942100810480"}
Sep 16 01:09:13.962559 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:13.962235    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-f29ccae3-6fdb5f549d-2jshw"
Sep 16 01:09:13.979329 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:13.979288    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-f29ccae3-6fdb5f549d-2jshw" podStartSLOduration=1.9792741839999999 podStartE2EDuration="1.979274184s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 01:09:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 01:09:13.978421931 +0000 UTC m=+1568.759923522" watchObservedRunningTime="2026-09-16 01:09:13.979274184 +0000 UTC m=+1568.760775776"
Sep 16 01:09:24.970118 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:24.970088    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-f29ccae3-6fdb5f549d-2jshw"
Sep 16 01:09:42.488550 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:42.488515    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-96c840/distinct-w0-96c840-kserve-c8989f89f-qwjjl"]
Sep 16 01:09:42.509653 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:42.509623    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-96c840/premium-w0-96c840-kserve-6d9c67476-t9n4q"]
Sep 16 01:09:53.282794 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:53.282761    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f29ccae3-6fdb5f549d-2jshw"]
Sep 16 01:09:53.283258 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:53.283129    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-f29ccae3-6fdb5f549d-2jshw" podUID="a82d7b06-73f3-49aa-9713-bff9f50ae190" containerName="payload-pre-processing" containerID="cri-o://e8ff32692043f056c208fa119e2e38bc2c80415338f8074ad276be725c8ffe22" gracePeriod=30
Sep 16 01:09:53.510740 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:53.510657    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-96c840-5bdd89c7f8-2c9h9"]
Sep 16 01:09:53.510958 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:53.510884    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-96c840-5bdd89c7f8-2c9h9" podUID="2e636703-6f9e-4043-961c-27b191e47d79" containerName="maas-api" containerID="cri-o://c42301cb3b3e10f006e30b3916c9a108fe28a9344800f6b9ec4c5b8330b83581" gracePeriod=30
Sep 16 01:09:53.641522 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:53.641490    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-f29ccae3-6fdb5f549d-2jshw"
Sep 16 01:09:53.653984 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:53.653372    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-96c840-76f5c8849f-ks9pz"]
Sep 16 01:09:53.653984 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:53.653621    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w0-96c840-76f5c8849f-ks9pz" podUID="fa9f4c4b-4e60-4638-aa94-b03b1877df9b" containerName="payload-processing" containerID="cri-o://bfd75d69293cde13b23f4266a6d88577b96760ad802c925934c5727e441a18e8" gracePeriod=30
Sep 16 01:09:53.655982 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:53.655950    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-96c840-56b9dd4654-5rp7j"]
Sep 16 01:09:53.656200 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:53.656172    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-96c840-56b9dd4654-5rp7j" podUID="2583e2cb-79b4-4b90-931f-99de7835d6e4" containerName="payload-pre-processing" containerID="cri-o://6389de8e731342a8ce369b868fb1aea0a2638d9431130a93f5c0aabfec575c23" gracePeriod=30
Sep 16 01:09:53.730812 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:53.730781    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a82d7b06-73f3-49aa-9713-bff9f50ae190-plugins-config-volume\") pod \"a82d7b06-73f3-49aa-9713-bff9f50ae190\" (UID: \"a82d7b06-73f3-49aa-9713-bff9f50ae190\") "
Sep 16 01:09:53.730994 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:53.730975    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tf459\" (UniqueName: \"kubernetes.io/projected/a82d7b06-73f3-49aa-9713-bff9f50ae190-kube-api-access-tf459\") pod \"a82d7b06-73f3-49aa-9713-bff9f50ae190\" (UID: \"a82d7b06-73f3-49aa-9713-bff9f50ae190\") "
Sep 16 01:09:53.731515 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:53.731204    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a82d7b06-73f3-49aa-9713-bff9f50ae190-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a82d7b06-73f3-49aa-9713-bff9f50ae190" (UID: "a82d7b06-73f3-49aa-9713-bff9f50ae190"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 01:09:53.731515 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:53.731448    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a82d7b06-73f3-49aa-9713-bff9f50ae190-plugins-config-volume\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:09:53.733591 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:53.733535    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a82d7b06-73f3-49aa-9713-bff9f50ae190-kube-api-access-tf459" (OuterVolumeSpecName: "kube-api-access-tf459") pod "a82d7b06-73f3-49aa-9713-bff9f50ae190" (UID: "a82d7b06-73f3-49aa-9713-bff9f50ae190"). InnerVolumeSpecName "kube-api-access-tf459". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 01:09:53.788007 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:53.787951    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-96c840-5bdd89c7f8-2c9h9"
Sep 16 01:09:53.832749 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:53.831919    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e636703-6f9e-4043-961c-27b191e47d79-metrics-certs\") pod \"2e636703-6f9e-4043-961c-27b191e47d79\" (UID: \"2e636703-6f9e-4043-961c-27b191e47d79\") "
Sep 16 01:09:53.832749 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:53.831982    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e636703-6f9e-4043-961c-27b191e47d79-openshift-service-ca\") pod \"2e636703-6f9e-4043-961c-27b191e47d79\" (UID: \"2e636703-6f9e-4043-961c-27b191e47d79\") "
Sep 16 01:09:53.832749 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:53.832027    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gsd65\" (UniqueName: \"kubernetes.io/projected/2e636703-6f9e-4043-961c-27b191e47d79-kube-api-access-gsd65\") pod \"2e636703-6f9e-4043-961c-27b191e47d79\" (UID: \"2e636703-6f9e-4043-961c-27b191e47d79\") "
Sep 16 01:09:53.832749 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:53.832059    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2e636703-6f9e-4043-961c-27b191e47d79-maas-api-tls\") pod \"2e636703-6f9e-4043-961c-27b191e47d79\" (UID: \"2e636703-6f9e-4043-961c-27b191e47d79\") "
Sep 16 01:09:53.832749 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:53.832237    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tf459\" (UniqueName: \"kubernetes.io/projected/a82d7b06-73f3-49aa-9713-bff9f50ae190-kube-api-access-tf459\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:09:53.833718 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:53.833662    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e636703-6f9e-4043-961c-27b191e47d79-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2e636703-6f9e-4043-961c-27b191e47d79" (UID: "2e636703-6f9e-4043-961c-27b191e47d79"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 01:09:53.836102 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:53.836049    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2e636703-6f9e-4043-961c-27b191e47d79-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "2e636703-6f9e-4043-961c-27b191e47d79" (UID: "2e636703-6f9e-4043-961c-27b191e47d79"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 01:09:53.836102 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:53.836086    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e636703-6f9e-4043-961c-27b191e47d79-kube-api-access-gsd65" (OuterVolumeSpecName: "kube-api-access-gsd65") pod "2e636703-6f9e-4043-961c-27b191e47d79" (UID: "2e636703-6f9e-4043-961c-27b191e47d79"). InnerVolumeSpecName "kube-api-access-gsd65". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 01:09:53.837380 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:53.837352    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2e636703-6f9e-4043-961c-27b191e47d79-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2e636703-6f9e-4043-961c-27b191e47d79" (UID: "2e636703-6f9e-4043-961c-27b191e47d79"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 01:09:53.926597 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:53.926569    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-96c840-76f5c8849f-ks9pz"
Sep 16 01:09:53.932980 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:53.932956    2556 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e636703-6f9e-4043-961c-27b191e47d79-metrics-certs\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:09:53.932980 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:53.932982    2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e636703-6f9e-4043-961c-27b191e47d79-openshift-service-ca\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:09:53.933152 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:53.932996    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gsd65\" (UniqueName: \"kubernetes.io/projected/2e636703-6f9e-4043-961c-27b191e47d79-kube-api-access-gsd65\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:09:53.933152 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:53.933008    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2e636703-6f9e-4043-961c-27b191e47d79-maas-api-tls\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:09:53.936138 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:53.936115    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-96c840-56b9dd4654-5rp7j"
Sep 16 01:09:54.033760 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:54.033717    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2583e2cb-79b4-4b90-931f-99de7835d6e4-plugins-config-volume\") pod \"2583e2cb-79b4-4b90-931f-99de7835d6e4\" (UID: \"2583e2cb-79b4-4b90-931f-99de7835d6e4\") "
Sep 16 01:09:54.033952 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:54.033785    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa9f4c4b-4e60-4638-aa94-b03b1877df9b-plugins-config-volume\") pod \"fa9f4c4b-4e60-4638-aa94-b03b1877df9b\" (UID: \"fa9f4c4b-4e60-4638-aa94-b03b1877df9b\") "
Sep 16 01:09:54.033952 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:54.033900    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tkwnp\" (UniqueName: \"kubernetes.io/projected/2583e2cb-79b4-4b90-931f-99de7835d6e4-kube-api-access-tkwnp\") pod \"2583e2cb-79b4-4b90-931f-99de7835d6e4\" (UID: \"2583e2cb-79b4-4b90-931f-99de7835d6e4\") "
Sep 16 01:09:54.033952 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:54.033931    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b7frt\" (UniqueName: \"kubernetes.io/projected/fa9f4c4b-4e60-4638-aa94-b03b1877df9b-kube-api-access-b7frt\") pod \"fa9f4c4b-4e60-4638-aa94-b03b1877df9b\" (UID: \"fa9f4c4b-4e60-4638-aa94-b03b1877df9b\") "
Sep 16 01:09:54.034133 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:54.034092    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2583e2cb-79b4-4b90-931f-99de7835d6e4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2583e2cb-79b4-4b90-931f-99de7835d6e4" (UID: "2583e2cb-79b4-4b90-931f-99de7835d6e4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 01:09:54.034229 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:54.034212    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2583e2cb-79b4-4b90-931f-99de7835d6e4-plugins-config-volume\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:09:54.034295 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:54.034207    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa9f4c4b-4e60-4638-aa94-b03b1877df9b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fa9f4c4b-4e60-4638-aa94-b03b1877df9b" (UID: "fa9f4c4b-4e60-4638-aa94-b03b1877df9b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 01:09:54.035804 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:54.035781    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2583e2cb-79b4-4b90-931f-99de7835d6e4-kube-api-access-tkwnp" (OuterVolumeSpecName: "kube-api-access-tkwnp") pod "2583e2cb-79b4-4b90-931f-99de7835d6e4" (UID: "2583e2cb-79b4-4b90-931f-99de7835d6e4"). InnerVolumeSpecName "kube-api-access-tkwnp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 01:09:54.035874 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:54.035817    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa9f4c4b-4e60-4638-aa94-b03b1877df9b-kube-api-access-b7frt" (OuterVolumeSpecName: "kube-api-access-b7frt") pod "fa9f4c4b-4e60-4638-aa94-b03b1877df9b" (UID: "fa9f4c4b-4e60-4638-aa94-b03b1877df9b"). InnerVolumeSpecName "kube-api-access-b7frt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 01:09:54.134672 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:54.134635    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa9f4c4b-4e60-4638-aa94-b03b1877df9b-plugins-config-volume\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:09:54.134672 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:54.134666    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tkwnp\" (UniqueName: \"kubernetes.io/projected/2583e2cb-79b4-4b90-931f-99de7835d6e4-kube-api-access-tkwnp\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:09:54.134672 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:54.134676    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b7frt\" (UniqueName: \"kubernetes.io/projected/fa9f4c4b-4e60-4638-aa94-b03b1877df9b-kube-api-access-b7frt\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:09:54.173430 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:54.173394    2556 generic.go:356] "Generic (PLEG): container finished" podID="2e636703-6f9e-4043-961c-27b191e47d79" containerID="c42301cb3b3e10f006e30b3916c9a108fe28a9344800f6b9ec4c5b8330b83581" exitCode=0
Sep 16 01:09:54.173631 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:54.173443    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-96c840-5bdd89c7f8-2c9h9"
Sep 16 01:09:54.173631 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:54.173494    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-96c840-5bdd89c7f8-2c9h9" event={"ID":"2e636703-6f9e-4043-961c-27b191e47d79","Type":"ContainerDied","Data":"c42301cb3b3e10f006e30b3916c9a108fe28a9344800f6b9ec4c5b8330b83581"}
Sep 16 01:09:54.173631 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:54.173560    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-96c840-5bdd89c7f8-2c9h9" event={"ID":"2e636703-6f9e-4043-961c-27b191e47d79","Type":"ContainerDied","Data":"9911d072888bf6c7a67938ea9a2faa808a89b225584a897164924748466d487e"}
Sep 16 01:09:54.173631 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:54.173586    2556 scope.go:117] "RemoveContainer" containerID="c42301cb3b3e10f006e30b3916c9a108fe28a9344800f6b9ec4c5b8330b83581"
Sep 16 01:09:54.174976 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:54.174946    2556 generic.go:356] "Generic (PLEG): container finished" podID="a82d7b06-73f3-49aa-9713-bff9f50ae190" containerID="e8ff32692043f056c208fa119e2e38bc2c80415338f8074ad276be725c8ffe22" exitCode=0
Sep 16 01:09:54.175082 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:54.174982    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-f29ccae3-6fdb5f549d-2jshw" event={"ID":"a82d7b06-73f3-49aa-9713-bff9f50ae190","Type":"ContainerDied","Data":"e8ff32692043f056c208fa119e2e38bc2c80415338f8074ad276be725c8ffe22"}
Sep 16 01:09:54.175082 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:54.175000    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-f29ccae3-6fdb5f549d-2jshw"
Sep 16 01:09:54.175082 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:54.175022    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-f29ccae3-6fdb5f549d-2jshw" event={"ID":"a82d7b06-73f3-49aa-9713-bff9f50ae190","Type":"ContainerDied","Data":"167d5d70377bab0774e61e6df44c09ea6e3cee77b4ea0248bf51942100810480"}
Sep 16 01:09:54.176677 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:54.176654    2556 generic.go:356] "Generic (PLEG): container finished" podID="fa9f4c4b-4e60-4638-aa94-b03b1877df9b" containerID="bfd75d69293cde13b23f4266a6d88577b96760ad802c925934c5727e441a18e8" exitCode=0
Sep 16 01:09:54.176677 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:54.176678    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-96c840-76f5c8849f-ks9pz" event={"ID":"fa9f4c4b-4e60-4638-aa94-b03b1877df9b","Type":"ContainerDied","Data":"bfd75d69293cde13b23f4266a6d88577b96760ad802c925934c5727e441a18e8"}
Sep 16 01:09:54.176836 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:54.176691    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-96c840-76f5c8849f-ks9pz"
Sep 16 01:09:54.176836 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:54.176732    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-96c840-76f5c8849f-ks9pz" event={"ID":"fa9f4c4b-4e60-4638-aa94-b03b1877df9b","Type":"ContainerDied","Data":"fc131c6f22313126cd419ec4416c1e89e205d49415806fa91adfe0399122f159"}
Sep 16 01:09:54.178548 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:54.178527    2556 generic.go:356] "Generic (PLEG): container finished" podID="2583e2cb-79b4-4b90-931f-99de7835d6e4" containerID="6389de8e731342a8ce369b868fb1aea0a2638d9431130a93f5c0aabfec575c23" exitCode=0
Sep 16 01:09:54.178642 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:54.178559    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-96c840-56b9dd4654-5rp7j" event={"ID":"2583e2cb-79b4-4b90-931f-99de7835d6e4","Type":"ContainerDied","Data":"6389de8e731342a8ce369b868fb1aea0a2638d9431130a93f5c0aabfec575c23"}
Sep 16 01:09:54.178642 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:54.178573    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-96c840-56b9dd4654-5rp7j"
Sep 16 01:09:54.178642 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:54.178595    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-96c840-56b9dd4654-5rp7j" event={"ID":"2583e2cb-79b4-4b90-931f-99de7835d6e4","Type":"ContainerDied","Data":"057da483159e7b7b6df189096ac008770d7d149694c7dfc0698343285239fe52"}
Sep 16 01:09:54.183882 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:54.183862    2556 scope.go:117] "RemoveContainer" containerID="c42301cb3b3e10f006e30b3916c9a108fe28a9344800f6b9ec4c5b8330b83581"
Sep 16 01:09:54.184167 ip-10-0-130-207 kubenswrapper[2556]: E0916 01:09:54.184146    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c42301cb3b3e10f006e30b3916c9a108fe28a9344800f6b9ec4c5b8330b83581\": container with ID starting with c42301cb3b3e10f006e30b3916c9a108fe28a9344800f6b9ec4c5b8330b83581 not found: ID does not exist" containerID="c42301cb3b3e10f006e30b3916c9a108fe28a9344800f6b9ec4c5b8330b83581"
Sep 16 01:09:54.184247 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:54.184179    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c42301cb3b3e10f006e30b3916c9a108fe28a9344800f6b9ec4c5b8330b83581"} err="failed to get container status \"c42301cb3b3e10f006e30b3916c9a108fe28a9344800f6b9ec4c5b8330b83581\": rpc error: code = NotFound desc = could not find container \"c42301cb3b3e10f006e30b3916c9a108fe28a9344800f6b9ec4c5b8330b83581\": container with ID starting with c42301cb3b3e10f006e30b3916c9a108fe28a9344800f6b9ec4c5b8330b83581 not found: ID does not exist"
Sep 16 01:09:54.184247 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:54.184201    2556 scope.go:117] "RemoveContainer" containerID="e8ff32692043f056c208fa119e2e38bc2c80415338f8074ad276be725c8ffe22"
Sep 16 01:09:54.194646 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:54.194627    2556 scope.go:117] "RemoveContainer" containerID="e8ff32692043f056c208fa119e2e38bc2c80415338f8074ad276be725c8ffe22"
Sep 16 01:09:54.195005 ip-10-0-130-207 kubenswrapper[2556]: E0916 01:09:54.194982    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e8ff32692043f056c208fa119e2e38bc2c80415338f8074ad276be725c8ffe22\": container with ID starting with e8ff32692043f056c208fa119e2e38bc2c80415338f8074ad276be725c8ffe22 not found: ID does not exist" containerID="e8ff32692043f056c208fa119e2e38bc2c80415338f8074ad276be725c8ffe22"
Sep 16 01:09:54.195068 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:54.195012    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8ff32692043f056c208fa119e2e38bc2c80415338f8074ad276be725c8ffe22"} err="failed to get container status \"e8ff32692043f056c208fa119e2e38bc2c80415338f8074ad276be725c8ffe22\": rpc error: code = NotFound desc = could not find container \"e8ff32692043f056c208fa119e2e38bc2c80415338f8074ad276be725c8ffe22\": container with ID starting with e8ff32692043f056c208fa119e2e38bc2c80415338f8074ad276be725c8ffe22 not found: ID does not exist"
Sep 16 01:09:54.195068 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:54.195029    2556 scope.go:117] "RemoveContainer" containerID="bfd75d69293cde13b23f4266a6d88577b96760ad802c925934c5727e441a18e8"
Sep 16 01:09:54.199978 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:54.199940    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f29ccae3-6fdb5f549d-2jshw"]
Sep 16 01:09:54.201549 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:54.201528    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f29ccae3-6fdb5f549d-2jshw"]
Sep 16 01:09:54.207962 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:54.207946    2556 scope.go:117] "RemoveContainer" containerID="bfd75d69293cde13b23f4266a6d88577b96760ad802c925934c5727e441a18e8"
Sep 16 01:09:54.208230 ip-10-0-130-207 kubenswrapper[2556]: E0916 01:09:54.208212    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bfd75d69293cde13b23f4266a6d88577b96760ad802c925934c5727e441a18e8\": container with ID starting with bfd75d69293cde13b23f4266a6d88577b96760ad802c925934c5727e441a18e8 not found: ID does not exist" containerID="bfd75d69293cde13b23f4266a6d88577b96760ad802c925934c5727e441a18e8"
Sep 16 01:09:54.208282 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:54.208238    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bfd75d69293cde13b23f4266a6d88577b96760ad802c925934c5727e441a18e8"} err="failed to get container status \"bfd75d69293cde13b23f4266a6d88577b96760ad802c925934c5727e441a18e8\": rpc error: code = NotFound desc = could not find container \"bfd75d69293cde13b23f4266a6d88577b96760ad802c925934c5727e441a18e8\": container with ID starting with bfd75d69293cde13b23f4266a6d88577b96760ad802c925934c5727e441a18e8 not found: ID does not exist"
Sep 16 01:09:54.208282 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:54.208254    2556 scope.go:117] "RemoveContainer" containerID="6389de8e731342a8ce369b868fb1aea0a2638d9431130a93f5c0aabfec575c23"
Sep 16 01:09:54.214551 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:54.214532    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-96c840-5bdd89c7f8-2c9h9"]
Sep 16 01:09:54.216174 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:54.216149    2556 scope.go:117] "RemoveContainer" containerID="6389de8e731342a8ce369b868fb1aea0a2638d9431130a93f5c0aabfec575c23"
Sep 16 01:09:54.216533 ip-10-0-130-207 kubenswrapper[2556]: E0916 01:09:54.216512    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6389de8e731342a8ce369b868fb1aea0a2638d9431130a93f5c0aabfec575c23\": container with ID starting with 6389de8e731342a8ce369b868fb1aea0a2638d9431130a93f5c0aabfec575c23 not found: ID does not exist" containerID="6389de8e731342a8ce369b868fb1aea0a2638d9431130a93f5c0aabfec575c23"
Sep 16 01:09:54.216615 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:54.216537    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6389de8e731342a8ce369b868fb1aea0a2638d9431130a93f5c0aabfec575c23"} err="failed to get container status \"6389de8e731342a8ce369b868fb1aea0a2638d9431130a93f5c0aabfec575c23\": rpc error: code = NotFound desc = could not find container \"6389de8e731342a8ce369b868fb1aea0a2638d9431130a93f5c0aabfec575c23\": container with ID starting with 6389de8e731342a8ce369b868fb1aea0a2638d9431130a93f5c0aabfec575c23 not found: ID does not exist"
Sep 16 01:09:54.218108 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:54.218091    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-96c840-5bdd89c7f8-2c9h9"]
Sep 16 01:09:54.228851 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:54.228832    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-96c840-56b9dd4654-5rp7j"]
Sep 16 01:09:54.232254 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:54.232232    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-96c840-56b9dd4654-5rp7j"]
Sep 16 01:09:54.240950 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:54.240916    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-96c840-76f5c8849f-ks9pz"]
Sep 16 01:09:54.242771 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:54.242751    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-96c840-76f5c8849f-ks9pz"]
Sep 16 01:09:55.736235 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:55.736203    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2583e2cb-79b4-4b90-931f-99de7835d6e4" path="/var/lib/kubelet/pods/2583e2cb-79b4-4b90-931f-99de7835d6e4/volumes"
Sep 16 01:09:55.736673 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:55.736582    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e636703-6f9e-4043-961c-27b191e47d79" path="/var/lib/kubelet/pods/2e636703-6f9e-4043-961c-27b191e47d79/volumes"
Sep 16 01:09:55.736895 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:55.736882    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a82d7b06-73f3-49aa-9713-bff9f50ae190" path="/var/lib/kubelet/pods/a82d7b06-73f3-49aa-9713-bff9f50ae190/volumes"
Sep 16 01:09:55.737459 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:55.737446    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa9f4c4b-4e60-4638-aa94-b03b1877df9b" path="/var/lib/kubelet/pods/fa9f4c4b-4e60-4638-aa94-b03b1877df9b/volumes"
Sep 16 01:09:57.507653 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:57.507613    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-96c840/distinct-w0-96c840-kserve-c8989f89f-qwjjl" podUID="e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3" containerName="main" containerID="cri-o://cb3b5f4e2c0a7195331a0fbf79ec5a67fc7b349ee4380344e7dde40726e1f19e" gracePeriod=45
Sep 16 01:09:57.529202 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:57.529177    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-96c840/premium-w0-96c840-kserve-6d9c67476-t9n4q" podUID="75077e26-5c4a-45dd-855d-8cc1a44cafd6" containerName="main" containerID="cri-o://216dc23fc2695e7389a25435c644623774f998849bdb75f2a69722f989bbdc3b" gracePeriod=45
Sep 16 01:09:57.765733 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:57.765676    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-96c840/distinct-w0-96c840-kserve-c8989f89f-qwjjl"
Sep 16 01:09:57.799410 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:57.799386    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-96c840/premium-w0-96c840-kserve-6d9c67476-t9n4q"
Sep 16 01:09:57.861715 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:57.861683    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3-tmp-dir\") pod \"e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3\" (UID: \"e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3\") "
Sep 16 01:09:57.861862 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:57.861723    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/75077e26-5c4a-45dd-855d-8cc1a44cafd6-home\") pod \"75077e26-5c4a-45dd-855d-8cc1a44cafd6\" (UID: \"75077e26-5c4a-45dd-855d-8cc1a44cafd6\") "
Sep 16 01:09:57.861862 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:57.861745    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75077e26-5c4a-45dd-855d-8cc1a44cafd6-tls-certs\") pod \"75077e26-5c4a-45dd-855d-8cc1a44cafd6\" (UID: \"75077e26-5c4a-45dd-855d-8cc1a44cafd6\") "
Sep 16 01:09:57.861862 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:57.861783    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3-tls-certs\") pod \"e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3\" (UID: \"e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3\") "
Sep 16 01:09:57.861862 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:57.861842    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/75077e26-5c4a-45dd-855d-8cc1a44cafd6-model-cache\") pod \"75077e26-5c4a-45dd-855d-8cc1a44cafd6\" (UID: \"75077e26-5c4a-45dd-855d-8cc1a44cafd6\") "
Sep 16 01:09:57.862081 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:57.861978    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3-model-cache\") pod \"e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3\" (UID: \"e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3\") "
Sep 16 01:09:57.862081 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:57.861987    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3" (UID: "e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 01:09:57.862081 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:57.862005    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75077e26-5c4a-45dd-855d-8cc1a44cafd6-home" (OuterVolumeSpecName: "home") pod "75077e26-5c4a-45dd-855d-8cc1a44cafd6" (UID: "75077e26-5c4a-45dd-855d-8cc1a44cafd6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 01:09:57.862081 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:57.862017    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3-home\") pod \"e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3\" (UID: \"e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3\") "
Sep 16 01:09:57.862081 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:57.862057    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75077e26-5c4a-45dd-855d-8cc1a44cafd6-tmp-dir\") pod \"75077e26-5c4a-45dd-855d-8cc1a44cafd6\" (UID: \"75077e26-5c4a-45dd-855d-8cc1a44cafd6\") "
Sep 16 01:09:57.862081 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:57.862078    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75077e26-5c4a-45dd-855d-8cc1a44cafd6-model-cache" (OuterVolumeSpecName: "model-cache") pod "75077e26-5c4a-45dd-855d-8cc1a44cafd6" (UID: "75077e26-5c4a-45dd-855d-8cc1a44cafd6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 01:09:57.862399 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:57.862085    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k2mg8\" (UniqueName: \"kubernetes.io/projected/75077e26-5c4a-45dd-855d-8cc1a44cafd6-kube-api-access-k2mg8\") pod \"75077e26-5c4a-45dd-855d-8cc1a44cafd6\" (UID: \"75077e26-5c4a-45dd-855d-8cc1a44cafd6\") "
Sep 16 01:09:57.862399 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:57.862163    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5fckd\" (UniqueName: \"kubernetes.io/projected/e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3-kube-api-access-5fckd\") pod \"e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3\" (UID: \"e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3\") "
Sep 16 01:09:57.862399 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:57.862180    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3-model-cache" (OuterVolumeSpecName: "model-cache") pod "e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3" (UID: "e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 01:09:57.862399 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:57.862206    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3-dshm\") pod \"e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3\" (UID: \"e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3\") "
Sep 16 01:09:57.862399 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:57.862274    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/75077e26-5c4a-45dd-855d-8cc1a44cafd6-dshm\") pod \"75077e26-5c4a-45dd-855d-8cc1a44cafd6\" (UID: \"75077e26-5c4a-45dd-855d-8cc1a44cafd6\") "
Sep 16 01:09:57.862685 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:57.862489    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3-home" (OuterVolumeSpecName: "home") pod "e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3" (UID: "e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 01:09:57.862685 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:57.862659    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75077e26-5c4a-45dd-855d-8cc1a44cafd6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "75077e26-5c4a-45dd-855d-8cc1a44cafd6" (UID: "75077e26-5c4a-45dd-855d-8cc1a44cafd6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 01:09:57.862797 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:57.862691    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3-tmp-dir\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:09:57.862797 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:57.862711    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/75077e26-5c4a-45dd-855d-8cc1a44cafd6-home\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:09:57.862797 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:57.862724    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/75077e26-5c4a-45dd-855d-8cc1a44cafd6-model-cache\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:09:57.862797 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:57.862736    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3-model-cache\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:09:57.862797 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:57.862748    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3-home\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:09:57.864624 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:57.864588    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3-dshm" (OuterVolumeSpecName: "dshm") pod "e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3" (UID: "e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 01:09:57.864624 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:57.864606    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3" (UID: "e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 01:09:57.864864 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:57.864836    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/75077e26-5c4a-45dd-855d-8cc1a44cafd6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "75077e26-5c4a-45dd-855d-8cc1a44cafd6" (UID: "75077e26-5c4a-45dd-855d-8cc1a44cafd6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 01:09:57.864924 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:57.864874    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75077e26-5c4a-45dd-855d-8cc1a44cafd6-dshm" (OuterVolumeSpecName: "dshm") pod "75077e26-5c4a-45dd-855d-8cc1a44cafd6" (UID: "75077e26-5c4a-45dd-855d-8cc1a44cafd6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 01:09:57.865025 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:57.864999    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3-kube-api-access-5fckd" (OuterVolumeSpecName: "kube-api-access-5fckd") pod "e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3" (UID: "e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3"). InnerVolumeSpecName "kube-api-access-5fckd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 01:09:57.865025 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:57.865010    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75077e26-5c4a-45dd-855d-8cc1a44cafd6-kube-api-access-k2mg8" (OuterVolumeSpecName: "kube-api-access-k2mg8") pod "75077e26-5c4a-45dd-855d-8cc1a44cafd6" (UID: "75077e26-5c4a-45dd-855d-8cc1a44cafd6"). InnerVolumeSpecName "kube-api-access-k2mg8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 01:09:57.963242 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:57.963202    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75077e26-5c4a-45dd-855d-8cc1a44cafd6-tmp-dir\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:09:57.963242 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:57.963242    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k2mg8\" (UniqueName: \"kubernetes.io/projected/75077e26-5c4a-45dd-855d-8cc1a44cafd6-kube-api-access-k2mg8\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:09:57.963442 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:57.963259    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5fckd\" (UniqueName: \"kubernetes.io/projected/e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3-kube-api-access-5fckd\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:09:57.963442 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:57.963271    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3-dshm\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:09:57.963442 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:57.963283    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/75077e26-5c4a-45dd-855d-8cc1a44cafd6-dshm\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:09:57.963442 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:57.963293    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75077e26-5c4a-45dd-855d-8cc1a44cafd6-tls-certs\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:09:57.963442 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:57.963303    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3-tls-certs\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:09:58.204984 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:58.204950    2556 generic.go:356] "Generic (PLEG): container finished" podID="e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3" containerID="cb3b5f4e2c0a7195331a0fbf79ec5a67fc7b349ee4380344e7dde40726e1f19e" exitCode=0
Sep 16 01:09:58.205168 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:58.204996    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-96c840/distinct-w0-96c840-kserve-c8989f89f-qwjjl"
Sep 16 01:09:58.205168 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:58.205042    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-96c840/distinct-w0-96c840-kserve-c8989f89f-qwjjl" event={"ID":"e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3","Type":"ContainerDied","Data":"cb3b5f4e2c0a7195331a0fbf79ec5a67fc7b349ee4380344e7dde40726e1f19e"}
Sep 16 01:09:58.205168 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:58.205082    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-96c840/distinct-w0-96c840-kserve-c8989f89f-qwjjl" event={"ID":"e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3","Type":"ContainerDied","Data":"ec72219fd212bd1f44718bdb4532ff65521ed7c2b27d95c9fcaf7ea917864254"}
Sep 16 01:09:58.205168 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:58.205103    2556 scope.go:117] "RemoveContainer" containerID="cb3b5f4e2c0a7195331a0fbf79ec5a67fc7b349ee4380344e7dde40726e1f19e"
Sep 16 01:09:58.206653 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:58.206628    2556 generic.go:356] "Generic (PLEG): container finished" podID="75077e26-5c4a-45dd-855d-8cc1a44cafd6" containerID="216dc23fc2695e7389a25435c644623774f998849bdb75f2a69722f989bbdc3b" exitCode=0
Sep 16 01:09:58.206781 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:58.206664    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-96c840/premium-w0-96c840-kserve-6d9c67476-t9n4q"
Sep 16 01:09:58.206781 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:58.206728    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-96c840/premium-w0-96c840-kserve-6d9c67476-t9n4q" event={"ID":"75077e26-5c4a-45dd-855d-8cc1a44cafd6","Type":"ContainerDied","Data":"216dc23fc2695e7389a25435c644623774f998849bdb75f2a69722f989bbdc3b"}
Sep 16 01:09:58.206781 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:58.206766    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-96c840/premium-w0-96c840-kserve-6d9c67476-t9n4q" event={"ID":"75077e26-5c4a-45dd-855d-8cc1a44cafd6","Type":"ContainerDied","Data":"8225bae385255574925697f47de6274477e3a695e85da45120c8b73204e27889"}
Sep 16 01:09:58.222137 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:58.222117    2556 scope.go:117] "RemoveContainer" containerID="cb3b5f4e2c0a7195331a0fbf79ec5a67fc7b349ee4380344e7dde40726e1f19e"
Sep 16 01:09:58.222408 ip-10-0-130-207 kubenswrapper[2556]: E0916 01:09:58.222387    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb3b5f4e2c0a7195331a0fbf79ec5a67fc7b349ee4380344e7dde40726e1f19e\": container with ID starting with cb3b5f4e2c0a7195331a0fbf79ec5a67fc7b349ee4380344e7dde40726e1f19e not found: ID does not exist" containerID="cb3b5f4e2c0a7195331a0fbf79ec5a67fc7b349ee4380344e7dde40726e1f19e"
Sep 16 01:09:58.222450 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:58.222416    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb3b5f4e2c0a7195331a0fbf79ec5a67fc7b349ee4380344e7dde40726e1f19e"} err="failed to get container status \"cb3b5f4e2c0a7195331a0fbf79ec5a67fc7b349ee4380344e7dde40726e1f19e\": rpc error: code = NotFound desc = could not find container \"cb3b5f4e2c0a7195331a0fbf79ec5a67fc7b349ee4380344e7dde40726e1f19e\": container with ID starting with cb3b5f4e2c0a7195331a0fbf79ec5a67fc7b349ee4380344e7dde40726e1f19e not found: ID does not exist"
Sep 16 01:09:58.222450 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:58.222432    2556 scope.go:117] "RemoveContainer" containerID="216dc23fc2695e7389a25435c644623774f998849bdb75f2a69722f989bbdc3b"
Sep 16 01:09:58.230562 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:58.230541    2556 scope.go:117] "RemoveContainer" containerID="216dc23fc2695e7389a25435c644623774f998849bdb75f2a69722f989bbdc3b"
Sep 16 01:09:58.230823 ip-10-0-130-207 kubenswrapper[2556]: E0916 01:09:58.230803    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"216dc23fc2695e7389a25435c644623774f998849bdb75f2a69722f989bbdc3b\": container with ID starting with 216dc23fc2695e7389a25435c644623774f998849bdb75f2a69722f989bbdc3b not found: ID does not exist" containerID="216dc23fc2695e7389a25435c644623774f998849bdb75f2a69722f989bbdc3b"
Sep 16 01:09:58.230897 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:58.230828    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"216dc23fc2695e7389a25435c644623774f998849bdb75f2a69722f989bbdc3b"} err="failed to get container status \"216dc23fc2695e7389a25435c644623774f998849bdb75f2a69722f989bbdc3b\": rpc error: code = NotFound desc = could not find container \"216dc23fc2695e7389a25435c644623774f998849bdb75f2a69722f989bbdc3b\": container with ID starting with 216dc23fc2695e7389a25435c644623774f998849bdb75f2a69722f989bbdc3b not found: ID does not exist"
Sep 16 01:09:58.232921 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:58.232899    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-96c840/premium-w0-96c840-kserve-6d9c67476-t9n4q"]
Sep 16 01:09:58.234866 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:58.234844    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-96c840/premium-w0-96c840-kserve-6d9c67476-t9n4q"]
Sep 16 01:09:58.242423 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:58.242399    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-96c840/distinct-w0-96c840-kserve-c8989f89f-qwjjl"]
Sep 16 01:09:58.245806 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:58.245787    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-96c840/distinct-w0-96c840-kserve-c8989f89f-qwjjl"]
Sep 16 01:09:59.735771 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:59.735734    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="75077e26-5c4a-45dd-855d-8cc1a44cafd6" path="/var/lib/kubelet/pods/75077e26-5c4a-45dd-855d-8cc1a44cafd6/volumes"
Sep 16 01:09:59.736146 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:09:59.736124    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3" path="/var/lib/kubelet/pods/e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3/volumes"
Sep 16 01:10:18.104607 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:18.104573    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-f39e09-7cc574497b-z6vrc"]
Sep 16 01:10:18.105157 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:18.104958    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa9f4c4b-4e60-4638-aa94-b03b1877df9b" containerName="payload-processing"
Sep 16 01:10:18.105157 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:18.104975    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa9f4c4b-4e60-4638-aa94-b03b1877df9b" containerName="payload-processing"
Sep 16 01:10:18.105157 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:18.104991    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3" containerName="main"
Sep 16 01:10:18.105157 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:18.105000    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3" containerName="main"
Sep 16 01:10:18.105157 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:18.105013    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e636703-6f9e-4043-961c-27b191e47d79" containerName="maas-api"
Sep 16 01:10:18.105157 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:18.105021    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e636703-6f9e-4043-961c-27b191e47d79" containerName="maas-api"
Sep 16 01:10:18.105157 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:18.105033    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a82d7b06-73f3-49aa-9713-bff9f50ae190" containerName="payload-pre-processing"
Sep 16 01:10:18.105157 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:18.105041    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="a82d7b06-73f3-49aa-9713-bff9f50ae190" containerName="payload-pre-processing"
Sep 16 01:10:18.105157 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:18.105053    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2583e2cb-79b4-4b90-931f-99de7835d6e4" containerName="payload-pre-processing"
Sep 16 01:10:18.105157 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:18.105061    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="2583e2cb-79b4-4b90-931f-99de7835d6e4" containerName="payload-pre-processing"
Sep 16 01:10:18.105157 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:18.105072    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75077e26-5c4a-45dd-855d-8cc1a44cafd6" containerName="main"
Sep 16 01:10:18.105157 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:18.105079    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="75077e26-5c4a-45dd-855d-8cc1a44cafd6" containerName="main"
Sep 16 01:10:18.105862 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:18.105198    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="75077e26-5c4a-45dd-855d-8cc1a44cafd6" containerName="main"
Sep 16 01:10:18.105862 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:18.105211    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa9f4c4b-4e60-4638-aa94-b03b1877df9b" containerName="payload-processing"
Sep 16 01:10:18.105862 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:18.105224    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7087dd0-00aa-46a9-ba1e-7f2f56c6d4c3" containerName="main"
Sep 16 01:10:18.105862 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:18.105229    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="a82d7b06-73f3-49aa-9713-bff9f50ae190" containerName="payload-pre-processing"
Sep 16 01:10:18.105862 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:18.105241    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e636703-6f9e-4043-961c-27b191e47d79" containerName="maas-api"
Sep 16 01:10:18.105862 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:18.105251    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="2583e2cb-79b4-4b90-931f-99de7835d6e4" containerName="payload-pre-processing"
Sep 16 01:10:18.111330 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:18.109956    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-f39e09-7cc574497b-z6vrc"
Sep 16 01:10:18.113144 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:18.113116    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w1-f39e09-serving-cert\""
Sep 16 01:10:18.119239 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:18.119215    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-f39e09-7cc574497b-z6vrc"]
Sep 16 01:10:18.231858 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:18.231828    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/26244697-ec2c-48f6-88f2-ce3e688e2add-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-f39e09-7cc574497b-z6vrc\" (UID: \"26244697-ec2c-48f6-88f2-ce3e688e2add\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-f39e09-7cc574497b-z6vrc"
Sep 16 01:10:18.231858 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:18.231862    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26244697-ec2c-48f6-88f2-ce3e688e2add-metrics-certs\") pod \"maas-api-e2e-shared-a-w1-f39e09-7cc574497b-z6vrc\" (UID: \"26244697-ec2c-48f6-88f2-ce3e688e2add\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-f39e09-7cc574497b-z6vrc"
Sep 16 01:10:18.232081 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:18.231885    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/26244697-ec2c-48f6-88f2-ce3e688e2add-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w1-f39e09-7cc574497b-z6vrc\" (UID: \"26244697-ec2c-48f6-88f2-ce3e688e2add\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-f39e09-7cc574497b-z6vrc"
Sep 16 01:10:18.232081 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:18.232013    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2pwz\" (UniqueName: \"kubernetes.io/projected/26244697-ec2c-48f6-88f2-ce3e688e2add-kube-api-access-q2pwz\") pod \"maas-api-e2e-shared-a-w1-f39e09-7cc574497b-z6vrc\" (UID: \"26244697-ec2c-48f6-88f2-ce3e688e2add\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-f39e09-7cc574497b-z6vrc"
Sep 16 01:10:18.333386 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:18.333361    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/26244697-ec2c-48f6-88f2-ce3e688e2add-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-f39e09-7cc574497b-z6vrc\" (UID: \"26244697-ec2c-48f6-88f2-ce3e688e2add\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-f39e09-7cc574497b-z6vrc"
Sep 16 01:10:18.333566 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:18.333393    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26244697-ec2c-48f6-88f2-ce3e688e2add-metrics-certs\") pod \"maas-api-e2e-shared-a-w1-f39e09-7cc574497b-z6vrc\" (UID: \"26244697-ec2c-48f6-88f2-ce3e688e2add\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-f39e09-7cc574497b-z6vrc"
Sep 16 01:10:18.333566 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:18.333413    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/26244697-ec2c-48f6-88f2-ce3e688e2add-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w1-f39e09-7cc574497b-z6vrc\" (UID: \"26244697-ec2c-48f6-88f2-ce3e688e2add\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-f39e09-7cc574497b-z6vrc"
Sep 16 01:10:18.333566 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:18.333444    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2pwz\" (UniqueName: \"kubernetes.io/projected/26244697-ec2c-48f6-88f2-ce3e688e2add-kube-api-access-q2pwz\") pod \"maas-api-e2e-shared-a-w1-f39e09-7cc574497b-z6vrc\" (UID: \"26244697-ec2c-48f6-88f2-ce3e688e2add\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-f39e09-7cc574497b-z6vrc"
Sep 16 01:10:18.333566 ip-10-0-130-207 kubenswrapper[2556]: E0916 01:10:18.333529    2556 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-f39e09-serving-cert: secret "maas-api-e2e-shared-a-w1-f39e09-serving-cert" not found
Sep 16 01:10:18.333799 ip-10-0-130-207 kubenswrapper[2556]: E0916 01:10:18.333599    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26244697-ec2c-48f6-88f2-ce3e688e2add-maas-api-tls podName:26244697-ec2c-48f6-88f2-ce3e688e2add nodeName:}" failed. No retries permitted until 2026-09-16 01:10:18.833582976 +0000 UTC m=+1633.615084547 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/26244697-ec2c-48f6-88f2-ce3e688e2add-maas-api-tls") pod "maas-api-e2e-shared-a-w1-f39e09-7cc574497b-z6vrc" (UID: "26244697-ec2c-48f6-88f2-ce3e688e2add") : secret "maas-api-e2e-shared-a-w1-f39e09-serving-cert" not found
Sep 16 01:10:18.334052 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:18.334027    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/26244697-ec2c-48f6-88f2-ce3e688e2add-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w1-f39e09-7cc574497b-z6vrc\" (UID: \"26244697-ec2c-48f6-88f2-ce3e688e2add\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-f39e09-7cc574497b-z6vrc"
Sep 16 01:10:18.335874 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:18.335858    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26244697-ec2c-48f6-88f2-ce3e688e2add-metrics-certs\") pod \"maas-api-e2e-shared-a-w1-f39e09-7cc574497b-z6vrc\" (UID: \"26244697-ec2c-48f6-88f2-ce3e688e2add\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-f39e09-7cc574497b-z6vrc"
Sep 16 01:10:18.341886 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:18.341860    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2pwz\" (UniqueName: \"kubernetes.io/projected/26244697-ec2c-48f6-88f2-ce3e688e2add-kube-api-access-q2pwz\") pod \"maas-api-e2e-shared-a-w1-f39e09-7cc574497b-z6vrc\" (UID: \"26244697-ec2c-48f6-88f2-ce3e688e2add\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-f39e09-7cc574497b-z6vrc"
Sep 16 01:10:18.704420 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:18.704386    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-f39e09-f5677f8cd-dj7jx"]
Sep 16 01:10:18.708561 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:18.708538    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-f39e09-f5677f8cd-dj7jx"
Sep 16 01:10:18.710980 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:18.710957    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w1-f39e09\""
Sep 16 01:10:18.717503 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:18.717337    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-f39e09-f5677f8cd-dj7jx"]
Sep 16 01:10:18.751934 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:18.751898    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-f39e09-5d74949579-6jqsj"]
Sep 16 01:10:18.755516 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:18.755497    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-f39e09-5d74949579-6jqsj"
Sep 16 01:10:18.758281 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:18.758257    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w1-f39e09-dockercfg-4g88v\""
Sep 16 01:10:18.768279 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:18.768255    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-f39e09-5d74949579-6jqsj"]
Sep 16 01:10:18.838872 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:18.838848    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/26244697-ec2c-48f6-88f2-ce3e688e2add-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-f39e09-7cc574497b-z6vrc\" (UID: \"26244697-ec2c-48f6-88f2-ce3e688e2add\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-f39e09-7cc574497b-z6vrc"
Sep 16 01:10:18.838998 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:18.838906    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a5b85f15-bbd3-4beb-97f0-22767c9571b9-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w1-f39e09-5d74949579-6jqsj\" (UID: \"a5b85f15-bbd3-4beb-97f0-22767c9571b9\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-f39e09-5d74949579-6jqsj"
Sep 16 01:10:18.838998 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:18.838935    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w65qg\" (UniqueName: \"kubernetes.io/projected/44e35ee6-c1f9-4d6c-9584-2ebea688add8-kube-api-access-w65qg\") pod \"payload-pre-processing-e2e-shared-a-w1-f39e09-f5677f8cd-dj7jx\" (UID: \"44e35ee6-c1f9-4d6c-9584-2ebea688add8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-f39e09-f5677f8cd-dj7jx"
Sep 16 01:10:18.838998 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:18.838989    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m56mh\" (UniqueName: \"kubernetes.io/projected/a5b85f15-bbd3-4beb-97f0-22767c9571b9-kube-api-access-m56mh\") pod \"payload-processing-e2e-shared-a-w1-f39e09-5d74949579-6jqsj\" (UID: \"a5b85f15-bbd3-4beb-97f0-22767c9571b9\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-f39e09-5d74949579-6jqsj"
Sep 16 01:10:18.839165 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:18.839007    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/44e35ee6-c1f9-4d6c-9584-2ebea688add8-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w1-f39e09-f5677f8cd-dj7jx\" (UID: \"44e35ee6-c1f9-4d6c-9584-2ebea688add8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-f39e09-f5677f8cd-dj7jx"
Sep 16 01:10:18.841136 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:18.841113    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/26244697-ec2c-48f6-88f2-ce3e688e2add-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-f39e09-7cc574497b-z6vrc\" (UID: \"26244697-ec2c-48f6-88f2-ce3e688e2add\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-f39e09-7cc574497b-z6vrc"
Sep 16 01:10:18.940093 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:18.940052    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a5b85f15-bbd3-4beb-97f0-22767c9571b9-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w1-f39e09-5d74949579-6jqsj\" (UID: \"a5b85f15-bbd3-4beb-97f0-22767c9571b9\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-f39e09-5d74949579-6jqsj"
Sep 16 01:10:18.940270 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:18.940103    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w65qg\" (UniqueName: \"kubernetes.io/projected/44e35ee6-c1f9-4d6c-9584-2ebea688add8-kube-api-access-w65qg\") pod \"payload-pre-processing-e2e-shared-a-w1-f39e09-f5677f8cd-dj7jx\" (UID: \"44e35ee6-c1f9-4d6c-9584-2ebea688add8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-f39e09-f5677f8cd-dj7jx"
Sep 16 01:10:18.940270 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:18.940148    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m56mh\" (UniqueName: \"kubernetes.io/projected/a5b85f15-bbd3-4beb-97f0-22767c9571b9-kube-api-access-m56mh\") pod \"payload-processing-e2e-shared-a-w1-f39e09-5d74949579-6jqsj\" (UID: \"a5b85f15-bbd3-4beb-97f0-22767c9571b9\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-f39e09-5d74949579-6jqsj"
Sep 16 01:10:18.940270 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:18.940172    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/44e35ee6-c1f9-4d6c-9584-2ebea688add8-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w1-f39e09-f5677f8cd-dj7jx\" (UID: \"44e35ee6-c1f9-4d6c-9584-2ebea688add8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-f39e09-f5677f8cd-dj7jx"
Sep 16 01:10:18.940767 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:18.940741    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a5b85f15-bbd3-4beb-97f0-22767c9571b9-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w1-f39e09-5d74949579-6jqsj\" (UID: \"a5b85f15-bbd3-4beb-97f0-22767c9571b9\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-f39e09-5d74949579-6jqsj"
Sep 16 01:10:18.940904 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:18.940884    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/44e35ee6-c1f9-4d6c-9584-2ebea688add8-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w1-f39e09-f5677f8cd-dj7jx\" (UID: \"44e35ee6-c1f9-4d6c-9584-2ebea688add8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-f39e09-f5677f8cd-dj7jx"
Sep 16 01:10:18.947592 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:18.947572    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w65qg\" (UniqueName: \"kubernetes.io/projected/44e35ee6-c1f9-4d6c-9584-2ebea688add8-kube-api-access-w65qg\") pod \"payload-pre-processing-e2e-shared-a-w1-f39e09-f5677f8cd-dj7jx\" (UID: \"44e35ee6-c1f9-4d6c-9584-2ebea688add8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-f39e09-f5677f8cd-dj7jx"
Sep 16 01:10:18.947690 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:18.947605    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m56mh\" (UniqueName: \"kubernetes.io/projected/a5b85f15-bbd3-4beb-97f0-22767c9571b9-kube-api-access-m56mh\") pod \"payload-processing-e2e-shared-a-w1-f39e09-5d74949579-6jqsj\" (UID: \"a5b85f15-bbd3-4beb-97f0-22767c9571b9\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-f39e09-5d74949579-6jqsj"
Sep 16 01:10:19.023297 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:19.023222    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-f39e09-f5677f8cd-dj7jx"
Sep 16 01:10:19.024317 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:19.024298    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-f39e09-7cc574497b-z6vrc"
Sep 16 01:10:19.066647 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:19.066605    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-f39e09-5d74949579-6jqsj"
Sep 16 01:10:19.300339 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:19.300311    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-f39e09-f5677f8cd-dj7jx"]
Sep 16 01:10:19.302168 ip-10-0-130-207 kubenswrapper[2556]: W0916 01:10:19.302141    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44e35ee6_c1f9_4d6c_9584_2ebea688add8.slice/crio-6d3af6d8f27fa43f882e862548e5e08cb18c83c9b86ef457901793b60861db9a WatchSource:0}: Error finding container 6d3af6d8f27fa43f882e862548e5e08cb18c83c9b86ef457901793b60861db9a: Status 404 returned error can't find the container with id 6d3af6d8f27fa43f882e862548e5e08cb18c83c9b86ef457901793b60861db9a
Sep 16 01:10:19.302403 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:19.302381    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-f39e09-7cc574497b-z6vrc"]
Sep 16 01:10:19.303750 ip-10-0-130-207 kubenswrapper[2556]: W0916 01:10:19.303710    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26244697_ec2c_48f6_88f2_ce3e688e2add.slice/crio-f85ab960fa36650369f266d9120eba532dbff0d4d3e8936a1091fbb29567c929 WatchSource:0}: Error finding container f85ab960fa36650369f266d9120eba532dbff0d4d3e8936a1091fbb29567c929: Status 404 returned error can't find the container with id f85ab960fa36650369f266d9120eba532dbff0d4d3e8936a1091fbb29567c929
Sep 16 01:10:19.305233 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:19.305216    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 01:10:19.314499 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:19.314460    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-f39e09-7cc574497b-z6vrc" event={"ID":"26244697-ec2c-48f6-88f2-ce3e688e2add","Type":"ContainerStarted","Data":"f85ab960fa36650369f266d9120eba532dbff0d4d3e8936a1091fbb29567c929"}
Sep 16 01:10:19.315694 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:19.315670    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-f39e09-f5677f8cd-dj7jx" event={"ID":"44e35ee6-c1f9-4d6c-9584-2ebea688add8","Type":"ContainerStarted","Data":"6d3af6d8f27fa43f882e862548e5e08cb18c83c9b86ef457901793b60861db9a"}
Sep 16 01:10:19.351823 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:19.351799    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-f39e09-5d74949579-6jqsj"]
Sep 16 01:10:19.360462 ip-10-0-130-207 kubenswrapper[2556]: W0916 01:10:19.360434    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5b85f15_bbd3_4beb_97f0_22767c9571b9.slice/crio-db202156beca452ebd084477db26a3534fb781c6b89e0b07b87a1f360f2487a7 WatchSource:0}: Error finding container db202156beca452ebd084477db26a3534fb781c6b89e0b07b87a1f360f2487a7: Status 404 returned error can't find the container with id db202156beca452ebd084477db26a3534fb781c6b89e0b07b87a1f360f2487a7
Sep 16 01:10:20.322276 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:20.322236    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-f39e09-f5677f8cd-dj7jx" event={"ID":"44e35ee6-c1f9-4d6c-9584-2ebea688add8","Type":"ContainerStarted","Data":"72abc7cd856be4b09f34310e896c365f7ac2353015e0192333572b347ec74fbf"}
Sep 16 01:10:20.322800 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:20.322308    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-f39e09-f5677f8cd-dj7jx"
Sep 16 01:10:20.323990 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:20.323957    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-f39e09-5d74949579-6jqsj" event={"ID":"a5b85f15-bbd3-4beb-97f0-22767c9571b9","Type":"ContainerStarted","Data":"ecbb2718463c54a836bca39d91ffc2624232f5b75ce561f2aaedbc0f80c8d002"}
Sep 16 01:10:20.324120 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:20.324015    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-f39e09-5d74949579-6jqsj"
Sep 16 01:10:20.324120 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:20.324041    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-f39e09-5d74949579-6jqsj" event={"ID":"a5b85f15-bbd3-4beb-97f0-22767c9571b9","Type":"ContainerStarted","Data":"db202156beca452ebd084477db26a3534fb781c6b89e0b07b87a1f360f2487a7"}
Sep 16 01:10:20.325563 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:20.325539    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-f39e09-7cc574497b-z6vrc" event={"ID":"26244697-ec2c-48f6-88f2-ce3e688e2add","Type":"ContainerStarted","Data":"fddb083e53ae7fac4d715ff6b2bf7958e5cc6e87ad1a0db187442abe7ba0f79e"}
Sep 16 01:10:20.325686 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:20.325596    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-f39e09-7cc574497b-z6vrc"
Sep 16 01:10:20.339365 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:20.339320    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-f39e09-f5677f8cd-dj7jx" podStartSLOduration=2.339307078 podStartE2EDuration="2.339307078s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 01:10:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 01:10:20.337630452 +0000 UTC m=+1635.119132044" watchObservedRunningTime="2026-09-16 01:10:20.339307078 +0000 UTC m=+1635.120808661"
Sep 16 01:10:20.354850 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:20.354804    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-f39e09-7cc574497b-z6vrc" podStartSLOduration=1.9977521280000001 podStartE2EDuration="2.354784362s" totalImagesPullingTime="357.032234ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 01:10:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 01:10:20.352930476 +0000 UTC m=+1635.134432075" watchObservedRunningTime="2026-09-16 01:10:20.354784362 +0000 UTC m=+1635.136285958"
Sep 16 01:10:20.369027 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:20.368978    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-f39e09-5d74949579-6jqsj" podStartSLOduration=2.368959977 podStartE2EDuration="2.368959977s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 01:10:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 01:10:20.368063592 +0000 UTC m=+1635.149565185" watchObservedRunningTime="2026-09-16 01:10:20.368959977 +0000 UTC m=+1635.150461572"
Sep 16 01:10:26.336394 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:26.336365    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-f39e09-7cc574497b-z6vrc"
Sep 16 01:10:31.333100 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:31.333066    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-f39e09-f5677f8cd-dj7jx"
Sep 16 01:10:31.333540 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:31.333288    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-f39e09-5d74949579-6jqsj"
Sep 16 01:10:46.991927 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:46.991869    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-45446b-57cd5466fb-n7fcv"]
Sep 16 01:10:47.003850 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:47.003819    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-45446b-57cd5466fb-n7fcv"
Sep 16 01:10:47.009313 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:47.009284    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w1-45446b-serving-cert\""
Sep 16 01:10:47.011029 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:47.011003    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-45446b-57cd5466fb-n7fcv"]
Sep 16 01:10:47.086043 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:47.086010    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5b8d636a-0055-4b3b-92b2-6c0c22f75a2a-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-45446b-57cd5466fb-n7fcv\" (UID: \"5b8d636a-0055-4b3b-92b2-6c0c22f75a2a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-45446b-57cd5466fb-n7fcv"
Sep 16 01:10:47.086207 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:47.086053    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c676k\" (UniqueName: \"kubernetes.io/projected/5b8d636a-0055-4b3b-92b2-6c0c22f75a2a-kube-api-access-c676k\") pod \"maas-api-e2e-shared-b-w1-45446b-57cd5466fb-n7fcv\" (UID: \"5b8d636a-0055-4b3b-92b2-6c0c22f75a2a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-45446b-57cd5466fb-n7fcv"
Sep 16 01:10:47.086207 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:47.086072    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b8d636a-0055-4b3b-92b2-6c0c22f75a2a-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-45446b-57cd5466fb-n7fcv\" (UID: \"5b8d636a-0055-4b3b-92b2-6c0c22f75a2a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-45446b-57cd5466fb-n7fcv"
Sep 16 01:10:47.086207 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:47.086184    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b8d636a-0055-4b3b-92b2-6c0c22f75a2a-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-45446b-57cd5466fb-n7fcv\" (UID: \"5b8d636a-0055-4b3b-92b2-6c0c22f75a2a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-45446b-57cd5466fb-n7fcv"
Sep 16 01:10:47.187510 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:47.187459    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b8d636a-0055-4b3b-92b2-6c0c22f75a2a-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-45446b-57cd5466fb-n7fcv\" (UID: \"5b8d636a-0055-4b3b-92b2-6c0c22f75a2a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-45446b-57cd5466fb-n7fcv"
Sep 16 01:10:47.187692 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:47.187529    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5b8d636a-0055-4b3b-92b2-6c0c22f75a2a-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-45446b-57cd5466fb-n7fcv\" (UID: \"5b8d636a-0055-4b3b-92b2-6c0c22f75a2a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-45446b-57cd5466fb-n7fcv"
Sep 16 01:10:47.187692 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:47.187561    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c676k\" (UniqueName: \"kubernetes.io/projected/5b8d636a-0055-4b3b-92b2-6c0c22f75a2a-kube-api-access-c676k\") pod \"maas-api-e2e-shared-b-w1-45446b-57cd5466fb-n7fcv\" (UID: \"5b8d636a-0055-4b3b-92b2-6c0c22f75a2a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-45446b-57cd5466fb-n7fcv"
Sep 16 01:10:47.187692 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:47.187579    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b8d636a-0055-4b3b-92b2-6c0c22f75a2a-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-45446b-57cd5466fb-n7fcv\" (UID: \"5b8d636a-0055-4b3b-92b2-6c0c22f75a2a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-45446b-57cd5466fb-n7fcv"
Sep 16 01:10:47.187868 ip-10-0-130-207 kubenswrapper[2556]: E0916 01:10:47.187757    2556 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-45446b-serving-cert: secret "maas-api-e2e-shared-b-w1-45446b-serving-cert" not found
Sep 16 01:10:47.187868 ip-10-0-130-207 kubenswrapper[2556]: E0916 01:10:47.187842    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b8d636a-0055-4b3b-92b2-6c0c22f75a2a-maas-api-tls podName:5b8d636a-0055-4b3b-92b2-6c0c22f75a2a nodeName:}" failed. No retries permitted until 2026-09-16 01:10:47.68782051 +0000 UTC m=+1662.469322097 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5b8d636a-0055-4b3b-92b2-6c0c22f75a2a-maas-api-tls") pod "maas-api-e2e-shared-b-w1-45446b-57cd5466fb-n7fcv" (UID: "5b8d636a-0055-4b3b-92b2-6c0c22f75a2a") : secret "maas-api-e2e-shared-b-w1-45446b-serving-cert" not found
Sep 16 01:10:47.188122 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:47.188094    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b8d636a-0055-4b3b-92b2-6c0c22f75a2a-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-45446b-57cd5466fb-n7fcv\" (UID: \"5b8d636a-0055-4b3b-92b2-6c0c22f75a2a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-45446b-57cd5466fb-n7fcv"
Sep 16 01:10:47.189997 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:47.189980    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5b8d636a-0055-4b3b-92b2-6c0c22f75a2a-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-45446b-57cd5466fb-n7fcv\" (UID: \"5b8d636a-0055-4b3b-92b2-6c0c22f75a2a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-45446b-57cd5466fb-n7fcv"
Sep 16 01:10:47.197496 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:47.197458    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c676k\" (UniqueName: \"kubernetes.io/projected/5b8d636a-0055-4b3b-92b2-6c0c22f75a2a-kube-api-access-c676k\") pod \"maas-api-e2e-shared-b-w1-45446b-57cd5466fb-n7fcv\" (UID: \"5b8d636a-0055-4b3b-92b2-6c0c22f75a2a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-45446b-57cd5466fb-n7fcv"
Sep 16 01:10:47.692214 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:47.692185    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b8d636a-0055-4b3b-92b2-6c0c22f75a2a-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-45446b-57cd5466fb-n7fcv\" (UID: \"5b8d636a-0055-4b3b-92b2-6c0c22f75a2a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-45446b-57cd5466fb-n7fcv"
Sep 16 01:10:47.694409 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:47.694390    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b8d636a-0055-4b3b-92b2-6c0c22f75a2a-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-45446b-57cd5466fb-n7fcv\" (UID: \"5b8d636a-0055-4b3b-92b2-6c0c22f75a2a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-45446b-57cd5466fb-n7fcv"
Sep 16 01:10:47.714302 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:47.714275    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-45446b-5f77bf4fbc-92529"]
Sep 16 01:10:47.718674 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:47.718652    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-45446b-5f77bf4fbc-92529"
Sep 16 01:10:47.720890 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:47.720869    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w1-45446b-dockercfg-xqq6l\""
Sep 16 01:10:47.720991 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:47.720933    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w1-45446b\""
Sep 16 01:10:47.727961 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:47.727899    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-45446b-5f77bf4fbc-92529"]
Sep 16 01:10:47.792825 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:47.792789    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-px6b7\" (UniqueName: \"kubernetes.io/projected/758e8dfc-7ff0-4716-8e79-24e9c5db468e-kube-api-access-px6b7\") pod \"payload-processing-e2e-shared-b-w1-45446b-5f77bf4fbc-92529\" (UID: \"758e8dfc-7ff0-4716-8e79-24e9c5db468e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-45446b-5f77bf4fbc-92529"
Sep 16 01:10:47.793208 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:47.793032    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/758e8dfc-7ff0-4716-8e79-24e9c5db468e-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w1-45446b-5f77bf4fbc-92529\" (UID: \"758e8dfc-7ff0-4716-8e79-24e9c5db468e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-45446b-5f77bf4fbc-92529"
Sep 16 01:10:47.894295 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:47.894266    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/758e8dfc-7ff0-4716-8e79-24e9c5db468e-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w1-45446b-5f77bf4fbc-92529\" (UID: \"758e8dfc-7ff0-4716-8e79-24e9c5db468e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-45446b-5f77bf4fbc-92529"
Sep 16 01:10:47.894440 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:47.894303    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-px6b7\" (UniqueName: \"kubernetes.io/projected/758e8dfc-7ff0-4716-8e79-24e9c5db468e-kube-api-access-px6b7\") pod \"payload-processing-e2e-shared-b-w1-45446b-5f77bf4fbc-92529\" (UID: \"758e8dfc-7ff0-4716-8e79-24e9c5db468e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-45446b-5f77bf4fbc-92529"
Sep 16 01:10:47.894941 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:47.894921    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/758e8dfc-7ff0-4716-8e79-24e9c5db468e-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w1-45446b-5f77bf4fbc-92529\" (UID: \"758e8dfc-7ff0-4716-8e79-24e9c5db468e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-45446b-5f77bf4fbc-92529"
Sep 16 01:10:47.901850 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:47.901823    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-px6b7\" (UniqueName: \"kubernetes.io/projected/758e8dfc-7ff0-4716-8e79-24e9c5db468e-kube-api-access-px6b7\") pod \"payload-processing-e2e-shared-b-w1-45446b-5f77bf4fbc-92529\" (UID: \"758e8dfc-7ff0-4716-8e79-24e9c5db468e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-45446b-5f77bf4fbc-92529"
Sep 16 01:10:47.919284 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:47.919267    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-45446b-57cd5466fb-n7fcv"
Sep 16 01:10:48.031305 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:48.031273    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-45446b-5f77bf4fbc-92529"
Sep 16 01:10:48.190897 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:48.190874    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-45446b-57cd5466fb-n7fcv"]
Sep 16 01:10:48.192434 ip-10-0-130-207 kubenswrapper[2556]: W0916 01:10:48.192406    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b8d636a_0055_4b3b_92b2_6c0c22f75a2a.slice/crio-9e3336be8537c2e87b77b1ee4bce4bb6b08b92e596113301fcb86601dee236c7 WatchSource:0}: Error finding container 9e3336be8537c2e87b77b1ee4bce4bb6b08b92e596113301fcb86601dee236c7: Status 404 returned error can't find the container with id 9e3336be8537c2e87b77b1ee4bce4bb6b08b92e596113301fcb86601dee236c7
Sep 16 01:10:48.302676 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:48.302655    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-45446b-5f77bf4fbc-92529"]
Sep 16 01:10:48.304251 ip-10-0-130-207 kubenswrapper[2556]: W0916 01:10:48.304225    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod758e8dfc_7ff0_4716_8e79_24e9c5db468e.slice/crio-c8770d7d6a73972b3d44a523051376b28432c43f87478097d5bfa1b57887b091 WatchSource:0}: Error finding container c8770d7d6a73972b3d44a523051376b28432c43f87478097d5bfa1b57887b091: Status 404 returned error can't find the container with id c8770d7d6a73972b3d44a523051376b28432c43f87478097d5bfa1b57887b091
Sep 16 01:10:48.481914 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:48.481884    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-45446b-57cd5466fb-n7fcv" event={"ID":"5b8d636a-0055-4b3b-92b2-6c0c22f75a2a","Type":"ContainerStarted","Data":"9e3336be8537c2e87b77b1ee4bce4bb6b08b92e596113301fcb86601dee236c7"}
Sep 16 01:10:48.483427 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:48.483398    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-45446b-5f77bf4fbc-92529" event={"ID":"758e8dfc-7ff0-4716-8e79-24e9c5db468e","Type":"ContainerStarted","Data":"6ca6f2574aa24819c88c988ff70b3d42d12fcea065e6d29d7b75dd3cb6ddf80d"}
Sep 16 01:10:48.483586 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:48.483446    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-45446b-5f77bf4fbc-92529" event={"ID":"758e8dfc-7ff0-4716-8e79-24e9c5db468e","Type":"ContainerStarted","Data":"c8770d7d6a73972b3d44a523051376b28432c43f87478097d5bfa1b57887b091"}
Sep 16 01:10:48.483652 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:48.483590    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-45446b-5f77bf4fbc-92529"
Sep 16 01:10:48.500054 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:48.499967    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-45446b-5f77bf4fbc-92529" podStartSLOduration=1.499952022 podStartE2EDuration="1.499952022s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 01:10:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 01:10:48.499716596 +0000 UTC m=+1663.281218189" watchObservedRunningTime="2026-09-16 01:10:48.499952022 +0000 UTC m=+1663.281453614"
Sep 16 01:10:49.490563 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:49.490522    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-45446b-57cd5466fb-n7fcv" event={"ID":"5b8d636a-0055-4b3b-92b2-6c0c22f75a2a","Type":"ContainerStarted","Data":"49b6d86773af5e6dcae8fdd3fef387df4bbae67ac859447bdc678591eba082f6"}
Sep 16 01:10:49.491039 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:49.490749    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-45446b-57cd5466fb-n7fcv"
Sep 16 01:10:49.507877 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:49.507831    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-45446b-57cd5466fb-n7fcv" podStartSLOduration=3.175622585 podStartE2EDuration="3.507817211s" totalImagesPullingTime="332.194626ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 01:10:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 01:10:49.506019873 +0000 UTC m=+1664.287521465" watchObservedRunningTime="2026-09-16 01:10:49.507817211 +0000 UTC m=+1664.289318811"
Sep 16 01:10:55.501754 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:55.501720    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-45446b-57cd5466fb-n7fcv"
Sep 16 01:10:59.493289 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:10:59.493259    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-45446b-5f77bf4fbc-92529"
Sep 16 01:11:26.556198 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:11:26.556162    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e7d6db19-5cd4d57545-9t7zp"]
Sep 16 01:11:26.560783 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:11:26.560760    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e7d6db19-5cd4d57545-9t7zp"
Sep 16 01:11:26.562852 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:11:26.562828    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-e7d6db19-dockercfg-bfzt8\""
Sep 16 01:11:26.563008 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:11:26.562828    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-e7d6db19\""
Sep 16 01:11:26.568116 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:11:26.568055    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e7d6db19-5cd4d57545-9t7zp"]
Sep 16 01:11:26.607828 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:11:26.607792    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccwp2\" (UniqueName: \"kubernetes.io/projected/886427c5-00a7-4508-81f0-a61d1d451f33-kube-api-access-ccwp2\") pod \"payload-processing-e2e-mt-e7d6db19-5cd4d57545-9t7zp\" (UID: \"886427c5-00a7-4508-81f0-a61d1d451f33\") " pod="openshift-ingress/payload-processing-e2e-mt-e7d6db19-5cd4d57545-9t7zp"
Sep 16 01:11:26.607982 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:11:26.607851    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/886427c5-00a7-4508-81f0-a61d1d451f33-plugins-config-volume\") pod \"payload-processing-e2e-mt-e7d6db19-5cd4d57545-9t7zp\" (UID: \"886427c5-00a7-4508-81f0-a61d1d451f33\") " pod="openshift-ingress/payload-processing-e2e-mt-e7d6db19-5cd4d57545-9t7zp"
Sep 16 01:11:26.709392 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:11:26.709361    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccwp2\" (UniqueName: \"kubernetes.io/projected/886427c5-00a7-4508-81f0-a61d1d451f33-kube-api-access-ccwp2\") pod \"payload-processing-e2e-mt-e7d6db19-5cd4d57545-9t7zp\" (UID: \"886427c5-00a7-4508-81f0-a61d1d451f33\") " pod="openshift-ingress/payload-processing-e2e-mt-e7d6db19-5cd4d57545-9t7zp"
Sep 16 01:11:26.709574 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:11:26.709407    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/886427c5-00a7-4508-81f0-a61d1d451f33-plugins-config-volume\") pod \"payload-processing-e2e-mt-e7d6db19-5cd4d57545-9t7zp\" (UID: \"886427c5-00a7-4508-81f0-a61d1d451f33\") " pod="openshift-ingress/payload-processing-e2e-mt-e7d6db19-5cd4d57545-9t7zp"
Sep 16 01:11:26.710162 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:11:26.710141    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/886427c5-00a7-4508-81f0-a61d1d451f33-plugins-config-volume\") pod \"payload-processing-e2e-mt-e7d6db19-5cd4d57545-9t7zp\" (UID: \"886427c5-00a7-4508-81f0-a61d1d451f33\") " pod="openshift-ingress/payload-processing-e2e-mt-e7d6db19-5cd4d57545-9t7zp"
Sep 16 01:11:26.716921 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:11:26.716896    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccwp2\" (UniqueName: \"kubernetes.io/projected/886427c5-00a7-4508-81f0-a61d1d451f33-kube-api-access-ccwp2\") pod \"payload-processing-e2e-mt-e7d6db19-5cd4d57545-9t7zp\" (UID: \"886427c5-00a7-4508-81f0-a61d1d451f33\") " pod="openshift-ingress/payload-processing-e2e-mt-e7d6db19-5cd4d57545-9t7zp"
Sep 16 01:11:26.873140 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:11:26.873105    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e7d6db19-5cd4d57545-9t7zp"
Sep 16 01:11:27.143851 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:11:27.143818    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e7d6db19-5cd4d57545-9t7zp"]
Sep 16 01:11:27.144762 ip-10-0-130-207 kubenswrapper[2556]: W0916 01:11:27.144736    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod886427c5_00a7_4508_81f0_a61d1d451f33.slice/crio-21499a5d578bce81f07d5f27dfb023109c5e7f265d7cdaa6d884115c34def54c WatchSource:0}: Error finding container 21499a5d578bce81f07d5f27dfb023109c5e7f265d7cdaa6d884115c34def54c: Status 404 returned error can't find the container with id 21499a5d578bce81f07d5f27dfb023109c5e7f265d7cdaa6d884115c34def54c
Sep 16 01:11:27.698193 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:11:27.698161    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e7d6db19-5cd4d57545-9t7zp" event={"ID":"886427c5-00a7-4508-81f0-a61d1d451f33","Type":"ContainerStarted","Data":"74ba78cd2bdf3d0b5d07d254a251065cb61f45f4cfed90d5ddc6ee47515216a0"}
Sep 16 01:11:27.698642 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:11:27.698210    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e7d6db19-5cd4d57545-9t7zp" event={"ID":"886427c5-00a7-4508-81f0-a61d1d451f33","Type":"ContainerStarted","Data":"21499a5d578bce81f07d5f27dfb023109c5e7f265d7cdaa6d884115c34def54c"}
Sep 16 01:11:27.698642 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:11:27.698287    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-e7d6db19-5cd4d57545-9t7zp"
Sep 16 01:11:27.715454 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:11:27.715408    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-e7d6db19-5cd4d57545-9t7zp" podStartSLOduration=1.715394679 podStartE2EDuration="1.715394679s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 01:11:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 01:11:27.712252344 +0000 UTC m=+1702.493753936" watchObservedRunningTime="2026-09-16 01:11:27.715394679 +0000 UTC m=+1702.496896270"
Sep 16 01:11:38.706917 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:11:38.706885    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-e7d6db19-5cd4d57545-9t7zp"
Sep 16 01:12:54.071630 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:12:54.071594    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e7d6db19-5cd4d57545-9t7zp"]
Sep 16 01:12:54.072145 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:12:54.071851    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-e7d6db19-5cd4d57545-9t7zp" podUID="886427c5-00a7-4508-81f0-a61d1d451f33" containerName="payload-processing" containerID="cri-o://74ba78cd2bdf3d0b5d07d254a251065cb61f45f4cfed90d5ddc6ee47515216a0" gracePeriod=30
Sep 16 01:12:54.324599 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:12:54.324540    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e7d6db19-5cd4d57545-9t7zp"
Sep 16 01:12:54.425816 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:12:54.425779    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/886427c5-00a7-4508-81f0-a61d1d451f33-plugins-config-volume\") pod \"886427c5-00a7-4508-81f0-a61d1d451f33\" (UID: \"886427c5-00a7-4508-81f0-a61d1d451f33\") "
Sep 16 01:12:54.425990 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:12:54.425896    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ccwp2\" (UniqueName: \"kubernetes.io/projected/886427c5-00a7-4508-81f0-a61d1d451f33-kube-api-access-ccwp2\") pod \"886427c5-00a7-4508-81f0-a61d1d451f33\" (UID: \"886427c5-00a7-4508-81f0-a61d1d451f33\") "
Sep 16 01:12:54.426242 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:12:54.426196    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/886427c5-00a7-4508-81f0-a61d1d451f33-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "886427c5-00a7-4508-81f0-a61d1d451f33" (UID: "886427c5-00a7-4508-81f0-a61d1d451f33"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 01:12:54.428009 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:12:54.427977    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/886427c5-00a7-4508-81f0-a61d1d451f33-kube-api-access-ccwp2" (OuterVolumeSpecName: "kube-api-access-ccwp2") pod "886427c5-00a7-4508-81f0-a61d1d451f33" (UID: "886427c5-00a7-4508-81f0-a61d1d451f33"). InnerVolumeSpecName "kube-api-access-ccwp2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 01:12:54.527299 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:12:54.527271    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/886427c5-00a7-4508-81f0-a61d1d451f33-plugins-config-volume\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:12:54.527299 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:12:54.527298    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ccwp2\" (UniqueName: \"kubernetes.io/projected/886427c5-00a7-4508-81f0-a61d1d451f33-kube-api-access-ccwp2\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:12:55.184345 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:12:55.184311    2556 generic.go:356] "Generic (PLEG): container finished" podID="886427c5-00a7-4508-81f0-a61d1d451f33" containerID="74ba78cd2bdf3d0b5d07d254a251065cb61f45f4cfed90d5ddc6ee47515216a0" exitCode=0
Sep 16 01:12:55.184849 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:12:55.184360    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e7d6db19-5cd4d57545-9t7zp"
Sep 16 01:12:55.184849 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:12:55.184381    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e7d6db19-5cd4d57545-9t7zp" event={"ID":"886427c5-00a7-4508-81f0-a61d1d451f33","Type":"ContainerDied","Data":"74ba78cd2bdf3d0b5d07d254a251065cb61f45f4cfed90d5ddc6ee47515216a0"}
Sep 16 01:12:55.184849 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:12:55.184422    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e7d6db19-5cd4d57545-9t7zp" event={"ID":"886427c5-00a7-4508-81f0-a61d1d451f33","Type":"ContainerDied","Data":"21499a5d578bce81f07d5f27dfb023109c5e7f265d7cdaa6d884115c34def54c"}
Sep 16 01:12:55.184849 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:12:55.184442    2556 scope.go:117] "RemoveContainer" containerID="74ba78cd2bdf3d0b5d07d254a251065cb61f45f4cfed90d5ddc6ee47515216a0"
Sep 16 01:12:55.194286 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:12:55.194269    2556 scope.go:117] "RemoveContainer" containerID="74ba78cd2bdf3d0b5d07d254a251065cb61f45f4cfed90d5ddc6ee47515216a0"
Sep 16 01:12:55.194558 ip-10-0-130-207 kubenswrapper[2556]: E0916 01:12:55.194540    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"74ba78cd2bdf3d0b5d07d254a251065cb61f45f4cfed90d5ddc6ee47515216a0\": container with ID starting with 74ba78cd2bdf3d0b5d07d254a251065cb61f45f4cfed90d5ddc6ee47515216a0 not found: ID does not exist" containerID="74ba78cd2bdf3d0b5d07d254a251065cb61f45f4cfed90d5ddc6ee47515216a0"
Sep 16 01:12:55.194608 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:12:55.194566    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74ba78cd2bdf3d0b5d07d254a251065cb61f45f4cfed90d5ddc6ee47515216a0"} err="failed to get container status \"74ba78cd2bdf3d0b5d07d254a251065cb61f45f4cfed90d5ddc6ee47515216a0\": rpc error: code = NotFound desc = could not find container \"74ba78cd2bdf3d0b5d07d254a251065cb61f45f4cfed90d5ddc6ee47515216a0\": container with ID starting with 74ba78cd2bdf3d0b5d07d254a251065cb61f45f4cfed90d5ddc6ee47515216a0 not found: ID does not exist"
Sep 16 01:12:55.205393 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:12:55.205367    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e7d6db19-5cd4d57545-9t7zp"]
Sep 16 01:12:55.211062 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:12:55.211039    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e7d6db19-5cd4d57545-9t7zp"]
Sep 16 01:12:55.736677 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:12:55.736643    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="886427c5-00a7-4508-81f0-a61d1d451f33" path="/var/lib/kubelet/pods/886427c5-00a7-4508-81f0-a61d1d451f33/volumes"
Sep 16 01:13:17.010093 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:13:17.010061    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w2-f831c5/rate-test-model-f831c5-kserve-555c6d9b7f-hz5g4"]
Sep 16 01:13:17.010457 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:13:17.010422    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="886427c5-00a7-4508-81f0-a61d1d451f33" containerName="payload-processing"
Sep 16 01:13:17.010457 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:13:17.010433    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="886427c5-00a7-4508-81f0-a61d1d451f33" containerName="payload-processing"
Sep 16 01:13:17.010560 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:13:17.010522    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="886427c5-00a7-4508-81f0-a61d1d451f33" containerName="payload-processing"
Sep 16 01:13:17.013520 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:13:17.013503    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-f831c5/rate-test-model-f831c5-kserve-555c6d9b7f-hz5g4"
Sep 16 01:13:17.015901 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:13:17.015880    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w2-f831c5\"/\"openshift-service-ca.crt\""
Sep 16 01:13:17.016868 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:13:17.016843    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w2-f831c5\"/\"kube-root-ca.crt\""
Sep 16 01:13:17.016969 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:13:17.016884    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w2-f831c5\"/\"default-dockercfg-kqqjd\""
Sep 16 01:13:17.016969 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:13:17.016884    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w2-f831c5\"/\"rate-test-model-f831c5-kserve-self-signed-certs\""
Sep 16 01:13:17.023794 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:13:17.023773    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w2-f831c5/rate-test-model-f831c5-kserve-555c6d9b7f-hz5g4"]
Sep 16 01:13:17.107813 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:13:17.107785    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c6d3a8fc-6308-43dd-a600-58f588028bf1-dshm\") pod \"rate-test-model-f831c5-kserve-555c6d9b7f-hz5g4\" (UID: \"c6d3a8fc-6308-43dd-a600-58f588028bf1\") " pod="ai-tenant-e2e-shared-a-w2-f831c5/rate-test-model-f831c5-kserve-555c6d9b7f-hz5g4"
Sep 16 01:13:17.107917 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:13:17.107816    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxqz6\" (UniqueName: \"kubernetes.io/projected/c6d3a8fc-6308-43dd-a600-58f588028bf1-kube-api-access-kxqz6\") pod \"rate-test-model-f831c5-kserve-555c6d9b7f-hz5g4\" (UID: \"c6d3a8fc-6308-43dd-a600-58f588028bf1\") " pod="ai-tenant-e2e-shared-a-w2-f831c5/rate-test-model-f831c5-kserve-555c6d9b7f-hz5g4"
Sep 16 01:13:17.107917 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:13:17.107852    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c6d3a8fc-6308-43dd-a600-58f588028bf1-model-cache\") pod \"rate-test-model-f831c5-kserve-555c6d9b7f-hz5g4\" (UID: \"c6d3a8fc-6308-43dd-a600-58f588028bf1\") " pod="ai-tenant-e2e-shared-a-w2-f831c5/rate-test-model-f831c5-kserve-555c6d9b7f-hz5g4"
Sep 16 01:13:17.108004 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:13:17.107936    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c6d3a8fc-6308-43dd-a600-58f588028bf1-home\") pod \"rate-test-model-f831c5-kserve-555c6d9b7f-hz5g4\" (UID: \"c6d3a8fc-6308-43dd-a600-58f588028bf1\") " pod="ai-tenant-e2e-shared-a-w2-f831c5/rate-test-model-f831c5-kserve-555c6d9b7f-hz5g4"
Sep 16 01:13:17.108004 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:13:17.107966    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6d3a8fc-6308-43dd-a600-58f588028bf1-tmp-dir\") pod \"rate-test-model-f831c5-kserve-555c6d9b7f-hz5g4\" (UID: \"c6d3a8fc-6308-43dd-a600-58f588028bf1\") " pod="ai-tenant-e2e-shared-a-w2-f831c5/rate-test-model-f831c5-kserve-555c6d9b7f-hz5g4"
Sep 16 01:13:17.108079 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:13:17.108006    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6d3a8fc-6308-43dd-a600-58f588028bf1-tls-certs\") pod \"rate-test-model-f831c5-kserve-555c6d9b7f-hz5g4\" (UID: \"c6d3a8fc-6308-43dd-a600-58f588028bf1\") " pod="ai-tenant-e2e-shared-a-w2-f831c5/rate-test-model-f831c5-kserve-555c6d9b7f-hz5g4"
Sep 16 01:13:17.209761 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:13:17.209734    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c6d3a8fc-6308-43dd-a600-58f588028bf1-model-cache\") pod \"rate-test-model-f831c5-kserve-555c6d9b7f-hz5g4\" (UID: \"c6d3a8fc-6308-43dd-a600-58f588028bf1\") " pod="ai-tenant-e2e-shared-a-w2-f831c5/rate-test-model-f831c5-kserve-555c6d9b7f-hz5g4"
Sep 16 01:13:17.209871 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:13:17.209797    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c6d3a8fc-6308-43dd-a600-58f588028bf1-home\") pod \"rate-test-model-f831c5-kserve-555c6d9b7f-hz5g4\" (UID: \"c6d3a8fc-6308-43dd-a600-58f588028bf1\") " pod="ai-tenant-e2e-shared-a-w2-f831c5/rate-test-model-f831c5-kserve-555c6d9b7f-hz5g4"
Sep 16 01:13:17.209871 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:13:17.209830    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6d3a8fc-6308-43dd-a600-58f588028bf1-tmp-dir\") pod \"rate-test-model-f831c5-kserve-555c6d9b7f-hz5g4\" (UID: \"c6d3a8fc-6308-43dd-a600-58f588028bf1\") " pod="ai-tenant-e2e-shared-a-w2-f831c5/rate-test-model-f831c5-kserve-555c6d9b7f-hz5g4"
Sep 16 01:13:17.209949 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:13:17.209878    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6d3a8fc-6308-43dd-a600-58f588028bf1-tls-certs\") pod \"rate-test-model-f831c5-kserve-555c6d9b7f-hz5g4\" (UID: \"c6d3a8fc-6308-43dd-a600-58f588028bf1\") " pod="ai-tenant-e2e-shared-a-w2-f831c5/rate-test-model-f831c5-kserve-555c6d9b7f-hz5g4"
Sep 16 01:13:17.209949 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:13:17.209924    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c6d3a8fc-6308-43dd-a600-58f588028bf1-dshm\") pod \"rate-test-model-f831c5-kserve-555c6d9b7f-hz5g4\" (UID: \"c6d3a8fc-6308-43dd-a600-58f588028bf1\") " pod="ai-tenant-e2e-shared-a-w2-f831c5/rate-test-model-f831c5-kserve-555c6d9b7f-hz5g4"
Sep 16 01:13:17.210027 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:13:17.209951    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxqz6\" (UniqueName: \"kubernetes.io/projected/c6d3a8fc-6308-43dd-a600-58f588028bf1-kube-api-access-kxqz6\") pod \"rate-test-model-f831c5-kserve-555c6d9b7f-hz5g4\" (UID: \"c6d3a8fc-6308-43dd-a600-58f588028bf1\") " pod="ai-tenant-e2e-shared-a-w2-f831c5/rate-test-model-f831c5-kserve-555c6d9b7f-hz5g4"
Sep 16 01:13:17.210154 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:13:17.210135    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c6d3a8fc-6308-43dd-a600-58f588028bf1-model-cache\") pod \"rate-test-model-f831c5-kserve-555c6d9b7f-hz5g4\" (UID: \"c6d3a8fc-6308-43dd-a600-58f588028bf1\") " pod="ai-tenant-e2e-shared-a-w2-f831c5/rate-test-model-f831c5-kserve-555c6d9b7f-hz5g4"
Sep 16 01:13:17.210224 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:13:17.210183    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c6d3a8fc-6308-43dd-a600-58f588028bf1-home\") pod \"rate-test-model-f831c5-kserve-555c6d9b7f-hz5g4\" (UID: \"c6d3a8fc-6308-43dd-a600-58f588028bf1\") " pod="ai-tenant-e2e-shared-a-w2-f831c5/rate-test-model-f831c5-kserve-555c6d9b7f-hz5g4"
Sep 16 01:13:17.210305 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:13:17.210287    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6d3a8fc-6308-43dd-a600-58f588028bf1-tmp-dir\") pod \"rate-test-model-f831c5-kserve-555c6d9b7f-hz5g4\" (UID: \"c6d3a8fc-6308-43dd-a600-58f588028bf1\") " pod="ai-tenant-e2e-shared-a-w2-f831c5/rate-test-model-f831c5-kserve-555c6d9b7f-hz5g4"
Sep 16 01:13:17.212119 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:13:17.212096    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c6d3a8fc-6308-43dd-a600-58f588028bf1-dshm\") pod \"rate-test-model-f831c5-kserve-555c6d9b7f-hz5g4\" (UID: \"c6d3a8fc-6308-43dd-a600-58f588028bf1\") " pod="ai-tenant-e2e-shared-a-w2-f831c5/rate-test-model-f831c5-kserve-555c6d9b7f-hz5g4"
Sep 16 01:13:17.212306 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:13:17.212287    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6d3a8fc-6308-43dd-a600-58f588028bf1-tls-certs\") pod \"rate-test-model-f831c5-kserve-555c6d9b7f-hz5g4\" (UID: \"c6d3a8fc-6308-43dd-a600-58f588028bf1\") " pod="ai-tenant-e2e-shared-a-w2-f831c5/rate-test-model-f831c5-kserve-555c6d9b7f-hz5g4"
Sep 16 01:13:17.216724 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:13:17.216703    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxqz6\" (UniqueName: \"kubernetes.io/projected/c6d3a8fc-6308-43dd-a600-58f588028bf1-kube-api-access-kxqz6\") pod \"rate-test-model-f831c5-kserve-555c6d9b7f-hz5g4\" (UID: \"c6d3a8fc-6308-43dd-a600-58f588028bf1\") " pod="ai-tenant-e2e-shared-a-w2-f831c5/rate-test-model-f831c5-kserve-555c6d9b7f-hz5g4"
Sep 16 01:13:17.323710 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:13:17.323650    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-f831c5/rate-test-model-f831c5-kserve-555c6d9b7f-hz5g4"
Sep 16 01:13:17.592055 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:13:17.592028    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w2-f831c5/rate-test-model-f831c5-kserve-555c6d9b7f-hz5g4"]
Sep 16 01:13:17.594409 ip-10-0-130-207 kubenswrapper[2556]: W0916 01:13:17.594382    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc6d3a8fc_6308_43dd_a600_58f588028bf1.slice/crio-47e035efe998799b9540f3bf26463446184afe86429e049185f91fc744ad0b96 WatchSource:0}: Error finding container 47e035efe998799b9540f3bf26463446184afe86429e049185f91fc744ad0b96: Status 404 returned error can't find the container with id 47e035efe998799b9540f3bf26463446184afe86429e049185f91fc744ad0b96
Sep 16 01:13:18.309053 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:13:18.309018    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-f831c5/rate-test-model-f831c5-kserve-555c6d9b7f-hz5g4" event={"ID":"c6d3a8fc-6308-43dd-a600-58f588028bf1","Type":"ContainerStarted","Data":"f047e685c417b84c03def8f7c4a3ba02c50dde79aa1d4a262a7099fd582c7c2c"}
Sep 16 01:13:18.309532 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:13:18.309064    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-f831c5/rate-test-model-f831c5-kserve-555c6d9b7f-hz5g4" event={"ID":"c6d3a8fc-6308-43dd-a600-58f588028bf1","Type":"ContainerStarted","Data":"47e035efe998799b9540f3bf26463446184afe86429e049185f91fc744ad0b96"}
Sep 16 01:13:18.324796 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:13:18.324750    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w2-f831c5/rate-test-model-f831c5-kserve-555c6d9b7f-hz5g4" podStartSLOduration=2.324737185 podStartE2EDuration="2.324737185s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 01:13:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 01:13:18.323886363 +0000 UTC m=+1813.105387954" watchObservedRunningTime="2026-09-16 01:13:18.324737185 +0000 UTC m=+1813.106238804"
Sep 16 01:13:27.324639 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:13:27.324608    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w2-f831c5/rate-test-model-f831c5-kserve-555c6d9b7f-hz5g4"
Sep 16 01:13:27.325009 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:13:27.324649    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w2-f831c5/rate-test-model-f831c5-kserve-555c6d9b7f-hz5g4"
Sep 16 01:13:27.336877 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:13:27.336852    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w2-f831c5/rate-test-model-f831c5-kserve-555c6d9b7f-hz5g4"
Sep 16 01:13:27.367326 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:13:27.367301    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w2-f831c5/rate-test-model-f831c5-kserve-555c6d9b7f-hz5g4"
Sep 16 01:14:33.188807 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:14:33.188774    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w2-f831c5/rate-test-model-f831c5-kserve-555c6d9b7f-hz5g4"]
Sep 16 01:14:48.208004 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:14:48.207937    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w2-f831c5/rate-test-model-f831c5-kserve-555c6d9b7f-hz5g4" podUID="c6d3a8fc-6308-43dd-a600-58f588028bf1" containerName="main" containerID="cri-o://f047e685c417b84c03def8f7c4a3ba02c50dde79aa1d4a262a7099fd582c7c2c" gracePeriod=45
Sep 16 01:14:48.457777 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:14:48.457754    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-f831c5/rate-test-model-f831c5-kserve-555c6d9b7f-hz5g4"
Sep 16 01:14:48.550402 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:14:48.550333    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6d3a8fc-6308-43dd-a600-58f588028bf1-tls-certs\") pod \"c6d3a8fc-6308-43dd-a600-58f588028bf1\" (UID: \"c6d3a8fc-6308-43dd-a600-58f588028bf1\") "
Sep 16 01:14:48.550402 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:14:48.550374    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c6d3a8fc-6308-43dd-a600-58f588028bf1-dshm\") pod \"c6d3a8fc-6308-43dd-a600-58f588028bf1\" (UID: \"c6d3a8fc-6308-43dd-a600-58f588028bf1\") "
Sep 16 01:14:48.550642 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:14:48.550410    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c6d3a8fc-6308-43dd-a600-58f588028bf1-model-cache\") pod \"c6d3a8fc-6308-43dd-a600-58f588028bf1\" (UID: \"c6d3a8fc-6308-43dd-a600-58f588028bf1\") "
Sep 16 01:14:48.550642 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:14:48.550442    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kxqz6\" (UniqueName: \"kubernetes.io/projected/c6d3a8fc-6308-43dd-a600-58f588028bf1-kube-api-access-kxqz6\") pod \"c6d3a8fc-6308-43dd-a600-58f588028bf1\" (UID: \"c6d3a8fc-6308-43dd-a600-58f588028bf1\") "
Sep 16 01:14:48.550642 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:14:48.550487    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c6d3a8fc-6308-43dd-a600-58f588028bf1-home\") pod \"c6d3a8fc-6308-43dd-a600-58f588028bf1\" (UID: \"c6d3a8fc-6308-43dd-a600-58f588028bf1\") "
Sep 16 01:14:48.550832 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:14:48.550741    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6d3a8fc-6308-43dd-a600-58f588028bf1-home" (OuterVolumeSpecName: "home") pod "c6d3a8fc-6308-43dd-a600-58f588028bf1" (UID: "c6d3a8fc-6308-43dd-a600-58f588028bf1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 01:14:48.550832 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:14:48.550758    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6d3a8fc-6308-43dd-a600-58f588028bf1-model-cache" (OuterVolumeSpecName: "model-cache") pod "c6d3a8fc-6308-43dd-a600-58f588028bf1" (UID: "c6d3a8fc-6308-43dd-a600-58f588028bf1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 01:14:48.550832 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:14:48.550521    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6d3a8fc-6308-43dd-a600-58f588028bf1-tmp-dir\") pod \"c6d3a8fc-6308-43dd-a600-58f588028bf1\" (UID: \"c6d3a8fc-6308-43dd-a600-58f588028bf1\") "
Sep 16 01:14:48.551394 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:14:48.551364    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6d3a8fc-6308-43dd-a600-58f588028bf1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c6d3a8fc-6308-43dd-a600-58f588028bf1" (UID: "c6d3a8fc-6308-43dd-a600-58f588028bf1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 01:14:48.551583 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:14:48.551548    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c6d3a8fc-6308-43dd-a600-58f588028bf1-model-cache\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:14:48.551672 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:14:48.551590    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c6d3a8fc-6308-43dd-a600-58f588028bf1-home\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:14:48.551672 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:14:48.551605    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6d3a8fc-6308-43dd-a600-58f588028bf1-tmp-dir\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:14:48.552831 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:14:48.552800    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6d3a8fc-6308-43dd-a600-58f588028bf1-kube-api-access-kxqz6" (OuterVolumeSpecName: "kube-api-access-kxqz6") pod "c6d3a8fc-6308-43dd-a600-58f588028bf1" (UID: "c6d3a8fc-6308-43dd-a600-58f588028bf1"). InnerVolumeSpecName "kube-api-access-kxqz6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 01:14:48.552929 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:14:48.552836    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6d3a8fc-6308-43dd-a600-58f588028bf1-dshm" (OuterVolumeSpecName: "dshm") pod "c6d3a8fc-6308-43dd-a600-58f588028bf1" (UID: "c6d3a8fc-6308-43dd-a600-58f588028bf1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 01:14:48.552929 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:14:48.552901    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6d3a8fc-6308-43dd-a600-58f588028bf1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c6d3a8fc-6308-43dd-a600-58f588028bf1" (UID: "c6d3a8fc-6308-43dd-a600-58f588028bf1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 01:14:48.652553 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:14:48.652529    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6d3a8fc-6308-43dd-a600-58f588028bf1-tls-certs\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:14:48.652553 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:14:48.652552    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c6d3a8fc-6308-43dd-a600-58f588028bf1-dshm\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:14:48.652721 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:14:48.652561    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kxqz6\" (UniqueName: \"kubernetes.io/projected/c6d3a8fc-6308-43dd-a600-58f588028bf1-kube-api-access-kxqz6\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:14:48.783017 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:14:48.782986    2556 generic.go:356] "Generic (PLEG): container finished" podID="c6d3a8fc-6308-43dd-a600-58f588028bf1" containerID="f047e685c417b84c03def8f7c4a3ba02c50dde79aa1d4a262a7099fd582c7c2c" exitCode=0
Sep 16 01:14:48.783177 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:14:48.783034    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-f831c5/rate-test-model-f831c5-kserve-555c6d9b7f-hz5g4"
Sep 16 01:14:48.783177 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:14:48.783069    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-f831c5/rate-test-model-f831c5-kserve-555c6d9b7f-hz5g4" event={"ID":"c6d3a8fc-6308-43dd-a600-58f588028bf1","Type":"ContainerDied","Data":"f047e685c417b84c03def8f7c4a3ba02c50dde79aa1d4a262a7099fd582c7c2c"}
Sep 16 01:14:48.783177 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:14:48.783110    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-f831c5/rate-test-model-f831c5-kserve-555c6d9b7f-hz5g4" event={"ID":"c6d3a8fc-6308-43dd-a600-58f588028bf1","Type":"ContainerDied","Data":"47e035efe998799b9540f3bf26463446184afe86429e049185f91fc744ad0b96"}
Sep 16 01:14:48.783177 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:14:48.783128    2556 scope.go:117] "RemoveContainer" containerID="f047e685c417b84c03def8f7c4a3ba02c50dde79aa1d4a262a7099fd582c7c2c"
Sep 16 01:14:48.796568 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:14:48.796503    2556 scope.go:117] "RemoveContainer" containerID="f047e685c417b84c03def8f7c4a3ba02c50dde79aa1d4a262a7099fd582c7c2c"
Sep 16 01:14:48.797179 ip-10-0-130-207 kubenswrapper[2556]: E0916 01:14:48.797152    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f047e685c417b84c03def8f7c4a3ba02c50dde79aa1d4a262a7099fd582c7c2c\": container with ID starting with f047e685c417b84c03def8f7c4a3ba02c50dde79aa1d4a262a7099fd582c7c2c not found: ID does not exist" containerID="f047e685c417b84c03def8f7c4a3ba02c50dde79aa1d4a262a7099fd582c7c2c"
Sep 16 01:14:48.797292 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:14:48.797193    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f047e685c417b84c03def8f7c4a3ba02c50dde79aa1d4a262a7099fd582c7c2c"} err="failed to get container status \"f047e685c417b84c03def8f7c4a3ba02c50dde79aa1d4a262a7099fd582c7c2c\": rpc error: code = NotFound desc = could not find container \"f047e685c417b84c03def8f7c4a3ba02c50dde79aa1d4a262a7099fd582c7c2c\": container with ID starting with f047e685c417b84c03def8f7c4a3ba02c50dde79aa1d4a262a7099fd582c7c2c not found: ID does not exist"
Sep 16 01:14:48.811445 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:14:48.811395    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w2-f831c5/rate-test-model-f831c5-kserve-555c6d9b7f-hz5g4"]
Sep 16 01:14:48.817713 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:14:48.817691    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w2-f831c5/rate-test-model-f831c5-kserve-555c6d9b7f-hz5g4"]
Sep 16 01:14:49.737136 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:14:49.737100    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6d3a8fc-6308-43dd-a600-58f588028bf1" path="/var/lib/kubelet/pods/c6d3a8fc-6308-43dd-a600-58f588028bf1/volumes"
Sep 16 01:14:52.916002 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:14:52.915970    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ad13e5-59468867bf-xjs9r"]
Sep 16 01:14:52.916345 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:14:52.916335    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6d3a8fc-6308-43dd-a600-58f588028bf1" containerName="main"
Sep 16 01:14:52.916387 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:14:52.916347    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6d3a8fc-6308-43dd-a600-58f588028bf1" containerName="main"
Sep 16 01:14:52.916435 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:14:52.916425    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6d3a8fc-6308-43dd-a600-58f588028bf1" containerName="main"
Sep 16 01:14:52.920750 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:14:52.920733    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-ad13e5-59468867bf-xjs9r"
Sep 16 01:14:52.923072 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:14:52.923040    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-ad13e5-dockercfg-w5hvl\""
Sep 16 01:14:52.923216 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:14:52.923048    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-ad13e5\""
Sep 16 01:14:52.933064 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:14:52.933029    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ad13e5-59468867bf-xjs9r"]
Sep 16 01:14:52.991324 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:14:52.991288    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa3f86c2-866b-416c-b3da-c31749d4b66f-plugins-config-volume\") pod \"payload-processing-e2e-api-a-ad13e5-59468867bf-xjs9r\" (UID: \"fa3f86c2-866b-416c-b3da-c31749d4b66f\") " pod="openshift-ingress/payload-processing-e2e-api-a-ad13e5-59468867bf-xjs9r"
Sep 16 01:14:52.991492 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:14:52.991367    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kddpl\" (UniqueName: \"kubernetes.io/projected/fa3f86c2-866b-416c-b3da-c31749d4b66f-kube-api-access-kddpl\") pod \"payload-processing-e2e-api-a-ad13e5-59468867bf-xjs9r\" (UID: \"fa3f86c2-866b-416c-b3da-c31749d4b66f\") " pod="openshift-ingress/payload-processing-e2e-api-a-ad13e5-59468867bf-xjs9r"
Sep 16 01:14:53.091876 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:14:53.091836    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa3f86c2-866b-416c-b3da-c31749d4b66f-plugins-config-volume\") pod \"payload-processing-e2e-api-a-ad13e5-59468867bf-xjs9r\" (UID: \"fa3f86c2-866b-416c-b3da-c31749d4b66f\") " pod="openshift-ingress/payload-processing-e2e-api-a-ad13e5-59468867bf-xjs9r"
Sep 16 01:14:53.092055 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:14:53.091898    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kddpl\" (UniqueName: \"kubernetes.io/projected/fa3f86c2-866b-416c-b3da-c31749d4b66f-kube-api-access-kddpl\") pod \"payload-processing-e2e-api-a-ad13e5-59468867bf-xjs9r\" (UID: \"fa3f86c2-866b-416c-b3da-c31749d4b66f\") " pod="openshift-ingress/payload-processing-e2e-api-a-ad13e5-59468867bf-xjs9r"
Sep 16 01:14:53.092557 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:14:53.092528    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa3f86c2-866b-416c-b3da-c31749d4b66f-plugins-config-volume\") pod \"payload-processing-e2e-api-a-ad13e5-59468867bf-xjs9r\" (UID: \"fa3f86c2-866b-416c-b3da-c31749d4b66f\") " pod="openshift-ingress/payload-processing-e2e-api-a-ad13e5-59468867bf-xjs9r"
Sep 16 01:14:53.098706 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:14:53.098678    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kddpl\" (UniqueName: \"kubernetes.io/projected/fa3f86c2-866b-416c-b3da-c31749d4b66f-kube-api-access-kddpl\") pod \"payload-processing-e2e-api-a-ad13e5-59468867bf-xjs9r\" (UID: \"fa3f86c2-866b-416c-b3da-c31749d4b66f\") " pod="openshift-ingress/payload-processing-e2e-api-a-ad13e5-59468867bf-xjs9r"
Sep 16 01:14:53.232611 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:14:53.232543    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-ad13e5-59468867bf-xjs9r"
Sep 16 01:14:53.500375 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:14:53.500347    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ad13e5-59468867bf-xjs9r"]
Sep 16 01:14:53.501893 ip-10-0-130-207 kubenswrapper[2556]: W0916 01:14:53.501854    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa3f86c2_866b_416c_b3da_c31749d4b66f.slice/crio-f8c17996e91897420e7aeebef08f7e832cec4d3d75ee00ce7a0b3eb663bcc331 WatchSource:0}: Error finding container f8c17996e91897420e7aeebef08f7e832cec4d3d75ee00ce7a0b3eb663bcc331: Status 404 returned error can't find the container with id f8c17996e91897420e7aeebef08f7e832cec4d3d75ee00ce7a0b3eb663bcc331
Sep 16 01:14:53.811863 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:14:53.811769    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-ad13e5-59468867bf-xjs9r" event={"ID":"fa3f86c2-866b-416c-b3da-c31749d4b66f","Type":"ContainerStarted","Data":"80f268240d8df2baef21e08207b81a9967bb0a6040f0f33abc8ee32aa7bdbed1"}
Sep 16 01:14:53.811863 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:14:53.811825    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-ad13e5-59468867bf-xjs9r" event={"ID":"fa3f86c2-866b-416c-b3da-c31749d4b66f","Type":"ContainerStarted","Data":"f8c17996e91897420e7aeebef08f7e832cec4d3d75ee00ce7a0b3eb663bcc331"}
Sep 16 01:14:53.812051 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:14:53.811862    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-ad13e5-59468867bf-xjs9r"
Sep 16 01:14:53.831315 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:14:53.831259    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-ad13e5-59468867bf-xjs9r" podStartSLOduration=1.831239843 podStartE2EDuration="1.831239843s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 01:14:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 01:14:53.827628873 +0000 UTC m=+1908.609130468" watchObservedRunningTime="2026-09-16 01:14:53.831239843 +0000 UTC m=+1908.612741438"
Sep 16 01:15:00.136648 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:00.136613    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-f831c5-29825355-9k4fb"]
Sep 16 01:15:00.140092 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:00.140075    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-f831c5-29825355-9k4fb"
Sep 16 01:15:00.142255 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:00.142237    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-j6blf\""
Sep 16 01:15:00.151723 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:00.151697    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-f831c5-29825355-9k4fb"]
Sep 16 01:15:00.151964 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:00.151823    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-f831c5-29825355-9k4fb"
Sep 16 01:15:00.157630 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:00.157608    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-ad13e5-29825355-xtfzt"]
Sep 16 01:15:00.161162 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:00.161141    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-ad13e5-29825355-xtfzt"
Sep 16 01:15:00.171617 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:00.171591    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-ad13e5-29825355-xtfzt"]
Sep 16 01:15:00.191884 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:00.191862    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-ad13e5-29825355-xtfzt"
Sep 16 01:15:00.436542 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:00.436515    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-f831c5-29825355-9k4fb"]
Sep 16 01:15:00.437502 ip-10-0-130-207 kubenswrapper[2556]: W0916 01:15:00.437452    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb97fa55f_e116_4e0f_9c9b_c2b3e7ca33ae.slice/crio-abcc5ec24c9d34a20c99e71e89ff39627a2f3f24593bc0cefaf6047b156d96d0 WatchSource:0}: Error finding container abcc5ec24c9d34a20c99e71e89ff39627a2f3f24593bc0cefaf6047b156d96d0: Status 404 returned error can't find the container with id abcc5ec24c9d34a20c99e71e89ff39627a2f3f24593bc0cefaf6047b156d96d0
Sep 16 01:15:00.471045 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:00.471022    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-ad13e5-29825355-xtfzt"]
Sep 16 01:15:00.471683 ip-10-0-130-207 kubenswrapper[2556]: W0916 01:15:00.471658    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2da8bdd6_c774_4b2c_a217_72fe60148040.slice/crio-e98e02a6e370319a38346922fe0cb952adfd3473d1cf72762e7947ea384c0bd5 WatchSource:0}: Error finding container e98e02a6e370319a38346922fe0cb952adfd3473d1cf72762e7947ea384c0bd5: Status 404 returned error can't find the container with id e98e02a6e370319a38346922fe0cb952adfd3473d1cf72762e7947ea384c0bd5
Sep 16 01:15:00.852558 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:00.852506    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-f831c5-29825355-9k4fb" event={"ID":"b97fa55f-e116-4e0f-9c9b-c2b3e7ca33ae","Type":"ContainerStarted","Data":"abcc5ec24c9d34a20c99e71e89ff39627a2f3f24593bc0cefaf6047b156d96d0"}
Sep 16 01:15:00.853714 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:00.853690    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-ad13e5-29825355-xtfzt" event={"ID":"2da8bdd6-c774-4b2c-a217-72fe60148040","Type":"ContainerStarted","Data":"e98e02a6e370319a38346922fe0cb952adfd3473d1cf72762e7947ea384c0bd5"}
Sep 16 01:15:03.881222 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:03.881191    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-f831c5-29825355-9k4fb" event={"ID":"b97fa55f-e116-4e0f-9c9b-c2b3e7ca33ae","Type":"ContainerStarted","Data":"41abf85c3ce748cfd9be27c577bcb9fa9d85cf86149497c72d206f1605780cae"}
Sep 16 01:15:03.896645 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:03.896604    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-f831c5-29825355-9k4fb" podStartSLOduration=0.839523353 podStartE2EDuration="3.896588447s" totalImagesPullingTime="3.057065094s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 01:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 01:15:03.894021111 +0000 UTC m=+1918.675522702" watchObservedRunningTime="2026-09-16 01:15:03.896588447 +0000 UTC m=+1918.678090041"
Sep 16 01:15:04.820685 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:04.820605    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-ad13e5-59468867bf-xjs9r"
Sep 16 01:15:04.889129 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:04.889094    2556 generic.go:356] "Generic (PLEG): container finished" podID="b97fa55f-e116-4e0f-9c9b-c2b3e7ca33ae" containerID="41abf85c3ce748cfd9be27c577bcb9fa9d85cf86149497c72d206f1605780cae" exitCode=0
Sep 16 01:15:04.889584 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:04.889203    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-f831c5-29825355-9k4fb" event={"ID":"b97fa55f-e116-4e0f-9c9b-c2b3e7ca33ae","Type":"ContainerDied","Data":"41abf85c3ce748cfd9be27c577bcb9fa9d85cf86149497c72d206f1605780cae"}
Sep 16 01:15:04.891015 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:04.890989    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-ad13e5-29825355-xtfzt" event={"ID":"2da8bdd6-c774-4b2c-a217-72fe60148040","Type":"ContainerStarted","Data":"8acc756a82178ac395d8e36308bad74199ae628bc4b40c0ba1b8f4556e47cbc7"}
Sep 16 01:15:04.919919 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:04.919871    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-ad13e5-29825355-xtfzt" podStartSLOduration=1.2529424900000001 podStartE2EDuration="4.919858431s" totalImagesPullingTime="3.666915941s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 01:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 01:15:04.917598168 +0000 UTC m=+1919.699099761" watchObservedRunningTime="2026-09-16 01:15:04.919858431 +0000 UTC m=+1919.701360049"
Sep 16 01:15:05.897519 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:05.897461    2556 generic.go:356] "Generic (PLEG): container finished" podID="2da8bdd6-c774-4b2c-a217-72fe60148040" containerID="8acc756a82178ac395d8e36308bad74199ae628bc4b40c0ba1b8f4556e47cbc7" exitCode=0
Sep 16 01:15:05.897938 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:05.897537    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-ad13e5-29825355-xtfzt" event={"ID":"2da8bdd6-c774-4b2c-a217-72fe60148040","Type":"ContainerDied","Data":"8acc756a82178ac395d8e36308bad74199ae628bc4b40c0ba1b8f4556e47cbc7"}
Sep 16 01:15:06.025429 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:06.025403    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-f831c5-29825355-9k4fb"
Sep 16 01:15:06.904576 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:06.904543    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-f831c5-29825355-9k4fb"
Sep 16 01:15:06.904954 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:06.904584    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-f831c5-29825355-9k4fb" event={"ID":"b97fa55f-e116-4e0f-9c9b-c2b3e7ca33ae","Type":"ContainerDied","Data":"abcc5ec24c9d34a20c99e71e89ff39627a2f3f24593bc0cefaf6047b156d96d0"}
Sep 16 01:15:06.904954 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:06.904630    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="abcc5ec24c9d34a20c99e71e89ff39627a2f3f24593bc0cefaf6047b156d96d0"
Sep 16 01:15:07.051631 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:07.051605    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-ad13e5-29825355-xtfzt"
Sep 16 01:15:07.577624 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:07.577578    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-195f51-5cc698659d-x8lzl"]
Sep 16 01:15:07.578005 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:07.577993    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b97fa55f-e116-4e0f-9c9b-c2b3e7ca33ae" containerName="cleanup"
Sep 16 01:15:07.578051 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:07.578007    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="b97fa55f-e116-4e0f-9c9b-c2b3e7ca33ae" containerName="cleanup"
Sep 16 01:15:07.578051 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:07.578028    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2da8bdd6-c774-4b2c-a217-72fe60148040" containerName="cleanup"
Sep 16 01:15:07.578051 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:07.578035    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="2da8bdd6-c774-4b2c-a217-72fe60148040" containerName="cleanup"
Sep 16 01:15:07.578150 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:07.578104    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="b97fa55f-e116-4e0f-9c9b-c2b3e7ca33ae" containerName="cleanup"
Sep 16 01:15:07.578150 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:07.578115    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="2da8bdd6-c774-4b2c-a217-72fe60148040" containerName="cleanup"
Sep 16 01:15:07.581122 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:07.581106    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-195f51-5cc698659d-x8lzl"
Sep 16 01:15:07.583138 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:07.583118    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-195f51-dockercfg-c56kw\""
Sep 16 01:15:07.583238 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:07.583162    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-195f51\""
Sep 16 01:15:07.588231 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:07.588207    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-195f51-5cc698659d-x8lzl"]
Sep 16 01:15:07.616979 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:07.616948    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-195f51-6c8f7975cc-gzrj6"]
Sep 16 01:15:07.620506 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:07.620488    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-195f51-6c8f7975cc-gzrj6"
Sep 16 01:15:07.630250 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:07.630221    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-195f51-6c8f7975cc-gzrj6"]
Sep 16 01:15:07.726584 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:07.726544    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbvfb\" (UniqueName: \"kubernetes.io/projected/ec919b37-44db-4ef5-90c9-b2791022b16b-kube-api-access-sbvfb\") pod \"payload-pre-processing-e2e-ipp-b-195f51-5cc698659d-x8lzl\" (UID: \"ec919b37-44db-4ef5-90c9-b2791022b16b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-195f51-5cc698659d-x8lzl"
Sep 16 01:15:07.726773 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:07.726671    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57v5p\" (UniqueName: \"kubernetes.io/projected/f7f9620c-309a-4e52-a3c5-0f7a4dfb7fed-kube-api-access-57v5p\") pod \"payload-processing-e2e-ipp-b-195f51-6c8f7975cc-gzrj6\" (UID: \"f7f9620c-309a-4e52-a3c5-0f7a4dfb7fed\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-195f51-6c8f7975cc-gzrj6"
Sep 16 01:15:07.726773 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:07.726762    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec919b37-44db-4ef5-90c9-b2791022b16b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-195f51-5cc698659d-x8lzl\" (UID: \"ec919b37-44db-4ef5-90c9-b2791022b16b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-195f51-5cc698659d-x8lzl"
Sep 16 01:15:07.726898 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:07.726817    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7f9620c-309a-4e52-a3c5-0f7a4dfb7fed-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-195f51-6c8f7975cc-gzrj6\" (UID: \"f7f9620c-309a-4e52-a3c5-0f7a4dfb7fed\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-195f51-6c8f7975cc-gzrj6"
Sep 16 01:15:07.827297 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:07.827262    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7f9620c-309a-4e52-a3c5-0f7a4dfb7fed-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-195f51-6c8f7975cc-gzrj6\" (UID: \"f7f9620c-309a-4e52-a3c5-0f7a4dfb7fed\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-195f51-6c8f7975cc-gzrj6"
Sep 16 01:15:07.827510 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:07.827322    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbvfb\" (UniqueName: \"kubernetes.io/projected/ec919b37-44db-4ef5-90c9-b2791022b16b-kube-api-access-sbvfb\") pod \"payload-pre-processing-e2e-ipp-b-195f51-5cc698659d-x8lzl\" (UID: \"ec919b37-44db-4ef5-90c9-b2791022b16b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-195f51-5cc698659d-x8lzl"
Sep 16 01:15:07.827510 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:07.827352    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57v5p\" (UniqueName: \"kubernetes.io/projected/f7f9620c-309a-4e52-a3c5-0f7a4dfb7fed-kube-api-access-57v5p\") pod \"payload-processing-e2e-ipp-b-195f51-6c8f7975cc-gzrj6\" (UID: \"f7f9620c-309a-4e52-a3c5-0f7a4dfb7fed\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-195f51-6c8f7975cc-gzrj6"
Sep 16 01:15:07.827510 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:07.827386    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec919b37-44db-4ef5-90c9-b2791022b16b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-195f51-5cc698659d-x8lzl\" (UID: \"ec919b37-44db-4ef5-90c9-b2791022b16b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-195f51-5cc698659d-x8lzl"
Sep 16 01:15:07.828048 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:07.827999    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7f9620c-309a-4e52-a3c5-0f7a4dfb7fed-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-195f51-6c8f7975cc-gzrj6\" (UID: \"f7f9620c-309a-4e52-a3c5-0f7a4dfb7fed\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-195f51-6c8f7975cc-gzrj6"
Sep 16 01:15:07.828186 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:07.828165    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec919b37-44db-4ef5-90c9-b2791022b16b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-195f51-5cc698659d-x8lzl\" (UID: \"ec919b37-44db-4ef5-90c9-b2791022b16b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-195f51-5cc698659d-x8lzl"
Sep 16 01:15:07.834647 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:07.834622    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbvfb\" (UniqueName: \"kubernetes.io/projected/ec919b37-44db-4ef5-90c9-b2791022b16b-kube-api-access-sbvfb\") pod \"payload-pre-processing-e2e-ipp-b-195f51-5cc698659d-x8lzl\" (UID: \"ec919b37-44db-4ef5-90c9-b2791022b16b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-195f51-5cc698659d-x8lzl"
Sep 16 01:15:07.834647 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:07.834642    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-57v5p\" (UniqueName: \"kubernetes.io/projected/f7f9620c-309a-4e52-a3c5-0f7a4dfb7fed-kube-api-access-57v5p\") pod \"payload-processing-e2e-ipp-b-195f51-6c8f7975cc-gzrj6\" (UID: \"f7f9620c-309a-4e52-a3c5-0f7a4dfb7fed\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-195f51-6c8f7975cc-gzrj6"
Sep 16 01:15:07.893094 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:07.893063    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-195f51-5cc698659d-x8lzl"
Sep 16 01:15:07.912942 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:07.912917    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-ad13e5-29825355-xtfzt"
Sep 16 01:15:07.913241 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:07.912945    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-ad13e5-29825355-xtfzt" event={"ID":"2da8bdd6-c774-4b2c-a217-72fe60148040","Type":"ContainerDied","Data":"e98e02a6e370319a38346922fe0cb952adfd3473d1cf72762e7947ea384c0bd5"}
Sep 16 01:15:07.913241 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:07.912998    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e98e02a6e370319a38346922fe0cb952adfd3473d1cf72762e7947ea384c0bd5"
Sep 16 01:15:07.931013 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:07.930988    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-195f51-6c8f7975cc-gzrj6"
Sep 16 01:15:08.176894 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:08.176869    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-195f51-5cc698659d-x8lzl"]
Sep 16 01:15:08.178377 ip-10-0-130-207 kubenswrapper[2556]: W0916 01:15:08.178347    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec919b37_44db_4ef5_90c9_b2791022b16b.slice/crio-c694952c133001852e5155bc0217c3f3b4a7cf8eb38b95f4c062e0c0b20d4bb0 WatchSource:0}: Error finding container c694952c133001852e5155bc0217c3f3b4a7cf8eb38b95f4c062e0c0b20d4bb0: Status 404 returned error can't find the container with id c694952c133001852e5155bc0217c3f3b4a7cf8eb38b95f4c062e0c0b20d4bb0
Sep 16 01:15:08.225677 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:08.225616    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-195f51-6c8f7975cc-gzrj6"]
Sep 16 01:15:08.227777 ip-10-0-130-207 kubenswrapper[2556]: W0916 01:15:08.227753    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7f9620c_309a_4e52_a3c5_0f7a4dfb7fed.slice/crio-e2d2b5cde0601282407306dbb9cd4378c87f318bb58ce7c262f9b6aafe9fa8eb WatchSource:0}: Error finding container e2d2b5cde0601282407306dbb9cd4378c87f318bb58ce7c262f9b6aafe9fa8eb: Status 404 returned error can't find the container with id e2d2b5cde0601282407306dbb9cd4378c87f318bb58ce7c262f9b6aafe9fa8eb
Sep 16 01:15:08.920707 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:08.920668    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-195f51-6c8f7975cc-gzrj6" event={"ID":"f7f9620c-309a-4e52-a3c5-0f7a4dfb7fed","Type":"ContainerStarted","Data":"3558bb58576739f437f1d6565e4674834a25861c6d0e6f6e1478d764565fc235"}
Sep 16 01:15:08.921157 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:08.920726    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-195f51-6c8f7975cc-gzrj6" event={"ID":"f7f9620c-309a-4e52-a3c5-0f7a4dfb7fed","Type":"ContainerStarted","Data":"e2d2b5cde0601282407306dbb9cd4378c87f318bb58ce7c262f9b6aafe9fa8eb"}
Sep 16 01:15:08.921157 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:08.920757    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-195f51-6c8f7975cc-gzrj6"
Sep 16 01:15:08.922457 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:08.922428    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-195f51-5cc698659d-x8lzl" event={"ID":"ec919b37-44db-4ef5-90c9-b2791022b16b","Type":"ContainerStarted","Data":"11211fcdff0c43e3ccc834500042c40e11ca8ec75a5c6df2b5b2b68e51e8992f"}
Sep 16 01:15:08.922626 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:08.922500    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-195f51-5cc698659d-x8lzl" event={"ID":"ec919b37-44db-4ef5-90c9-b2791022b16b","Type":"ContainerStarted","Data":"c694952c133001852e5155bc0217c3f3b4a7cf8eb38b95f4c062e0c0b20d4bb0"}
Sep 16 01:15:08.922626 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:08.922541    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-195f51-5cc698659d-x8lzl"
Sep 16 01:15:08.936413 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:08.936366    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-195f51-6c8f7975cc-gzrj6" podStartSLOduration=1.936351926 podStartE2EDuration="1.936351926s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 01:15:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 01:15:08.934131527 +0000 UTC m=+1923.715633120" watchObservedRunningTime="2026-09-16 01:15:08.936351926 +0000 UTC m=+1923.717853518"
Sep 16 01:15:08.947879 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:08.947839    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-195f51-5cc698659d-x8lzl" podStartSLOduration=1.947826503 podStartE2EDuration="1.947826503s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 01:15:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 01:15:08.947044996 +0000 UTC m=+1923.728546587" watchObservedRunningTime="2026-09-16 01:15:08.947826503 +0000 UTC m=+1923.729328101"
Sep 16 01:15:18.041996 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:18.041960    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-00f5c9-5b985bd6d7-bnkdx"]
Sep 16 01:15:18.045595 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:18.045575    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-00f5c9-5b985bd6d7-bnkdx"
Sep 16 01:15:18.047936 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:18.047908    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-00f5c9\""
Sep 16 01:15:18.048045 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:18.047940    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-00f5c9-dockercfg-2ck77\""
Sep 16 01:15:18.055911 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:18.055882    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-00f5c9-5b985bd6d7-bnkdx"]
Sep 16 01:15:18.128536 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:18.128498    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/589f427b-d265-4a13-b266-9127f54be415-plugins-config-volume\") pod \"payload-processing-e2e-api-b-00f5c9-5b985bd6d7-bnkdx\" (UID: \"589f427b-d265-4a13-b266-9127f54be415\") " pod="openshift-ingress/payload-processing-e2e-api-b-00f5c9-5b985bd6d7-bnkdx"
Sep 16 01:15:18.128707 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:18.128594    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2sbw\" (UniqueName: \"kubernetes.io/projected/589f427b-d265-4a13-b266-9127f54be415-kube-api-access-t2sbw\") pod \"payload-processing-e2e-api-b-00f5c9-5b985bd6d7-bnkdx\" (UID: \"589f427b-d265-4a13-b266-9127f54be415\") " pod="openshift-ingress/payload-processing-e2e-api-b-00f5c9-5b985bd6d7-bnkdx"
Sep 16 01:15:18.229593 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:18.229553    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2sbw\" (UniqueName: \"kubernetes.io/projected/589f427b-d265-4a13-b266-9127f54be415-kube-api-access-t2sbw\") pod \"payload-processing-e2e-api-b-00f5c9-5b985bd6d7-bnkdx\" (UID: \"589f427b-d265-4a13-b266-9127f54be415\") " pod="openshift-ingress/payload-processing-e2e-api-b-00f5c9-5b985bd6d7-bnkdx"
Sep 16 01:15:18.229763 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:18.229704    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/589f427b-d265-4a13-b266-9127f54be415-plugins-config-volume\") pod \"payload-processing-e2e-api-b-00f5c9-5b985bd6d7-bnkdx\" (UID: \"589f427b-d265-4a13-b266-9127f54be415\") " pod="openshift-ingress/payload-processing-e2e-api-b-00f5c9-5b985bd6d7-bnkdx"
Sep 16 01:15:18.230582 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:18.230559    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/589f427b-d265-4a13-b266-9127f54be415-plugins-config-volume\") pod \"payload-processing-e2e-api-b-00f5c9-5b985bd6d7-bnkdx\" (UID: \"589f427b-d265-4a13-b266-9127f54be415\") " pod="openshift-ingress/payload-processing-e2e-api-b-00f5c9-5b985bd6d7-bnkdx"
Sep 16 01:15:18.237082 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:18.237058    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2sbw\" (UniqueName: \"kubernetes.io/projected/589f427b-d265-4a13-b266-9127f54be415-kube-api-access-t2sbw\") pod \"payload-processing-e2e-api-b-00f5c9-5b985bd6d7-bnkdx\" (UID: \"589f427b-d265-4a13-b266-9127f54be415\") " pod="openshift-ingress/payload-processing-e2e-api-b-00f5c9-5b985bd6d7-bnkdx"
Sep 16 01:15:18.357020 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:18.356987    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-00f5c9-5b985bd6d7-bnkdx"
Sep 16 01:15:18.629023 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:18.628998    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-00f5c9-5b985bd6d7-bnkdx"]
Sep 16 01:15:18.630437 ip-10-0-130-207 kubenswrapper[2556]: W0916 01:15:18.630407    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod589f427b_d265_4a13_b266_9127f54be415.slice/crio-407af49e63df72ce757fa5dbeefbc626e830ebd5381a572e34291462c412372d WatchSource:0}: Error finding container 407af49e63df72ce757fa5dbeefbc626e830ebd5381a572e34291462c412372d: Status 404 returned error can't find the container with id 407af49e63df72ce757fa5dbeefbc626e830ebd5381a572e34291462c412372d
Sep 16 01:15:18.978842 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:18.978758    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-00f5c9-5b985bd6d7-bnkdx" event={"ID":"589f427b-d265-4a13-b266-9127f54be415","Type":"ContainerStarted","Data":"d907cc15d4d96c7dfed06e8c8f40e900dd9a162e1e1aa60bd8f1f12ad732b15c"}
Sep 16 01:15:18.978842 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:18.978818    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-00f5c9-5b985bd6d7-bnkdx" event={"ID":"589f427b-d265-4a13-b266-9127f54be415","Type":"ContainerStarted","Data":"407af49e63df72ce757fa5dbeefbc626e830ebd5381a572e34291462c412372d"}
Sep 16 01:15:18.979048 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:18.978847    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-00f5c9-5b985bd6d7-bnkdx"
Sep 16 01:15:18.995005 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:18.994962    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-00f5c9-5b985bd6d7-bnkdx" podStartSLOduration=0.994946443 podStartE2EDuration="994.946443ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 01:15:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 01:15:18.992490277 +0000 UTC m=+1933.773991869" watchObservedRunningTime="2026-09-16 01:15:18.994946443 +0000 UTC m=+1933.776448075"
Sep 16 01:15:19.930106 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:19.930075    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-195f51-5cc698659d-x8lzl"
Sep 16 01:15:19.930502 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:19.930140    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-195f51-6c8f7975cc-gzrj6"
Sep 16 01:15:29.987153 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:29.987124    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-00f5c9-5b985bd6d7-bnkdx"
Sep 16 01:15:38.382192 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:38.382166    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-195f51-6c8f7975cc-gzrj6_f7f9620c-309a-4e52-a3c5-0f7a4dfb7fed/payload-processing/0.log"
Sep 16 01:15:51.551008 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:51.550965    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-ad13e5-29825355-xtfzt"]
Sep 16 01:15:51.553325 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:51.553301    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-ad13e5-29825355-xtfzt"]
Sep 16 01:15:51.578786 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:51.578755    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ad13e5-59468867bf-xjs9r"]
Sep 16 01:15:51.579057 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:51.578987    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-ad13e5-59468867bf-xjs9r" podUID="fa3f86c2-866b-416c-b3da-c31749d4b66f" containerName="payload-processing" containerID="cri-o://80f268240d8df2baef21e08207b81a9967bb0a6040f0f33abc8ee32aa7bdbed1" gracePeriod=30
Sep 16 01:15:51.737337 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:51.737304    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2da8bdd6-c774-4b2c-a217-72fe60148040" path="/var/lib/kubelet/pods/2da8bdd6-c774-4b2c-a217-72fe60148040/volumes"
Sep 16 01:15:51.834144 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:51.834120    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-ad13e5-59468867bf-xjs9r"
Sep 16 01:15:51.917508 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:51.917409    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa3f86c2-866b-416c-b3da-c31749d4b66f-plugins-config-volume\") pod \"fa3f86c2-866b-416c-b3da-c31749d4b66f\" (UID: \"fa3f86c2-866b-416c-b3da-c31749d4b66f\") "
Sep 16 01:15:51.917745 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:51.917719    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kddpl\" (UniqueName: \"kubernetes.io/projected/fa3f86c2-866b-416c-b3da-c31749d4b66f-kube-api-access-kddpl\") pod \"fa3f86c2-866b-416c-b3da-c31749d4b66f\" (UID: \"fa3f86c2-866b-416c-b3da-c31749d4b66f\") "
Sep 16 01:15:51.917903 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:51.917877    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa3f86c2-866b-416c-b3da-c31749d4b66f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fa3f86c2-866b-416c-b3da-c31749d4b66f" (UID: "fa3f86c2-866b-416c-b3da-c31749d4b66f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 01:15:51.918049 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:51.918035    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa3f86c2-866b-416c-b3da-c31749d4b66f-plugins-config-volume\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:15:51.919952 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:51.919911    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa3f86c2-866b-416c-b3da-c31749d4b66f-kube-api-access-kddpl" (OuterVolumeSpecName: "kube-api-access-kddpl") pod "fa3f86c2-866b-416c-b3da-c31749d4b66f" (UID: "fa3f86c2-866b-416c-b3da-c31749d4b66f"). InnerVolumeSpecName "kube-api-access-kddpl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 01:15:52.019312 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:52.019279    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kddpl\" (UniqueName: \"kubernetes.io/projected/fa3f86c2-866b-416c-b3da-c31749d4b66f-kube-api-access-kddpl\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:15:52.164323 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:52.164293    2556 generic.go:356] "Generic (PLEG): container finished" podID="fa3f86c2-866b-416c-b3da-c31749d4b66f" containerID="80f268240d8df2baef21e08207b81a9967bb0a6040f0f33abc8ee32aa7bdbed1" exitCode=0
Sep 16 01:15:52.164505 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:52.164336    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-ad13e5-59468867bf-xjs9r"
Sep 16 01:15:52.164505 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:52.164383    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-ad13e5-59468867bf-xjs9r" event={"ID":"fa3f86c2-866b-416c-b3da-c31749d4b66f","Type":"ContainerDied","Data":"80f268240d8df2baef21e08207b81a9967bb0a6040f0f33abc8ee32aa7bdbed1"}
Sep 16 01:15:52.164505 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:52.164431    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-ad13e5-59468867bf-xjs9r" event={"ID":"fa3f86c2-866b-416c-b3da-c31749d4b66f","Type":"ContainerDied","Data":"f8c17996e91897420e7aeebef08f7e832cec4d3d75ee00ce7a0b3eb663bcc331"}
Sep 16 01:15:52.164505 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:52.164455    2556 scope.go:117] "RemoveContainer" containerID="80f268240d8df2baef21e08207b81a9967bb0a6040f0f33abc8ee32aa7bdbed1"
Sep 16 01:15:52.174134 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:52.174122    2556 scope.go:117] "RemoveContainer" containerID="80f268240d8df2baef21e08207b81a9967bb0a6040f0f33abc8ee32aa7bdbed1"
Sep 16 01:15:52.174416 ip-10-0-130-207 kubenswrapper[2556]: E0916 01:15:52.174393    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80f268240d8df2baef21e08207b81a9967bb0a6040f0f33abc8ee32aa7bdbed1\": container with ID starting with 80f268240d8df2baef21e08207b81a9967bb0a6040f0f33abc8ee32aa7bdbed1 not found: ID does not exist" containerID="80f268240d8df2baef21e08207b81a9967bb0a6040f0f33abc8ee32aa7bdbed1"
Sep 16 01:15:52.174509 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:52.174423    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80f268240d8df2baef21e08207b81a9967bb0a6040f0f33abc8ee32aa7bdbed1"} err="failed to get container status \"80f268240d8df2baef21e08207b81a9967bb0a6040f0f33abc8ee32aa7bdbed1\": rpc error: code = NotFound desc = could not find container \"80f268240d8df2baef21e08207b81a9967bb0a6040f0f33abc8ee32aa7bdbed1\": container with ID starting with 80f268240d8df2baef21e08207b81a9967bb0a6040f0f33abc8ee32aa7bdbed1 not found: ID does not exist"
Sep 16 01:15:52.184935 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:52.184914    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ad13e5-59468867bf-xjs9r"]
Sep 16 01:15:52.188777 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:52.188759    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ad13e5-59468867bf-xjs9r"]
Sep 16 01:15:53.736500 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:53.736450    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa3f86c2-866b-416c-b3da-c31749d4b66f" path="/var/lib/kubelet/pods/fa3f86c2-866b-416c-b3da-c31749d4b66f/volumes"
Sep 16 01:15:54.590684 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:54.590647    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-089563/ipp-route-089563-kserve-6c5d46dc9f-l58vh"]
Sep 16 01:15:54.591072 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:54.591058    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa3f86c2-866b-416c-b3da-c31749d4b66f" containerName="payload-processing"
Sep 16 01:15:54.591131 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:54.591074    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa3f86c2-866b-416c-b3da-c31749d4b66f" containerName="payload-processing"
Sep 16 01:15:54.591170 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:54.591150    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa3f86c2-866b-416c-b3da-c31749d4b66f" containerName="payload-processing"
Sep 16 01:15:54.594340 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:54.594324    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-089563/ipp-route-089563-kserve-6c5d46dc9f-l58vh"
Sep 16 01:15:54.596540 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:54.596519    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-089563\"/\"openshift-service-ca.crt\""
Sep 16 01:15:54.597265 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:54.597244    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-089563\"/\"ipp-route-089563-kserve-self-signed-certs\""
Sep 16 01:15:54.597373 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:54.597272    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-089563\"/\"default-dockercfg-sq9gl\""
Sep 16 01:15:54.597373 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:54.597318    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-089563\"/\"kube-root-ca.crt\""
Sep 16 01:15:54.601490 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:54.601457    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-089563/ipp-route-089563-kserve-6c5d46dc9f-l58vh"]
Sep 16 01:15:54.743026 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:54.742996    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81f46a16-db8b-4387-89f4-5657401de37f-tls-certs\") pod \"ipp-route-089563-kserve-6c5d46dc9f-l58vh\" (UID: \"81f46a16-db8b-4387-89f4-5657401de37f\") " pod="ai-tenant-e2e-ipp-a-089563/ipp-route-089563-kserve-6c5d46dc9f-l58vh"
Sep 16 01:15:54.743344 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:54.743037    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81f46a16-db8b-4387-89f4-5657401de37f-model-cache\") pod \"ipp-route-089563-kserve-6c5d46dc9f-l58vh\" (UID: \"81f46a16-db8b-4387-89f4-5657401de37f\") " pod="ai-tenant-e2e-ipp-a-089563/ipp-route-089563-kserve-6c5d46dc9f-l58vh"
Sep 16 01:15:54.743344 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:54.743107    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81f46a16-db8b-4387-89f4-5657401de37f-home\") pod \"ipp-route-089563-kserve-6c5d46dc9f-l58vh\" (UID: \"81f46a16-db8b-4387-89f4-5657401de37f\") " pod="ai-tenant-e2e-ipp-a-089563/ipp-route-089563-kserve-6c5d46dc9f-l58vh"
Sep 16 01:15:54.743344 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:54.743127    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81f46a16-db8b-4387-89f4-5657401de37f-dshm\") pod \"ipp-route-089563-kserve-6c5d46dc9f-l58vh\" (UID: \"81f46a16-db8b-4387-89f4-5657401de37f\") " pod="ai-tenant-e2e-ipp-a-089563/ipp-route-089563-kserve-6c5d46dc9f-l58vh"
Sep 16 01:15:54.743344 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:54.743158    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81f46a16-db8b-4387-89f4-5657401de37f-tmp-dir\") pod \"ipp-route-089563-kserve-6c5d46dc9f-l58vh\" (UID: \"81f46a16-db8b-4387-89f4-5657401de37f\") " pod="ai-tenant-e2e-ipp-a-089563/ipp-route-089563-kserve-6c5d46dc9f-l58vh"
Sep 16 01:15:54.743344 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:54.743197    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwrtz\" (UniqueName: \"kubernetes.io/projected/81f46a16-db8b-4387-89f4-5657401de37f-kube-api-access-hwrtz\") pod \"ipp-route-089563-kserve-6c5d46dc9f-l58vh\" (UID: \"81f46a16-db8b-4387-89f4-5657401de37f\") " pod="ai-tenant-e2e-ipp-a-089563/ipp-route-089563-kserve-6c5d46dc9f-l58vh"
Sep 16 01:15:54.843587 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:54.843516    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwrtz\" (UniqueName: \"kubernetes.io/projected/81f46a16-db8b-4387-89f4-5657401de37f-kube-api-access-hwrtz\") pod \"ipp-route-089563-kserve-6c5d46dc9f-l58vh\" (UID: \"81f46a16-db8b-4387-89f4-5657401de37f\") " pod="ai-tenant-e2e-ipp-a-089563/ipp-route-089563-kserve-6c5d46dc9f-l58vh"
Sep 16 01:15:54.843587 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:54.843565    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81f46a16-db8b-4387-89f4-5657401de37f-tls-certs\") pod \"ipp-route-089563-kserve-6c5d46dc9f-l58vh\" (UID: \"81f46a16-db8b-4387-89f4-5657401de37f\") " pod="ai-tenant-e2e-ipp-a-089563/ipp-route-089563-kserve-6c5d46dc9f-l58vh"
Sep 16 01:15:54.843799 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:54.843686    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81f46a16-db8b-4387-89f4-5657401de37f-model-cache\") pod \"ipp-route-089563-kserve-6c5d46dc9f-l58vh\" (UID: \"81f46a16-db8b-4387-89f4-5657401de37f\") " pod="ai-tenant-e2e-ipp-a-089563/ipp-route-089563-kserve-6c5d46dc9f-l58vh"
Sep 16 01:15:54.843861 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:54.843811    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81f46a16-db8b-4387-89f4-5657401de37f-home\") pod \"ipp-route-089563-kserve-6c5d46dc9f-l58vh\" (UID: \"81f46a16-db8b-4387-89f4-5657401de37f\") " pod="ai-tenant-e2e-ipp-a-089563/ipp-route-089563-kserve-6c5d46dc9f-l58vh"
Sep 16 01:15:54.843924 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:54.843860    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81f46a16-db8b-4387-89f4-5657401de37f-dshm\") pod \"ipp-route-089563-kserve-6c5d46dc9f-l58vh\" (UID: \"81f46a16-db8b-4387-89f4-5657401de37f\") " pod="ai-tenant-e2e-ipp-a-089563/ipp-route-089563-kserve-6c5d46dc9f-l58vh"
Sep 16 01:15:54.843924 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:54.843895    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81f46a16-db8b-4387-89f4-5657401de37f-tmp-dir\") pod \"ipp-route-089563-kserve-6c5d46dc9f-l58vh\" (UID: \"81f46a16-db8b-4387-89f4-5657401de37f\") " pod="ai-tenant-e2e-ipp-a-089563/ipp-route-089563-kserve-6c5d46dc9f-l58vh"
Sep 16 01:15:54.844083 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:54.844060    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81f46a16-db8b-4387-89f4-5657401de37f-model-cache\") pod \"ipp-route-089563-kserve-6c5d46dc9f-l58vh\" (UID: \"81f46a16-db8b-4387-89f4-5657401de37f\") " pod="ai-tenant-e2e-ipp-a-089563/ipp-route-089563-kserve-6c5d46dc9f-l58vh"
Sep 16 01:15:54.844158 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:54.844130    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81f46a16-db8b-4387-89f4-5657401de37f-home\") pod \"ipp-route-089563-kserve-6c5d46dc9f-l58vh\" (UID: \"81f46a16-db8b-4387-89f4-5657401de37f\") " pod="ai-tenant-e2e-ipp-a-089563/ipp-route-089563-kserve-6c5d46dc9f-l58vh"
Sep 16 01:15:54.844343 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:54.844325    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81f46a16-db8b-4387-89f4-5657401de37f-tmp-dir\") pod \"ipp-route-089563-kserve-6c5d46dc9f-l58vh\" (UID: \"81f46a16-db8b-4387-89f4-5657401de37f\") " pod="ai-tenant-e2e-ipp-a-089563/ipp-route-089563-kserve-6c5d46dc9f-l58vh"
Sep 16 01:15:54.846008 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:54.845984    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81f46a16-db8b-4387-89f4-5657401de37f-dshm\") pod \"ipp-route-089563-kserve-6c5d46dc9f-l58vh\" (UID: \"81f46a16-db8b-4387-89f4-5657401de37f\") " pod="ai-tenant-e2e-ipp-a-089563/ipp-route-089563-kserve-6c5d46dc9f-l58vh"
Sep 16 01:15:54.846079 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:54.846065    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81f46a16-db8b-4387-89f4-5657401de37f-tls-certs\") pod \"ipp-route-089563-kserve-6c5d46dc9f-l58vh\" (UID: \"81f46a16-db8b-4387-89f4-5657401de37f\") " pod="ai-tenant-e2e-ipp-a-089563/ipp-route-089563-kserve-6c5d46dc9f-l58vh"
Sep 16 01:15:54.850538 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:54.850519    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwrtz\" (UniqueName: \"kubernetes.io/projected/81f46a16-db8b-4387-89f4-5657401de37f-kube-api-access-hwrtz\") pod \"ipp-route-089563-kserve-6c5d46dc9f-l58vh\" (UID: \"81f46a16-db8b-4387-89f4-5657401de37f\") " pod="ai-tenant-e2e-ipp-a-089563/ipp-route-089563-kserve-6c5d46dc9f-l58vh"
Sep 16 01:15:54.905254 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:54.905221    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-089563/ipp-route-089563-kserve-6c5d46dc9f-l58vh"
Sep 16 01:15:55.178369 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:55.178337    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-089563/ipp-route-089563-kserve-6c5d46dc9f-l58vh"]
Sep 16 01:15:55.179851 ip-10-0-130-207 kubenswrapper[2556]: W0916 01:15:55.179819    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod81f46a16_db8b_4387_89f4_5657401de37f.slice/crio-33b9633218426daaf371cbd2398760442bf3439a941d1cda34ae4c5d7d5d06c1 WatchSource:0}: Error finding container 33b9633218426daaf371cbd2398760442bf3439a941d1cda34ae4c5d7d5d06c1: Status 404 returned error can't find the container with id 33b9633218426daaf371cbd2398760442bf3439a941d1cda34ae4c5d7d5d06c1
Sep 16 01:15:56.193053 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:56.193022    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-089563/ipp-route-089563-kserve-6c5d46dc9f-l58vh" event={"ID":"81f46a16-db8b-4387-89f4-5657401de37f","Type":"ContainerStarted","Data":"a84d2f629af484f2dec69c4e81db994025ca8f726789db8b5781b317d84c4826"}
Sep 16 01:15:56.193484 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:56.193061    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-089563/ipp-route-089563-kserve-6c5d46dc9f-l58vh" event={"ID":"81f46a16-db8b-4387-89f4-5657401de37f","Type":"ContainerStarted","Data":"33b9633218426daaf371cbd2398760442bf3439a941d1cda34ae4c5d7d5d06c1"}
Sep 16 01:15:56.206452 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:15:56.206406    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-089563/ipp-route-089563-kserve-6c5d46dc9f-l58vh" podStartSLOduration=2.20638974 podStartE2EDuration="2.20638974s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 01:15:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 01:15:56.205227259 +0000 UTC m=+1970.986728854" watchObservedRunningTime="2026-09-16 01:15:56.20638974 +0000 UTC m=+1970.987891377"
Sep 16 01:16:04.905424 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:04.905339    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-089563/ipp-route-089563-kserve-6c5d46dc9f-l58vh"
Sep 16 01:16:04.905424 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:04.905385    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-089563/ipp-route-089563-kserve-6c5d46dc9f-l58vh"
Sep 16 01:16:04.917825 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:04.917799    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-089563/ipp-route-089563-kserve-6c5d46dc9f-l58vh"
Sep 16 01:16:05.251927 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:05.251844    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-089563/ipp-route-089563-kserve-6c5d46dc9f-l58vh"
Sep 16 01:16:14.132458 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:14.132405    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-00f5c9-5b985bd6d7-bnkdx"]
Sep 16 01:16:14.133371 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:14.133303    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-00f5c9-5b985bd6d7-bnkdx" podUID="589f427b-d265-4a13-b266-9127f54be415" containerName="payload-processing" containerID="cri-o://d907cc15d4d96c7dfed06e8c8f40e900dd9a162e1e1aa60bd8f1f12ad732b15c" gracePeriod=30
Sep 16 01:16:14.293588 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:14.293429    2556 generic.go:356] "Generic (PLEG): container finished" podID="589f427b-d265-4a13-b266-9127f54be415" containerID="d907cc15d4d96c7dfed06e8c8f40e900dd9a162e1e1aa60bd8f1f12ad732b15c" exitCode=0
Sep 16 01:16:14.293588 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:14.293506    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-00f5c9-5b985bd6d7-bnkdx" event={"ID":"589f427b-d265-4a13-b266-9127f54be415","Type":"ContainerDied","Data":"d907cc15d4d96c7dfed06e8c8f40e900dd9a162e1e1aa60bd8f1f12ad732b15c"}
Sep 16 01:16:14.398738 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:14.398682    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-00f5c9-5b985bd6d7-bnkdx"
Sep 16 01:16:14.508144 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:14.508109    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/589f427b-d265-4a13-b266-9127f54be415-plugins-config-volume\") pod \"589f427b-d265-4a13-b266-9127f54be415\" (UID: \"589f427b-d265-4a13-b266-9127f54be415\") "
Sep 16 01:16:14.508300 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:14.508245    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t2sbw\" (UniqueName: \"kubernetes.io/projected/589f427b-d265-4a13-b266-9127f54be415-kube-api-access-t2sbw\") pod \"589f427b-d265-4a13-b266-9127f54be415\" (UID: \"589f427b-d265-4a13-b266-9127f54be415\") "
Sep 16 01:16:14.508559 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:14.508533    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/589f427b-d265-4a13-b266-9127f54be415-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "589f427b-d265-4a13-b266-9127f54be415" (UID: "589f427b-d265-4a13-b266-9127f54be415"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 01:16:14.510191 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:14.510161    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/589f427b-d265-4a13-b266-9127f54be415-kube-api-access-t2sbw" (OuterVolumeSpecName: "kube-api-access-t2sbw") pod "589f427b-d265-4a13-b266-9127f54be415" (UID: "589f427b-d265-4a13-b266-9127f54be415"). InnerVolumeSpecName "kube-api-access-t2sbw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 01:16:14.609364 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:14.609337    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/589f427b-d265-4a13-b266-9127f54be415-plugins-config-volume\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:16:14.609364 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:14.609363    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t2sbw\" (UniqueName: \"kubernetes.io/projected/589f427b-d265-4a13-b266-9127f54be415-kube-api-access-t2sbw\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:16:15.300171 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:15.300140    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-00f5c9-5b985bd6d7-bnkdx"
Sep 16 01:16:15.300171 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:15.300163    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-00f5c9-5b985bd6d7-bnkdx" event={"ID":"589f427b-d265-4a13-b266-9127f54be415","Type":"ContainerDied","Data":"407af49e63df72ce757fa5dbeefbc626e830ebd5381a572e34291462c412372d"}
Sep 16 01:16:15.300662 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:15.300226    2556 scope.go:117] "RemoveContainer" containerID="d907cc15d4d96c7dfed06e8c8f40e900dd9a162e1e1aa60bd8f1f12ad732b15c"
Sep 16 01:16:15.337569 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:15.337539    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-00f5c9-5b985bd6d7-bnkdx"]
Sep 16 01:16:15.344215 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:15.344192    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-00f5c9-5b985bd6d7-bnkdx"]
Sep 16 01:16:15.742213 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:15.742169    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="589f427b-d265-4a13-b266-9127f54be415" path="/var/lib/kubelet/pods/589f427b-d265-4a13-b266-9127f54be415/volumes"
Sep 16 01:16:31.521841 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:31.521806    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-f39e09-7cc574497b-z6vrc"]
Sep 16 01:16:31.522293 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:31.522092    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-f39e09-7cc574497b-z6vrc" podUID="26244697-ec2c-48f6-88f2-ce3e688e2add" containerName="maas-api" containerID="cri-o://fddb083e53ae7fac4d715ff6b2bf7958e5cc6e87ad1a0db187442abe7ba0f79e" gracePeriod=30
Sep 16 01:16:31.577058 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:31.577026    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-f39e09-5d74949579-6jqsj"]
Sep 16 01:16:31.577317 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:31.577260    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-f39e09-5d74949579-6jqsj" podUID="a5b85f15-bbd3-4beb-97f0-22767c9571b9" containerName="payload-processing" containerID="cri-o://ecbb2718463c54a836bca39d91ffc2624232f5b75ce561f2aaedbc0f80c8d002" gracePeriod=30
Sep 16 01:16:31.586697 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:31.586672    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-f39e09-f5677f8cd-dj7jx"]
Sep 16 01:16:31.586902 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:31.586875    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-f39e09-f5677f8cd-dj7jx" podUID="44e35ee6-c1f9-4d6c-9584-2ebea688add8" containerName="payload-pre-processing" containerID="cri-o://72abc7cd856be4b09f34310e896c365f7ac2353015e0192333572b347ec74fbf" gracePeriod=30
Sep 16 01:16:31.853524 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:31.853458    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-f39e09-7cc574497b-z6vrc"
Sep 16 01:16:31.900346 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:31.900314    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-f39e09-f5677f8cd-dj7jx"
Sep 16 01:16:31.903457 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:31.903437    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-f39e09-5d74949579-6jqsj"
Sep 16 01:16:31.957031 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:31.957000    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w65qg\" (UniqueName: \"kubernetes.io/projected/44e35ee6-c1f9-4d6c-9584-2ebea688add8-kube-api-access-w65qg\") pod \"44e35ee6-c1f9-4d6c-9584-2ebea688add8\" (UID: \"44e35ee6-c1f9-4d6c-9584-2ebea688add8\") "
Sep 16 01:16:31.957182 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:31.957058    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q2pwz\" (UniqueName: \"kubernetes.io/projected/26244697-ec2c-48f6-88f2-ce3e688e2add-kube-api-access-q2pwz\") pod \"26244697-ec2c-48f6-88f2-ce3e688e2add\" (UID: \"26244697-ec2c-48f6-88f2-ce3e688e2add\") "
Sep 16 01:16:31.957182 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:31.957111    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26244697-ec2c-48f6-88f2-ce3e688e2add-metrics-certs\") pod \"26244697-ec2c-48f6-88f2-ce3e688e2add\" (UID: \"26244697-ec2c-48f6-88f2-ce3e688e2add\") "
Sep 16 01:16:31.957182 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:31.957140    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a5b85f15-bbd3-4beb-97f0-22767c9571b9-plugins-config-volume\") pod \"a5b85f15-bbd3-4beb-97f0-22767c9571b9\" (UID: \"a5b85f15-bbd3-4beb-97f0-22767c9571b9\") "
Sep 16 01:16:31.957182 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:31.957165    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/44e35ee6-c1f9-4d6c-9584-2ebea688add8-plugins-config-volume\") pod \"44e35ee6-c1f9-4d6c-9584-2ebea688add8\" (UID: \"44e35ee6-c1f9-4d6c-9584-2ebea688add8\") "
Sep 16 01:16:31.957417 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:31.957207    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/26244697-ec2c-48f6-88f2-ce3e688e2add-openshift-service-ca\") pod \"26244697-ec2c-48f6-88f2-ce3e688e2add\" (UID: \"26244697-ec2c-48f6-88f2-ce3e688e2add\") "
Sep 16 01:16:31.957417 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:31.957237    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/26244697-ec2c-48f6-88f2-ce3e688e2add-maas-api-tls\") pod \"26244697-ec2c-48f6-88f2-ce3e688e2add\" (UID: \"26244697-ec2c-48f6-88f2-ce3e688e2add\") "
Sep 16 01:16:31.957417 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:31.957311    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m56mh\" (UniqueName: \"kubernetes.io/projected/a5b85f15-bbd3-4beb-97f0-22767c9571b9-kube-api-access-m56mh\") pod \"a5b85f15-bbd3-4beb-97f0-22767c9571b9\" (UID: \"a5b85f15-bbd3-4beb-97f0-22767c9571b9\") "
Sep 16 01:16:31.957852 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:31.957686    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26244697-ec2c-48f6-88f2-ce3e688e2add-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "26244697-ec2c-48f6-88f2-ce3e688e2add" (UID: "26244697-ec2c-48f6-88f2-ce3e688e2add"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 01:16:31.957975 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:31.957874    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5b85f15-bbd3-4beb-97f0-22767c9571b9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a5b85f15-bbd3-4beb-97f0-22767c9571b9" (UID: "a5b85f15-bbd3-4beb-97f0-22767c9571b9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 01:16:31.958241 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:31.958185    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44e35ee6-c1f9-4d6c-9584-2ebea688add8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "44e35ee6-c1f9-4d6c-9584-2ebea688add8" (UID: "44e35ee6-c1f9-4d6c-9584-2ebea688add8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 01:16:31.959396 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:31.959371    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/26244697-ec2c-48f6-88f2-ce3e688e2add-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "26244697-ec2c-48f6-88f2-ce3e688e2add" (UID: "26244697-ec2c-48f6-88f2-ce3e688e2add"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 01:16:31.959650 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:31.959633    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26244697-ec2c-48f6-88f2-ce3e688e2add-kube-api-access-q2pwz" (OuterVolumeSpecName: "kube-api-access-q2pwz") pod "26244697-ec2c-48f6-88f2-ce3e688e2add" (UID: "26244697-ec2c-48f6-88f2-ce3e688e2add"). InnerVolumeSpecName "kube-api-access-q2pwz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 01:16:31.959722 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:31.959698    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5b85f15-bbd3-4beb-97f0-22767c9571b9-kube-api-access-m56mh" (OuterVolumeSpecName: "kube-api-access-m56mh") pod "a5b85f15-bbd3-4beb-97f0-22767c9571b9" (UID: "a5b85f15-bbd3-4beb-97f0-22767c9571b9"). InnerVolumeSpecName "kube-api-access-m56mh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 01:16:31.960110 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:31.960080    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/26244697-ec2c-48f6-88f2-ce3e688e2add-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "26244697-ec2c-48f6-88f2-ce3e688e2add" (UID: "26244697-ec2c-48f6-88f2-ce3e688e2add"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 01:16:31.960196 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:31.960113    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44e35ee6-c1f9-4d6c-9584-2ebea688add8-kube-api-access-w65qg" (OuterVolumeSpecName: "kube-api-access-w65qg") pod "44e35ee6-c1f9-4d6c-9584-2ebea688add8" (UID: "44e35ee6-c1f9-4d6c-9584-2ebea688add8"). InnerVolumeSpecName "kube-api-access-w65qg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 01:16:32.058694 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:32.058663    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w65qg\" (UniqueName: \"kubernetes.io/projected/44e35ee6-c1f9-4d6c-9584-2ebea688add8-kube-api-access-w65qg\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:16:32.058694 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:32.058690    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q2pwz\" (UniqueName: \"kubernetes.io/projected/26244697-ec2c-48f6-88f2-ce3e688e2add-kube-api-access-q2pwz\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:16:32.058694 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:32.058700    2556 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26244697-ec2c-48f6-88f2-ce3e688e2add-metrics-certs\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:16:32.058906 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:32.058709    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a5b85f15-bbd3-4beb-97f0-22767c9571b9-plugins-config-volume\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:16:32.058906 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:32.058719    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/44e35ee6-c1f9-4d6c-9584-2ebea688add8-plugins-config-volume\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:16:32.058906 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:32.058727    2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/26244697-ec2c-48f6-88f2-ce3e688e2add-openshift-service-ca\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:16:32.058906 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:32.058735    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/26244697-ec2c-48f6-88f2-ce3e688e2add-maas-api-tls\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:16:32.058906 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:32.058743    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m56mh\" (UniqueName: \"kubernetes.io/projected/a5b85f15-bbd3-4beb-97f0-22767c9571b9-kube-api-access-m56mh\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:16:32.397701 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:32.397663    2556 generic.go:356] "Generic (PLEG): container finished" podID="a5b85f15-bbd3-4beb-97f0-22767c9571b9" containerID="ecbb2718463c54a836bca39d91ffc2624232f5b75ce561f2aaedbc0f80c8d002" exitCode=0
Sep 16 01:16:32.397880 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:32.397712    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-f39e09-5d74949579-6jqsj"
Sep 16 01:16:32.397880 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:32.397751    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-f39e09-5d74949579-6jqsj" event={"ID":"a5b85f15-bbd3-4beb-97f0-22767c9571b9","Type":"ContainerDied","Data":"ecbb2718463c54a836bca39d91ffc2624232f5b75ce561f2aaedbc0f80c8d002"}
Sep 16 01:16:32.397880 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:32.397822    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-f39e09-5d74949579-6jqsj" event={"ID":"a5b85f15-bbd3-4beb-97f0-22767c9571b9","Type":"ContainerDied","Data":"db202156beca452ebd084477db26a3534fb781c6b89e0b07b87a1f360f2487a7"}
Sep 16 01:16:32.397880 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:32.397844    2556 scope.go:117] "RemoveContainer" containerID="ecbb2718463c54a836bca39d91ffc2624232f5b75ce561f2aaedbc0f80c8d002"
Sep 16 01:16:32.399085 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:32.399064    2556 generic.go:356] "Generic (PLEG): container finished" podID="26244697-ec2c-48f6-88f2-ce3e688e2add" containerID="fddb083e53ae7fac4d715ff6b2bf7958e5cc6e87ad1a0db187442abe7ba0f79e" exitCode=0
Sep 16 01:16:32.399165 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:32.399122    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-f39e09-7cc574497b-z6vrc"
Sep 16 01:16:32.399165 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:32.399139    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-f39e09-7cc574497b-z6vrc" event={"ID":"26244697-ec2c-48f6-88f2-ce3e688e2add","Type":"ContainerDied","Data":"fddb083e53ae7fac4d715ff6b2bf7958e5cc6e87ad1a0db187442abe7ba0f79e"}
Sep 16 01:16:32.399270 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:32.399180    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-f39e09-7cc574497b-z6vrc" event={"ID":"26244697-ec2c-48f6-88f2-ce3e688e2add","Type":"ContainerDied","Data":"f85ab960fa36650369f266d9120eba532dbff0d4d3e8936a1091fbb29567c929"}
Sep 16 01:16:32.400630 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:32.400605    2556 generic.go:356] "Generic (PLEG): container finished" podID="44e35ee6-c1f9-4d6c-9584-2ebea688add8" containerID="72abc7cd856be4b09f34310e896c365f7ac2353015e0192333572b347ec74fbf" exitCode=0
Sep 16 01:16:32.400759 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:32.400653    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-f39e09-f5677f8cd-dj7jx"
Sep 16 01:16:32.400759 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:32.400659    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-f39e09-f5677f8cd-dj7jx" event={"ID":"44e35ee6-c1f9-4d6c-9584-2ebea688add8","Type":"ContainerDied","Data":"72abc7cd856be4b09f34310e896c365f7ac2353015e0192333572b347ec74fbf"}
Sep 16 01:16:32.400759 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:32.400694    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-f39e09-f5677f8cd-dj7jx" event={"ID":"44e35ee6-c1f9-4d6c-9584-2ebea688add8","Type":"ContainerDied","Data":"6d3af6d8f27fa43f882e862548e5e08cb18c83c9b86ef457901793b60861db9a"}
Sep 16 01:16:32.409132 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:32.409110    2556 scope.go:117] "RemoveContainer" containerID="ecbb2718463c54a836bca39d91ffc2624232f5b75ce561f2aaedbc0f80c8d002"
Sep 16 01:16:32.409443 ip-10-0-130-207 kubenswrapper[2556]: E0916 01:16:32.409425    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ecbb2718463c54a836bca39d91ffc2624232f5b75ce561f2aaedbc0f80c8d002\": container with ID starting with ecbb2718463c54a836bca39d91ffc2624232f5b75ce561f2aaedbc0f80c8d002 not found: ID does not exist" containerID="ecbb2718463c54a836bca39d91ffc2624232f5b75ce561f2aaedbc0f80c8d002"
Sep 16 01:16:32.409567 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:32.409454    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ecbb2718463c54a836bca39d91ffc2624232f5b75ce561f2aaedbc0f80c8d002"} err="failed to get container status \"ecbb2718463c54a836bca39d91ffc2624232f5b75ce561f2aaedbc0f80c8d002\": rpc error: code = NotFound desc = could not find container \"ecbb2718463c54a836bca39d91ffc2624232f5b75ce561f2aaedbc0f80c8d002\": container with ID starting with ecbb2718463c54a836bca39d91ffc2624232f5b75ce561f2aaedbc0f80c8d002 not found: ID does not exist"
Sep 16 01:16:32.409567 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:32.409490    2556 scope.go:117] "RemoveContainer" containerID="fddb083e53ae7fac4d715ff6b2bf7958e5cc6e87ad1a0db187442abe7ba0f79e"
Sep 16 01:16:32.418120 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:32.418103    2556 scope.go:117] "RemoveContainer" containerID="fddb083e53ae7fac4d715ff6b2bf7958e5cc6e87ad1a0db187442abe7ba0f79e"
Sep 16 01:16:32.418362 ip-10-0-130-207 kubenswrapper[2556]: E0916 01:16:32.418342    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fddb083e53ae7fac4d715ff6b2bf7958e5cc6e87ad1a0db187442abe7ba0f79e\": container with ID starting with fddb083e53ae7fac4d715ff6b2bf7958e5cc6e87ad1a0db187442abe7ba0f79e not found: ID does not exist" containerID="fddb083e53ae7fac4d715ff6b2bf7958e5cc6e87ad1a0db187442abe7ba0f79e"
Sep 16 01:16:32.418442 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:32.418368    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fddb083e53ae7fac4d715ff6b2bf7958e5cc6e87ad1a0db187442abe7ba0f79e"} err="failed to get container status \"fddb083e53ae7fac4d715ff6b2bf7958e5cc6e87ad1a0db187442abe7ba0f79e\": rpc error: code = NotFound desc = could not find container \"fddb083e53ae7fac4d715ff6b2bf7958e5cc6e87ad1a0db187442abe7ba0f79e\": container with ID starting with fddb083e53ae7fac4d715ff6b2bf7958e5cc6e87ad1a0db187442abe7ba0f79e not found: ID does not exist"
Sep 16 01:16:32.418442 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:32.418385    2556 scope.go:117] "RemoveContainer" containerID="72abc7cd856be4b09f34310e896c365f7ac2353015e0192333572b347ec74fbf"
Sep 16 01:16:32.425462 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:32.425438    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-f39e09-7cc574497b-z6vrc"]
Sep 16 01:16:32.426869 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:32.426854    2556 scope.go:117] "RemoveContainer" containerID="72abc7cd856be4b09f34310e896c365f7ac2353015e0192333572b347ec74fbf"
Sep 16 01:16:32.427096 ip-10-0-130-207 kubenswrapper[2556]: E0916 01:16:32.427079    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72abc7cd856be4b09f34310e896c365f7ac2353015e0192333572b347ec74fbf\": container with ID starting with 72abc7cd856be4b09f34310e896c365f7ac2353015e0192333572b347ec74fbf not found: ID does not exist" containerID="72abc7cd856be4b09f34310e896c365f7ac2353015e0192333572b347ec74fbf"
Sep 16 01:16:32.427168 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:32.427102    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72abc7cd856be4b09f34310e896c365f7ac2353015e0192333572b347ec74fbf"} err="failed to get container status \"72abc7cd856be4b09f34310e896c365f7ac2353015e0192333572b347ec74fbf\": rpc error: code = NotFound desc = could not find container \"72abc7cd856be4b09f34310e896c365f7ac2353015e0192333572b347ec74fbf\": container with ID starting with 72abc7cd856be4b09f34310e896c365f7ac2353015e0192333572b347ec74fbf not found: ID does not exist"
Sep 16 01:16:32.430713 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:32.430695    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-f39e09-7cc574497b-z6vrc"]
Sep 16 01:16:32.442049 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:32.442025    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-f39e09-f5677f8cd-dj7jx"]
Sep 16 01:16:32.447828 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:32.447807    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-f39e09-f5677f8cd-dj7jx"]
Sep 16 01:16:32.456366 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:32.456343    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-f39e09-5d74949579-6jqsj"]
Sep 16 01:16:32.462976 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:32.462953    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-f39e09-5d74949579-6jqsj"]
Sep 16 01:16:32.660771 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:32.660714    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-195f51-6c8f7975cc-gzrj6_f7f9620c-309a-4e52-a3c5-0f7a4dfb7fed/payload-processing/0.log"
Sep 16 01:16:32.778396 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:32.778373    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-195f51-6c8f7975cc-gzrj6_f7f9620c-309a-4e52-a3c5-0f7a4dfb7fed/payload-processing/0.log"
Sep 16 01:16:33.737053 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:33.737012    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26244697-ec2c-48f6-88f2-ce3e688e2add" path="/var/lib/kubelet/pods/26244697-ec2c-48f6-88f2-ce3e688e2add/volumes"
Sep 16 01:16:33.737602 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:33.737580    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44e35ee6-c1f9-4d6c-9584-2ebea688add8" path="/var/lib/kubelet/pods/44e35ee6-c1f9-4d6c-9584-2ebea688add8/volumes"
Sep 16 01:16:33.737964 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:33.737946    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5b85f15-bbd3-4beb-97f0-22767c9571b9" path="/var/lib/kubelet/pods/a5b85f15-bbd3-4beb-97f0-22767c9571b9/volumes"
Sep 16 01:16:40.495330 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:40.495289    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f46871-bbb9b964b-42slq"]
Sep 16 01:16:40.495978 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:40.495954    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5b85f15-bbd3-4beb-97f0-22767c9571b9" containerName="payload-processing"
Sep 16 01:16:40.495978 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:40.495978    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5b85f15-bbd3-4beb-97f0-22767c9571b9" containerName="payload-processing"
Sep 16 01:16:40.496140 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:40.495990    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="589f427b-d265-4a13-b266-9127f54be415" containerName="payload-processing"
Sep 16 01:16:40.496140 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:40.495999    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="589f427b-d265-4a13-b266-9127f54be415" containerName="payload-processing"
Sep 16 01:16:40.496140 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:40.496019    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26244697-ec2c-48f6-88f2-ce3e688e2add" containerName="maas-api"
Sep 16 01:16:40.496140 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:40.496028    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="26244697-ec2c-48f6-88f2-ce3e688e2add" containerName="maas-api"
Sep 16 01:16:40.496140 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:40.496039    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44e35ee6-c1f9-4d6c-9584-2ebea688add8" containerName="payload-pre-processing"
Sep 16 01:16:40.496140 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:40.496047    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="44e35ee6-c1f9-4d6c-9584-2ebea688add8" containerName="payload-pre-processing"
Sep 16 01:16:40.496425 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:40.496157    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="26244697-ec2c-48f6-88f2-ce3e688e2add" containerName="maas-api"
Sep 16 01:16:40.496425 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:40.496170    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="44e35ee6-c1f9-4d6c-9584-2ebea688add8" containerName="payload-pre-processing"
Sep 16 01:16:40.496425 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:40.496182    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5b85f15-bbd3-4beb-97f0-22767c9571b9" containerName="payload-processing"
Sep 16 01:16:40.496425 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:40.496192    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="589f427b-d265-4a13-b266-9127f54be415" containerName="payload-processing"
Sep 16 01:16:40.499256 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:40.499232    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f46871-bbb9b964b-42slq"
Sep 16 01:16:40.501744 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:40.501720    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-f46871-serving-cert\""
Sep 16 01:16:40.508322 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:40.508261    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f46871-bbb9b964b-42slq"]
Sep 16 01:16:40.630924 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:40.630894    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rz9jl\" (UniqueName: \"kubernetes.io/projected/85ff90b1-cee8-4ecc-84b2-3258928d9d66-kube-api-access-rz9jl\") pod \"maas-api-e2e-ipp-cleanup-f46871-bbb9b964b-42slq\" (UID: \"85ff90b1-cee8-4ecc-84b2-3258928d9d66\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f46871-bbb9b964b-42slq"
Sep 16 01:16:40.631054 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:40.630950    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85ff90b1-cee8-4ecc-84b2-3258928d9d66-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-f46871-bbb9b964b-42slq\" (UID: \"85ff90b1-cee8-4ecc-84b2-3258928d9d66\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f46871-bbb9b964b-42slq"
Sep 16 01:16:40.631054 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:40.631021    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/85ff90b1-cee8-4ecc-84b2-3258928d9d66-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-f46871-bbb9b964b-42slq\" (UID: \"85ff90b1-cee8-4ecc-84b2-3258928d9d66\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f46871-bbb9b964b-42slq"
Sep 16 01:16:40.631138 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:40.631058    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85ff90b1-cee8-4ecc-84b2-3258928d9d66-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-f46871-bbb9b964b-42slq\" (UID: \"85ff90b1-cee8-4ecc-84b2-3258928d9d66\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f46871-bbb9b964b-42slq"
Sep 16 01:16:40.731954 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:40.731917    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85ff90b1-cee8-4ecc-84b2-3258928d9d66-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-f46871-bbb9b964b-42slq\" (UID: \"85ff90b1-cee8-4ecc-84b2-3258928d9d66\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f46871-bbb9b964b-42slq"
Sep 16 01:16:40.732078 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:40.731961    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/85ff90b1-cee8-4ecc-84b2-3258928d9d66-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-f46871-bbb9b964b-42slq\" (UID: \"85ff90b1-cee8-4ecc-84b2-3258928d9d66\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f46871-bbb9b964b-42slq"
Sep 16 01:16:40.732078 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:40.731983    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85ff90b1-cee8-4ecc-84b2-3258928d9d66-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-f46871-bbb9b964b-42slq\" (UID: \"85ff90b1-cee8-4ecc-84b2-3258928d9d66\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f46871-bbb9b964b-42slq"
Sep 16 01:16:40.732078 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:40.732034    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rz9jl\" (UniqueName: \"kubernetes.io/projected/85ff90b1-cee8-4ecc-84b2-3258928d9d66-kube-api-access-rz9jl\") pod \"maas-api-e2e-ipp-cleanup-f46871-bbb9b964b-42slq\" (UID: \"85ff90b1-cee8-4ecc-84b2-3258928d9d66\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f46871-bbb9b964b-42slq"
Sep 16 01:16:40.732718 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:40.732694    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/85ff90b1-cee8-4ecc-84b2-3258928d9d66-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-f46871-bbb9b964b-42slq\" (UID: \"85ff90b1-cee8-4ecc-84b2-3258928d9d66\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f46871-bbb9b964b-42slq"
Sep 16 01:16:40.734424 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:40.734401    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85ff90b1-cee8-4ecc-84b2-3258928d9d66-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-f46871-bbb9b964b-42slq\" (UID: \"85ff90b1-cee8-4ecc-84b2-3258928d9d66\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f46871-bbb9b964b-42slq"
Sep 16 01:16:40.734539 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:40.734444    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85ff90b1-cee8-4ecc-84b2-3258928d9d66-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-f46871-bbb9b964b-42slq\" (UID: \"85ff90b1-cee8-4ecc-84b2-3258928d9d66\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f46871-bbb9b964b-42slq"
Sep 16 01:16:40.740195 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:40.740170    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rz9jl\" (UniqueName: \"kubernetes.io/projected/85ff90b1-cee8-4ecc-84b2-3258928d9d66-kube-api-access-rz9jl\") pod \"maas-api-e2e-ipp-cleanup-f46871-bbb9b964b-42slq\" (UID: \"85ff90b1-cee8-4ecc-84b2-3258928d9d66\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f46871-bbb9b964b-42slq"
Sep 16 01:16:40.811842 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:40.811778    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f46871-bbb9b964b-42slq"
Sep 16 01:16:41.085789 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:41.085750    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f46871-bbb9b964b-42slq"]
Sep 16 01:16:41.092970 ip-10-0-130-207 kubenswrapper[2556]: W0916 01:16:41.092920    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85ff90b1_cee8_4ecc_84b2_3258928d9d66.slice/crio-4189f5f49410c0c1d9ecda83d097b1ebd6d4ea91d2a7b4b24403c9c85c9e1d07 WatchSource:0}: Error finding container 4189f5f49410c0c1d9ecda83d097b1ebd6d4ea91d2a7b4b24403c9c85c9e1d07: Status 404 returned error can't find the container with id 4189f5f49410c0c1d9ecda83d097b1ebd6d4ea91d2a7b4b24403c9c85c9e1d07
Sep 16 01:16:41.095135 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:41.095109    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 01:16:41.097327 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:41.097305    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f46871-5fb4b547f8-2mc6p"]
Sep 16 01:16:41.101933 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:41.101918    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f46871-5fb4b547f8-2mc6p"
Sep 16 01:16:41.104057 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:41.104034    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-f46871\""
Sep 16 01:16:41.104146 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:41.104041    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-f46871-dockercfg-rwq48\""
Sep 16 01:16:41.109453 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:41.109425    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f46871-5fb4b547f8-2mc6p"]
Sep 16 01:16:41.117989 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:41.117968    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-f46871-788665f459-4p8zg"]
Sep 16 01:16:41.121766 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:41.121513    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f46871-788665f459-4p8zg"
Sep 16 01:16:41.133954 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:41.133935    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-f46871-788665f459-4p8zg"]
Sep 16 01:16:41.236553 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:41.236521    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnt59\" (UniqueName: \"kubernetes.io/projected/d5c4fa18-0783-4ef0-8836-c85416112d72-kube-api-access-vnt59\") pod \"payload-processing-e2e-ipp-cleanup-f46871-788665f459-4p8zg\" (UID: \"d5c4fa18-0783-4ef0-8836-c85416112d72\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f46871-788665f459-4p8zg"
Sep 16 01:16:41.236704 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:41.236557    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d5c4fa18-0783-4ef0-8836-c85416112d72-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-f46871-788665f459-4p8zg\" (UID: \"d5c4fa18-0783-4ef0-8836-c85416112d72\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f46871-788665f459-4p8zg"
Sep 16 01:16:41.236704 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:41.236678    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7lzf\" (UniqueName: \"kubernetes.io/projected/f8f04ece-84fa-4078-860c-0aab1b1a293f-kube-api-access-k7lzf\") pod \"payload-pre-processing-e2e-ipp-cleanup-f46871-5fb4b547f8-2mc6p\" (UID: \"f8f04ece-84fa-4078-860c-0aab1b1a293f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f46871-5fb4b547f8-2mc6p"
Sep 16 01:16:41.236778 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:41.236756    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8f04ece-84fa-4078-860c-0aab1b1a293f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-f46871-5fb4b547f8-2mc6p\" (UID: \"f8f04ece-84fa-4078-860c-0aab1b1a293f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f46871-5fb4b547f8-2mc6p"
Sep 16 01:16:41.337996 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:41.337926    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7lzf\" (UniqueName: \"kubernetes.io/projected/f8f04ece-84fa-4078-860c-0aab1b1a293f-kube-api-access-k7lzf\") pod \"payload-pre-processing-e2e-ipp-cleanup-f46871-5fb4b547f8-2mc6p\" (UID: \"f8f04ece-84fa-4078-860c-0aab1b1a293f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f46871-5fb4b547f8-2mc6p"
Sep 16 01:16:41.337996 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:41.337990    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8f04ece-84fa-4078-860c-0aab1b1a293f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-f46871-5fb4b547f8-2mc6p\" (UID: \"f8f04ece-84fa-4078-860c-0aab1b1a293f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f46871-5fb4b547f8-2mc6p"
Sep 16 01:16:41.338171 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:41.338018    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnt59\" (UniqueName: \"kubernetes.io/projected/d5c4fa18-0783-4ef0-8836-c85416112d72-kube-api-access-vnt59\") pod \"payload-processing-e2e-ipp-cleanup-f46871-788665f459-4p8zg\" (UID: \"d5c4fa18-0783-4ef0-8836-c85416112d72\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f46871-788665f459-4p8zg"
Sep 16 01:16:41.338171 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:41.338035    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d5c4fa18-0783-4ef0-8836-c85416112d72-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-f46871-788665f459-4p8zg\" (UID: \"d5c4fa18-0783-4ef0-8836-c85416112d72\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f46871-788665f459-4p8zg"
Sep 16 01:16:41.338747 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:41.338729    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d5c4fa18-0783-4ef0-8836-c85416112d72-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-f46871-788665f459-4p8zg\" (UID: \"d5c4fa18-0783-4ef0-8836-c85416112d72\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f46871-788665f459-4p8zg"
Sep 16 01:16:41.338793 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:41.338742    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8f04ece-84fa-4078-860c-0aab1b1a293f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-f46871-5fb4b547f8-2mc6p\" (UID: \"f8f04ece-84fa-4078-860c-0aab1b1a293f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f46871-5fb4b547f8-2mc6p"
Sep 16 01:16:41.345372 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:41.345351    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7lzf\" (UniqueName: \"kubernetes.io/projected/f8f04ece-84fa-4078-860c-0aab1b1a293f-kube-api-access-k7lzf\") pod \"payload-pre-processing-e2e-ipp-cleanup-f46871-5fb4b547f8-2mc6p\" (UID: \"f8f04ece-84fa-4078-860c-0aab1b1a293f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f46871-5fb4b547f8-2mc6p"
Sep 16 01:16:41.346037 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:41.346017    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnt59\" (UniqueName: \"kubernetes.io/projected/d5c4fa18-0783-4ef0-8836-c85416112d72-kube-api-access-vnt59\") pod \"payload-processing-e2e-ipp-cleanup-f46871-788665f459-4p8zg\" (UID: \"d5c4fa18-0783-4ef0-8836-c85416112d72\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f46871-788665f459-4p8zg"
Sep 16 01:16:41.413875 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:41.413859    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f46871-5fb4b547f8-2mc6p"
Sep 16 01:16:41.433183 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:41.433163    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f46871-788665f459-4p8zg"
Sep 16 01:16:41.464746 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:41.464709    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f46871-bbb9b964b-42slq" event={"ID":"85ff90b1-cee8-4ecc-84b2-3258928d9d66","Type":"ContainerStarted","Data":"88566d00009c3a1efb3f8cf1e19affed9fdb7bfe67a197d69ce4c3ca11530235"}
Sep 16 01:16:41.464908 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:41.464775    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f46871-bbb9b964b-42slq" event={"ID":"85ff90b1-cee8-4ecc-84b2-3258928d9d66","Type":"ContainerStarted","Data":"4189f5f49410c0c1d9ecda83d097b1ebd6d4ea91d2a7b4b24403c9c85c9e1d07"}
Sep 16 01:16:41.464908 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:41.464867    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f46871-bbb9b964b-42slq"
Sep 16 01:16:41.484602 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:41.484543    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f46871-bbb9b964b-42slq" podStartSLOduration=1.2015440179999999 podStartE2EDuration="1.484519072s" totalImagesPullingTime="282.975054ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 01:16:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 01:16:41.479955457 +0000 UTC m=+2016.261457048" watchObservedRunningTime="2026-09-16 01:16:41.484519072 +0000 UTC m=+2016.266020666"
Sep 16 01:16:41.695057 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:41.695030    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f46871-5fb4b547f8-2mc6p"]
Sep 16 01:16:41.696619 ip-10-0-130-207 kubenswrapper[2556]: W0916 01:16:41.696587    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8f04ece_84fa_4078_860c_0aab1b1a293f.slice/crio-1811c88b31211ce90cf658e90d91bfac699d97c955b3a815833c192364f1e9a3 WatchSource:0}: Error finding container 1811c88b31211ce90cf658e90d91bfac699d97c955b3a815833c192364f1e9a3: Status 404 returned error can't find the container with id 1811c88b31211ce90cf658e90d91bfac699d97c955b3a815833c192364f1e9a3
Sep 16 01:16:41.720923 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:41.720895    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-f46871-788665f459-4p8zg"]
Sep 16 01:16:41.732857 ip-10-0-130-207 kubenswrapper[2556]: W0916 01:16:41.732829    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5c4fa18_0783_4ef0_8836_c85416112d72.slice/crio-62d40832f0ebde2292447b5c25740fb18f1b73b00937c0fa81062adaf86b8073 WatchSource:0}: Error finding container 62d40832f0ebde2292447b5c25740fb18f1b73b00937c0fa81062adaf86b8073: Status 404 returned error can't find the container with id 62d40832f0ebde2292447b5c25740fb18f1b73b00937c0fa81062adaf86b8073
Sep 16 01:16:42.472635 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:42.472599    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f46871-788665f459-4p8zg" event={"ID":"d5c4fa18-0783-4ef0-8836-c85416112d72","Type":"ContainerStarted","Data":"2fe13d9b0948638d4dbbbc7d9b5c608e5de8488c96aa41e8b1fe84b1fdf1bf1e"}
Sep 16 01:16:42.472838 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:42.472662    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f46871-788665f459-4p8zg" event={"ID":"d5c4fa18-0783-4ef0-8836-c85416112d72","Type":"ContainerStarted","Data":"62d40832f0ebde2292447b5c25740fb18f1b73b00937c0fa81062adaf86b8073"}
Sep 16 01:16:42.472838 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:42.472796    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f46871-788665f459-4p8zg"
Sep 16 01:16:42.474222 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:42.474199    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f46871-5fb4b547f8-2mc6p" event={"ID":"f8f04ece-84fa-4078-860c-0aab1b1a293f","Type":"ContainerStarted","Data":"011e57516b495b750d2a13641ab9ee5f6a898399aeca146dca21aeea16c470c9"}
Sep 16 01:16:42.474333 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:42.474241    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f46871-5fb4b547f8-2mc6p" event={"ID":"f8f04ece-84fa-4078-860c-0aab1b1a293f","Type":"ContainerStarted","Data":"1811c88b31211ce90cf658e90d91bfac699d97c955b3a815833c192364f1e9a3"}
Sep 16 01:16:42.474333 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:42.474299    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f46871-5fb4b547f8-2mc6p"
Sep 16 01:16:42.488207 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:42.488166    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f46871-788665f459-4p8zg" podStartSLOduration=1.488153138 podStartE2EDuration="1.488153138s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 01:16:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 01:16:42.487020232 +0000 UTC m=+2017.268521825" watchObservedRunningTime="2026-09-16 01:16:42.488153138 +0000 UTC m=+2017.269654730"
Sep 16 01:16:42.502643 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:42.502601    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f46871-5fb4b547f8-2mc6p" podStartSLOduration=1.5025893620000002 podStartE2EDuration="1.502589362s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 01:16:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 01:16:42.500609017 +0000 UTC m=+2017.282110637" watchObservedRunningTime="2026-09-16 01:16:42.502589362 +0000 UTC m=+2017.284090953"
Sep 16 01:16:47.480228 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:47.480192    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f46871-bbb9b964b-42slq"
Sep 16 01:16:48.790883 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:48.790848    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-45446b-57cd5466fb-n7fcv"]
Sep 16 01:16:48.791616 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:48.791579    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-45446b-57cd5466fb-n7fcv" podUID="5b8d636a-0055-4b3b-92b2-6c0c22f75a2a" containerName="maas-api" containerID="cri-o://49b6d86773af5e6dcae8fdd3fef387df4bbae67ac859447bdc678591eba082f6" gracePeriod=30
Sep 16 01:16:48.838932 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:48.838904    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-45446b-5f77bf4fbc-92529"]
Sep 16 01:16:48.839201 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:48.839175    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-45446b-5f77bf4fbc-92529" podUID="758e8dfc-7ff0-4716-8e79-24e9c5db468e" containerName="payload-processing" containerID="cri-o://6ca6f2574aa24819c88c988ff70b3d42d12fcea065e6d29d7b75dd3cb6ddf80d" gracePeriod=30
Sep 16 01:16:49.087892 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:49.087869    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-45446b-57cd5466fb-n7fcv"
Sep 16 01:16:49.120715 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:49.120688    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-45446b-5f77bf4fbc-92529"
Sep 16 01:16:49.210095 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:49.210065    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b8d636a-0055-4b3b-92b2-6c0c22f75a2a-openshift-service-ca\") pod \"5b8d636a-0055-4b3b-92b2-6c0c22f75a2a\" (UID: \"5b8d636a-0055-4b3b-92b2-6c0c22f75a2a\") "
Sep 16 01:16:49.210217 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:49.210100    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c676k\" (UniqueName: \"kubernetes.io/projected/5b8d636a-0055-4b3b-92b2-6c0c22f75a2a-kube-api-access-c676k\") pod \"5b8d636a-0055-4b3b-92b2-6c0c22f75a2a\" (UID: \"5b8d636a-0055-4b3b-92b2-6c0c22f75a2a\") "
Sep 16 01:16:49.210217 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:49.210182    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5b8d636a-0055-4b3b-92b2-6c0c22f75a2a-metrics-certs\") pod \"5b8d636a-0055-4b3b-92b2-6c0c22f75a2a\" (UID: \"5b8d636a-0055-4b3b-92b2-6c0c22f75a2a\") "
Sep 16 01:16:49.210217 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:49.210203    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b8d636a-0055-4b3b-92b2-6c0c22f75a2a-maas-api-tls\") pod \"5b8d636a-0055-4b3b-92b2-6c0c22f75a2a\" (UID: \"5b8d636a-0055-4b3b-92b2-6c0c22f75a2a\") "
Sep 16 01:16:49.210406 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:49.210383    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b8d636a-0055-4b3b-92b2-6c0c22f75a2a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5b8d636a-0055-4b3b-92b2-6c0c22f75a2a" (UID: "5b8d636a-0055-4b3b-92b2-6c0c22f75a2a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 01:16:49.210561 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:49.210544    2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b8d636a-0055-4b3b-92b2-6c0c22f75a2a-openshift-service-ca\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:16:49.212244 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:49.212223    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b8d636a-0055-4b3b-92b2-6c0c22f75a2a-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "5b8d636a-0055-4b3b-92b2-6c0c22f75a2a" (UID: "5b8d636a-0055-4b3b-92b2-6c0c22f75a2a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 01:16:49.212320 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:49.212268    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b8d636a-0055-4b3b-92b2-6c0c22f75a2a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5b8d636a-0055-4b3b-92b2-6c0c22f75a2a" (UID: "5b8d636a-0055-4b3b-92b2-6c0c22f75a2a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 01:16:49.212320 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:49.212279    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b8d636a-0055-4b3b-92b2-6c0c22f75a2a-kube-api-access-c676k" (OuterVolumeSpecName: "kube-api-access-c676k") pod "5b8d636a-0055-4b3b-92b2-6c0c22f75a2a" (UID: "5b8d636a-0055-4b3b-92b2-6c0c22f75a2a"). InnerVolumeSpecName "kube-api-access-c676k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 01:16:49.311369 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:49.311308    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/758e8dfc-7ff0-4716-8e79-24e9c5db468e-plugins-config-volume\") pod \"758e8dfc-7ff0-4716-8e79-24e9c5db468e\" (UID: \"758e8dfc-7ff0-4716-8e79-24e9c5db468e\") "
Sep 16 01:16:49.311530 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:49.311419    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-px6b7\" (UniqueName: \"kubernetes.io/projected/758e8dfc-7ff0-4716-8e79-24e9c5db468e-kube-api-access-px6b7\") pod \"758e8dfc-7ff0-4716-8e79-24e9c5db468e\" (UID: \"758e8dfc-7ff0-4716-8e79-24e9c5db468e\") "
Sep 16 01:16:49.311678 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:49.311660    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c676k\" (UniqueName: \"kubernetes.io/projected/5b8d636a-0055-4b3b-92b2-6c0c22f75a2a-kube-api-access-c676k\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:16:49.311678 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:49.311677    2556 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5b8d636a-0055-4b3b-92b2-6c0c22f75a2a-metrics-certs\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:16:49.311817 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:49.311686    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b8d636a-0055-4b3b-92b2-6c0c22f75a2a-maas-api-tls\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:16:49.311817 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:49.311708    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/758e8dfc-7ff0-4716-8e79-24e9c5db468e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "758e8dfc-7ff0-4716-8e79-24e9c5db468e" (UID: "758e8dfc-7ff0-4716-8e79-24e9c5db468e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 01:16:49.313282 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:49.313262    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/758e8dfc-7ff0-4716-8e79-24e9c5db468e-kube-api-access-px6b7" (OuterVolumeSpecName: "kube-api-access-px6b7") pod "758e8dfc-7ff0-4716-8e79-24e9c5db468e" (UID: "758e8dfc-7ff0-4716-8e79-24e9c5db468e"). InnerVolumeSpecName "kube-api-access-px6b7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 01:16:49.413082 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:49.413056    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-px6b7\" (UniqueName: \"kubernetes.io/projected/758e8dfc-7ff0-4716-8e79-24e9c5db468e-kube-api-access-px6b7\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:16:49.413082 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:49.413078    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/758e8dfc-7ff0-4716-8e79-24e9c5db468e-plugins-config-volume\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:16:49.515825 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:49.515799    2556 generic.go:356] "Generic (PLEG): container finished" podID="5b8d636a-0055-4b3b-92b2-6c0c22f75a2a" containerID="49b6d86773af5e6dcae8fdd3fef387df4bbae67ac859447bdc678591eba082f6" exitCode=0
Sep 16 01:16:49.515972 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:49.515851    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-45446b-57cd5466fb-n7fcv"
Sep 16 01:16:49.515972 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:49.515894    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-45446b-57cd5466fb-n7fcv" event={"ID":"5b8d636a-0055-4b3b-92b2-6c0c22f75a2a","Type":"ContainerDied","Data":"49b6d86773af5e6dcae8fdd3fef387df4bbae67ac859447bdc678591eba082f6"}
Sep 16 01:16:49.515972 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:49.515957    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-45446b-57cd5466fb-n7fcv" event={"ID":"5b8d636a-0055-4b3b-92b2-6c0c22f75a2a","Type":"ContainerDied","Data":"9e3336be8537c2e87b77b1ee4bce4bb6b08b92e596113301fcb86601dee236c7"}
Sep 16 01:16:49.516149 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:49.515986    2556 scope.go:117] "RemoveContainer" containerID="49b6d86773af5e6dcae8fdd3fef387df4bbae67ac859447bdc678591eba082f6"
Sep 16 01:16:49.517302 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:49.517284    2556 generic.go:356] "Generic (PLEG): container finished" podID="758e8dfc-7ff0-4716-8e79-24e9c5db468e" containerID="6ca6f2574aa24819c88c988ff70b3d42d12fcea065e6d29d7b75dd3cb6ddf80d" exitCode=0
Sep 16 01:16:49.517410 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:49.517341    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-45446b-5f77bf4fbc-92529"
Sep 16 01:16:49.517410 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:49.517363    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-45446b-5f77bf4fbc-92529" event={"ID":"758e8dfc-7ff0-4716-8e79-24e9c5db468e","Type":"ContainerDied","Data":"6ca6f2574aa24819c88c988ff70b3d42d12fcea065e6d29d7b75dd3cb6ddf80d"}
Sep 16 01:16:49.517552 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:49.517419    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-45446b-5f77bf4fbc-92529" event={"ID":"758e8dfc-7ff0-4716-8e79-24e9c5db468e","Type":"ContainerDied","Data":"c8770d7d6a73972b3d44a523051376b28432c43f87478097d5bfa1b57887b091"}
Sep 16 01:16:49.525522 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:49.525503    2556 scope.go:117] "RemoveContainer" containerID="49b6d86773af5e6dcae8fdd3fef387df4bbae67ac859447bdc678591eba082f6"
Sep 16 01:16:49.525780 ip-10-0-130-207 kubenswrapper[2556]: E0916 01:16:49.525760    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49b6d86773af5e6dcae8fdd3fef387df4bbae67ac859447bdc678591eba082f6\": container with ID starting with 49b6d86773af5e6dcae8fdd3fef387df4bbae67ac859447bdc678591eba082f6 not found: ID does not exist" containerID="49b6d86773af5e6dcae8fdd3fef387df4bbae67ac859447bdc678591eba082f6"
Sep 16 01:16:49.525844 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:49.525788    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49b6d86773af5e6dcae8fdd3fef387df4bbae67ac859447bdc678591eba082f6"} err="failed to get container status \"49b6d86773af5e6dcae8fdd3fef387df4bbae67ac859447bdc678591eba082f6\": rpc error: code = NotFound desc = could not find container \"49b6d86773af5e6dcae8fdd3fef387df4bbae67ac859447bdc678591eba082f6\": container with ID starting with 49b6d86773af5e6dcae8fdd3fef387df4bbae67ac859447bdc678591eba082f6 not found: ID does not exist"
Sep 16 01:16:49.525844 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:49.525804    2556 scope.go:117] "RemoveContainer" containerID="6ca6f2574aa24819c88c988ff70b3d42d12fcea065e6d29d7b75dd3cb6ddf80d"
Sep 16 01:16:49.533669 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:49.533652    2556 scope.go:117] "RemoveContainer" containerID="6ca6f2574aa24819c88c988ff70b3d42d12fcea065e6d29d7b75dd3cb6ddf80d"
Sep 16 01:16:49.533906 ip-10-0-130-207 kubenswrapper[2556]: E0916 01:16:49.533888    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ca6f2574aa24819c88c988ff70b3d42d12fcea065e6d29d7b75dd3cb6ddf80d\": container with ID starting with 6ca6f2574aa24819c88c988ff70b3d42d12fcea065e6d29d7b75dd3cb6ddf80d not found: ID does not exist" containerID="6ca6f2574aa24819c88c988ff70b3d42d12fcea065e6d29d7b75dd3cb6ddf80d"
Sep 16 01:16:49.533967 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:49.533916    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ca6f2574aa24819c88c988ff70b3d42d12fcea065e6d29d7b75dd3cb6ddf80d"} err="failed to get container status \"6ca6f2574aa24819c88c988ff70b3d42d12fcea065e6d29d7b75dd3cb6ddf80d\": rpc error: code = NotFound desc = could not find container \"6ca6f2574aa24819c88c988ff70b3d42d12fcea065e6d29d7b75dd3cb6ddf80d\": container with ID starting with 6ca6f2574aa24819c88c988ff70b3d42d12fcea065e6d29d7b75dd3cb6ddf80d not found: ID does not exist"
Sep 16 01:16:49.540223 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:49.540202    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-45446b-5f77bf4fbc-92529"]
Sep 16 01:16:49.543316 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:49.543295    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-45446b-5f77bf4fbc-92529"]
Sep 16 01:16:49.551641 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:49.551621    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-45446b-57cd5466fb-n7fcv"]
Sep 16 01:16:49.557061 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:49.557041    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-45446b-57cd5466fb-n7fcv"]
Sep 16 01:16:49.741760 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:49.738830    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b8d636a-0055-4b3b-92b2-6c0c22f75a2a" path="/var/lib/kubelet/pods/5b8d636a-0055-4b3b-92b2-6c0c22f75a2a/volumes"
Sep 16 01:16:49.741760 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:49.739407    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="758e8dfc-7ff0-4716-8e79-24e9c5db468e" path="/var/lib/kubelet/pods/758e8dfc-7ff0-4716-8e79-24e9c5db468e/volumes"
Sep 16 01:16:53.482486 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:53.482440    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f46871-5fb4b547f8-2mc6p"
Sep 16 01:16:53.482832 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:16:53.482621    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f46871-788665f459-4p8zg"
Sep 16 01:17:07.333190 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:07.333148    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f46871-bbb9b964b-42slq"]
Sep 16 01:17:07.333733 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:07.333389    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f46871-bbb9b964b-42slq" podUID="85ff90b1-cee8-4ecc-84b2-3258928d9d66" containerName="maas-api" containerID="cri-o://88566d00009c3a1efb3f8cf1e19affed9fdb7bfe67a197d69ce4c3ca11530235" gracePeriod=30
Sep 16 01:17:07.389266 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:07.389240    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-f46871-788665f459-4p8zg"]
Sep 16 01:17:07.389668 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:07.389492    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f46871-788665f459-4p8zg" podUID="d5c4fa18-0783-4ef0-8836-c85416112d72" containerName="payload-processing" containerID="cri-o://2fe13d9b0948638d4dbbbc7d9b5c608e5de8488c96aa41e8b1fe84b1fdf1bf1e" gracePeriod=30
Sep 16 01:17:07.402318 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:07.402244    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f46871-5fb4b547f8-2mc6p"]
Sep 16 01:17:07.402612 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:07.402575    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f46871-5fb4b547f8-2mc6p" podUID="f8f04ece-84fa-4078-860c-0aab1b1a293f" containerName="payload-pre-processing" containerID="cri-o://011e57516b495b750d2a13641ab9ee5f6a898399aeca146dca21aeea16c470c9" gracePeriod=30
Sep 16 01:17:07.474977 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:07.474937    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f46871-bbb9b964b-42slq" podUID="85ff90b1-cee8-4ecc-84b2-3258928d9d66" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.74:8443/health\": dial tcp 10.133.0.74:8443: connect: connection refused"
Sep 16 01:17:07.632973 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:07.632936    2556 generic.go:356] "Generic (PLEG): container finished" podID="d5c4fa18-0783-4ef0-8836-c85416112d72" containerID="2fe13d9b0948638d4dbbbc7d9b5c608e5de8488c96aa41e8b1fe84b1fdf1bf1e" exitCode=0
Sep 16 01:17:07.633150 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:07.633054    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f46871-788665f459-4p8zg" event={"ID":"d5c4fa18-0783-4ef0-8836-c85416112d72","Type":"ContainerDied","Data":"2fe13d9b0948638d4dbbbc7d9b5c608e5de8488c96aa41e8b1fe84b1fdf1bf1e"}
Sep 16 01:17:07.635366 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:07.635335    2556 generic.go:356] "Generic (PLEG): container finished" podID="f8f04ece-84fa-4078-860c-0aab1b1a293f" containerID="011e57516b495b750d2a13641ab9ee5f6a898399aeca146dca21aeea16c470c9" exitCode=0
Sep 16 01:17:07.635492 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:07.635422    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f46871-5fb4b547f8-2mc6p" event={"ID":"f8f04ece-84fa-4078-860c-0aab1b1a293f","Type":"ContainerDied","Data":"011e57516b495b750d2a13641ab9ee5f6a898399aeca146dca21aeea16c470c9"}
Sep 16 01:17:07.637460 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:07.637433    2556 generic.go:356] "Generic (PLEG): container finished" podID="85ff90b1-cee8-4ecc-84b2-3258928d9d66" containerID="88566d00009c3a1efb3f8cf1e19affed9fdb7bfe67a197d69ce4c3ca11530235" exitCode=0
Sep 16 01:17:07.637460 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:07.637493    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f46871-bbb9b964b-42slq" event={"ID":"85ff90b1-cee8-4ecc-84b2-3258928d9d66","Type":"ContainerDied","Data":"88566d00009c3a1efb3f8cf1e19affed9fdb7bfe67a197d69ce4c3ca11530235"}
Sep 16 01:17:07.692726 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:07.692704    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f46871-788665f459-4p8zg"
Sep 16 01:17:07.695858 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:07.695833    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f46871-bbb9b964b-42slq"
Sep 16 01:17:07.706635 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:07.706617    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f46871-5fb4b547f8-2mc6p"
Sep 16 01:17:07.754658 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:07.754627    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8f04ece-84fa-4078-860c-0aab1b1a293f-plugins-config-volume\") pod \"f8f04ece-84fa-4078-860c-0aab1b1a293f\" (UID: \"f8f04ece-84fa-4078-860c-0aab1b1a293f\") "
Sep 16 01:17:07.754826 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:07.754689    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rz9jl\" (UniqueName: \"kubernetes.io/projected/85ff90b1-cee8-4ecc-84b2-3258928d9d66-kube-api-access-rz9jl\") pod \"85ff90b1-cee8-4ecc-84b2-3258928d9d66\" (UID: \"85ff90b1-cee8-4ecc-84b2-3258928d9d66\") "
Sep 16 01:17:07.754826 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:07.754708    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85ff90b1-cee8-4ecc-84b2-3258928d9d66-metrics-certs\") pod \"85ff90b1-cee8-4ecc-84b2-3258928d9d66\" (UID: \"85ff90b1-cee8-4ecc-84b2-3258928d9d66\") "
Sep 16 01:17:07.754826 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:07.754736    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85ff90b1-cee8-4ecc-84b2-3258928d9d66-maas-api-tls\") pod \"85ff90b1-cee8-4ecc-84b2-3258928d9d66\" (UID: \"85ff90b1-cee8-4ecc-84b2-3258928d9d66\") "
Sep 16 01:17:07.754826 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:07.754764    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k7lzf\" (UniqueName: \"kubernetes.io/projected/f8f04ece-84fa-4078-860c-0aab1b1a293f-kube-api-access-k7lzf\") pod \"f8f04ece-84fa-4078-860c-0aab1b1a293f\" (UID: \"f8f04ece-84fa-4078-860c-0aab1b1a293f\") "
Sep 16 01:17:07.754826 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:07.754786    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vnt59\" (UniqueName: \"kubernetes.io/projected/d5c4fa18-0783-4ef0-8836-c85416112d72-kube-api-access-vnt59\") pod \"d5c4fa18-0783-4ef0-8836-c85416112d72\" (UID: \"d5c4fa18-0783-4ef0-8836-c85416112d72\") "
Sep 16 01:17:07.754826 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:07.754823    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/85ff90b1-cee8-4ecc-84b2-3258928d9d66-openshift-service-ca\") pod \"85ff90b1-cee8-4ecc-84b2-3258928d9d66\" (UID: \"85ff90b1-cee8-4ecc-84b2-3258928d9d66\") "
Sep 16 01:17:07.755174 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:07.754852    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d5c4fa18-0783-4ef0-8836-c85416112d72-plugins-config-volume\") pod \"d5c4fa18-0783-4ef0-8836-c85416112d72\" (UID: \"d5c4fa18-0783-4ef0-8836-c85416112d72\") "
Sep 16 01:17:07.755174 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:07.755113    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8f04ece-84fa-4078-860c-0aab1b1a293f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f8f04ece-84fa-4078-860c-0aab1b1a293f" (UID: "f8f04ece-84fa-4078-860c-0aab1b1a293f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 01:17:07.755647 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:07.755606    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85ff90b1-cee8-4ecc-84b2-3258928d9d66-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "85ff90b1-cee8-4ecc-84b2-3258928d9d66" (UID: "85ff90b1-cee8-4ecc-84b2-3258928d9d66"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 01:17:07.755971 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:07.755937    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5c4fa18-0783-4ef0-8836-c85416112d72-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d5c4fa18-0783-4ef0-8836-c85416112d72" (UID: "d5c4fa18-0783-4ef0-8836-c85416112d72"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 01:17:07.757214 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:07.757183    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85ff90b1-cee8-4ecc-84b2-3258928d9d66-kube-api-access-rz9jl" (OuterVolumeSpecName: "kube-api-access-rz9jl") pod "85ff90b1-cee8-4ecc-84b2-3258928d9d66" (UID: "85ff90b1-cee8-4ecc-84b2-3258928d9d66"). InnerVolumeSpecName "kube-api-access-rz9jl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 01:17:07.757831 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:07.757790    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/85ff90b1-cee8-4ecc-84b2-3258928d9d66-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "85ff90b1-cee8-4ecc-84b2-3258928d9d66" (UID: "85ff90b1-cee8-4ecc-84b2-3258928d9d66"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 01:17:07.757992 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:07.757965    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8f04ece-84fa-4078-860c-0aab1b1a293f-kube-api-access-k7lzf" (OuterVolumeSpecName: "kube-api-access-k7lzf") pod "f8f04ece-84fa-4078-860c-0aab1b1a293f" (UID: "f8f04ece-84fa-4078-860c-0aab1b1a293f"). InnerVolumeSpecName "kube-api-access-k7lzf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 01:17:07.757992 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:07.757980    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5c4fa18-0783-4ef0-8836-c85416112d72-kube-api-access-vnt59" (OuterVolumeSpecName: "kube-api-access-vnt59") pod "d5c4fa18-0783-4ef0-8836-c85416112d72" (UID: "d5c4fa18-0783-4ef0-8836-c85416112d72"). InnerVolumeSpecName "kube-api-access-vnt59". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 01:17:07.758272 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:07.758242    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/85ff90b1-cee8-4ecc-84b2-3258928d9d66-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "85ff90b1-cee8-4ecc-84b2-3258928d9d66" (UID: "85ff90b1-cee8-4ecc-84b2-3258928d9d66"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 01:17:07.855770 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:07.855708    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8f04ece-84fa-4078-860c-0aab1b1a293f-plugins-config-volume\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:17:07.855770 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:07.855733    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rz9jl\" (UniqueName: \"kubernetes.io/projected/85ff90b1-cee8-4ecc-84b2-3258928d9d66-kube-api-access-rz9jl\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:17:07.855770 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:07.855744    2556 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85ff90b1-cee8-4ecc-84b2-3258928d9d66-metrics-certs\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:17:07.855770 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:07.855752    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85ff90b1-cee8-4ecc-84b2-3258928d9d66-maas-api-tls\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:17:07.855770 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:07.855761    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k7lzf\" (UniqueName: \"kubernetes.io/projected/f8f04ece-84fa-4078-860c-0aab1b1a293f-kube-api-access-k7lzf\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:17:07.855770 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:07.855770    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vnt59\" (UniqueName: \"kubernetes.io/projected/d5c4fa18-0783-4ef0-8836-c85416112d72-kube-api-access-vnt59\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:17:07.856043 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:07.855779    2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/85ff90b1-cee8-4ecc-84b2-3258928d9d66-openshift-service-ca\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:17:07.856043 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:07.855787    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d5c4fa18-0783-4ef0-8836-c85416112d72-plugins-config-volume\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:17:08.643561 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:08.643495    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f46871-bbb9b964b-42slq"
Sep 16 01:17:08.643561 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:08.643518    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f46871-bbb9b964b-42slq" event={"ID":"85ff90b1-cee8-4ecc-84b2-3258928d9d66","Type":"ContainerDied","Data":"4189f5f49410c0c1d9ecda83d097b1ebd6d4ea91d2a7b4b24403c9c85c9e1d07"}
Sep 16 01:17:08.644018 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:08.643586    2556 scope.go:117] "RemoveContainer" containerID="88566d00009c3a1efb3f8cf1e19affed9fdb7bfe67a197d69ce4c3ca11530235"
Sep 16 01:17:08.645062 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:08.645039    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f46871-788665f459-4p8zg" event={"ID":"d5c4fa18-0783-4ef0-8836-c85416112d72","Type":"ContainerDied","Data":"62d40832f0ebde2292447b5c25740fb18f1b73b00937c0fa81062adaf86b8073"}
Sep 16 01:17:08.645146 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:08.645044    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f46871-788665f459-4p8zg"
Sep 16 01:17:08.646529 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:08.646501    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f46871-5fb4b547f8-2mc6p" event={"ID":"f8f04ece-84fa-4078-860c-0aab1b1a293f","Type":"ContainerDied","Data":"1811c88b31211ce90cf658e90d91bfac699d97c955b3a815833c192364f1e9a3"}
Sep 16 01:17:08.646687 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:08.646506    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f46871-5fb4b547f8-2mc6p"
Sep 16 01:17:08.654334 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:08.654311    2556 scope.go:117] "RemoveContainer" containerID="2fe13d9b0948638d4dbbbc7d9b5c608e5de8488c96aa41e8b1fe84b1fdf1bf1e"
Sep 16 01:17:08.662593 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:08.662577    2556 scope.go:117] "RemoveContainer" containerID="011e57516b495b750d2a13641ab9ee5f6a898399aeca146dca21aeea16c470c9"
Sep 16 01:17:08.668989 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:08.668967    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-f46871-788665f459-4p8zg"]
Sep 16 01:17:08.672330 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:08.672308    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-f46871-788665f459-4p8zg"]
Sep 16 01:17:08.681158 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:08.681135    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f46871-bbb9b964b-42slq"]
Sep 16 01:17:08.684776 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:08.684752    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f46871-bbb9b964b-42slq"]
Sep 16 01:17:08.693635 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:08.693606    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f46871-5fb4b547f8-2mc6p"]
Sep 16 01:17:08.695632 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:08.695612    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f46871-5fb4b547f8-2mc6p"]
Sep 16 01:17:09.736071 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:09.736035    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="85ff90b1-cee8-4ecc-84b2-3258928d9d66" path="/var/lib/kubelet/pods/85ff90b1-cee8-4ecc-84b2-3258928d9d66/volumes"
Sep 16 01:17:09.736457 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:09.736373    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d5c4fa18-0783-4ef0-8836-c85416112d72" path="/var/lib/kubelet/pods/d5c4fa18-0783-4ef0-8836-c85416112d72/volumes"
Sep 16 01:17:09.736735 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:09.736720    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f8f04ece-84fa-4078-860c-0aab1b1a293f" path="/var/lib/kubelet/pods/f8f04ece-84fa-4078-860c-0aab1b1a293f/volumes"
Sep 16 01:17:38.787377 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:38.787344    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-089563/ipp-route-089563-kserve-6c5d46dc9f-l58vh"]
Sep 16 01:17:53.806100 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:53.806050    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-089563/ipp-route-089563-kserve-6c5d46dc9f-l58vh" podUID="81f46a16-db8b-4387-89f4-5657401de37f" containerName="main" containerID="cri-o://a84d2f629af484f2dec69c4e81db994025ca8f726789db8b5781b317d84c4826" gracePeriod=45
Sep 16 01:17:54.060046 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:54.059992    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-089563/ipp-route-089563-kserve-6c5d46dc9f-l58vh"
Sep 16 01:17:54.231016 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:54.230986    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81f46a16-db8b-4387-89f4-5657401de37f-tmp-dir\") pod \"81f46a16-db8b-4387-89f4-5657401de37f\" (UID: \"81f46a16-db8b-4387-89f4-5657401de37f\") "
Sep 16 01:17:54.231199 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:54.231024    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81f46a16-db8b-4387-89f4-5657401de37f-model-cache\") pod \"81f46a16-db8b-4387-89f4-5657401de37f\" (UID: \"81f46a16-db8b-4387-89f4-5657401de37f\") "
Sep 16 01:17:54.231199 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:54.231045    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81f46a16-db8b-4387-89f4-5657401de37f-home\") pod \"81f46a16-db8b-4387-89f4-5657401de37f\" (UID: \"81f46a16-db8b-4387-89f4-5657401de37f\") "
Sep 16 01:17:54.231199 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:54.231072    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81f46a16-db8b-4387-89f4-5657401de37f-dshm\") pod \"81f46a16-db8b-4387-89f4-5657401de37f\" (UID: \"81f46a16-db8b-4387-89f4-5657401de37f\") "
Sep 16 01:17:54.231199 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:54.231112    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81f46a16-db8b-4387-89f4-5657401de37f-tls-certs\") pod \"81f46a16-db8b-4387-89f4-5657401de37f\" (UID: \"81f46a16-db8b-4387-89f4-5657401de37f\") "
Sep 16 01:17:54.231422 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:54.231203    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hwrtz\" (UniqueName: \"kubernetes.io/projected/81f46a16-db8b-4387-89f4-5657401de37f-kube-api-access-hwrtz\") pod \"81f46a16-db8b-4387-89f4-5657401de37f\" (UID: \"81f46a16-db8b-4387-89f4-5657401de37f\") "
Sep 16 01:17:54.231422 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:54.231313    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81f46a16-db8b-4387-89f4-5657401de37f-home" (OuterVolumeSpecName: "home") pod "81f46a16-db8b-4387-89f4-5657401de37f" (UID: "81f46a16-db8b-4387-89f4-5657401de37f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 01:17:54.231422 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:54.231318    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81f46a16-db8b-4387-89f4-5657401de37f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "81f46a16-db8b-4387-89f4-5657401de37f" (UID: "81f46a16-db8b-4387-89f4-5657401de37f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 01:17:54.231422 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:54.231342    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81f46a16-db8b-4387-89f4-5657401de37f-model-cache" (OuterVolumeSpecName: "model-cache") pod "81f46a16-db8b-4387-89f4-5657401de37f" (UID: "81f46a16-db8b-4387-89f4-5657401de37f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 01:17:54.231631 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:54.231534    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81f46a16-db8b-4387-89f4-5657401de37f-tmp-dir\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:17:54.231631 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:54.231556    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81f46a16-db8b-4387-89f4-5657401de37f-model-cache\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:17:54.231631 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:54.231570    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81f46a16-db8b-4387-89f4-5657401de37f-home\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:17:54.233352 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:54.233329    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81f46a16-db8b-4387-89f4-5657401de37f-kube-api-access-hwrtz" (OuterVolumeSpecName: "kube-api-access-hwrtz") pod "81f46a16-db8b-4387-89f4-5657401de37f" (UID: "81f46a16-db8b-4387-89f4-5657401de37f"). InnerVolumeSpecName "kube-api-access-hwrtz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 01:17:54.233454 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:54.233348    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81f46a16-db8b-4387-89f4-5657401de37f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "81f46a16-db8b-4387-89f4-5657401de37f" (UID: "81f46a16-db8b-4387-89f4-5657401de37f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 01:17:54.233454 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:54.233348    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81f46a16-db8b-4387-89f4-5657401de37f-dshm" (OuterVolumeSpecName: "dshm") pod "81f46a16-db8b-4387-89f4-5657401de37f" (UID: "81f46a16-db8b-4387-89f4-5657401de37f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 01:17:54.332863 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:54.332840    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hwrtz\" (UniqueName: \"kubernetes.io/projected/81f46a16-db8b-4387-89f4-5657401de37f-kube-api-access-hwrtz\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:17:54.332863 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:54.332862    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81f46a16-db8b-4387-89f4-5657401de37f-dshm\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:17:54.333007 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:54.332873    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81f46a16-db8b-4387-89f4-5657401de37f-tls-certs\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:17:54.899768 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:54.899739    2556 generic.go:356] "Generic (PLEG): container finished" podID="81f46a16-db8b-4387-89f4-5657401de37f" containerID="a84d2f629af484f2dec69c4e81db994025ca8f726789db8b5781b317d84c4826" exitCode=0
Sep 16 01:17:54.900235 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:54.899789    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-089563/ipp-route-089563-kserve-6c5d46dc9f-l58vh"
Sep 16 01:17:54.900235 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:54.899829    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-089563/ipp-route-089563-kserve-6c5d46dc9f-l58vh" event={"ID":"81f46a16-db8b-4387-89f4-5657401de37f","Type":"ContainerDied","Data":"a84d2f629af484f2dec69c4e81db994025ca8f726789db8b5781b317d84c4826"}
Sep 16 01:17:54.900235 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:54.899869    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-089563/ipp-route-089563-kserve-6c5d46dc9f-l58vh" event={"ID":"81f46a16-db8b-4387-89f4-5657401de37f","Type":"ContainerDied","Data":"33b9633218426daaf371cbd2398760442bf3439a941d1cda34ae4c5d7d5d06c1"}
Sep 16 01:17:54.900235 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:54.899886    2556 scope.go:117] "RemoveContainer" containerID="a84d2f629af484f2dec69c4e81db994025ca8f726789db8b5781b317d84c4826"
Sep 16 01:17:54.909067 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:54.909050    2556 scope.go:117] "RemoveContainer" containerID="a84d2f629af484f2dec69c4e81db994025ca8f726789db8b5781b317d84c4826"
Sep 16 01:17:54.909307 ip-10-0-130-207 kubenswrapper[2556]: E0916 01:17:54.909288    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a84d2f629af484f2dec69c4e81db994025ca8f726789db8b5781b317d84c4826\": container with ID starting with a84d2f629af484f2dec69c4e81db994025ca8f726789db8b5781b317d84c4826 not found: ID does not exist" containerID="a84d2f629af484f2dec69c4e81db994025ca8f726789db8b5781b317d84c4826"
Sep 16 01:17:54.909360 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:54.909313    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a84d2f629af484f2dec69c4e81db994025ca8f726789db8b5781b317d84c4826"} err="failed to get container status \"a84d2f629af484f2dec69c4e81db994025ca8f726789db8b5781b317d84c4826\": rpc error: code = NotFound desc = could not find container \"a84d2f629af484f2dec69c4e81db994025ca8f726789db8b5781b317d84c4826\": container with ID starting with a84d2f629af484f2dec69c4e81db994025ca8f726789db8b5781b317d84c4826 not found: ID does not exist"
Sep 16 01:17:54.919658 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:54.919635    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-089563/ipp-route-089563-kserve-6c5d46dc9f-l58vh"]
Sep 16 01:17:54.925044 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:54.925025    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-089563/ipp-route-089563-kserve-6c5d46dc9f-l58vh"]
Sep 16 01:17:55.735722 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:17:55.735692    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81f46a16-db8b-4387-89f4-5657401de37f" path="/var/lib/kubelet/pods/81f46a16-db8b-4387-89f4-5657401de37f/volumes"
Sep 16 01:18:32.217076 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:18:32.217043    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-195f51-6c8f7975cc-gzrj6"]
Sep 16 01:18:32.217567 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:18:32.217238    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-195f51-6c8f7975cc-gzrj6" podUID="f7f9620c-309a-4e52-a3c5-0f7a4dfb7fed" containerName="payload-processing" containerID="cri-o://3558bb58576739f437f1d6565e4674834a25861c6d0e6f6e1478d764565fc235" gracePeriod=30
Sep 16 01:18:32.223983 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:18:32.223953    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-195f51-5cc698659d-x8lzl"]
Sep 16 01:18:32.225121 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:18:32.224168    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-195f51-5cc698659d-x8lzl" podUID="ec919b37-44db-4ef5-90c9-b2791022b16b" containerName="payload-pre-processing" containerID="cri-o://11211fcdff0c43e3ccc834500042c40e11ca8ec75a5c6df2b5b2b68e51e8992f" gracePeriod=30
Sep 16 01:18:32.505624 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:18:32.505595    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-195f51-6c8f7975cc-gzrj6"
Sep 16 01:18:32.508968 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:18:32.508944    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-195f51-5cc698659d-x8lzl"
Sep 16 01:18:32.645565 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:18:32.645534    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-57v5p\" (UniqueName: \"kubernetes.io/projected/f7f9620c-309a-4e52-a3c5-0f7a4dfb7fed-kube-api-access-57v5p\") pod \"f7f9620c-309a-4e52-a3c5-0f7a4dfb7fed\" (UID: \"f7f9620c-309a-4e52-a3c5-0f7a4dfb7fed\") "
Sep 16 01:18:32.645565 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:18:32.645564    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7f9620c-309a-4e52-a3c5-0f7a4dfb7fed-plugins-config-volume\") pod \"f7f9620c-309a-4e52-a3c5-0f7a4dfb7fed\" (UID: \"f7f9620c-309a-4e52-a3c5-0f7a4dfb7fed\") "
Sep 16 01:18:32.645806 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:18:32.645589    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sbvfb\" (UniqueName: \"kubernetes.io/projected/ec919b37-44db-4ef5-90c9-b2791022b16b-kube-api-access-sbvfb\") pod \"ec919b37-44db-4ef5-90c9-b2791022b16b\" (UID: \"ec919b37-44db-4ef5-90c9-b2791022b16b\") "
Sep 16 01:18:32.645806 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:18:32.645613    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec919b37-44db-4ef5-90c9-b2791022b16b-plugins-config-volume\") pod \"ec919b37-44db-4ef5-90c9-b2791022b16b\" (UID: \"ec919b37-44db-4ef5-90c9-b2791022b16b\") "
Sep 16 01:18:32.646032 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:18:32.645996    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7f9620c-309a-4e52-a3c5-0f7a4dfb7fed-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f7f9620c-309a-4e52-a3c5-0f7a4dfb7fed" (UID: "f7f9620c-309a-4e52-a3c5-0f7a4dfb7fed"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 01:18:32.646163 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:18:32.646062    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec919b37-44db-4ef5-90c9-b2791022b16b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ec919b37-44db-4ef5-90c9-b2791022b16b" (UID: "ec919b37-44db-4ef5-90c9-b2791022b16b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 01:18:32.647798 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:18:32.647773    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7f9620c-309a-4e52-a3c5-0f7a4dfb7fed-kube-api-access-57v5p" (OuterVolumeSpecName: "kube-api-access-57v5p") pod "f7f9620c-309a-4e52-a3c5-0f7a4dfb7fed" (UID: "f7f9620c-309a-4e52-a3c5-0f7a4dfb7fed"). InnerVolumeSpecName "kube-api-access-57v5p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 01:18:32.647798 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:18:32.647785    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec919b37-44db-4ef5-90c9-b2791022b16b-kube-api-access-sbvfb" (OuterVolumeSpecName: "kube-api-access-sbvfb") pod "ec919b37-44db-4ef5-90c9-b2791022b16b" (UID: "ec919b37-44db-4ef5-90c9-b2791022b16b"). InnerVolumeSpecName "kube-api-access-sbvfb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 01:18:32.746480 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:18:32.746411    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-57v5p\" (UniqueName: \"kubernetes.io/projected/f7f9620c-309a-4e52-a3c5-0f7a4dfb7fed-kube-api-access-57v5p\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:18:32.746480 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:18:32.746432    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7f9620c-309a-4e52-a3c5-0f7a4dfb7fed-plugins-config-volume\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:18:32.746480 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:18:32.746444    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sbvfb\" (UniqueName: \"kubernetes.io/projected/ec919b37-44db-4ef5-90c9-b2791022b16b-kube-api-access-sbvfb\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:18:32.746480 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:18:32.746455    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec919b37-44db-4ef5-90c9-b2791022b16b-plugins-config-volume\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:18:33.102972 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:18:33.102939    2556 generic.go:356] "Generic (PLEG): container finished" podID="ec919b37-44db-4ef5-90c9-b2791022b16b" containerID="11211fcdff0c43e3ccc834500042c40e11ca8ec75a5c6df2b5b2b68e51e8992f" exitCode=0
Sep 16 01:18:33.103146 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:18:33.102985    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-195f51-5cc698659d-x8lzl"
Sep 16 01:18:33.103146 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:18:33.103020    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-195f51-5cc698659d-x8lzl" event={"ID":"ec919b37-44db-4ef5-90c9-b2791022b16b","Type":"ContainerDied","Data":"11211fcdff0c43e3ccc834500042c40e11ca8ec75a5c6df2b5b2b68e51e8992f"}
Sep 16 01:18:33.103146 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:18:33.103076    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-195f51-5cc698659d-x8lzl" event={"ID":"ec919b37-44db-4ef5-90c9-b2791022b16b","Type":"ContainerDied","Data":"c694952c133001852e5155bc0217c3f3b4a7cf8eb38b95f4c062e0c0b20d4bb0"}
Sep 16 01:18:33.103146 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:18:33.103100    2556 scope.go:117] "RemoveContainer" containerID="11211fcdff0c43e3ccc834500042c40e11ca8ec75a5c6df2b5b2b68e51e8992f"
Sep 16 01:18:33.104497 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:18:33.104451    2556 generic.go:356] "Generic (PLEG): container finished" podID="f7f9620c-309a-4e52-a3c5-0f7a4dfb7fed" containerID="3558bb58576739f437f1d6565e4674834a25861c6d0e6f6e1478d764565fc235" exitCode=0
Sep 16 01:18:33.104613 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:18:33.104521    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-195f51-6c8f7975cc-gzrj6"
Sep 16 01:18:33.104613 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:18:33.104517    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-195f51-6c8f7975cc-gzrj6" event={"ID":"f7f9620c-309a-4e52-a3c5-0f7a4dfb7fed","Type":"ContainerDied","Data":"3558bb58576739f437f1d6565e4674834a25861c6d0e6f6e1478d764565fc235"}
Sep 16 01:18:33.104727 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:18:33.104633    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-195f51-6c8f7975cc-gzrj6" event={"ID":"f7f9620c-309a-4e52-a3c5-0f7a4dfb7fed","Type":"ContainerDied","Data":"e2d2b5cde0601282407306dbb9cd4378c87f318bb58ce7c262f9b6aafe9fa8eb"}
Sep 16 01:18:33.115985 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:18:33.115963    2556 scope.go:117] "RemoveContainer" containerID="11211fcdff0c43e3ccc834500042c40e11ca8ec75a5c6df2b5b2b68e51e8992f"
Sep 16 01:18:33.116517 ip-10-0-130-207 kubenswrapper[2556]: E0916 01:18:33.116487    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11211fcdff0c43e3ccc834500042c40e11ca8ec75a5c6df2b5b2b68e51e8992f\": container with ID starting with 11211fcdff0c43e3ccc834500042c40e11ca8ec75a5c6df2b5b2b68e51e8992f not found: ID does not exist" containerID="11211fcdff0c43e3ccc834500042c40e11ca8ec75a5c6df2b5b2b68e51e8992f"
Sep 16 01:18:33.116717 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:18:33.116537    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11211fcdff0c43e3ccc834500042c40e11ca8ec75a5c6df2b5b2b68e51e8992f"} err="failed to get container status \"11211fcdff0c43e3ccc834500042c40e11ca8ec75a5c6df2b5b2b68e51e8992f\": rpc error: code = NotFound desc = could not find container \"11211fcdff0c43e3ccc834500042c40e11ca8ec75a5c6df2b5b2b68e51e8992f\": container with ID starting with 11211fcdff0c43e3ccc834500042c40e11ca8ec75a5c6df2b5b2b68e51e8992f not found: ID does not exist"
Sep 16 01:18:33.116717 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:18:33.116563    2556 scope.go:117] "RemoveContainer" containerID="3558bb58576739f437f1d6565e4674834a25861c6d0e6f6e1478d764565fc235"
Sep 16 01:18:33.124807 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:18:33.124791    2556 scope.go:117] "RemoveContainer" containerID="3558bb58576739f437f1d6565e4674834a25861c6d0e6f6e1478d764565fc235"
Sep 16 01:18:33.125050 ip-10-0-130-207 kubenswrapper[2556]: E0916 01:18:33.125025    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3558bb58576739f437f1d6565e4674834a25861c6d0e6f6e1478d764565fc235\": container with ID starting with 3558bb58576739f437f1d6565e4674834a25861c6d0e6f6e1478d764565fc235 not found: ID does not exist" containerID="3558bb58576739f437f1d6565e4674834a25861c6d0e6f6e1478d764565fc235"
Sep 16 01:18:33.125134 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:18:33.125049    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3558bb58576739f437f1d6565e4674834a25861c6d0e6f6e1478d764565fc235"} err="failed to get container status \"3558bb58576739f437f1d6565e4674834a25861c6d0e6f6e1478d764565fc235\": rpc error: code = NotFound desc = could not find container \"3558bb58576739f437f1d6565e4674834a25861c6d0e6f6e1478d764565fc235\": container with ID starting with 3558bb58576739f437f1d6565e4674834a25861c6d0e6f6e1478d764565fc235 not found: ID does not exist"
Sep 16 01:18:33.131197 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:18:33.131172    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-195f51-6c8f7975cc-gzrj6"]
Sep 16 01:18:33.134823 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:18:33.134801    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-195f51-6c8f7975cc-gzrj6"]
Sep 16 01:18:33.144195 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:18:33.144174    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-195f51-5cc698659d-x8lzl"]
Sep 16 01:18:33.148355 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:18:33.148334    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-195f51-5cc698659d-x8lzl"]
Sep 16 01:18:33.737132 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:18:33.737101    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec919b37-44db-4ef5-90c9-b2791022b16b" path="/var/lib/kubelet/pods/ec919b37-44db-4ef5-90c9-b2791022b16b/volumes"
Sep 16 01:18:33.737505 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:18:33.737436    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7f9620c-309a-4e52-a3c5-0f7a4dfb7fed" path="/var/lib/kubelet/pods/f7f9620c-309a-4e52-a3c5-0f7a4dfb7fed/volumes"
Sep 16 01:18:49.586233 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:18:49.586190    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-f831c5-29825355-9k4fb"]
Sep 16 01:18:49.588247 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:18:49.588222    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-f831c5-29825355-9k4fb"]
Sep 16 01:18:49.617415 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:18:49.617387    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f831c5-7fcf9bb6f8-86vjn"]
Sep 16 01:18:49.617759 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:18:49.617696    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f831c5-7fcf9bb6f8-86vjn" podUID="fdd181b6-f35b-4862-8988-23d546bd0c36" containerName="payload-pre-processing" containerID="cri-o://f7f38adadee82603606abfe93bb96fc40bf039eb0dd1560cd94f08bb6f044809" gracePeriod=30
Sep 16 01:18:49.737212 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:18:49.737177    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b97fa55f-e116-4e0f-9c9b-c2b3e7ca33ae" path="/var/lib/kubelet/pods/b97fa55f-e116-4e0f-9c9b-c2b3e7ca33ae/volumes"
Sep 16 01:18:49.865891 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:18:49.865863    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f831c5-7fcf9bb6f8-86vjn"
Sep 16 01:18:49.990880 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:18:49.990847    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9pfjm\" (UniqueName: \"kubernetes.io/projected/fdd181b6-f35b-4862-8988-23d546bd0c36-kube-api-access-9pfjm\") pod \"fdd181b6-f35b-4862-8988-23d546bd0c36\" (UID: \"fdd181b6-f35b-4862-8988-23d546bd0c36\") "
Sep 16 01:18:49.991033 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:18:49.990959    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fdd181b6-f35b-4862-8988-23d546bd0c36-plugins-config-volume\") pod \"fdd181b6-f35b-4862-8988-23d546bd0c36\" (UID: \"fdd181b6-f35b-4862-8988-23d546bd0c36\") "
Sep 16 01:18:49.991330 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:18:49.991292    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fdd181b6-f35b-4862-8988-23d546bd0c36-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fdd181b6-f35b-4862-8988-23d546bd0c36" (UID: "fdd181b6-f35b-4862-8988-23d546bd0c36"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 01:18:49.992855 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:18:49.992830    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fdd181b6-f35b-4862-8988-23d546bd0c36-kube-api-access-9pfjm" (OuterVolumeSpecName: "kube-api-access-9pfjm") pod "fdd181b6-f35b-4862-8988-23d546bd0c36" (UID: "fdd181b6-f35b-4862-8988-23d546bd0c36"). InnerVolumeSpecName "kube-api-access-9pfjm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 01:18:50.091584 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:18:50.091554    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fdd181b6-f35b-4862-8988-23d546bd0c36-plugins-config-volume\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:18:50.091584 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:18:50.091580    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9pfjm\" (UniqueName: \"kubernetes.io/projected/fdd181b6-f35b-4862-8988-23d546bd0c36-kube-api-access-9pfjm\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:18:50.196395 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:18:50.196317    2556 generic.go:356] "Generic (PLEG): container finished" podID="fdd181b6-f35b-4862-8988-23d546bd0c36" containerID="f7f38adadee82603606abfe93bb96fc40bf039eb0dd1560cd94f08bb6f044809" exitCode=0
Sep 16 01:18:50.196395 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:18:50.196365    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f831c5-7fcf9bb6f8-86vjn"
Sep 16 01:18:50.196610 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:18:50.196407    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f831c5-7fcf9bb6f8-86vjn" event={"ID":"fdd181b6-f35b-4862-8988-23d546bd0c36","Type":"ContainerDied","Data":"f7f38adadee82603606abfe93bb96fc40bf039eb0dd1560cd94f08bb6f044809"}
Sep 16 01:18:50.196610 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:18:50.196490    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f831c5-7fcf9bb6f8-86vjn" event={"ID":"fdd181b6-f35b-4862-8988-23d546bd0c36","Type":"ContainerDied","Data":"514150212c6fc7476777f6efb709d0c1648426fd5643a6a05471b44370fa892e"}
Sep 16 01:18:50.196610 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:18:50.196498    2556 scope.go:117] "RemoveContainer" containerID="f7f38adadee82603606abfe93bb96fc40bf039eb0dd1560cd94f08bb6f044809"
Sep 16 01:18:50.205332 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:18:50.205312    2556 scope.go:117] "RemoveContainer" containerID="f7f38adadee82603606abfe93bb96fc40bf039eb0dd1560cd94f08bb6f044809"
Sep 16 01:18:50.205615 ip-10-0-130-207 kubenswrapper[2556]: E0916 01:18:50.205598    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7f38adadee82603606abfe93bb96fc40bf039eb0dd1560cd94f08bb6f044809\": container with ID starting with f7f38adadee82603606abfe93bb96fc40bf039eb0dd1560cd94f08bb6f044809 not found: ID does not exist" containerID="f7f38adadee82603606abfe93bb96fc40bf039eb0dd1560cd94f08bb6f044809"
Sep 16 01:18:50.205666 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:18:50.205625    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7f38adadee82603606abfe93bb96fc40bf039eb0dd1560cd94f08bb6f044809"} err="failed to get container status \"f7f38adadee82603606abfe93bb96fc40bf039eb0dd1560cd94f08bb6f044809\": rpc error: code = NotFound desc = could not find container \"f7f38adadee82603606abfe93bb96fc40bf039eb0dd1560cd94f08bb6f044809\": container with ID starting with f7f38adadee82603606abfe93bb96fc40bf039eb0dd1560cd94f08bb6f044809 not found: ID does not exist"
Sep 16 01:18:50.218048 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:18:50.218007    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f831c5-7fcf9bb6f8-86vjn"]
Sep 16 01:18:50.220082 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:18:50.220065    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f831c5-7fcf9bb6f8-86vjn"]
Sep 16 01:18:51.736293 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:18:51.736254    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fdd181b6-f35b-4862-8988-23d546bd0c36" path="/var/lib/kubelet/pods/fdd181b6-f35b-4862-8988-23d546bd0c36/volumes"
Sep 16 01:19:07.370714 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:19:07.370681    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-0ee1bf-9d8459445-ddpbs"]
Sep 16 01:19:07.371157 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:19:07.370916    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-0ee1bf-9d8459445-ddpbs" podUID="26bcde78-012a-4c98-9d68-1d56904af202" containerName="payload-processing" containerID="cri-o://6c4bb89dd4efb7f65b15cf05e14afda9895256a175a80935b5ca73108691082a" gracePeriod=30
Sep 16 01:19:07.376912 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:19:07.376887    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0ee1bf-589b478bf4-tmjrm"]
Sep 16 01:19:07.377402 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:19:07.377352    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0ee1bf-589b478bf4-tmjrm" podUID="49eeadd0-10a0-4130-b174-0088166e8d60" containerName="payload-pre-processing" containerID="cri-o://966a7f60535e2a3f43cf2881190ebd4633874812b0494803d9e574f5bc58e6f2" gracePeriod=30
Sep 16 01:19:07.618014 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:19:07.617983    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-0ee1bf-9d8459445-ddpbs"
Sep 16 01:19:07.654805 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:19:07.654781    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0ee1bf-589b478bf4-tmjrm"
Sep 16 01:19:07.727082 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:19:07.727056    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26bcde78-012a-4c98-9d68-1d56904af202-plugins-config-volume\") pod \"26bcde78-012a-4c98-9d68-1d56904af202\" (UID: \"26bcde78-012a-4c98-9d68-1d56904af202\") "
Sep 16 01:19:07.727243 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:19:07.727148    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-trrrk\" (UniqueName: \"kubernetes.io/projected/26bcde78-012a-4c98-9d68-1d56904af202-kube-api-access-trrrk\") pod \"26bcde78-012a-4c98-9d68-1d56904af202\" (UID: \"26bcde78-012a-4c98-9d68-1d56904af202\") "
Sep 16 01:19:07.727243 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:19:07.727209    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49eeadd0-10a0-4130-b174-0088166e8d60-plugins-config-volume\") pod \"49eeadd0-10a0-4130-b174-0088166e8d60\" (UID: \"49eeadd0-10a0-4130-b174-0088166e8d60\") "
Sep 16 01:19:07.727243 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:19:07.727232    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xpgc4\" (UniqueName: \"kubernetes.io/projected/49eeadd0-10a0-4130-b174-0088166e8d60-kube-api-access-xpgc4\") pod \"49eeadd0-10a0-4130-b174-0088166e8d60\" (UID: \"49eeadd0-10a0-4130-b174-0088166e8d60\") "
Sep 16 01:19:07.727588 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:19:07.727561    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26bcde78-012a-4c98-9d68-1d56904af202-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "26bcde78-012a-4c98-9d68-1d56904af202" (UID: "26bcde78-012a-4c98-9d68-1d56904af202"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 01:19:07.727690 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:19:07.727607    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49eeadd0-10a0-4130-b174-0088166e8d60-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "49eeadd0-10a0-4130-b174-0088166e8d60" (UID: "49eeadd0-10a0-4130-b174-0088166e8d60"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 01:19:07.729245 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:19:07.729217    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26bcde78-012a-4c98-9d68-1d56904af202-kube-api-access-trrrk" (OuterVolumeSpecName: "kube-api-access-trrrk") pod "26bcde78-012a-4c98-9d68-1d56904af202" (UID: "26bcde78-012a-4c98-9d68-1d56904af202"). InnerVolumeSpecName "kube-api-access-trrrk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 01:19:07.729618 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:19:07.729269    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49eeadd0-10a0-4130-b174-0088166e8d60-kube-api-access-xpgc4" (OuterVolumeSpecName: "kube-api-access-xpgc4") pod "49eeadd0-10a0-4130-b174-0088166e8d60" (UID: "49eeadd0-10a0-4130-b174-0088166e8d60"). InnerVolumeSpecName "kube-api-access-xpgc4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 01:19:07.827909 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:19:07.827886    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-trrrk\" (UniqueName: \"kubernetes.io/projected/26bcde78-012a-4c98-9d68-1d56904af202-kube-api-access-trrrk\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:19:07.827909 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:19:07.827909    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49eeadd0-10a0-4130-b174-0088166e8d60-plugins-config-volume\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:19:07.828045 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:19:07.827920    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xpgc4\" (UniqueName: \"kubernetes.io/projected/49eeadd0-10a0-4130-b174-0088166e8d60-kube-api-access-xpgc4\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:19:07.828045 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:19:07.827934    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26bcde78-012a-4c98-9d68-1d56904af202-plugins-config-volume\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:19:08.294585 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:19:08.294556    2556 generic.go:356] "Generic (PLEG): container finished" podID="49eeadd0-10a0-4130-b174-0088166e8d60" containerID="966a7f60535e2a3f43cf2881190ebd4633874812b0494803d9e574f5bc58e6f2" exitCode=0
Sep 16 01:19:08.294757 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:19:08.294603    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0ee1bf-589b478bf4-tmjrm"
Sep 16 01:19:08.294757 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:19:08.294641    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0ee1bf-589b478bf4-tmjrm" event={"ID":"49eeadd0-10a0-4130-b174-0088166e8d60","Type":"ContainerDied","Data":"966a7f60535e2a3f43cf2881190ebd4633874812b0494803d9e574f5bc58e6f2"}
Sep 16 01:19:08.294757 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:19:08.294691    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0ee1bf-589b478bf4-tmjrm" event={"ID":"49eeadd0-10a0-4130-b174-0088166e8d60","Type":"ContainerDied","Data":"5cca7506b1065543513d177a70c3d8078cb8554b84277568b9683b61045084bc"}
Sep 16 01:19:08.294757 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:19:08.294716    2556 scope.go:117] "RemoveContainer" containerID="966a7f60535e2a3f43cf2881190ebd4633874812b0494803d9e574f5bc58e6f2"
Sep 16 01:19:08.296092 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:19:08.296072    2556 generic.go:356] "Generic (PLEG): container finished" podID="26bcde78-012a-4c98-9d68-1d56904af202" containerID="6c4bb89dd4efb7f65b15cf05e14afda9895256a175a80935b5ca73108691082a" exitCode=0
Sep 16 01:19:08.296217 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:19:08.296119    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-0ee1bf-9d8459445-ddpbs" event={"ID":"26bcde78-012a-4c98-9d68-1d56904af202","Type":"ContainerDied","Data":"6c4bb89dd4efb7f65b15cf05e14afda9895256a175a80935b5ca73108691082a"}
Sep 16 01:19:08.296217 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:19:08.296123    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-0ee1bf-9d8459445-ddpbs"
Sep 16 01:19:08.296217 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:19:08.296152    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-0ee1bf-9d8459445-ddpbs" event={"ID":"26bcde78-012a-4c98-9d68-1d56904af202","Type":"ContainerDied","Data":"28955dc954c1939ef0cf7546552f75e675f4adb3e1fa3455b059a3c809934875"}
Sep 16 01:19:08.303519 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:19:08.303495    2556 scope.go:117] "RemoveContainer" containerID="966a7f60535e2a3f43cf2881190ebd4633874812b0494803d9e574f5bc58e6f2"
Sep 16 01:19:08.303785 ip-10-0-130-207 kubenswrapper[2556]: E0916 01:19:08.303766    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"966a7f60535e2a3f43cf2881190ebd4633874812b0494803d9e574f5bc58e6f2\": container with ID starting with 966a7f60535e2a3f43cf2881190ebd4633874812b0494803d9e574f5bc58e6f2 not found: ID does not exist" containerID="966a7f60535e2a3f43cf2881190ebd4633874812b0494803d9e574f5bc58e6f2"
Sep 16 01:19:08.303848 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:19:08.303793    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"966a7f60535e2a3f43cf2881190ebd4633874812b0494803d9e574f5bc58e6f2"} err="failed to get container status \"966a7f60535e2a3f43cf2881190ebd4633874812b0494803d9e574f5bc58e6f2\": rpc error: code = NotFound desc = could not find container \"966a7f60535e2a3f43cf2881190ebd4633874812b0494803d9e574f5bc58e6f2\": container with ID starting with 966a7f60535e2a3f43cf2881190ebd4633874812b0494803d9e574f5bc58e6f2 not found: ID does not exist"
Sep 16 01:19:08.303848 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:19:08.303810    2556 scope.go:117] "RemoveContainer" containerID="6c4bb89dd4efb7f65b15cf05e14afda9895256a175a80935b5ca73108691082a"
Sep 16 01:19:08.311675 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:19:08.311658    2556 scope.go:117] "RemoveContainer" containerID="6c4bb89dd4efb7f65b15cf05e14afda9895256a175a80935b5ca73108691082a"
Sep 16 01:19:08.311993 ip-10-0-130-207 kubenswrapper[2556]: E0916 01:19:08.311973    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c4bb89dd4efb7f65b15cf05e14afda9895256a175a80935b5ca73108691082a\": container with ID starting with 6c4bb89dd4efb7f65b15cf05e14afda9895256a175a80935b5ca73108691082a not found: ID does not exist" containerID="6c4bb89dd4efb7f65b15cf05e14afda9895256a175a80935b5ca73108691082a"
Sep 16 01:19:08.312060 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:19:08.312000    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-0ee1bf-9d8459445-ddpbs"]
Sep 16 01:19:08.312060 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:19:08.312002    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c4bb89dd4efb7f65b15cf05e14afda9895256a175a80935b5ca73108691082a"} err="failed to get container status \"6c4bb89dd4efb7f65b15cf05e14afda9895256a175a80935b5ca73108691082a\": rpc error: code = NotFound desc = could not find container \"6c4bb89dd4efb7f65b15cf05e14afda9895256a175a80935b5ca73108691082a\": container with ID starting with 6c4bb89dd4efb7f65b15cf05e14afda9895256a175a80935b5ca73108691082a not found: ID does not exist"
Sep 16 01:19:08.315075 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:19:08.315054    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-0ee1bf-9d8459445-ddpbs"]
Sep 16 01:19:08.325197 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:19:08.325175    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0ee1bf-589b478bf4-tmjrm"]
Sep 16 01:19:08.334864 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:19:08.332520    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0ee1bf-589b478bf4-tmjrm"]
Sep 16 01:19:09.736043 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:19:09.735998    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26bcde78-012a-4c98-9d68-1d56904af202" path="/var/lib/kubelet/pods/26bcde78-012a-4c98-9d68-1d56904af202/volumes"
Sep 16 01:19:09.736570 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:19:09.736524    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49eeadd0-10a0-4130-b174-0088166e8d60" path="/var/lib/kubelet/pods/49eeadd0-10a0-4130-b174-0088166e8d60/volumes"
Sep 16 01:19:16.417139 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:19:16.417104    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6b69bf8998-k7cr8"]
Sep 16 01:19:16.417534 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:19:16.417356    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6b69bf8998-k7cr8" podUID="11cd5078-9a7c-4208-8805-f9c7b6711521" containerName="manager" containerID="cri-o://fc204e3dabbfbeba1aff559eb359912da35c35910a9285cb3c9fa8f4d8346986" gracePeriod=30
Sep 16 01:19:16.652425 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:19:16.652398    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b69bf8998-k7cr8"
Sep 16 01:19:16.690365 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:19:16.690293    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nk7wc\" (UniqueName: \"kubernetes.io/projected/11cd5078-9a7c-4208-8805-f9c7b6711521-kube-api-access-nk7wc\") pod \"11cd5078-9a7c-4208-8805-f9c7b6711521\" (UID: \"11cd5078-9a7c-4208-8805-f9c7b6711521\") "
Sep 16 01:19:16.690365 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:19:16.690345    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/11cd5078-9a7c-4208-8805-f9c7b6711521-metrics-certs\") pod \"11cd5078-9a7c-4208-8805-f9c7b6711521\" (UID: \"11cd5078-9a7c-4208-8805-f9c7b6711521\") "
Sep 16 01:19:16.690572 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:19:16.690374    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/11cd5078-9a7c-4208-8805-f9c7b6711521-webhook-cert\") pod \"11cd5078-9a7c-4208-8805-f9c7b6711521\" (UID: \"11cd5078-9a7c-4208-8805-f9c7b6711521\") "
Sep 16 01:19:16.692523 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:19:16.692441    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11cd5078-9a7c-4208-8805-f9c7b6711521-kube-api-access-nk7wc" (OuterVolumeSpecName: "kube-api-access-nk7wc") pod "11cd5078-9a7c-4208-8805-f9c7b6711521" (UID: "11cd5078-9a7c-4208-8805-f9c7b6711521"). InnerVolumeSpecName "kube-api-access-nk7wc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 01:19:16.692523 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:19:16.692496    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11cd5078-9a7c-4208-8805-f9c7b6711521-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "11cd5078-9a7c-4208-8805-f9c7b6711521" (UID: "11cd5078-9a7c-4208-8805-f9c7b6711521"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 01:19:16.692675 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:19:16.692657    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11cd5078-9a7c-4208-8805-f9c7b6711521-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "11cd5078-9a7c-4208-8805-f9c7b6711521" (UID: "11cd5078-9a7c-4208-8805-f9c7b6711521"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 01:19:16.792175 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:19:16.792140    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nk7wc\" (UniqueName: \"kubernetes.io/projected/11cd5078-9a7c-4208-8805-f9c7b6711521-kube-api-access-nk7wc\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:19:16.792175 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:19:16.792171    2556 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/11cd5078-9a7c-4208-8805-f9c7b6711521-metrics-certs\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:19:16.792329 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:19:16.792184    2556 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/11cd5078-9a7c-4208-8805-f9c7b6711521-webhook-cert\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:19:17.346287 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:19:17.346248    2556 generic.go:356] "Generic (PLEG): container finished" podID="11cd5078-9a7c-4208-8805-f9c7b6711521" containerID="fc204e3dabbfbeba1aff559eb359912da35c35910a9285cb3c9fa8f4d8346986" exitCode=0
Sep 16 01:19:17.346456 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:19:17.346304    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b69bf8998-k7cr8"
Sep 16 01:19:17.346456 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:19:17.346332    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b69bf8998-k7cr8" event={"ID":"11cd5078-9a7c-4208-8805-f9c7b6711521","Type":"ContainerDied","Data":"fc204e3dabbfbeba1aff559eb359912da35c35910a9285cb3c9fa8f4d8346986"}
Sep 16 01:19:17.346456 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:19:17.346398    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b69bf8998-k7cr8" event={"ID":"11cd5078-9a7c-4208-8805-f9c7b6711521","Type":"ContainerDied","Data":"d2fc46660d1dfc759f865d32f0bc152283c282ab186fce21b20c77db91520f48"}
Sep 16 01:19:17.346456 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:19:17.346429    2556 scope.go:117] "RemoveContainer" containerID="fc204e3dabbfbeba1aff559eb359912da35c35910a9285cb3c9fa8f4d8346986"
Sep 16 01:19:17.356660 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:19:17.356617    2556 scope.go:117] "RemoveContainer" containerID="fc204e3dabbfbeba1aff559eb359912da35c35910a9285cb3c9fa8f4d8346986"
Sep 16 01:19:17.356914 ip-10-0-130-207 kubenswrapper[2556]: E0916 01:19:17.356896    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc204e3dabbfbeba1aff559eb359912da35c35910a9285cb3c9fa8f4d8346986\": container with ID starting with fc204e3dabbfbeba1aff559eb359912da35c35910a9285cb3c9fa8f4d8346986 not found: ID does not exist" containerID="fc204e3dabbfbeba1aff559eb359912da35c35910a9285cb3c9fa8f4d8346986"
Sep 16 01:19:17.356961 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:19:17.356923    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc204e3dabbfbeba1aff559eb359912da35c35910a9285cb3c9fa8f4d8346986"} err="failed to get container status \"fc204e3dabbfbeba1aff559eb359912da35c35910a9285cb3c9fa8f4d8346986\": rpc error: code = NotFound desc = could not find container \"fc204e3dabbfbeba1aff559eb359912da35c35910a9285cb3c9fa8f4d8346986\": container with ID starting with fc204e3dabbfbeba1aff559eb359912da35c35910a9285cb3c9fa8f4d8346986 not found: ID does not exist"
Sep 16 01:19:17.366619 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:19:17.366593    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6b69bf8998-k7cr8"]
Sep 16 01:19:17.369694 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:19:17.369673    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6b69bf8998-k7cr8"]
Sep 16 01:19:17.735676 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:19:17.735611    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11cd5078-9a7c-4208-8805-f9c7b6711521" path="/var/lib/kubelet/pods/11cd5078-9a7c-4208-8805-f9c7b6711521/volumes"
Sep 16 01:20:18.247651 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:18.247616    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6b69bf8998-m76l8"]
Sep 16 01:20:18.248129 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:18.247920    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fdd181b6-f35b-4862-8988-23d546bd0c36" containerName="payload-pre-processing"
Sep 16 01:20:18.248129 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:18.247933    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="fdd181b6-f35b-4862-8988-23d546bd0c36" containerName="payload-pre-processing"
Sep 16 01:20:18.248129 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:18.247942    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8f04ece-84fa-4078-860c-0aab1b1a293f" containerName="payload-pre-processing"
Sep 16 01:20:18.248129 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:18.247947    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8f04ece-84fa-4078-860c-0aab1b1a293f" containerName="payload-pre-processing"
Sep 16 01:20:18.248129 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:18.247954    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85ff90b1-cee8-4ecc-84b2-3258928d9d66" containerName="maas-api"
Sep 16 01:20:18.248129 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:18.247960    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="85ff90b1-cee8-4ecc-84b2-3258928d9d66" containerName="maas-api"
Sep 16 01:20:18.248129 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:18.247967    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5c4fa18-0783-4ef0-8836-c85416112d72" containerName="payload-processing"
Sep 16 01:20:18.248129 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:18.247972    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5c4fa18-0783-4ef0-8836-c85416112d72" containerName="payload-processing"
Sep 16 01:20:18.248129 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:18.247981    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7f9620c-309a-4e52-a3c5-0f7a4dfb7fed" containerName="payload-processing"
Sep 16 01:20:18.248129 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:18.247987    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7f9620c-309a-4e52-a3c5-0f7a4dfb7fed" containerName="payload-processing"
Sep 16 01:20:18.248129 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:18.247995    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49eeadd0-10a0-4130-b174-0088166e8d60" containerName="payload-pre-processing"
Sep 16 01:20:18.248129 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:18.248000    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="49eeadd0-10a0-4130-b174-0088166e8d60" containerName="payload-pre-processing"
Sep 16 01:20:18.248129 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:18.248007    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26bcde78-012a-4c98-9d68-1d56904af202" containerName="payload-processing"
Sep 16 01:20:18.248129 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:18.248012    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="26bcde78-012a-4c98-9d68-1d56904af202" containerName="payload-processing"
Sep 16 01:20:18.248129 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:18.248019    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="758e8dfc-7ff0-4716-8e79-24e9c5db468e" containerName="payload-processing"
Sep 16 01:20:18.248129 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:18.248024    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="758e8dfc-7ff0-4716-8e79-24e9c5db468e" containerName="payload-processing"
Sep 16 01:20:18.248129 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:18.248032    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11cd5078-9a7c-4208-8805-f9c7b6711521" containerName="manager"
Sep 16 01:20:18.248129 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:18.248037    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="11cd5078-9a7c-4208-8805-f9c7b6711521" containerName="manager"
Sep 16 01:20:18.248129 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:18.248052    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b8d636a-0055-4b3b-92b2-6c0c22f75a2a" containerName="maas-api"
Sep 16 01:20:18.248129 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:18.248057    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b8d636a-0055-4b3b-92b2-6c0c22f75a2a" containerName="maas-api"
Sep 16 01:20:18.248129 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:18.248063    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec919b37-44db-4ef5-90c9-b2791022b16b" containerName="payload-pre-processing"
Sep 16 01:20:18.248129 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:18.248068    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec919b37-44db-4ef5-90c9-b2791022b16b" containerName="payload-pre-processing"
Sep 16 01:20:18.248129 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:18.248075    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81f46a16-db8b-4387-89f4-5657401de37f" containerName="main"
Sep 16 01:20:18.248129 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:18.248080    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="81f46a16-db8b-4387-89f4-5657401de37f" containerName="main"
Sep 16 01:20:18.248129 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:18.248126    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7f9620c-309a-4e52-a3c5-0f7a4dfb7fed" containerName="payload-processing"
Sep 16 01:20:18.248129 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:18.248133    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8f04ece-84fa-4078-860c-0aab1b1a293f" containerName="payload-pre-processing"
Sep 16 01:20:18.248129 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:18.248139    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="fdd181b6-f35b-4862-8988-23d546bd0c36" containerName="payload-pre-processing"
Sep 16 01:20:18.248129 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:18.248147    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="11cd5078-9a7c-4208-8805-f9c7b6711521" containerName="manager"
Sep 16 01:20:18.249238 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:18.248153    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="49eeadd0-10a0-4130-b174-0088166e8d60" containerName="payload-pre-processing"
Sep 16 01:20:18.249238 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:18.248159    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5c4fa18-0783-4ef0-8836-c85416112d72" containerName="payload-processing"
Sep 16 01:20:18.249238 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:18.248164    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="26bcde78-012a-4c98-9d68-1d56904af202" containerName="payload-processing"
Sep 16 01:20:18.249238 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:18.248170    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="85ff90b1-cee8-4ecc-84b2-3258928d9d66" containerName="maas-api"
Sep 16 01:20:18.249238 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:18.248175    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="758e8dfc-7ff0-4716-8e79-24e9c5db468e" containerName="payload-processing"
Sep 16 01:20:18.249238 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:18.248179    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b8d636a-0055-4b3b-92b2-6c0c22f75a2a" containerName="maas-api"
Sep 16 01:20:18.249238 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:18.248183    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec919b37-44db-4ef5-90c9-b2791022b16b" containerName="payload-pre-processing"
Sep 16 01:20:18.249238 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:18.248189    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="81f46a16-db8b-4387-89f4-5657401de37f" containerName="main"
Sep 16 01:20:18.251151 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:18.251134    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b69bf8998-m76l8"
Sep 16 01:20:18.254007 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:18.253985    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 16 01:20:18.254007 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:18.253993    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 16 01:20:18.254007 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:18.253991    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-q5q6q\""
Sep 16 01:20:18.254238 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:18.254048    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 16 01:20:18.259738 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:18.259706    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/69c0ed48-9477-4878-b6ed-9ddf402e5809-webhook-cert\") pod \"maas-controller-6b69bf8998-m76l8\" (UID: \"69c0ed48-9477-4878-b6ed-9ddf402e5809\") " pod="opendatahub/maas-controller-6b69bf8998-m76l8"
Sep 16 01:20:18.259851 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:18.259782    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vq68\" (UniqueName: \"kubernetes.io/projected/69c0ed48-9477-4878-b6ed-9ddf402e5809-kube-api-access-6vq68\") pod \"maas-controller-6b69bf8998-m76l8\" (UID: \"69c0ed48-9477-4878-b6ed-9ddf402e5809\") " pod="opendatahub/maas-controller-6b69bf8998-m76l8"
Sep 16 01:20:18.259929 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:18.259856    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/69c0ed48-9477-4878-b6ed-9ddf402e5809-metrics-certs\") pod \"maas-controller-6b69bf8998-m76l8\" (UID: \"69c0ed48-9477-4878-b6ed-9ddf402e5809\") " pod="opendatahub/maas-controller-6b69bf8998-m76l8"
Sep 16 01:20:18.261156 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:18.261130    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6b69bf8998-m76l8"]
Sep 16 01:20:18.360225 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:18.360199    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/69c0ed48-9477-4878-b6ed-9ddf402e5809-webhook-cert\") pod \"maas-controller-6b69bf8998-m76l8\" (UID: \"69c0ed48-9477-4878-b6ed-9ddf402e5809\") " pod="opendatahub/maas-controller-6b69bf8998-m76l8"
Sep 16 01:20:18.360337 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:18.360241    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vq68\" (UniqueName: \"kubernetes.io/projected/69c0ed48-9477-4878-b6ed-9ddf402e5809-kube-api-access-6vq68\") pod \"maas-controller-6b69bf8998-m76l8\" (UID: \"69c0ed48-9477-4878-b6ed-9ddf402e5809\") " pod="opendatahub/maas-controller-6b69bf8998-m76l8"
Sep 16 01:20:18.360337 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:18.360267    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/69c0ed48-9477-4878-b6ed-9ddf402e5809-metrics-certs\") pod \"maas-controller-6b69bf8998-m76l8\" (UID: \"69c0ed48-9477-4878-b6ed-9ddf402e5809\") " pod="opendatahub/maas-controller-6b69bf8998-m76l8"
Sep 16 01:20:18.362686 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:18.362662    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/69c0ed48-9477-4878-b6ed-9ddf402e5809-webhook-cert\") pod \"maas-controller-6b69bf8998-m76l8\" (UID: \"69c0ed48-9477-4878-b6ed-9ddf402e5809\") " pod="opendatahub/maas-controller-6b69bf8998-m76l8"
Sep 16 01:20:18.362818 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:18.362796    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/69c0ed48-9477-4878-b6ed-9ddf402e5809-metrics-certs\") pod \"maas-controller-6b69bf8998-m76l8\" (UID: \"69c0ed48-9477-4878-b6ed-9ddf402e5809\") " pod="opendatahub/maas-controller-6b69bf8998-m76l8"
Sep 16 01:20:18.367179 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:18.367160    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vq68\" (UniqueName: \"kubernetes.io/projected/69c0ed48-9477-4878-b6ed-9ddf402e5809-kube-api-access-6vq68\") pod \"maas-controller-6b69bf8998-m76l8\" (UID: \"69c0ed48-9477-4878-b6ed-9ddf402e5809\") " pod="opendatahub/maas-controller-6b69bf8998-m76l8"
Sep 16 01:20:18.561845 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:18.561760    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b69bf8998-m76l8"
Sep 16 01:20:18.830354 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:18.830326    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6b69bf8998-m76l8"]
Sep 16 01:20:18.832293 ip-10-0-130-207 kubenswrapper[2556]: W0916 01:20:18.832260    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69c0ed48_9477_4878_b6ed_9ddf402e5809.slice/crio-71031abe3142eaa51b2cdeada413e52be09536689cd6b5ba8ec77c8b7726829d WatchSource:0}: Error finding container 71031abe3142eaa51b2cdeada413e52be09536689cd6b5ba8ec77c8b7726829d: Status 404 returned error can't find the container with id 71031abe3142eaa51b2cdeada413e52be09536689cd6b5ba8ec77c8b7726829d
Sep 16 01:20:19.662744 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:19.662704    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b69bf8998-m76l8" event={"ID":"69c0ed48-9477-4878-b6ed-9ddf402e5809","Type":"ContainerStarted","Data":"8f7810defaee663b11268acf2147181b9d30fbcba97b511b282c02dee21d7733"}
Sep 16 01:20:19.663121 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:19.662765    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b69bf8998-m76l8" event={"ID":"69c0ed48-9477-4878-b6ed-9ddf402e5809","Type":"ContainerStarted","Data":"71031abe3142eaa51b2cdeada413e52be09536689cd6b5ba8ec77c8b7726829d"}
Sep 16 01:20:19.663121 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:19.662797    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6b69bf8998-m76l8"
Sep 16 01:20:19.678031 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:19.677979    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6b69bf8998-m76l8" podStartSLOduration=1.23390202 podStartE2EDuration="1.677964254s" totalImagesPullingTime="444.062234ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 01:20:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 01:20:19.677206387 +0000 UTC m=+2234.458707980" watchObservedRunningTime="2026-09-16 01:20:19.677964254 +0000 UTC m=+2234.459465843"
Sep 16 01:20:30.669708 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:30.669676    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6b69bf8998-m76l8"
Sep 16 01:20:40.058640 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:40.058605    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6b69bf8998-m76l8"]
Sep 16 01:20:40.059100 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:40.058812    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6b69bf8998-m76l8" podUID="69c0ed48-9477-4878-b6ed-9ddf402e5809" containerName="manager" containerID="cri-o://8f7810defaee663b11268acf2147181b9d30fbcba97b511b282c02dee21d7733" gracePeriod=30
Sep 16 01:20:40.301824 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:40.301800    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b69bf8998-m76l8"
Sep 16 01:20:40.425000 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:40.424970    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/69c0ed48-9477-4878-b6ed-9ddf402e5809-metrics-certs\") pod \"69c0ed48-9477-4878-b6ed-9ddf402e5809\" (UID: \"69c0ed48-9477-4878-b6ed-9ddf402e5809\") "
Sep 16 01:20:40.425172 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:40.425008    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/69c0ed48-9477-4878-b6ed-9ddf402e5809-webhook-cert\") pod \"69c0ed48-9477-4878-b6ed-9ddf402e5809\" (UID: \"69c0ed48-9477-4878-b6ed-9ddf402e5809\") "
Sep 16 01:20:40.425172 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:40.425050    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6vq68\" (UniqueName: \"kubernetes.io/projected/69c0ed48-9477-4878-b6ed-9ddf402e5809-kube-api-access-6vq68\") pod \"69c0ed48-9477-4878-b6ed-9ddf402e5809\" (UID: \"69c0ed48-9477-4878-b6ed-9ddf402e5809\") "
Sep 16 01:20:40.427023 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:40.426993    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69c0ed48-9477-4878-b6ed-9ddf402e5809-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "69c0ed48-9477-4878-b6ed-9ddf402e5809" (UID: "69c0ed48-9477-4878-b6ed-9ddf402e5809"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 01:20:40.427138 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:40.427052    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69c0ed48-9477-4878-b6ed-9ddf402e5809-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "69c0ed48-9477-4878-b6ed-9ddf402e5809" (UID: "69c0ed48-9477-4878-b6ed-9ddf402e5809"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 01:20:40.427181 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:40.427140    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69c0ed48-9477-4878-b6ed-9ddf402e5809-kube-api-access-6vq68" (OuterVolumeSpecName: "kube-api-access-6vq68") pod "69c0ed48-9477-4878-b6ed-9ddf402e5809" (UID: "69c0ed48-9477-4878-b6ed-9ddf402e5809"). InnerVolumeSpecName "kube-api-access-6vq68". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 01:20:40.525790 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:40.525765    2556 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/69c0ed48-9477-4878-b6ed-9ddf402e5809-metrics-certs\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:20:40.525790 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:40.525788    2556 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/69c0ed48-9477-4878-b6ed-9ddf402e5809-webhook-cert\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:20:40.525940 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:40.525798    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6vq68\" (UniqueName: \"kubernetes.io/projected/69c0ed48-9477-4878-b6ed-9ddf402e5809-kube-api-access-6vq68\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:20:40.763318 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:40.763247    2556 generic.go:356] "Generic (PLEG): container finished" podID="69c0ed48-9477-4878-b6ed-9ddf402e5809" containerID="8f7810defaee663b11268acf2147181b9d30fbcba97b511b282c02dee21d7733" exitCode=0
Sep 16 01:20:40.763318 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:40.763292    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b69bf8998-m76l8"
Sep 16 01:20:40.763460 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:40.763325    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b69bf8998-m76l8" event={"ID":"69c0ed48-9477-4878-b6ed-9ddf402e5809","Type":"ContainerDied","Data":"8f7810defaee663b11268acf2147181b9d30fbcba97b511b282c02dee21d7733"}
Sep 16 01:20:40.763460 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:40.763372    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b69bf8998-m76l8" event={"ID":"69c0ed48-9477-4878-b6ed-9ddf402e5809","Type":"ContainerDied","Data":"71031abe3142eaa51b2cdeada413e52be09536689cd6b5ba8ec77c8b7726829d"}
Sep 16 01:20:40.763460 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:40.763394    2556 scope.go:117] "RemoveContainer" containerID="8f7810defaee663b11268acf2147181b9d30fbcba97b511b282c02dee21d7733"
Sep 16 01:20:40.772603 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:40.772584    2556 scope.go:117] "RemoveContainer" containerID="8f7810defaee663b11268acf2147181b9d30fbcba97b511b282c02dee21d7733"
Sep 16 01:20:40.772882 ip-10-0-130-207 kubenswrapper[2556]: E0916 01:20:40.772862    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f7810defaee663b11268acf2147181b9d30fbcba97b511b282c02dee21d7733\": container with ID starting with 8f7810defaee663b11268acf2147181b9d30fbcba97b511b282c02dee21d7733 not found: ID does not exist" containerID="8f7810defaee663b11268acf2147181b9d30fbcba97b511b282c02dee21d7733"
Sep 16 01:20:40.773495 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:40.773188    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f7810defaee663b11268acf2147181b9d30fbcba97b511b282c02dee21d7733"} err="failed to get container status \"8f7810defaee663b11268acf2147181b9d30fbcba97b511b282c02dee21d7733\": rpc error: code = NotFound desc = could not find container \"8f7810defaee663b11268acf2147181b9d30fbcba97b511b282c02dee21d7733\": container with ID starting with 8f7810defaee663b11268acf2147181b9d30fbcba97b511b282c02dee21d7733 not found: ID does not exist"
Sep 16 01:20:40.788131 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:40.788099    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6b69bf8998-m76l8"]
Sep 16 01:20:40.793017 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:40.792995    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6b69bf8998-m76l8"]
Sep 16 01:20:41.673063 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:41.673027    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6b69bf8998-lrpcj"]
Sep 16 01:20:41.673428 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:41.673354    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69c0ed48-9477-4878-b6ed-9ddf402e5809" containerName="manager"
Sep 16 01:20:41.673428 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:41.673365    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="69c0ed48-9477-4878-b6ed-9ddf402e5809" containerName="manager"
Sep 16 01:20:41.673428 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:41.673426    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="69c0ed48-9477-4878-b6ed-9ddf402e5809" containerName="manager"
Sep 16 01:20:41.676286 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:41.676261    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b69bf8998-lrpcj"
Sep 16 01:20:41.679233 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:41.679212    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 16 01:20:41.679347 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:41.679212    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-q5q6q\""
Sep 16 01:20:41.679347 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:41.679216    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 16 01:20:41.679347 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:41.679214    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 16 01:20:41.684454 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:41.684425    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6b69bf8998-lrpcj"]
Sep 16 01:20:41.733045 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:41.733021    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7cf9\" (UniqueName: \"kubernetes.io/projected/c42b4555-b040-42e8-a3fe-a51d9ffdb55f-kube-api-access-r7cf9\") pod \"maas-controller-6b69bf8998-lrpcj\" (UID: \"c42b4555-b040-42e8-a3fe-a51d9ffdb55f\") " pod="opendatahub/maas-controller-6b69bf8998-lrpcj"
Sep 16 01:20:41.733168 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:41.733069    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c42b4555-b040-42e8-a3fe-a51d9ffdb55f-metrics-certs\") pod \"maas-controller-6b69bf8998-lrpcj\" (UID: \"c42b4555-b040-42e8-a3fe-a51d9ffdb55f\") " pod="opendatahub/maas-controller-6b69bf8998-lrpcj"
Sep 16 01:20:41.733168 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:41.733128    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c42b4555-b040-42e8-a3fe-a51d9ffdb55f-webhook-cert\") pod \"maas-controller-6b69bf8998-lrpcj\" (UID: \"c42b4555-b040-42e8-a3fe-a51d9ffdb55f\") " pod="opendatahub/maas-controller-6b69bf8998-lrpcj"
Sep 16 01:20:41.735334 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:41.735310    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69c0ed48-9477-4878-b6ed-9ddf402e5809" path="/var/lib/kubelet/pods/69c0ed48-9477-4878-b6ed-9ddf402e5809/volumes"
Sep 16 01:20:41.834320 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:41.834293    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c42b4555-b040-42e8-a3fe-a51d9ffdb55f-webhook-cert\") pod \"maas-controller-6b69bf8998-lrpcj\" (UID: \"c42b4555-b040-42e8-a3fe-a51d9ffdb55f\") " pod="opendatahub/maas-controller-6b69bf8998-lrpcj"
Sep 16 01:20:41.834427 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:41.834345    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7cf9\" (UniqueName: \"kubernetes.io/projected/c42b4555-b040-42e8-a3fe-a51d9ffdb55f-kube-api-access-r7cf9\") pod \"maas-controller-6b69bf8998-lrpcj\" (UID: \"c42b4555-b040-42e8-a3fe-a51d9ffdb55f\") " pod="opendatahub/maas-controller-6b69bf8998-lrpcj"
Sep 16 01:20:41.834427 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:41.834379    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c42b4555-b040-42e8-a3fe-a51d9ffdb55f-metrics-certs\") pod \"maas-controller-6b69bf8998-lrpcj\" (UID: \"c42b4555-b040-42e8-a3fe-a51d9ffdb55f\") " pod="opendatahub/maas-controller-6b69bf8998-lrpcj"
Sep 16 01:20:41.836760 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:41.836737    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c42b4555-b040-42e8-a3fe-a51d9ffdb55f-webhook-cert\") pod \"maas-controller-6b69bf8998-lrpcj\" (UID: \"c42b4555-b040-42e8-a3fe-a51d9ffdb55f\") " pod="opendatahub/maas-controller-6b69bf8998-lrpcj"
Sep 16 01:20:41.836760 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:41.836756    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c42b4555-b040-42e8-a3fe-a51d9ffdb55f-metrics-certs\") pod \"maas-controller-6b69bf8998-lrpcj\" (UID: \"c42b4555-b040-42e8-a3fe-a51d9ffdb55f\") " pod="opendatahub/maas-controller-6b69bf8998-lrpcj"
Sep 16 01:20:41.841479 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:41.841437    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7cf9\" (UniqueName: \"kubernetes.io/projected/c42b4555-b040-42e8-a3fe-a51d9ffdb55f-kube-api-access-r7cf9\") pod \"maas-controller-6b69bf8998-lrpcj\" (UID: \"c42b4555-b040-42e8-a3fe-a51d9ffdb55f\") " pod="opendatahub/maas-controller-6b69bf8998-lrpcj"
Sep 16 01:20:41.986721 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:41.986652    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b69bf8998-lrpcj"
Sep 16 01:20:42.258529 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:42.258505    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6b69bf8998-lrpcj"]
Sep 16 01:20:42.260288 ip-10-0-130-207 kubenswrapper[2556]: W0916 01:20:42.260261    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc42b4555_b040_42e8_a3fe_a51d9ffdb55f.slice/crio-f727f43ef650b6721f3f0779d4c2483778d03db071dd2920322b65955aa5bf7d WatchSource:0}: Error finding container f727f43ef650b6721f3f0779d4c2483778d03db071dd2920322b65955aa5bf7d: Status 404 returned error can't find the container with id f727f43ef650b6721f3f0779d4c2483778d03db071dd2920322b65955aa5bf7d
Sep 16 01:20:42.774939 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:42.774906    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b69bf8998-lrpcj" event={"ID":"c42b4555-b040-42e8-a3fe-a51d9ffdb55f","Type":"ContainerStarted","Data":"c0f2b55e11115fd1a39b91b29e664191f23e5a5a2fe44067519e0f2a10213ed6"}
Sep 16 01:20:42.775496 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:42.774954    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b69bf8998-lrpcj" event={"ID":"c42b4555-b040-42e8-a3fe-a51d9ffdb55f","Type":"ContainerStarted","Data":"f727f43ef650b6721f3f0779d4c2483778d03db071dd2920322b65955aa5bf7d"}
Sep 16 01:20:42.775496 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:42.775023    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6b69bf8998-lrpcj"
Sep 16 01:20:42.792113 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:42.792071    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6b69bf8998-lrpcj" podStartSLOduration=1.398398448 podStartE2EDuration="1.792053959s" totalImagesPullingTime="393.655511ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 01:20:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 01:20:42.789839239 +0000 UTC m=+2257.571340831" watchObservedRunningTime="2026-09-16 01:20:42.792053959 +0000 UTC m=+2257.573555559"
Sep 16 01:20:53.781376 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:20:53.781344    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6b69bf8998-lrpcj"
Sep 16 01:21:06.027880 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:21:06.027851    2556 scope.go:117] "RemoveContainer" containerID="41abf85c3ce748cfd9be27c577bcb9fa9d85cf86149497c72d206f1605780cae"
Sep 16 01:21:06.035508 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:21:06.035482    2556 scope.go:117] "RemoveContainer" containerID="8acc756a82178ac395d8e36308bad74199ae628bc4b40c0ba1b8f4556e47cbc7"
Sep 16 01:24:19.379972 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:24:19.379935    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-d04d108c-kserve-699b654cc5-sjv75"]
Sep 16 01:24:19.383273 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:24:19.383256    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-d04d108c-kserve-699b654cc5-sjv75"
Sep 16 01:24:19.385223 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:24:19.385198    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-d04d108c-kserve-self-signed-certs\""
Sep 16 01:24:19.392225 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:24:19.392203    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-d04d108c-kserve-699b654cc5-sjv75"]
Sep 16 01:24:19.488890 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:24:19.488860    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sf6s7\" (UniqueName: \"kubernetes.io/projected/a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9-kube-api-access-sf6s7\") pod \"e2e-same-id-b-d04d108c-kserve-699b654cc5-sjv75\" (UID: \"a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9\") " pod="llm/e2e-same-id-b-d04d108c-kserve-699b654cc5-sjv75"
Sep 16 01:24:19.489033 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:24:19.488903    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9-model-cache\") pod \"e2e-same-id-b-d04d108c-kserve-699b654cc5-sjv75\" (UID: \"a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9\") " pod="llm/e2e-same-id-b-d04d108c-kserve-699b654cc5-sjv75"
Sep 16 01:24:19.489033 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:24:19.488921    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9-tls-certs\") pod \"e2e-same-id-b-d04d108c-kserve-699b654cc5-sjv75\" (UID: \"a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9\") " pod="llm/e2e-same-id-b-d04d108c-kserve-699b654cc5-sjv75"
Sep 16 01:24:19.489033 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:24:19.488984    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9-home\") pod \"e2e-same-id-b-d04d108c-kserve-699b654cc5-sjv75\" (UID: \"a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9\") " pod="llm/e2e-same-id-b-d04d108c-kserve-699b654cc5-sjv75"
Sep 16 01:24:19.489033 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:24:19.489015    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9-dshm\") pod \"e2e-same-id-b-d04d108c-kserve-699b654cc5-sjv75\" (UID: \"a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9\") " pod="llm/e2e-same-id-b-d04d108c-kserve-699b654cc5-sjv75"
Sep 16 01:24:19.489200 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:24:19.489070    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9-tmp-dir\") pod \"e2e-same-id-b-d04d108c-kserve-699b654cc5-sjv75\" (UID: \"a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9\") " pod="llm/e2e-same-id-b-d04d108c-kserve-699b654cc5-sjv75"
Sep 16 01:24:19.589394 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:24:19.589363    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9-tls-certs\") pod \"e2e-same-id-b-d04d108c-kserve-699b654cc5-sjv75\" (UID: \"a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9\") " pod="llm/e2e-same-id-b-d04d108c-kserve-699b654cc5-sjv75"
Sep 16 01:24:19.589574 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:24:19.589403    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9-home\") pod \"e2e-same-id-b-d04d108c-kserve-699b654cc5-sjv75\" (UID: \"a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9\") " pod="llm/e2e-same-id-b-d04d108c-kserve-699b654cc5-sjv75"
Sep 16 01:24:19.589574 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:24:19.589419    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9-dshm\") pod \"e2e-same-id-b-d04d108c-kserve-699b654cc5-sjv75\" (UID: \"a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9\") " pod="llm/e2e-same-id-b-d04d108c-kserve-699b654cc5-sjv75"
Sep 16 01:24:19.589574 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:24:19.589457    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9-tmp-dir\") pod \"e2e-same-id-b-d04d108c-kserve-699b654cc5-sjv75\" (UID: \"a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9\") " pod="llm/e2e-same-id-b-d04d108c-kserve-699b654cc5-sjv75"
Sep 16 01:24:19.589574 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:24:19.589496    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sf6s7\" (UniqueName: \"kubernetes.io/projected/a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9-kube-api-access-sf6s7\") pod \"e2e-same-id-b-d04d108c-kserve-699b654cc5-sjv75\" (UID: \"a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9\") " pod="llm/e2e-same-id-b-d04d108c-kserve-699b654cc5-sjv75"
Sep 16 01:24:19.589801 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:24:19.589650    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9-model-cache\") pod \"e2e-same-id-b-d04d108c-kserve-699b654cc5-sjv75\" (UID: \"a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9\") " pod="llm/e2e-same-id-b-d04d108c-kserve-699b654cc5-sjv75"
Sep 16 01:24:19.589862 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:24:19.589831    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9-home\") pod \"e2e-same-id-b-d04d108c-kserve-699b654cc5-sjv75\" (UID: \"a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9\") " pod="llm/e2e-same-id-b-d04d108c-kserve-699b654cc5-sjv75"
Sep 16 01:24:19.589917 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:24:19.589860    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9-tmp-dir\") pod \"e2e-same-id-b-d04d108c-kserve-699b654cc5-sjv75\" (UID: \"a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9\") " pod="llm/e2e-same-id-b-d04d108c-kserve-699b654cc5-sjv75"
Sep 16 01:24:19.590048 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:24:19.590029    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9-model-cache\") pod \"e2e-same-id-b-d04d108c-kserve-699b654cc5-sjv75\" (UID: \"a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9\") " pod="llm/e2e-same-id-b-d04d108c-kserve-699b654cc5-sjv75"
Sep 16 01:24:19.591690 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:24:19.591668    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9-dshm\") pod \"e2e-same-id-b-d04d108c-kserve-699b654cc5-sjv75\" (UID: \"a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9\") " pod="llm/e2e-same-id-b-d04d108c-kserve-699b654cc5-sjv75"
Sep 16 01:24:19.592124 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:24:19.592104    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9-tls-certs\") pod \"e2e-same-id-b-d04d108c-kserve-699b654cc5-sjv75\" (UID: \"a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9\") " pod="llm/e2e-same-id-b-d04d108c-kserve-699b654cc5-sjv75"
Sep 16 01:24:19.599164 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:24:19.599142    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sf6s7\" (UniqueName: \"kubernetes.io/projected/a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9-kube-api-access-sf6s7\") pod \"e2e-same-id-b-d04d108c-kserve-699b654cc5-sjv75\" (UID: \"a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9\") " pod="llm/e2e-same-id-b-d04d108c-kserve-699b654cc5-sjv75"
Sep 16 01:24:19.693818 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:24:19.693765    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-d04d108c-kserve-699b654cc5-sjv75"
Sep 16 01:24:19.966805 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:24:19.964336    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-d04d108c-kserve-699b654cc5-sjv75"]
Sep 16 01:24:19.967187 ip-10-0-130-207 kubenswrapper[2556]: W0916 01:24:19.967149    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda89fd8eb_0ee0_459c_b18b_d0ee73dc3ab9.slice/crio-e5e848ab3a92968cca36aadbedfc605340d28cca5e9c69ec4af585a3ecaac662 WatchSource:0}: Error finding container e5e848ab3a92968cca36aadbedfc605340d28cca5e9c69ec4af585a3ecaac662: Status 404 returned error can't find the container with id e5e848ab3a92968cca36aadbedfc605340d28cca5e9c69ec4af585a3ecaac662
Sep 16 01:24:20.817419 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:24:20.817383    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-d04d108c-kserve-699b654cc5-sjv75" event={"ID":"a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9","Type":"ContainerStarted","Data":"6822f43821dc64541e25b47e79f0e9fb6c0aca5ace2cabd16707fbd8fa81a16f"}
Sep 16 01:24:20.817419 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:24:20.817424    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-d04d108c-kserve-699b654cc5-sjv75" event={"ID":"a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9","Type":"ContainerStarted","Data":"e5e848ab3a92968cca36aadbedfc605340d28cca5e9c69ec4af585a3ecaac662"}
Sep 16 01:24:20.832357 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:24:20.832310    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-d04d108c-kserve-699b654cc5-sjv75" podStartSLOduration=1.832294179 podStartE2EDuration="1.832294179s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 01:24:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 01:24:20.831094586 +0000 UTC m=+2475.612596180" watchObservedRunningTime="2026-09-16 01:24:20.832294179 +0000 UTC m=+2475.613795781"
Sep 16 01:24:29.694541 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:24:29.694503    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-d04d108c-kserve-699b654cc5-sjv75"
Sep 16 01:24:29.694541 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:24:29.694551    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-d04d108c-kserve-699b654cc5-sjv75"
Sep 16 01:24:29.707248 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:24:29.707219    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-d04d108c-kserve-699b654cc5-sjv75"
Sep 16 01:24:29.870298 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:24:29.870270    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-d04d108c-kserve-699b654cc5-sjv75"
Sep 16 01:24:42.570146 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:24:42.570112    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-d04d108c-kserve-699b654cc5-sjv75"]
Sep 16 01:24:57.589723 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:24:57.589681    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-d04d108c-kserve-699b654cc5-sjv75" podUID="a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9" containerName="main" containerID="cri-o://6822f43821dc64541e25b47e79f0e9fb6c0aca5ace2cabd16707fbd8fa81a16f" gracePeriod=45
Sep 16 01:24:57.835859 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:24:57.835837    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-d04d108c-kserve-699b654cc5-sjv75"
Sep 16 01:24:57.988683 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:24:57.988599    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9-dshm\") pod \"a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9\" (UID: \"a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9\") "
Sep 16 01:24:57.988683 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:24:57.988638    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9-tmp-dir\") pod \"a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9\" (UID: \"a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9\") "
Sep 16 01:24:57.988683 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:24:57.988681    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sf6s7\" (UniqueName: \"kubernetes.io/projected/a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9-kube-api-access-sf6s7\") pod \"a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9\" (UID: \"a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9\") "
Sep 16 01:24:57.988949 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:24:57.988717    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9-model-cache\") pod \"a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9\" (UID: \"a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9\") "
Sep 16 01:24:57.988949 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:24:57.988737    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9-tls-certs\") pod \"a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9\" (UID: \"a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9\") "
Sep 16 01:24:57.988949 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:24:57.988782    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9-home\") pod \"a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9\" (UID: \"a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9\") "
Sep 16 01:24:57.989118 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:24:57.988929    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9" (UID: "a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 01:24:57.989118 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:24:57.988960    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9-model-cache" (OuterVolumeSpecName: "model-cache") pod "a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9" (UID: "a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 01:24:57.989118 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:24:57.989035    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9-model-cache\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:24:57.989118 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:24:57.989053    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9-tmp-dir\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:24:57.989353 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:24:57.989152    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9-home" (OuterVolumeSpecName: "home") pod "a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9" (UID: "a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 01:24:57.990950 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:24:57.990919    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9-dshm" (OuterVolumeSpecName: "dshm") pod "a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9" (UID: "a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 01:24:57.991113 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:24:57.991085    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9-kube-api-access-sf6s7" (OuterVolumeSpecName: "kube-api-access-sf6s7") pod "a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9" (UID: "a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9"). InnerVolumeSpecName "kube-api-access-sf6s7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 01:24:57.991188 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:24:57.991132    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9" (UID: "a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 01:24:57.995362 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:24:57.995336    2556 generic.go:356] "Generic (PLEG): container finished" podID="a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9" containerID="6822f43821dc64541e25b47e79f0e9fb6c0aca5ace2cabd16707fbd8fa81a16f" exitCode=0
Sep 16 01:24:57.995567 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:24:57.995371    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-d04d108c-kserve-699b654cc5-sjv75"
Sep 16 01:24:57.995567 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:24:57.995384    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-d04d108c-kserve-699b654cc5-sjv75" event={"ID":"a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9","Type":"ContainerDied","Data":"6822f43821dc64541e25b47e79f0e9fb6c0aca5ace2cabd16707fbd8fa81a16f"}
Sep 16 01:24:57.995567 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:24:57.995421    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-d04d108c-kserve-699b654cc5-sjv75" event={"ID":"a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9","Type":"ContainerDied","Data":"e5e848ab3a92968cca36aadbedfc605340d28cca5e9c69ec4af585a3ecaac662"}
Sep 16 01:24:57.995567 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:24:57.995442    2556 scope.go:117] "RemoveContainer" containerID="6822f43821dc64541e25b47e79f0e9fb6c0aca5ace2cabd16707fbd8fa81a16f"
Sep 16 01:24:58.007160 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:24:58.007142    2556 scope.go:117] "RemoveContainer" containerID="6822f43821dc64541e25b47e79f0e9fb6c0aca5ace2cabd16707fbd8fa81a16f"
Sep 16 01:24:58.007427 ip-10-0-130-207 kubenswrapper[2556]: E0916 01:24:58.007405    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6822f43821dc64541e25b47e79f0e9fb6c0aca5ace2cabd16707fbd8fa81a16f\": container with ID starting with 6822f43821dc64541e25b47e79f0e9fb6c0aca5ace2cabd16707fbd8fa81a16f not found: ID does not exist" containerID="6822f43821dc64541e25b47e79f0e9fb6c0aca5ace2cabd16707fbd8fa81a16f"
Sep 16 01:24:58.007500 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:24:58.007435    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6822f43821dc64541e25b47e79f0e9fb6c0aca5ace2cabd16707fbd8fa81a16f"} err="failed to get container status \"6822f43821dc64541e25b47e79f0e9fb6c0aca5ace2cabd16707fbd8fa81a16f\": rpc error: code = NotFound desc = could not find container \"6822f43821dc64541e25b47e79f0e9fb6c0aca5ace2cabd16707fbd8fa81a16f\": container with ID starting with 6822f43821dc64541e25b47e79f0e9fb6c0aca5ace2cabd16707fbd8fa81a16f not found: ID does not exist"
Sep 16 01:24:58.016605 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:24:58.016585    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-d04d108c-kserve-699b654cc5-sjv75"]
Sep 16 01:24:58.019768 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:24:58.019746    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-d04d108c-kserve-699b654cc5-sjv75"]
Sep 16 01:24:58.090266 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:24:58.090245    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9-home\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:24:58.090266 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:24:58.090264    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9-dshm\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:24:58.090389 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:24:58.090273    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sf6s7\" (UniqueName: \"kubernetes.io/projected/a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9-kube-api-access-sf6s7\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:24:58.090389 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:24:58.090282    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9-tls-certs\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 01:24:59.736176 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:24:59.736143    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9" path="/var/lib/kubelet/pods/a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9/volumes"
Sep 16 01:28:33.115198 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:28:33.115166    2556 ???:1] "http: TLS handshake error from 10.0.130.4:51874: EOF"
Sep 16 01:28:33.119955 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:28:33.119929    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-wk9f6_c23418e8-6326-481d-9a85-2cbda835d5f6/global-pull-secret-syncer/0.log"
Sep 16 01:28:33.239669 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:28:33.239646    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-sjrmq_f1c71941-4979-4074-beb1-cf727004f45d/konnectivity-agent/0.log"
Sep 16 01:28:33.256726 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:28:33.256705    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-207.ec2.internal_4054c0090de8f54aed802a5d555be580/haproxy/0.log"
Sep 16 01:28:38.090035 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:28:38.090002    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-jfnkb_6aa9394a-32fe-4a8f-8d27-baa2b6eeb98f/manager/0.log"
Sep 16 01:28:38.125393 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:28:38.125367    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-zrmwp_5a57e258-33e9-4fb5-b808-41021dfb50d6/manager/0.log"
Sep 16 01:28:38.302609 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:28:38.302566    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-cssg6_24200bd2-1f1f-48e9-a8c4-2bca32931b5c/manager/0.log"
Sep 16 01:28:40.374160 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:28:40.374133    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xvms9_dfce583b-e647-4ca3-bbec-34ce267f5b56/node-exporter/0.log"
Sep 16 01:28:40.393683 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:28:40.393663    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xvms9_dfce583b-e647-4ca3-bbec-34ce267f5b56/kube-rbac-proxy/0.log"
Sep 16 01:28:40.414144 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:28:40.414118    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xvms9_dfce583b-e647-4ca3-bbec-34ce267f5b56/init-textfile/0.log"
Sep 16 01:28:43.208286 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:28:43.208251    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-xll2x/perf-node-gather-daemonset-szrp8"]
Sep 16 01:28:43.208768 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:28:43.208575    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9" containerName="main"
Sep 16 01:28:43.208768 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:28:43.208587    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9" containerName="main"
Sep 16 01:28:43.211489 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:28:43.209347    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="a89fd8eb-0ee0-459c-b18b-d0ee73dc3ab9" containerName="main"
Sep 16 01:28:43.213773 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:28:43.213746    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xll2x/perf-node-gather-daemonset-szrp8"
Sep 16 01:28:43.216397 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:28:43.216375    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xll2x\"/\"openshift-service-ca.crt\""
Sep 16 01:28:43.217662 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:28:43.217641    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xll2x\"/\"kube-root-ca.crt\""
Sep 16 01:28:43.217782 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:28:43.217669    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-xll2x\"/\"default-dockercfg-fdxcj\""
Sep 16 01:28:43.219760 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:28:43.219735    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xll2x/perf-node-gather-daemonset-szrp8"]
Sep 16 01:28:43.282620 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:28:43.282592    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7bddbf77-baec-41ce-899f-e59eaba3e1fd-lib-modules\") pod \"perf-node-gather-daemonset-szrp8\" (UID: \"7bddbf77-baec-41ce-899f-e59eaba3e1fd\") " pod="openshift-must-gather-xll2x/perf-node-gather-daemonset-szrp8"
Sep 16 01:28:43.282725 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:28:43.282623    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7bddbf77-baec-41ce-899f-e59eaba3e1fd-podres\") pod \"perf-node-gather-daemonset-szrp8\" (UID: \"7bddbf77-baec-41ce-899f-e59eaba3e1fd\") " pod="openshift-must-gather-xll2x/perf-node-gather-daemonset-szrp8"
Sep 16 01:28:43.282725 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:28:43.282648    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7bddbf77-baec-41ce-899f-e59eaba3e1fd-proc\") pod \"perf-node-gather-daemonset-szrp8\" (UID: \"7bddbf77-baec-41ce-899f-e59eaba3e1fd\") " pod="openshift-must-gather-xll2x/perf-node-gather-daemonset-szrp8"
Sep 16 01:28:43.282823 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:28:43.282783    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7bddbf77-baec-41ce-899f-e59eaba3e1fd-sys\") pod \"perf-node-gather-daemonset-szrp8\" (UID: \"7bddbf77-baec-41ce-899f-e59eaba3e1fd\") " pod="openshift-must-gather-xll2x/perf-node-gather-daemonset-szrp8"
Sep 16 01:28:43.282872 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:28:43.282834    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztd4v\" (UniqueName: \"kubernetes.io/projected/7bddbf77-baec-41ce-899f-e59eaba3e1fd-kube-api-access-ztd4v\") pod \"perf-node-gather-daemonset-szrp8\" (UID: \"7bddbf77-baec-41ce-899f-e59eaba3e1fd\") " pod="openshift-must-gather-xll2x/perf-node-gather-daemonset-szrp8"
Sep 16 01:28:43.383260 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:28:43.383236    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7bddbf77-baec-41ce-899f-e59eaba3e1fd-lib-modules\") pod \"perf-node-gather-daemonset-szrp8\" (UID: \"7bddbf77-baec-41ce-899f-e59eaba3e1fd\") " pod="openshift-must-gather-xll2x/perf-node-gather-daemonset-szrp8"
Sep 16 01:28:43.383366 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:28:43.383262    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7bddbf77-baec-41ce-899f-e59eaba3e1fd-podres\") pod \"perf-node-gather-daemonset-szrp8\" (UID: \"7bddbf77-baec-41ce-899f-e59eaba3e1fd\") " pod="openshift-must-gather-xll2x/perf-node-gather-daemonset-szrp8"
Sep 16 01:28:43.383366 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:28:43.383286    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7bddbf77-baec-41ce-899f-e59eaba3e1fd-proc\") pod \"perf-node-gather-daemonset-szrp8\" (UID: \"7bddbf77-baec-41ce-899f-e59eaba3e1fd\") " pod="openshift-must-gather-xll2x/perf-node-gather-daemonset-szrp8"
Sep 16 01:28:43.383366 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:28:43.383325    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7bddbf77-baec-41ce-899f-e59eaba3e1fd-sys\") pod \"perf-node-gather-daemonset-szrp8\" (UID: \"7bddbf77-baec-41ce-899f-e59eaba3e1fd\") " pod="openshift-must-gather-xll2x/perf-node-gather-daemonset-szrp8"
Sep 16 01:28:43.383366 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:28:43.383349    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztd4v\" (UniqueName: \"kubernetes.io/projected/7bddbf77-baec-41ce-899f-e59eaba3e1fd-kube-api-access-ztd4v\") pod \"perf-node-gather-daemonset-szrp8\" (UID: \"7bddbf77-baec-41ce-899f-e59eaba3e1fd\") " pod="openshift-must-gather-xll2x/perf-node-gather-daemonset-szrp8"
Sep 16 01:28:43.383580 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:28:43.383379    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7bddbf77-baec-41ce-899f-e59eaba3e1fd-proc\") pod \"perf-node-gather-daemonset-szrp8\" (UID: \"7bddbf77-baec-41ce-899f-e59eaba3e1fd\") " pod="openshift-must-gather-xll2x/perf-node-gather-daemonset-szrp8"
Sep 16 01:28:43.383580 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:28:43.383399    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7bddbf77-baec-41ce-899f-e59eaba3e1fd-lib-modules\") pod \"perf-node-gather-daemonset-szrp8\" (UID: \"7bddbf77-baec-41ce-899f-e59eaba3e1fd\") " pod="openshift-must-gather-xll2x/perf-node-gather-daemonset-szrp8"
Sep 16 01:28:43.383580 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:28:43.383411    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7bddbf77-baec-41ce-899f-e59eaba3e1fd-podres\") pod \"perf-node-gather-daemonset-szrp8\" (UID: \"7bddbf77-baec-41ce-899f-e59eaba3e1fd\") " pod="openshift-must-gather-xll2x/perf-node-gather-daemonset-szrp8"
Sep 16 01:28:43.383580 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:28:43.383442    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7bddbf77-baec-41ce-899f-e59eaba3e1fd-sys\") pod \"perf-node-gather-daemonset-szrp8\" (UID: \"7bddbf77-baec-41ce-899f-e59eaba3e1fd\") " pod="openshift-must-gather-xll2x/perf-node-gather-daemonset-szrp8"
Sep 16 01:28:43.390729 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:28:43.390710    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztd4v\" (UniqueName: \"kubernetes.io/projected/7bddbf77-baec-41ce-899f-e59eaba3e1fd-kube-api-access-ztd4v\") pod \"perf-node-gather-daemonset-szrp8\" (UID: \"7bddbf77-baec-41ce-899f-e59eaba3e1fd\") " pod="openshift-must-gather-xll2x/perf-node-gather-daemonset-szrp8"
Sep 16 01:28:43.524969 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:28:43.524913    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xll2x/perf-node-gather-daemonset-szrp8"
Sep 16 01:28:43.567637 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:28:43.567612    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7cd4c5986d-zsddx_c02cd606-1a1a-4789-aeee-4c831943babe/console/0.log"
Sep 16 01:28:43.796502 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:28:43.796462    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xll2x/perf-node-gather-daemonset-szrp8"]
Sep 16 01:28:43.798968 ip-10-0-130-207 kubenswrapper[2556]: W0916 01:28:43.798940    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod7bddbf77_baec_41ce_899f_e59eaba3e1fd.slice/crio-5efffc36d88b15ddbbb3bf34614da10346e2afe33da7d030b59d4d9a2d27a3bb WatchSource:0}: Error finding container 5efffc36d88b15ddbbb3bf34614da10346e2afe33da7d030b59d4d9a2d27a3bb: Status 404 returned error can't find the container with id 5efffc36d88b15ddbbb3bf34614da10346e2afe33da7d030b59d4d9a2d27a3bb
Sep 16 01:28:44.079578 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:28:44.079548    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xll2x/perf-node-gather-daemonset-szrp8" event={"ID":"7bddbf77-baec-41ce-899f-e59eaba3e1fd","Type":"ContainerStarted","Data":"8221324c54e9844df6978fe6a22334c9edc2d94e436dfc484306874787feed49"}
Sep 16 01:28:44.079746 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:28:44.079596    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xll2x/perf-node-gather-daemonset-szrp8" event={"ID":"7bddbf77-baec-41ce-899f-e59eaba3e1fd","Type":"ContainerStarted","Data":"5efffc36d88b15ddbbb3bf34614da10346e2afe33da7d030b59d4d9a2d27a3bb"}
Sep 16 01:28:44.079746 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:28:44.079663    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-xll2x/perf-node-gather-daemonset-szrp8"
Sep 16 01:28:44.094094 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:28:44.094051    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-xll2x/perf-node-gather-daemonset-szrp8" podStartSLOduration=1.094038181 podStartE2EDuration="1.094038181s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 01:28:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 01:28:44.09301238 +0000 UTC m=+2738.874513973" watchObservedRunningTime="2026-09-16 01:28:44.094038181 +0000 UTC m=+2738.875539773"
Sep 16 01:28:45.549202 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:28:45.549171    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rsqrc_d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80/dns/0.log"
Sep 16 01:28:45.570273 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:28:45.570247    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rsqrc_d2f6824d-c5ca-4cde-9c75-c7e4d4a63f80/kube-rbac-proxy/0.log"
Sep 16 01:28:45.614253 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:28:45.614229    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-q29p2_21802a49-7d39-41db-a85c-0b6c56472a56/dns-node-resolver/0.log"
Sep 16 01:28:46.201236 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:28:46.201208    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-jvxd7_07a10807-5530-48f1-9aeb-8becb1380db5/node-ca/0.log"
Sep 16 01:28:47.495203 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:28:47.495173    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-7v9tv_3e004a11-dc0e-4c29-ade3-ad5b9afbbab2/discovery/0.log"
Sep 16 01:28:47.517101 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:28:47.517059    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5bfdbc6f8d-j5r2k_c3689323-4bf5-447c-9f4e-2d5054c5ff85/kube-auth-proxy/0.log"
Sep 16 01:28:48.522732 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:28:48.522702    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jt2l9_aa0a0489-cc60-43ea-89dc-cb3d97276b9d/serve-healthcheck-canary/0.log"
Sep 16 01:28:49.107685 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:28:49.107656    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mkbc5_10b542a2-e2cd-48f8-bb46-9fe3b0711e83/kube-rbac-proxy/0.log"
Sep 16 01:28:49.126039 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:28:49.126016    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mkbc5_10b542a2-e2cd-48f8-bb46-9fe3b0711e83/exporter/0.log"
Sep 16 01:28:49.144741 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:28:49.144721    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mkbc5_10b542a2-e2cd-48f8-bb46-9fe3b0711e83/extractor/0.log"
Sep 16 01:28:50.093732 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:28:50.093707    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-xll2x/perf-node-gather-daemonset-szrp8"
Sep 16 01:28:51.666696 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:28:51.666667    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6b69bf8998-lrpcj_c42b4555-b040-42e8-a3fe-a51d9ffdb55f/manager/0.log"
Sep 16 01:28:51.881807 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:28:51.881785    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9778fc5bd-rvf5d_e20f5a6a-518f-498c-bcb0-1abe83414bb9/manager/0.log"
Sep 16 01:28:51.900503 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:28:51.900462    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9778fc5bd-rvf5d_e20f5a6a-518f-498c-bcb0-1abe83414bb9/copy-manifests/0.log"
Sep 16 01:28:53.609347 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:28:53.609317    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-hpbj5_97e3a04a-9639-4970-8fc2-f92dd1cd9274/openshift-lws-operator/0.log"
Sep 16 01:28:54.382649 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:28:54.382620    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-7v9tv_3e004a11-dc0e-4c29-ade3-ad5b9afbbab2/discovery/0.log"
Sep 16 01:28:54.398682 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:28:54.398667    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5bfdbc6f8d-j5r2k_c3689323-4bf5-447c-9f4e-2d5054c5ff85/kube-auth-proxy/0.log"
Sep 16 01:28:56.270281 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:28:56.270252    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-7v9tv_3e004a11-dc0e-4c29-ade3-ad5b9afbbab2/discovery/0.log"
Sep 16 01:28:56.284683 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:28:56.284659    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5bfdbc6f8d-j5r2k_c3689323-4bf5-447c-9f4e-2d5054c5ff85/kube-auth-proxy/0.log"
Sep 16 01:28:59.928714 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:28:59.928679    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-m9fb7_e94d6aa4-58be-4a67-bd8b-5670201fc0af/kube-multus-additional-cni-plugins/0.log"
Sep 16 01:28:59.947323 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:28:59.947302    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-m9fb7_e94d6aa4-58be-4a67-bd8b-5670201fc0af/egress-router-binary-copy/0.log"
Sep 16 01:28:59.968704 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:28:59.968680    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-m9fb7_e94d6aa4-58be-4a67-bd8b-5670201fc0af/cni-plugins/0.log"
Sep 16 01:28:59.988236 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:28:59.988218    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-m9fb7_e94d6aa4-58be-4a67-bd8b-5670201fc0af/bond-cni-plugin/0.log"
Sep 16 01:29:00.006411 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:29:00.006394    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-m9fb7_e94d6aa4-58be-4a67-bd8b-5670201fc0af/routeoverride-cni/0.log"
Sep 16 01:29:00.025346 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:29:00.025327    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-m9fb7_e94d6aa4-58be-4a67-bd8b-5670201fc0af/whereabouts-cni-bincopy/0.log"
Sep 16 01:29:00.043241 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:29:00.043221    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-m9fb7_e94d6aa4-58be-4a67-bd8b-5670201fc0af/whereabouts-cni/0.log"
Sep 16 01:29:00.367442 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:29:00.367416    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-l7fbr_94d2085e-48d2-4ae9-9112-6f4fa7897b43/kube-multus/0.log"
Sep 16 01:29:00.494192 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:29:00.494168    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-hnw6l_f2569a02-edf9-4b33-a7df-a8d0fdce0580/network-metrics-daemon/0.log"
Sep 16 01:29:00.509657 ip-10-0-130-207 kubenswrapper[2556]: I0916 01:29:00.509635    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-hnw6l_f2569a02-edf9-4b33-a7df-a8d0fdce0580/kube-rbac-proxy/0.log"