Sep 16 14:02:18.792592 ip-10-0-129-95 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 16 14:02:18.792603 ip-10-0-129-95 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 16 14:02:18.792611 ip-10-0-129-95 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 16 14:02:18.792800 ip-10-0-129-95 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 16 14:02:30.069442 ip-10-0-129-95 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 16 14:02:30.069456 ip-10-0-129-95 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot d73aebc90d564dabbaa75429d73c6fa5 --
Sep 16 14:04:25.786144 ip-10-0-129-95 systemd[1]: Starting Kubernetes Kubelet...
Sep 16 14:04:26.102508 ip-10-0-129-95 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 16 14:04:26.102508 ip-10-0-129-95 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 16 14:04:26.102508 ip-10-0-129-95 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 16 14:04:26.102508 ip-10-0-129-95 kubenswrapper[2580]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 16 14:04:26.102508 ip-10-0-129-95 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 16 14:04:26.103388 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.103302    2580 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 16 14:04:26.106276 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106242    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 14:04:26.106276 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106270    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 14:04:26.106276 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106275    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 14:04:26.106276 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106279    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 14:04:26.106276 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106284    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 14:04:26.106276 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106288    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 14:04:26.106276 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106291    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 14:04:26.106649 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106295    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 14:04:26.106649 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106300    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 14:04:26.106649 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106303    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 14:04:26.106649 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106308    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 14:04:26.106649 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106312    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 14:04:26.106649 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106315    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 14:04:26.106649 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106319    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 14:04:26.106649 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106322    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 14:04:26.106649 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106326    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 14:04:26.106649 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106330    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 14:04:26.106649 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106334    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 14:04:26.106649 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106338    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 14:04:26.106649 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106341    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 14:04:26.106649 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106345    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 14:04:26.106649 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106356    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 14:04:26.106649 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106361    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 14:04:26.106649 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106364    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 14:04:26.106649 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106368    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 14:04:26.106649 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106372    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 14:04:26.106649 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106379    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 14:04:26.107487 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106385    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 14:04:26.107487 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106390    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 14:04:26.107487 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106394    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 14:04:26.107487 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106399    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 14:04:26.107487 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106404    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 14:04:26.107487 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106408    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 14:04:26.107487 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106413    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 14:04:26.107487 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106418    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 14:04:26.107487 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106422    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 14:04:26.107487 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106427    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 14:04:26.107487 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106432    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 14:04:26.107487 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106436    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 14:04:26.107487 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106440    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 14:04:26.107487 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106445    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 14:04:26.107487 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106450    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 14:04:26.107487 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106454    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 14:04:26.107487 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106458    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 14:04:26.107487 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106471    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 14:04:26.107487 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106475    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 14:04:26.107487 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106479    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 14:04:26.108453 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106483    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 14:04:26.108453 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106487    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 14:04:26.108453 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106492    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 14:04:26.108453 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106496    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 14:04:26.108453 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106500    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 14:04:26.108453 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106505    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 14:04:26.108453 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106509    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 14:04:26.108453 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106521    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 14:04:26.108453 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106525    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 14:04:26.108453 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106529    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 14:04:26.108453 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106534    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 14:04:26.108453 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106540    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 14:04:26.108453 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106546    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 14:04:26.108453 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106551    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 14:04:26.108453 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106556    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 14:04:26.108453 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106561    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 14:04:26.108453 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106565    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 14:04:26.108453 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106569    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 14:04:26.108453 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106574    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 14:04:26.109229 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106578    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 14:04:26.109229 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106583    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 14:04:26.109229 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106588    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 14:04:26.109229 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106593    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 14:04:26.109229 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106598    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 14:04:26.109229 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106602    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 14:04:26.109229 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106607    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 14:04:26.109229 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106614    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 14:04:26.109229 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106618    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 14:04:26.109229 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106623    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 14:04:26.109229 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106627    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 14:04:26.109229 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106631    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 14:04:26.109229 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106636    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 14:04:26.109229 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106640    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 14:04:26.109229 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106644    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 14:04:26.109229 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106648    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 14:04:26.109229 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106652    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 14:04:26.109229 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106656    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 14:04:26.109229 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106660    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 14:04:26.109229 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106664    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 14:04:26.109902 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106668    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 14:04:26.109902 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106679    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 14:04:26.109902 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106684    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 14:04:26.109902 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106688    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 14:04:26.109902 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106692    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 14:04:26.109902 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106698    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 14:04:26.109902 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106702    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 14:04:26.109902 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106706    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 14:04:26.109902 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106713    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 14:04:26.109902 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106718    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 14:04:26.109902 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106722    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 14:04:26.109902 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106726    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 14:04:26.109902 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.106730    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 14:04:26.109902 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.107959    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 14:04:26.109902 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.107971    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 14:04:26.109902 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.107976    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 14:04:26.109902 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.107980    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 14:04:26.109902 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.107985    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 14:04:26.109902 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.107989    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 14:04:26.110395 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.107994    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 14:04:26.110395 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.107998    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 14:04:26.110395 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108002    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 14:04:26.110395 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108007    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 14:04:26.110395 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108011    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 14:04:26.110395 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108015    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 14:04:26.110395 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108019    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 14:04:26.110395 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108024    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 14:04:26.110395 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108028    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 14:04:26.110395 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108032    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 14:04:26.110395 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108036    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 14:04:26.110395 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108041    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 14:04:26.110395 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108045    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 14:04:26.110395 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108050    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 14:04:26.110395 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108054    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 14:04:26.110395 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108066    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 14:04:26.110395 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108071    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 14:04:26.110395 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108075    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 14:04:26.110395 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108082    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 14:04:26.110861 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108088    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 14:04:26.110861 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108094    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 14:04:26.110861 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108100    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 14:04:26.110861 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108105    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 14:04:26.110861 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108110    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 14:04:26.110861 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108114    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 14:04:26.110861 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108118    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 14:04:26.110861 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108123    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 14:04:26.110861 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108127    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 14:04:26.110861 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108131    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 14:04:26.110861 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108135    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 14:04:26.110861 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108139    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 14:04:26.110861 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108144    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 14:04:26.110861 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108148    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 14:04:26.110861 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108152    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 14:04:26.110861 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108156    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 14:04:26.110861 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108160    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 14:04:26.110861 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108164    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 14:04:26.110861 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108168    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 14:04:26.111339 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108173    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 14:04:26.111339 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108177    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 14:04:26.111339 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108181    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 14:04:26.111339 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108185    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 14:04:26.111339 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108189    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 14:04:26.111339 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108193    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 14:04:26.111339 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108197    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 14:04:26.111339 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108201    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 14:04:26.111339 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108205    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 14:04:26.111339 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108209    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 14:04:26.111339 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108221    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 14:04:26.111339 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108227    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 14:04:26.111339 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108231    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 14:04:26.111339 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108235    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 14:04:26.111339 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108239    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 14:04:26.111339 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108243    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 14:04:26.111339 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108247    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 14:04:26.111339 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108251    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 14:04:26.111339 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108255    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 14:04:26.111339 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108260    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 14:04:26.111815 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108265    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 14:04:26.111815 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108269    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 14:04:26.111815 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108273    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 14:04:26.111815 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108279    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 14:04:26.111815 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108285    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 14:04:26.111815 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108290    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 14:04:26.111815 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108294    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 14:04:26.111815 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108298    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 14:04:26.111815 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108303    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 14:04:26.111815 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108307    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 14:04:26.111815 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108311    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 14:04:26.111815 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108315    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 14:04:26.111815 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108319    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 14:04:26.111815 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108323    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 14:04:26.111815 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108327    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 14:04:26.111815 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108331    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 14:04:26.111815 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108336    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 14:04:26.111815 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108340    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 14:04:26.111815 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108344    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 14:04:26.111815 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108349    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 14:04:26.112321 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108352    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 14:04:26.112321 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108359    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 14:04:26.112321 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108363    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 14:04:26.112321 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108377    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 14:04:26.112321 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108382    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 14:04:26.112321 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108387    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 14:04:26.112321 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108391    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 14:04:26.112321 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108395    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 14:04:26.112321 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108399    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 14:04:26.112321 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108403    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 14:04:26.112321 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108407    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 14:04:26.112321 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108412    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 14:04:26.112321 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108416    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 14:04:26.112321 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108420    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 14:04:26.112321 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.108425    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 14:04:26.112321 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.109958    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 14:04:26.112321 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.109970    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 14:04:26.112321 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.109974    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 14:04:26.112321 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.109977    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 14:04:26.112776 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.109980    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 14:04:26.112776 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.109983    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 14:04:26.112776 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.109986    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 14:04:26.112776 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.109989    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 14:04:26.112776 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.109994    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 14:04:26.112776 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.109998    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 14:04:26.112776 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110001    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 14:04:26.112776 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110004    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 14:04:26.112776 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110007    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 14:04:26.112776 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110009    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 14:04:26.112776 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110012    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 14:04:26.112776 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110015    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 14:04:26.112776 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110017    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 14:04:26.112776 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110021    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 14:04:26.112776 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110026    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 14:04:26.112776 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110029    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 14:04:26.112776 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110032    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 14:04:26.112776 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110035    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 14:04:26.112776 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110037    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 14:04:26.113259 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110040    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 14:04:26.113259 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110043    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 14:04:26.113259 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110046    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 14:04:26.113259 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110049    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 14:04:26.113259 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110052    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 14:04:26.113259 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110055    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 14:04:26.113259 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110059    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 14:04:26.113259 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110062    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 14:04:26.113259 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110065    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 14:04:26.113259 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110068    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 14:04:26.113259 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110072    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 14:04:26.113259 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110075    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 14:04:26.113259 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110078    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 14:04:26.113259 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110081    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 14:04:26.113259 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110083    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 14:04:26.113259 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110086    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 14:04:26.113259 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110089    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 14:04:26.113259 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110091    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 14:04:26.113259 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110094    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 14:04:26.113714 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110096    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 14:04:26.113714 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110099    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 14:04:26.113714 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110102    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 14:04:26.113714 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110104    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 14:04:26.113714 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110107    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 14:04:26.113714 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110110    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 14:04:26.113714 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110112    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 14:04:26.113714 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110115    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 14:04:26.113714 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110118    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 14:04:26.113714 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110121    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 14:04:26.113714 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110124    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 14:04:26.113714 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110126    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 14:04:26.113714 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110130    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 14:04:26.113714 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110133    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 14:04:26.113714 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110136    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 14:04:26.113714 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110138    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 14:04:26.113714 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110141    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 14:04:26.113714 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110143    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 14:04:26.113714 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110146    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 14:04:26.113714 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110148    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 14:04:26.114217 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110151    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 14:04:26.114217 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110153    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 14:04:26.114217 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110156    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 14:04:26.114217 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110162    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 14:04:26.114217 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110164    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 14:04:26.114217 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110167    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 14:04:26.114217 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110170    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 14:04:26.114217 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110172    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 14:04:26.114217 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110175    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 14:04:26.114217 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110178    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 14:04:26.114217 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110181    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 14:04:26.114217 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110184    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 14:04:26.114217 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110186    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 14:04:26.114217 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110189    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 14:04:26.114217 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110191    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 14:04:26.114217 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110194    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 14:04:26.114217 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110196    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 14:04:26.114217 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110199    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 14:04:26.114217 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110202    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 14:04:26.114217 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110204    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 14:04:26.114689 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110207    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 14:04:26.114689 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110210    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 14:04:26.114689 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110213    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 14:04:26.114689 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110215    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 14:04:26.114689 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110218    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 14:04:26.114689 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110221    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 14:04:26.114689 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110223    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 14:04:26.114689 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110226    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 14:04:26.114689 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110228    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 14:04:26.114689 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110231    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 14:04:26.114689 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110233    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 14:04:26.114689 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110236    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 14:04:26.114689 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110238    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 14:04:26.114689 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110241    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 14:04:26.114689 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110243    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 14:04:26.114689 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110249    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 14:04:26.114689 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110252    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 14:04:26.114689 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110324    2580 flags.go:64] FLAG: --address="0.0.0.0"
Sep 16 14:04:26.114689 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110358    2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 16 14:04:26.114689 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110365    2580 flags.go:64] FLAG: --anonymous-auth="true"
Sep 16 14:04:26.115225 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110370    2580 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 16 14:04:26.115225 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110374    2580 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 16 14:04:26.115225 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110377    2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 16 14:04:26.115225 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110382    2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 16 14:04:26.115225 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110386    2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 16 14:04:26.115225 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110389    2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 16 14:04:26.115225 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110393    2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 16 14:04:26.115225 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110396    2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 16 14:04:26.115225 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110400    2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 16 14:04:26.115225 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110404    2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 16 14:04:26.115225 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110407    2580 flags.go:64] FLAG: --cgroup-root=""
Sep 16 14:04:26.115225 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110410    2580 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 16 14:04:26.115225 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110413    2580 flags.go:64] FLAG: --client-ca-file=""
Sep 16 14:04:26.115225 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110416    2580 flags.go:64] FLAG: --cloud-provider="external"
Sep 16 14:04:26.115225 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110420    2580 flags.go:64] FLAG: --cluster-dns="[]"
Sep 16 14:04:26.115225 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110424    2580 flags.go:64] FLAG: --cluster-domain=""
Sep 16 14:04:26.115225 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110427    2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 16 14:04:26.115225 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110430    2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 16 14:04:26.115225 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110434    2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 16 14:04:26.115225 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110437    2580 flags.go:64] FLAG: --container-log-max-files="5"
Sep 16 14:04:26.115225 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110441    2580 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 16 14:04:26.115225 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110443    2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 16 14:04:26.115225 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110447    2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 16 14:04:26.115770 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110450    2580 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 16 14:04:26.115770 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110453    2580 flags.go:64] FLAG: --contention-profiling="false"
Sep 16 14:04:26.115770 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110456    2580 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 16 14:04:26.115770 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110459    2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 16 14:04:26.115770 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110464    2580 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 16 14:04:26.115770 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110467    2580 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 16 14:04:26.115770 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110471    2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 16 14:04:26.115770 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110474    2580 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 16 14:04:26.115770 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110477    2580 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 16 14:04:26.115770 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110480    2580 flags.go:64] FLAG: --enable-load-reader="false"
Sep 16 14:04:26.115770 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110483    2580 flags.go:64] FLAG: --enable-server="true"
Sep 16 14:04:26.115770 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110486    2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 16 14:04:26.115770 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110490    2580 flags.go:64] FLAG: --event-burst="100"
Sep 16 14:04:26.115770 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110493    2580 flags.go:64] FLAG: --event-qps="50"
Sep 16 14:04:26.115770 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110496    2580 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 16 14:04:26.115770 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110499    2580 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 16 14:04:26.115770 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110502    2580 flags.go:64] FLAG: --eviction-hard=""
Sep 16 14:04:26.115770 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110506    2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 16 14:04:26.115770 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110509    2580 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 16 14:04:26.115770 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110512    2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 16 14:04:26.115770 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110515    2580 flags.go:64] FLAG: --eviction-soft=""
Sep 16 14:04:26.115770 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110517    2580 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 16 14:04:26.115770 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110520    2580 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 16 14:04:26.115770 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110523    2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 16 14:04:26.115770 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110526    2580 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 16 14:04:26.116410 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110529    2580 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 16 14:04:26.116410 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110533    2580 flags.go:64] FLAG: --fail-swap-on="true"
Sep 16 14:04:26.116410 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110535    2580 flags.go:64] FLAG: --feature-gates=""
Sep 16 14:04:26.116410 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110539    2580 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 16 14:04:26.116410 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110542    2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 16 14:04:26.116410 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110545    2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 16 14:04:26.116410 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110548    2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 16 14:04:26.116410 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110551    2580 flags.go:64] FLAG: --healthz-port="10248"
Sep 16 14:04:26.116410 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110554    2580 flags.go:64] FLAG: --help="false"
Sep 16 14:04:26.116410 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110557    2580 flags.go:64] FLAG: --hostname-override="ip-10-0-129-95.ec2.internal"
Sep 16 14:04:26.116410 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110560    2580 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 16 14:04:26.116410 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110564    2580 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 16 14:04:26.116410 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110567    2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 16 14:04:26.116410 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110571    2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 16 14:04:26.116410 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110574    2580 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 16 14:04:26.116410 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110577    2580 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 16 14:04:26.116410 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110580    2580 flags.go:64] FLAG: --image-service-endpoint=""
Sep 16 14:04:26.116410 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110583    2580 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 16 14:04:26.116410 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110586    2580 flags.go:64] FLAG: --kube-api-burst="100"
Sep 16 14:04:26.116410 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110589    2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 16 14:04:26.116410 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110592    2580 flags.go:64] FLAG: --kube-api-qps="50"
Sep 16 14:04:26.116410 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110594    2580 flags.go:64] FLAG: --kube-reserved=""
Sep 16 14:04:26.116410 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110597    2580 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 16 14:04:26.116410 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110600    2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 16 14:04:26.117019 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110604    2580 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 16 14:04:26.117019 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110606    2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 16 14:04:26.117019 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110609    2580 flags.go:64] FLAG: --lock-file=""
Sep 16 14:04:26.117019 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110612    2580 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 16 14:04:26.117019 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110615    2580 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 16 14:04:26.117019 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110618    2580 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 16 14:04:26.117019 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110624    2580 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 16 14:04:26.117019 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110627    2580 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 16 14:04:26.117019 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110632    2580 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 16 14:04:26.117019 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110635    2580 flags.go:64] FLAG: --logging-format="text"
Sep 16 14:04:26.117019 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110637    2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 16 14:04:26.117019 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110641    2580 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 16 14:04:26.117019 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110643    2580 flags.go:64] FLAG: --manifest-url=""
Sep 16 14:04:26.117019 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110646    2580 flags.go:64] FLAG: --manifest-url-header=""
Sep 16 14:04:26.117019 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110651    2580 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 16 14:04:26.117019 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110654    2580 flags.go:64] FLAG: --max-open-files="1000000"
Sep 16 14:04:26.117019 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110659    2580 flags.go:64] FLAG: --max-pods="110"
Sep 16 14:04:26.117019 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110661    2580 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 16 14:04:26.117019 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110664    2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 16 14:04:26.117019 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110669    2580 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 16 14:04:26.117019 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110672    2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 16 14:04:26.117019 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110675    2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 16 14:04:26.117019 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110678    2580 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 16 14:04:26.117019 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110681    2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 16 14:04:26.117019 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110689    2580 flags.go:64] FLAG: --node-status-max-images="50"
Sep 16 14:04:26.117639 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110692    2580 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 16 14:04:26.117639 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110695    2580 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 16 14:04:26.117639 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110698    2580 flags.go:64] FLAG: --pod-cidr=""
Sep 16 14:04:26.117639 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110701    2580 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 16 14:04:26.117639 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110706    2580 flags.go:64] FLAG: --pod-manifest-path=""
Sep 16 14:04:26.117639 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110709    2580 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 16 14:04:26.117639 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110712    2580 flags.go:64] FLAG: --pods-per-core="0"
Sep 16 14:04:26.117639 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110715    2580 flags.go:64] FLAG: --port="10250"
Sep 16 14:04:26.117639 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110718    2580 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 16 14:04:26.117639 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110721    2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0dec4ec1c0e12db5e"
Sep 16 14:04:26.117639 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110725    2580 flags.go:64] FLAG: --qos-reserved=""
Sep 16 14:04:26.117639 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110728    2580 flags.go:64] FLAG: --read-only-port="10255"
Sep 16 14:04:26.117639 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110730    2580 flags.go:64] FLAG: --register-node="true"
Sep 16 14:04:26.117639 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110734    2580 flags.go:64] FLAG: --register-with-taints=""
Sep 16 14:04:26.117639 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110738    2580 flags.go:64] FLAG: --registry-burst="10"
Sep 16 14:04:26.117639 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110741    2580 flags.go:64] FLAG: --registry-qps="5"
Sep 16 14:04:26.117639 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110744    2580 flags.go:64] FLAG: --reserved-cpus=""
Sep 16 14:04:26.117639 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110747    2580 flags.go:64] FLAG: --reserved-memory=""
Sep 16 14:04:26.117639 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110751    2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 16 14:04:26.117639 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110754    2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 16 14:04:26.117639 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110757    2580 flags.go:64] FLAG: --rotate-certificates="false"
Sep 16 14:04:26.117639 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110759    2580 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 16 14:04:26.117639 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110762    2580 flags.go:64] FLAG: --runonce="false"
Sep 16 14:04:26.117639 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110765    2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 16 14:04:26.117639 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110768    2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 16 14:04:26.118257 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110771    2580 flags.go:64] FLAG: --seccomp-default="false"
Sep 16 14:04:26.118257 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110774    2580 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 16 14:04:26.118257 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110778    2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 16 14:04:26.118257 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110781    2580 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 16 14:04:26.118257 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110783    2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 16 14:04:26.118257 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110787    2580 flags.go:64] FLAG: --storage-driver-password="root"
Sep 16 14:04:26.118257 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110789    2580 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 16 14:04:26.118257 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110792    2580 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 16 14:04:26.118257 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110795    2580 flags.go:64] FLAG: --storage-driver-user="root"
Sep 16 14:04:26.118257 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110798    2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 16 14:04:26.118257 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110801    2580 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 16 14:04:26.118257 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110804    2580 flags.go:64] FLAG: --system-cgroups=""
Sep 16 14:04:26.118257 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110806    2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 16 14:04:26.118257 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110812    2580 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 16 14:04:26.118257 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110815    2580 flags.go:64] FLAG: --tls-cert-file=""
Sep 16 14:04:26.118257 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110817    2580 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 16 14:04:26.118257 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110821    2580 flags.go:64] FLAG: --tls-min-version=""
Sep 16 14:04:26.118257 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110824    2580 flags.go:64] FLAG: --tls-private-key-file=""
Sep 16 14:04:26.118257 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110827    2580 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 16 14:04:26.118257 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110830    2580 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 16 14:04:26.118257 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110834    2580 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 16 14:04:26.118257 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110837    2580 flags.go:64] FLAG: --v="2"
Sep 16 14:04:26.118257 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110841    2580 flags.go:64] FLAG: --version="false"
Sep 16 14:04:26.118257 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110844    2580 flags.go:64] FLAG: --vmodule=""
Sep 16 14:04:26.118257 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110848    2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 16 14:04:26.118902 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.110852    2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 16 14:04:26.118902 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110980    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 14:04:26.118902 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110984    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 14:04:26.118902 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110987    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 14:04:26.118902 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110990    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 14:04:26.118902 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110993    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 14:04:26.118902 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110995    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 14:04:26.118902 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.110998    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 14:04:26.118902 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111001    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 14:04:26.118902 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111005    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 14:04:26.118902 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111007    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 14:04:26.118902 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111010    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 14:04:26.118902 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111013    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 14:04:26.118902 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111016    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 14:04:26.118902 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111018    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 14:04:26.118902 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111021    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 14:04:26.118902 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111024    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 14:04:26.118902 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111027    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 14:04:26.118902 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111029    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 14:04:26.119368 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111032    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 14:04:26.119368 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111034    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 14:04:26.119368 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111037    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 14:04:26.119368 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111040    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 14:04:26.119368 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111043    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 14:04:26.119368 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111045    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 14:04:26.119368 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111047    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 14:04:26.119368 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111050    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 14:04:26.119368 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111054    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 14:04:26.119368 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111057    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 14:04:26.119368 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111059    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 14:04:26.119368 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111062    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 14:04:26.119368 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111065    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 14:04:26.119368 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111068    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 14:04:26.119368 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111071    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 14:04:26.119368 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111073    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 14:04:26.119368 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111076    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 14:04:26.119368 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111078    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 14:04:26.119368 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111081    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 14:04:26.119368 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111083    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 14:04:26.119847 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111086    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 14:04:26.119847 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111088    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 14:04:26.119847 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111091    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 14:04:26.119847 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111094    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 14:04:26.119847 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111097    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 14:04:26.119847 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111099    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 14:04:26.119847 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111101    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 14:04:26.119847 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111104    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 14:04:26.119847 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111106    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 14:04:26.119847 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111109    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 14:04:26.119847 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111112    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 14:04:26.119847 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111114    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 14:04:26.119847 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111118    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 14:04:26.119847 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111122    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 14:04:26.119847 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111125    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 14:04:26.119847 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111127    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 14:04:26.119847 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111130    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 14:04:26.119847 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111133    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 14:04:26.119847 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111136    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 14:04:26.120349 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111138    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 14:04:26.120349 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111142    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 14:04:26.120349 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111145    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 14:04:26.120349 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111148    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 14:04:26.120349 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111150    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 14:04:26.120349 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111153    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 14:04:26.120349 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111155    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 14:04:26.120349 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111158    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 14:04:26.120349 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111161    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 14:04:26.120349 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111164    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 14:04:26.120349 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111168    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 14:04:26.120349 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111171    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 14:04:26.120349 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111173    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 14:04:26.120349 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111176    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 14:04:26.120349 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111179    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 14:04:26.120349 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111182    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 14:04:26.120349 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111184    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 14:04:26.120349 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111187    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 14:04:26.120349 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111190    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 14:04:26.120349 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111192    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 14:04:26.120834 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111195    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 14:04:26.120834 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111197    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 14:04:26.120834 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111200    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 14:04:26.120834 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111203    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 14:04:26.120834 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111205    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 14:04:26.120834 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111208    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 14:04:26.120834 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111210    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 14:04:26.120834 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111213    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 14:04:26.120834 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111217    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 14:04:26.120834 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111220    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 14:04:26.120834 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111223    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 14:04:26.120834 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111226    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 14:04:26.120834 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111229    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 14:04:26.120834 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111233    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 14:04:26.120834 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111236    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 14:04:26.120834 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111238    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 14:04:26.120834 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111241    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 14:04:26.120834 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111244    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 14:04:26.120834 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111246    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 14:04:26.120834 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111249    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 14:04:26.121346 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111252    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 14:04:26.121346 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.111255    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 14:04:26.121346 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.111751    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 16 14:04:26.121346 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.118761    2580 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 16 14:04:26.121346 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.118778    2580 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 16 14:04:26.121346 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.119436    2580 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 16 14:04:26.121346 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.119444    2580 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 16 14:04:26.121346 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119499    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 14:04:26.121346 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119505    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 14:04:26.121346 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119508    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 14:04:26.121346 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119512    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 14:04:26.121346 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119515    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 14:04:26.121346 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119517    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 14:04:26.121346 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119520    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 14:04:26.121346 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119523    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 14:04:26.121723 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119526    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 14:04:26.121723 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119528    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 14:04:26.121723 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119531    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 14:04:26.121723 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119534    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 14:04:26.121723 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119536    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 14:04:26.121723 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119539    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 14:04:26.121723 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119542    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 14:04:26.121723 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119544    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 14:04:26.121723 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119547    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 14:04:26.121723 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119549    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 14:04:26.121723 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119552    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 14:04:26.121723 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119555    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 14:04:26.121723 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119558    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 14:04:26.121723 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119561    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 14:04:26.121723 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119564    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 14:04:26.121723 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119566    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 14:04:26.121723 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119569    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 14:04:26.121723 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119571    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 14:04:26.121723 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119574    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 14:04:26.121723 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119577    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 14:04:26.122228 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119580    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 14:04:26.122228 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119584    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 14:04:26.122228 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119586    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 14:04:26.122228 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119589    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 14:04:26.122228 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119591    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 14:04:26.122228 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119594    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 14:04:26.122228 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119597    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 14:04:26.122228 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119599    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 14:04:26.122228 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119602    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 14:04:26.122228 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119605    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 14:04:26.122228 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119607    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 14:04:26.122228 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119610    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 14:04:26.122228 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119613    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 14:04:26.122228 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119615    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 14:04:26.122228 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119618    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 14:04:26.122228 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119620    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 14:04:26.122228 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119623    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 14:04:26.122228 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119626    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 14:04:26.122228 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119628    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 14:04:26.122697 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119631    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 14:04:26.122697 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119633    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 14:04:26.122697 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119636    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 14:04:26.122697 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119639    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 14:04:26.122697 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119641    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 14:04:26.122697 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119644    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 14:04:26.122697 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119647    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 14:04:26.122697 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119650    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 14:04:26.122697 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119652    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 14:04:26.122697 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119655    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 14:04:26.122697 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119657    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 14:04:26.122697 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119660    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 14:04:26.122697 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119663    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 14:04:26.122697 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119666    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 14:04:26.122697 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119669    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 14:04:26.122697 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119673    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 14:04:26.122697 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119677    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 14:04:26.122697 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119680    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 14:04:26.122697 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119684    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 14:04:26.122697 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119688    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 14:04:26.123207 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119692    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 14:04:26.123207 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119695    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 14:04:26.123207 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119698    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 14:04:26.123207 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119701    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 14:04:26.123207 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119704    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 14:04:26.123207 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119706    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 14:04:26.123207 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119709    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 14:04:26.123207 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119712    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 14:04:26.123207 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119715    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 14:04:26.123207 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119717    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 14:04:26.123207 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119720    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 14:04:26.123207 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119722    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 14:04:26.123207 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119725    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 14:04:26.123207 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119727    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 14:04:26.123207 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119730    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 14:04:26.123207 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119733    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 14:04:26.123207 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119735    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 14:04:26.123207 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119738    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 14:04:26.123207 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119740    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 14:04:26.123207 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119743    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 14:04:26.123687 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119746    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 14:04:26.123687 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119748    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 14:04:26.123687 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119751    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 14:04:26.123687 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119753    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 14:04:26.123687 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119756    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 14:04:26.123687 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119759    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 14:04:26.123687 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119761    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 14:04:26.123687 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119765    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 14:04:26.123687 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119772    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 14:04:26.123687 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119775    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 14:04:26.123687 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119778    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 14:04:26.123687 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119781    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 14:04:26.123687 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.119786    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 16 14:04:26.123687 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119954    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 14:04:26.124041 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119961    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 14:04:26.124041 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119964    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 14:04:26.124041 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119968    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 14:04:26.124041 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119971    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 14:04:26.124041 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119973    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 14:04:26.124041 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119976    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 14:04:26.124041 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119979    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 14:04:26.124041 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119983    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 14:04:26.124041 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119985    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 14:04:26.124041 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119988    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 14:04:26.124041 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119990    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 14:04:26.124041 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119993    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 14:04:26.124041 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119996    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 14:04:26.124041 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.119998    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 14:04:26.124041 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120001    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 14:04:26.124041 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120003    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 14:04:26.124041 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120006    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 14:04:26.124041 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120008    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 14:04:26.124041 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120023    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 14:04:26.124041 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120026    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 14:04:26.124508 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120030    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 14:04:26.124508 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120033    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 14:04:26.124508 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120036    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 14:04:26.124508 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120038    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 14:04:26.124508 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120041    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 14:04:26.124508 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120044    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 14:04:26.124508 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120047    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 14:04:26.124508 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120050    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 14:04:26.124508 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120053    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 14:04:26.124508 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120055    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 14:04:26.124508 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120058    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 14:04:26.124508 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120060    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 14:04:26.124508 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120063    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 14:04:26.124508 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120066    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 14:04:26.124508 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120068    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 14:04:26.124508 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120071    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 14:04:26.124508 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120073    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 14:04:26.124508 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120076    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 14:04:26.124508 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120078    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 14:04:26.124508 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120081    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 14:04:26.124995 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120083    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 14:04:26.124995 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120085    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 14:04:26.124995 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120088    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 14:04:26.124995 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120092    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 14:04:26.124995 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120095    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 14:04:26.124995 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120098    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 14:04:26.124995 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120101    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 14:04:26.124995 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120104    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 14:04:26.124995 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120106    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 14:04:26.124995 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120109    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 14:04:26.124995 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120111    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 14:04:26.124995 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120114    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 14:04:26.124995 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120116    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 14:04:26.124995 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120119    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 14:04:26.124995 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120121    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 14:04:26.124995 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120124    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 14:04:26.124995 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120126    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 14:04:26.124995 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120129    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 14:04:26.124995 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120132    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 14:04:26.125480 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120134    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 14:04:26.125480 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120137    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 14:04:26.125480 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120140    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 14:04:26.125480 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120142    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 14:04:26.125480 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120145    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 14:04:26.125480 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120147    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 14:04:26.125480 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120150    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 14:04:26.125480 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120153    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 14:04:26.125480 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120155    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 14:04:26.125480 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120158    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 14:04:26.125480 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120160    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 14:04:26.125480 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120164    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 14:04:26.125480 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120167    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 14:04:26.125480 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120170    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 14:04:26.125480 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120172    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 14:04:26.125480 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120175    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 14:04:26.125480 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120178    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 14:04:26.125480 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120180    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 14:04:26.125480 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120183    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 14:04:26.125480 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120185    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 14:04:26.125974 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120188    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 14:04:26.125974 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120190    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 14:04:26.125974 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120193    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 14:04:26.125974 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120195    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 14:04:26.125974 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120198    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 14:04:26.125974 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120202    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 14:04:26.125974 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120205    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 14:04:26.125974 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120208    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 14:04:26.125974 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120211    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 14:04:26.125974 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120214    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 14:04:26.125974 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120220    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 14:04:26.125974 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120222    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 14:04:26.125974 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120225    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 14:04:26.125974 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120228    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 14:04:26.125974 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120231    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 14:04:26.125974 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120233    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 14:04:26.125974 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120236    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 14:04:26.125974 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120239    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 14:04:26.125974 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:26.120242    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 14:04:26.126446 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.120246    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 16 14:04:26.126446 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.120346    2580 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 16 14:04:26.126446 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.122260    2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 16 14:04:26.126446 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.123018    2580 server.go:1019] "Starting client certificate rotation"
Sep 16 14:04:26.126446 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.123118    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 16 14:04:26.126446 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.123745    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 16 14:04:26.126446 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.125225    2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 16 14:04:26.126446 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.126347    2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 16 14:04:26.136005 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.135990    2580 log.go:25] "Validated CRI v1 runtime API"
Sep 16 14:04:26.138545 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.138526    2580 log.go:25] "Validated CRI v1 image API"
Sep 16 14:04:26.141597 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.141580    2580 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 16 14:04:26.145266 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.145249    2580 fs.go:135] Filesystem UUIDs: map[76341c2b-067a-4fb9-b365-c415b8e384a1:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 7f1f5c02-2fa0-43dc-aaf2-3267cbead8e8:/dev/nvme0n1p4]
Sep 16 14:04:26.145318 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.145265    2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 16 14:04:26.150897 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.150876    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 16 14:04:26.152000 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.151893    2580 manager.go:216] Machine: {Timestamp:2026-09-16 14:04:26.150482725 +0000 UTC m=+0.271222078 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100354 MemoryCapacity:33164476416 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2303f66353677bc6140fa0249c6dcc SystemUUID:ec2303f6-6353-677b-c614-0fa0249c6dcc BootID:d73aebc9-0d56-4dab-baa7-5429d73c6fa5 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:ed:b2:8a:8d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:ed:b2:8a:8d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:3e:8e:20:a3:23:cb Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164476416 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 16 14:04:26.152000 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.151995    2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 16 14:04:26.152111 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.152067    2580 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 16 14:04:26.154695 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.154669    2580 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 16 14:04:26.154854 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.154698    2580 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-95.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 16 14:04:26.155973 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.155960    2580 topology_manager.go:138] "Creating topology manager with none policy"
Sep 16 14:04:26.156031 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.155981    2580 container_manager_linux.go:309] "Creating device plugin manager"
Sep 16 14:04:26.156031 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.155993    2580 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 16 14:04:26.156532 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.156520    2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 16 14:04:26.157367 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.157355    2580 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 16 14:04:26.161389 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.161375    2580 state_mem.go:36] "Initialized new in-memory state store"
Sep 16 14:04:26.161535 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.161524    2580 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 16 14:04:26.161602 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.161561    2580 kubelet.go:486] "Attempting to sync node with API server"
Sep 16 14:04:26.161602 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.161575    2580 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 16 14:04:26.161602 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.161594    2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 16 14:04:26.161726 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.161607    2580 kubelet.go:398] "Adding apiserver pod source"
Sep 16 14:04:26.161726 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.161620    2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 16 14:04:26.161994 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.161981    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 16 14:04:26.162056 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.162005    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 16 14:04:26.163748 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.163729    2580 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 16 14:04:26.166052 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.166038    2580 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 16 14:04:26.166108 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.166069    2580 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 16 14:04:26.167705 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.167694    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 16 14:04:26.167758 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.167710    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 16 14:04:26.167758 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.167716    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 16 14:04:26.167758 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.167721    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 16 14:04:26.167758 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.167729    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 16 14:04:26.167758 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.167734    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 16 14:04:26.167758 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.167750    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 16 14:04:26.167758 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.167758    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 16 14:04:26.167953 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.167766    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 16 14:04:26.167953 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.167773    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 16 14:04:26.167953 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.167783    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 16 14:04:26.167953 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.167788    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 16 14:04:26.169002 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.168990    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 16 14:04:26.169002 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.169001    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 16 14:04:26.170022 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:26.170001    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-95.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 16 14:04:26.170022 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:26.170012    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 16 14:04:26.172523 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.172509    2580 server.go:1268] "Started kubelet"
Sep 16 14:04:26.172612 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.172572    2580 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 16 14:04:26.172664 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.172597    2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 16 14:04:26.172710 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.172691    2580 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 16 14:04:26.173421 ip-10-0-129-95 systemd[1]: Started Kubernetes Kubelet.
Sep 16 14:04:26.174374 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.173938    2580 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 16 14:04:26.175667 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.175643    2580 server.go:310] "Adding debug handlers to kubelet server"
Sep 16 14:04:26.177471 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.177448    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rlhnl"
Sep 16 14:04:26.179385 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:26.179340    2580 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 16 14:04:26.179563 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.179544    2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-95.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 16 14:04:26.179717 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.179695    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 16 14:04:26.180129 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.180116    2580 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 16 14:04:26.180242 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:26.180226    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-95.ec2.internal\" not found"
Sep 16 14:04:26.180429 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:26.179595    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-95.ec2.internal.18d5d22102ce838d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-95.ec2.internal,UID:ip-10-0-129-95.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-95.ec2.internal,},FirstTimestamp:2026-09-16 14:04:26.172482445 +0000 UTC m=+0.293221800,LastTimestamp:2026-09-16 14:04:26.172482445 +0000 UTC m=+0.293221800,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-95.ec2.internal,}"
Sep 16 14:04:26.180651 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.180637    2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 16 14:04:26.180715 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.180638    2580 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 16 14:04:26.180715 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.180672    2580 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 16 14:04:26.181054 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.181040    2580 reconstruct.go:97] "Volume reconstruction finished"
Sep 16 14:04:26.181054 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.181054    2580 reconciler.go:29] "Reconciler: start to sync state"
Sep 16 14:04:26.183923 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.182465    2580 factory.go:55] Registering systemd factory
Sep 16 14:04:26.183923 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.182489    2580 factory.go:223] Registration of the systemd container factory successfully
Sep 16 14:04:26.183923 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.182746    2580 factory.go:153] Registering CRI-O factory
Sep 16 14:04:26.183923 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.182762    2580 factory.go:223] Registration of the crio container factory successfully
Sep 16 14:04:26.183923 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.182823    2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 16 14:04:26.183923 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.182841    2580 factory.go:103] Registering Raw factory
Sep 16 14:04:26.183923 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.182856    2580 manager.go:1196] Started watching for new ooms in manager
Sep 16 14:04:26.183923 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.183255    2580 manager.go:349] Starting recovery of all containers
Sep 16 14:04:26.185205 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.185175    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rlhnl"
Sep 16 14:04:26.185438 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:26.185419    2580 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-95.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 16 14:04:26.185570 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:26.185550    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 16 14:04:26.190525 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.190492    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 16 14:04:26.195110 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.195063    2580 manager.go:354] Recovery completed
Sep 16 14:04:26.200972 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.200956    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 14:04:26.202838 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.202823    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-95.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 14:04:26.202909 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.202849    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-95.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 14:04:26.202909 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.202861    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-95.ec2.internal" event="NodeHasSufficientPID"
Sep 16 14:04:26.203449 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.203434    2580 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 16 14:04:26.203449 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.203446    2580 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 16 14:04:26.203588 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.203452    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 14:04:26.203588 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.203460    2580 state_mem.go:36] "Initialized new in-memory state store"
Sep 16 14:04:26.204293 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.204271    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-95.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 14:04:26.204469 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.204311    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-95.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 14:04:26.204469 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.204326    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-95.ec2.internal" event="NodeHasSufficientPID"
Sep 16 14:04:26.205442 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.205424    2580 policy_none.go:49] "None policy: Start"
Sep 16 14:04:26.205442 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.205443    2580 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 16 14:04:26.205556 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.205454    2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 16 14:04:26.206596 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:26.205765    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-95.ec2.internal.18d5d221049db1a7  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-95.ec2.internal,UID:ip-10-0-129-95.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-95.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-95.ec2.internal,},FirstTimestamp:2026-09-16 14:04:26.202837415 +0000 UTC m=+0.323576769,LastTimestamp:2026-09-16 14:04:26.202837415 +0000 UTC m=+0.323576769,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-95.ec2.internal,}"
Sep 16 14:04:26.207479 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.207469    2580 policy_none.go:47] "Start"
Sep 16 14:04:26.231449 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.231435    2580 manager.go:337] "Starting Device Plugin manager"
Sep 16 14:04:26.255148 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:26.231468    2580 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 16 14:04:26.255148 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.231478    2580 server.go:87] "Starting device plugin registration server"
Sep 16 14:04:26.255148 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.231744    2580 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 16 14:04:26.255148 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.231754    2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 16 14:04:26.255148 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.231883    2580 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 16 14:04:26.255148 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.231994    2580 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 16 14:04:26.255148 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.232003    2580 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 16 14:04:26.255148 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:26.232493    2580 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 16 14:04:26.255148 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:26.232540    2580 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-95.ec2.internal\" not found"
Sep 16 14:04:26.318057 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.318036    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 16 14:04:26.318184 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.318086    2580 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 16 14:04:26.318184 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.318109    2580 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 16 14:04:26.318184 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:26.318148    2580 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 16 14:04:26.320769 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.320751    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 14:04:26.332783 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.332767    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 14:04:26.333616 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.333600    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-95.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 14:04:26.333691 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.333628    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-95.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 14:04:26.333691 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.333637    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-95.ec2.internal" event="NodeHasSufficientPID"
Sep 16 14:04:26.333691 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.333666    2580 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-95.ec2.internal"
Sep 16 14:04:26.343922 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.343903    2580 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-95.ec2.internal"
Sep 16 14:04:26.343991 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:26.343923    2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-95.ec2.internal\": node \"ip-10-0-129-95.ec2.internal\" not found"
Sep 16 14:04:26.398377 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:26.398314    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-95.ec2.internal\" not found"
Sep 16 14:04:26.419011 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.418968    2580 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-95.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-95.ec2.internal"]
Sep 16 14:04:26.419077 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.419050    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 14:04:26.419838 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.419820    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-95.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 14:04:26.419945 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.419845    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-95.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 14:04:26.419945 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.419856    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-95.ec2.internal" event="NodeHasSufficientPID"
Sep 16 14:04:26.422150 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.422138    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 14:04:26.422308 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.422293    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-95.ec2.internal"
Sep 16 14:04:26.422344 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.422323    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 14:04:26.422905 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.422891    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-95.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 14:04:26.422996 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.422911    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-95.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 14:04:26.422996 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.422895    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-95.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 14:04:26.422996 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.422921    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-95.ec2.internal" event="NodeHasSufficientPID"
Sep 16 14:04:26.422996 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.422938    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-95.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 14:04:26.422996 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.422950    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-95.ec2.internal" event="NodeHasSufficientPID"
Sep 16 14:04:26.425188 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.425174    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-95.ec2.internal"
Sep 16 14:04:26.425264 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.425199    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 14:04:26.425843 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.425819    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-95.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 14:04:26.425931 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.425851    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-95.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 14:04:26.425931 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.425878    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-95.ec2.internal" event="NodeHasSufficientPID"
Sep 16 14:04:26.448426 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:26.448412    2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-95.ec2.internal\" not found" node="ip-10-0-129-95.ec2.internal"
Sep 16 14:04:26.452687 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:26.452673    2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-95.ec2.internal\" not found" node="ip-10-0-129-95.ec2.internal"
Sep 16 14:04:26.482346 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.482328    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/cf3700a3bae4d7dda42c2b6a53ad43be-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-95.ec2.internal\" (UID: \"cf3700a3bae4d7dda42c2b6a53ad43be\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-95.ec2.internal"
Sep 16 14:04:26.482434 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.482352    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cf3700a3bae4d7dda42c2b6a53ad43be-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-95.ec2.internal\" (UID: \"cf3700a3bae4d7dda42c2b6a53ad43be\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-95.ec2.internal"
Sep 16 14:04:26.482434 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.482367    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/38046ded5fcb5dec29d3f7bce0f86c40-config\") pod \"kube-apiserver-proxy-ip-10-0-129-95.ec2.internal\" (UID: \"38046ded5fcb5dec29d3f7bce0f86c40\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-95.ec2.internal"
Sep 16 14:04:26.498696 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:26.498680    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-95.ec2.internal\" not found"
Sep 16 14:04:26.583361 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.583336    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/cf3700a3bae4d7dda42c2b6a53ad43be-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-95.ec2.internal\" (UID: \"cf3700a3bae4d7dda42c2b6a53ad43be\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-95.ec2.internal"
Sep 16 14:04:26.583460 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.583371    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cf3700a3bae4d7dda42c2b6a53ad43be-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-95.ec2.internal\" (UID: \"cf3700a3bae4d7dda42c2b6a53ad43be\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-95.ec2.internal"
Sep 16 14:04:26.583460 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.583388    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/38046ded5fcb5dec29d3f7bce0f86c40-config\") pod \"kube-apiserver-proxy-ip-10-0-129-95.ec2.internal\" (UID: \"38046ded5fcb5dec29d3f7bce0f86c40\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-95.ec2.internal"
Sep 16 14:04:26.583460 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.583448    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/cf3700a3bae4d7dda42c2b6a53ad43be-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-95.ec2.internal\" (UID: \"cf3700a3bae4d7dda42c2b6a53ad43be\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-95.ec2.internal"
Sep 16 14:04:26.583588 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.583463    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cf3700a3bae4d7dda42c2b6a53ad43be-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-95.ec2.internal\" (UID: \"cf3700a3bae4d7dda42c2b6a53ad43be\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-95.ec2.internal"
Sep 16 14:04:26.583588 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.583524    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/38046ded5fcb5dec29d3f7bce0f86c40-config\") pod \"kube-apiserver-proxy-ip-10-0-129-95.ec2.internal\" (UID: \"38046ded5fcb5dec29d3f7bce0f86c40\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-95.ec2.internal"
Sep 16 14:04:26.598980 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:26.598960    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-95.ec2.internal\" not found"
Sep 16 14:04:26.699682 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:26.699613    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-95.ec2.internal\" not found"
Sep 16 14:04:26.751818 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.751777    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-95.ec2.internal"
Sep 16 14:04:26.755100 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:26.755074    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-95.ec2.internal"
Sep 16 14:04:26.800046 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:26.800027    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-95.ec2.internal\" not found"
Sep 16 14:04:26.900468 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:26.900452    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-95.ec2.internal\" not found"
Sep 16 14:04:27.001053 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:27.000986    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-95.ec2.internal\" not found"
Sep 16 14:04:28.207059 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:28.207037    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-95.ec2.internal\" not found"
Sep 16 14:04:28.229200 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:28.229185    2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 16 14:04:28.229562 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:28.229325    2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 16 14:04:28.296127 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:28.296097    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-15 13:59:27 +0000 UTC" deadline="2028-03-17 08:32:02.383963436 +0000 UTC"
Sep 16 14:04:28.296127 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:28.296122    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13146h27m34.087844055s"
Sep 16 14:04:28.307213 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:28.307197    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-95.ec2.internal\" not found"
Sep 16 14:04:28.407847 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:28.407769    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-95.ec2.internal\" not found"
Sep 16 14:04:28.508186 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:28.508156    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-95.ec2.internal\" not found"
Sep 16 14:04:28.608655 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:28.608624    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-95.ec2.internal\" not found"
Sep 16 14:04:28.709101 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:28.709076    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-95.ec2.internal\" not found"
Sep 16 14:04:28.725845 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:28.725828    2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 14:04:28.786831 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:28.786810    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-95.ec2.internal"
Sep 16 14:04:28.786923 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:28.786909    2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 14:04:28.801427 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:28.801412    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 16 14:04:28.802852 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:28.802834    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-95.ec2.internal"
Sep 16 14:04:28.815275 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:28.815253    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 16 14:04:28.881940 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:28.881914    2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 14:04:29.268243 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.268221    2580 apiserver.go:52] "Watching apiserver"
Sep 16 14:04:29.272727 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.272711    2580 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 16 14:04:29.273083 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.273065    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-wg92s","kube-system/konnectivity-agent-cxcmv","kube-system/kube-apiserver-proxy-ip-10-0-129-95.ec2.internal","openshift-image-registry/node-ca-r9k4l","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-95.ec2.internal","openshift-multus/multus-7xg75","openshift-multus/multus-additional-cni-plugins-c5xjg","openshift-multus/network-metrics-daemon-pxv9d","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbrr5","openshift-cluster-node-tuning-operator/tuned-r62q2","openshift-dns/node-resolver-s544c","openshift-network-diagnostics/network-check-target-tgkvk","openshift-network-operator/iptables-alerter-kwf42","openshift-ovn-kubernetes/ovnkube-node-z4h56"]
Sep 16 14:04:29.275606 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.275593    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-c5xjg"
Sep 16 14:04:29.277337 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.277322    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 16 14:04:29.277538 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.277523    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 16 14:04:29.277649 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.277623    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 16 14:04:29.277785 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.277665    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 16 14:04:29.277785 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.277679    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 16 14:04:29.277785 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.277770    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-s544c"
Sep 16 14:04:29.277941 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.277903    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-mgvg9\""
Sep 16 14:04:29.280538 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.279397    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-swwlx\""
Sep 16 14:04:29.280538 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.279658    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 16 14:04:29.280538 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.279853    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 16 14:04:29.281064 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.281049    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-z4h56"
Sep 16 14:04:29.282508 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.282484    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 16 14:04:29.282576 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.282510    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 16 14:04:29.282703 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.282690    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 16 14:04:29.282738 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.282700    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 16 14:04:29.283015 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.283002    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-9cqmq\""
Sep 16 14:04:29.283049 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.283003    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 16 14:04:29.283083 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.283062    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 16 14:04:29.283341 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.283326    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-r9k4l"
Sep 16 14:04:29.285109 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.285094    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-4fxz6\""
Sep 16 14:04:29.285109 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.285106    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 16 14:04:29.285293 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.285282    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 16 14:04:29.285484 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.285469    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 16 14:04:29.285569 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.285492    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pxv9d"
Sep 16 14:04:29.285569 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:29.285551    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pxv9d" podUID="f0346ba8-7f08-4472-933c-7238392887b6"
Sep 16 14:04:29.286131 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.286114    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 16 14:04:29.287727 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.287711    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7xg75"
Sep 16 14:04:29.289318 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.289300    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-xj5ng\""
Sep 16 14:04:29.289385 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.289344    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 16 14:04:29.289954 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.289940    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbrr5"
Sep 16 14:04:29.292102 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.292089    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kwf42"
Sep 16 14:04:29.292515 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.292487    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 16 14:04:29.292599 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.292566    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-grknj\""
Sep 16 14:04:29.292748 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.292735    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 16 14:04:29.292816 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.292769    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 16 14:04:29.293365 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.293352    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 16 14:04:29.293852 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.293834    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 16 14:04:29.293916 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.293903    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-tpfr6\""
Sep 16 14:04:29.294103 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.294086    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 16 14:04:29.294305 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.294291    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 16 14:04:29.294356 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.294330    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-cxcmv"
Sep 16 14:04:29.295211 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.295197    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e046d878-c8a6-4278-bde8-238f63fb96b2-tmp-dir\") pod \"node-resolver-s544c\" (UID: \"e046d878-c8a6-4278-bde8-238f63fb96b2\") " pod="openshift-dns/node-resolver-s544c"
Sep 16 14:04:29.295275 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.295224    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f1c44fe4-da28-4517-a5ce-b47ea2592ba4-log-socket\") pod \"ovnkube-node-z4h56\" (UID: \"f1c44fe4-da28-4517-a5ce-b47ea2592ba4\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4h56"
Sep 16 14:04:29.295275 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.295239    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fa3e87c7-91e7-4767-b43d-6f00261f1d56-host\") pod \"node-ca-r9k4l\" (UID: \"fa3e87c7-91e7-4767-b43d-6f00261f1d56\") " pod="openshift-image-registry/node-ca-r9k4l"
Sep 16 14:04:29.295275 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.295253    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d89c07ec-f200-4033-9fe1-c8dab7ef1d48-host-run-k8s-cni-cncf-io\") pod \"multus-7xg75\" (UID: \"d89c07ec-f200-4033-9fe1-c8dab7ef1d48\") " pod="openshift-multus/multus-7xg75"
Sep 16 14:04:29.295380 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.295287    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b586j\" (UniqueName: \"kubernetes.io/projected/9840e56c-3a33-49c0-a917-70f641aabf3b-kube-api-access-b586j\") pod \"multus-additional-cni-plugins-c5xjg\" (UID: \"9840e56c-3a33-49c0-a917-70f641aabf3b\") " pod="openshift-multus/multus-additional-cni-plugins-c5xjg"
Sep 16 14:04:29.295380 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.295311    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f1c44fe4-da28-4517-a5ce-b47ea2592ba4-host-slash\") pod \"ovnkube-node-z4h56\" (UID: \"f1c44fe4-da28-4517-a5ce-b47ea2592ba4\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4h56"
Sep 16 14:04:29.295380 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.295327    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d89c07ec-f200-4033-9fe1-c8dab7ef1d48-cnibin\") pod \"multus-7xg75\" (UID: \"d89c07ec-f200-4033-9fe1-c8dab7ef1d48\") " pod="openshift-multus/multus-7xg75"
Sep 16 14:04:29.295380 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.295355    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d89c07ec-f200-4033-9fe1-c8dab7ef1d48-cni-binary-copy\") pod \"multus-7xg75\" (UID: \"d89c07ec-f200-4033-9fe1-c8dab7ef1d48\") " pod="openshift-multus/multus-7xg75"
Sep 16 14:04:29.295380 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.295370    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d89c07ec-f200-4033-9fe1-c8dab7ef1d48-host-var-lib-cni-bin\") pod \"multus-7xg75\" (UID: \"d89c07ec-f200-4033-9fe1-c8dab7ef1d48\") " pod="openshift-multus/multus-7xg75"
Sep 16 14:04:29.295629 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.295382    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d89c07ec-f200-4033-9fe1-c8dab7ef1d48-multus-conf-dir\") pod \"multus-7xg75\" (UID: \"d89c07ec-f200-4033-9fe1-c8dab7ef1d48\") " pod="openshift-multus/multus-7xg75"
Sep 16 14:04:29.295629 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.295404    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f1c44fe4-da28-4517-a5ce-b47ea2592ba4-systemd-units\") pod \"ovnkube-node-z4h56\" (UID: \"f1c44fe4-da28-4517-a5ce-b47ea2592ba4\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4h56"
Sep 16 14:04:29.295629 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.295447    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f1c44fe4-da28-4517-a5ce-b47ea2592ba4-var-lib-openvswitch\") pod \"ovnkube-node-z4h56\" (UID: \"f1c44fe4-da28-4517-a5ce-b47ea2592ba4\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4h56"
Sep 16 14:04:29.295629 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.295477    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f1c44fe4-da28-4517-a5ce-b47ea2592ba4-etc-openvswitch\") pod \"ovnkube-node-z4h56\" (UID: \"f1c44fe4-da28-4517-a5ce-b47ea2592ba4\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4h56"
Sep 16 14:04:29.295629 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.295516    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f1c44fe4-da28-4517-a5ce-b47ea2592ba4-run-openvswitch\") pod \"ovnkube-node-z4h56\" (UID: \"f1c44fe4-da28-4517-a5ce-b47ea2592ba4\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4h56"
Sep 16 14:04:29.295629 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.295539    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f1c44fe4-da28-4517-a5ce-b47ea2592ba4-run-ovn\") pod \"ovnkube-node-z4h56\" (UID: \"f1c44fe4-da28-4517-a5ce-b47ea2592ba4\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4h56"
Sep 16 14:04:29.295629 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.295564    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47qh2\" (UniqueName: \"kubernetes.io/projected/f0346ba8-7f08-4472-933c-7238392887b6-kube-api-access-47qh2\") pod \"network-metrics-daemon-pxv9d\" (UID: \"f0346ba8-7f08-4472-933c-7238392887b6\") " pod="openshift-multus/network-metrics-daemon-pxv9d"
Sep 16 14:04:29.295629 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.295588    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d89c07ec-f200-4033-9fe1-c8dab7ef1d48-system-cni-dir\") pod \"multus-7xg75\" (UID: \"d89c07ec-f200-4033-9fe1-c8dab7ef1d48\") " pod="openshift-multus/multus-7xg75"
Sep 16 14:04:29.295629 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.295610    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d89c07ec-f200-4033-9fe1-c8dab7ef1d48-multus-cni-dir\") pod \"multus-7xg75\" (UID: \"d89c07ec-f200-4033-9fe1-c8dab7ef1d48\") " pod="openshift-multus/multus-7xg75"
Sep 16 14:04:29.295937 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.295632    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9840e56c-3a33-49c0-a917-70f641aabf3b-cni-binary-copy\") pod \"multus-additional-cni-plugins-c5xjg\" (UID: \"9840e56c-3a33-49c0-a917-70f641aabf3b\") " pod="openshift-multus/multus-additional-cni-plugins-c5xjg"
Sep 16 14:04:29.295937 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.295654    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f1c44fe4-da28-4517-a5ce-b47ea2592ba4-host-kubelet\") pod \"ovnkube-node-z4h56\" (UID: \"f1c44fe4-da28-4517-a5ce-b47ea2592ba4\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4h56"
Sep 16 14:04:29.295937 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.295678    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f1c44fe4-da28-4517-a5ce-b47ea2592ba4-host-run-ovn-kubernetes\") pod \"ovnkube-node-z4h56\" (UID: \"f1c44fe4-da28-4517-a5ce-b47ea2592ba4\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4h56"
Sep 16 14:04:29.295937 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.295699    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f1c44fe4-da28-4517-a5ce-b47ea2592ba4-host-cni-netd\") pod \"ovnkube-node-z4h56\" (UID: \"f1c44fe4-da28-4517-a5ce-b47ea2592ba4\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4h56"
Sep 16 14:04:29.295937 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.295729    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d89c07ec-f200-4033-9fe1-c8dab7ef1d48-host-var-lib-kubelet\") pod \"multus-7xg75\" (UID: \"d89c07ec-f200-4033-9fe1-c8dab7ef1d48\") " pod="openshift-multus/multus-7xg75"
Sep 16 14:04:29.295937 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.295748    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d89c07ec-f200-4033-9fe1-c8dab7ef1d48-multus-daemon-config\") pod \"multus-7xg75\" (UID: \"d89c07ec-f200-4033-9fe1-c8dab7ef1d48\") " pod="openshift-multus/multus-7xg75"
Sep 16 14:04:29.295937 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.295761    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d89c07ec-f200-4033-9fe1-c8dab7ef1d48-host-run-multus-certs\") pod \"multus-7xg75\" (UID: \"d89c07ec-f200-4033-9fe1-c8dab7ef1d48\") " pod="openshift-multus/multus-7xg75"
Sep 16 14:04:29.295937 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.295780    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9840e56c-3a33-49c0-a917-70f641aabf3b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-c5xjg\" (UID: \"9840e56c-3a33-49c0-a917-70f641aabf3b\") " pod="openshift-multus/multus-additional-cni-plugins-c5xjg"
Sep 16 14:04:29.295937 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.295806    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e046d878-c8a6-4278-bde8-238f63fb96b2-hosts-file\") pod \"node-resolver-s544c\" (UID: \"e046d878-c8a6-4278-bde8-238f63fb96b2\") " pod="openshift-dns/node-resolver-s544c"
Sep 16 14:04:29.295937 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.295836    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njl4p\" (UniqueName: \"kubernetes.io/projected/e046d878-c8a6-4278-bde8-238f63fb96b2-kube-api-access-njl4p\") pod \"node-resolver-s544c\" (UID: \"e046d878-c8a6-4278-bde8-238f63fb96b2\") " pod="openshift-dns/node-resolver-s544c"
Sep 16 14:04:29.295937 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.295854    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f1c44fe4-da28-4517-a5ce-b47ea2592ba4-run-systemd\") pod \"ovnkube-node-z4h56\" (UID: \"f1c44fe4-da28-4517-a5ce-b47ea2592ba4\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4h56"
Sep 16 14:04:29.295937 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.295885    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0346ba8-7f08-4472-933c-7238392887b6-metrics-certs\") pod \"network-metrics-daemon-pxv9d\" (UID: \"f0346ba8-7f08-4472-933c-7238392887b6\") " pod="openshift-multus/network-metrics-daemon-pxv9d"
Sep 16 14:04:29.295937 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.295907    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d89c07ec-f200-4033-9fe1-c8dab7ef1d48-multus-socket-dir-parent\") pod \"multus-7xg75\" (UID: \"d89c07ec-f200-4033-9fe1-c8dab7ef1d48\") " pod="openshift-multus/multus-7xg75"
Sep 16 14:04:29.295937 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.295924    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d89c07ec-f200-4033-9fe1-c8dab7ef1d48-etc-kubernetes\") pod \"multus-7xg75\" (UID: \"d89c07ec-f200-4033-9fe1-c8dab7ef1d48\") " pod="openshift-multus/multus-7xg75"
Sep 16 14:04:29.295937 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.295940    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9840e56c-3a33-49c0-a917-70f641aabf3b-system-cni-dir\") pod \"multus-additional-cni-plugins-c5xjg\" (UID: \"9840e56c-3a33-49c0-a917-70f641aabf3b\") " pod="openshift-multus/multus-additional-cni-plugins-c5xjg"
Sep 16 14:04:29.296391 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.295955    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9840e56c-3a33-49c0-a917-70f641aabf3b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-c5xjg\" (UID: \"9840e56c-3a33-49c0-a917-70f641aabf3b\") " pod="openshift-multus/multus-additional-cni-plugins-c5xjg"
Sep 16 14:04:29.296391 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.295980    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f1c44fe4-da28-4517-a5ce-b47ea2592ba4-host-run-netns\") pod \"ovnkube-node-z4h56\" (UID: \"f1c44fe4-da28-4517-a5ce-b47ea2592ba4\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4h56"
Sep 16 14:04:29.296391 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.295999    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f1c44fe4-da28-4517-a5ce-b47ea2592ba4-node-log\") pod \"ovnkube-node-z4h56\" (UID: \"f1c44fe4-da28-4517-a5ce-b47ea2592ba4\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4h56"
Sep 16 14:04:29.296391 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.296034    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f1c44fe4-da28-4517-a5ce-b47ea2592ba4-ovn-node-metrics-cert\") pod \"ovnkube-node-z4h56\" (UID: \"f1c44fe4-da28-4517-a5ce-b47ea2592ba4\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4h56"
Sep 16 14:04:29.296391 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.296055    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gsj8s\" (UniqueName: \"kubernetes.io/projected/f1c44fe4-da28-4517-a5ce-b47ea2592ba4-kube-api-access-gsj8s\") pod \"ovnkube-node-z4h56\" (UID: \"f1c44fe4-da28-4517-a5ce-b47ea2592ba4\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4h56"
Sep 16 14:04:29.296391 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.296072    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d89c07ec-f200-4033-9fe1-c8dab7ef1d48-hostroot\") pod \"multus-7xg75\" (UID: \"d89c07ec-f200-4033-9fe1-c8dab7ef1d48\") " pod="openshift-multus/multus-7xg75"
Sep 16 14:04:29.296391 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.296090    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9840e56c-3a33-49c0-a917-70f641aabf3b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-c5xjg\" (UID: \"9840e56c-3a33-49c0-a917-70f641aabf3b\") " pod="openshift-multus/multus-additional-cni-plugins-c5xjg"
Sep 16 14:04:29.296391 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.296104    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f1c44fe4-da28-4517-a5ce-b47ea2592ba4-host-cni-bin\") pod \"ovnkube-node-z4h56\" (UID: \"f1c44fe4-da28-4517-a5ce-b47ea2592ba4\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4h56"
Sep 16 14:04:29.296391 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.296119    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f1c44fe4-da28-4517-a5ce-b47ea2592ba4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-z4h56\" (UID: \"f1c44fe4-da28-4517-a5ce-b47ea2592ba4\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4h56"
Sep 16 14:04:29.296391 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.296138    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f1c44fe4-da28-4517-a5ce-b47ea2592ba4-env-overrides\") pod \"ovnkube-node-z4h56\" (UID: \"f1c44fe4-da28-4517-a5ce-b47ea2592ba4\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4h56"
Sep 16 14:04:29.296391 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.296168    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fa3e87c7-91e7-4767-b43d-6f00261f1d56-serviceca\") pod \"node-ca-r9k4l\" (UID: \"fa3e87c7-91e7-4767-b43d-6f00261f1d56\") " pod="openshift-image-registry/node-ca-r9k4l"
Sep 16 14:04:29.296391 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.296181    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mxzd\" (UniqueName: \"kubernetes.io/projected/fa3e87c7-91e7-4767-b43d-6f00261f1d56-kube-api-access-6mxzd\") pod \"node-ca-r9k4l\" (UID: \"fa3e87c7-91e7-4767-b43d-6f00261f1d56\") " pod="openshift-image-registry/node-ca-r9k4l"
Sep 16 14:04:29.296391 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.296197    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d89c07ec-f200-4033-9fe1-c8dab7ef1d48-host-run-netns\") pod \"multus-7xg75\" (UID: \"d89c07ec-f200-4033-9fe1-c8dab7ef1d48\") " pod="openshift-multus/multus-7xg75"
Sep 16 14:04:29.296391 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.296227    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f1c44fe4-da28-4517-a5ce-b47ea2592ba4-ovnkube-config\") pod \"ovnkube-node-z4h56\" (UID: \"f1c44fe4-da28-4517-a5ce-b47ea2592ba4\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4h56"
Sep 16 14:04:29.296391 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.296245    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f1c44fe4-da28-4517-a5ce-b47ea2592ba4-ovnkube-script-lib\") pod \"ovnkube-node-z4h56\" (UID: \"f1c44fe4-da28-4517-a5ce-b47ea2592ba4\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4h56"
Sep 16 14:04:29.296391 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.296261    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d89c07ec-f200-4033-9fe1-c8dab7ef1d48-host-var-lib-cni-multus\") pod \"multus-7xg75\" (UID: \"d89c07ec-f200-4033-9fe1-c8dab7ef1d48\") " pod="openshift-multus/multus-7xg75"
Sep 16 14:04:29.296862 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.296290    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkf9d\" (UniqueName: \"kubernetes.io/projected/d89c07ec-f200-4033-9fe1-c8dab7ef1d48-kube-api-access-kkf9d\") pod \"multus-7xg75\" (UID: \"d89c07ec-f200-4033-9fe1-c8dab7ef1d48\") " pod="openshift-multus/multus-7xg75"
Sep 16 14:04:29.296862 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.296309    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9840e56c-3a33-49c0-a917-70f641aabf3b-cnibin\") pod \"multus-additional-cni-plugins-c5xjg\" (UID: \"9840e56c-3a33-49c0-a917-70f641aabf3b\") " pod="openshift-multus/multus-additional-cni-plugins-c5xjg"
Sep 16 14:04:29.296862 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.296647    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 16 14:04:29.296862 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.296695    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-r62q2"
Sep 16 14:04:29.296862 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.296840    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-hfsls\""
Sep 16 14:04:29.297022 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.296978    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 16 14:04:29.299108 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.299090    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wg92s"
Sep 16 14:04:29.299192 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:29.299138    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wg92s" podUID="832cddc0-66b7-45a7-8cad-2a8609131f31"
Sep 16 14:04:29.299296 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.299280    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 16 14:04:29.299393 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.299380    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-d6ttt\""
Sep 16 14:04:29.300088 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.300074    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 16 14:04:29.301374 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.301359    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tgkvk"
Sep 16 14:04:29.301440 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:29.301410    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tgkvk" podUID="43276d8f-034a-4f4d-ad3d-2ca7f3ddfd38"
Sep 16 14:04:29.315559 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.315544    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 14:04:29.326050 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.326037    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-4mgcz"
Sep 16 14:04:29.334982 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.334964    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-4mgcz"
Sep 16 14:04:29.387385 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.387365    2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 16 14:04:29.396612 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.396587    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d89c07ec-f200-4033-9fe1-c8dab7ef1d48-multus-daemon-config\") pod \"multus-7xg75\" (UID: \"d89c07ec-f200-4033-9fe1-c8dab7ef1d48\") " pod="openshift-multus/multus-7xg75"
Sep 16 14:04:29.396702 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.396620    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/602525a8-0cf7-46fe-b669-1d38f6e15d0e-host-slash\") pod \"iptables-alerter-kwf42\" (UID: \"602525a8-0cf7-46fe-b669-1d38f6e15d0e\") " pod="openshift-network-operator/iptables-alerter-kwf42"
Sep 16 14:04:29.396702 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.396636    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e046d878-c8a6-4278-bde8-238f63fb96b2-hosts-file\") pod \"node-resolver-s544c\" (UID: \"e046d878-c8a6-4278-bde8-238f63fb96b2\") " pod="openshift-dns/node-resolver-s544c"
Sep 16 14:04:29.396702 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.396653    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f1c44fe4-da28-4517-a5ce-b47ea2592ba4-run-systemd\") pod \"ovnkube-node-z4h56\" (UID: \"f1c44fe4-da28-4517-a5ce-b47ea2592ba4\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4h56"
Sep 16 14:04:29.396821 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.396696    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d89c07ec-f200-4033-9fe1-c8dab7ef1d48-etc-kubernetes\") pod \"multus-7xg75\" (UID: \"d89c07ec-f200-4033-9fe1-c8dab7ef1d48\") " pod="openshift-multus/multus-7xg75"
Sep 16 14:04:29.396821 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.396723    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f1c44fe4-da28-4517-a5ce-b47ea2592ba4-run-systemd\") pod \"ovnkube-node-z4h56\" (UID: \"f1c44fe4-da28-4517-a5ce-b47ea2592ba4\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4h56"
Sep 16 14:04:29.396821 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.396726    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9840e56c-3a33-49c0-a917-70f641aabf3b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-c5xjg\" (UID: \"9840e56c-3a33-49c0-a917-70f641aabf3b\") " pod="openshift-multus/multus-additional-cni-plugins-c5xjg"
Sep 16 14:04:29.396821 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.396703    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e046d878-c8a6-4278-bde8-238f63fb96b2-hosts-file\") pod \"node-resolver-s544c\" (UID: \"e046d878-c8a6-4278-bde8-238f63fb96b2\") " pod="openshift-dns/node-resolver-s544c"
Sep 16 14:04:29.396821 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.396755    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d89c07ec-f200-4033-9fe1-c8dab7ef1d48-etc-kubernetes\") pod \"multus-7xg75\" (UID: \"d89c07ec-f200-4033-9fe1-c8dab7ef1d48\") " pod="openshift-multus/multus-7xg75"
Sep 16 14:04:29.396821 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.396749    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f1c44fe4-da28-4517-a5ce-b47ea2592ba4-host-run-netns\") pod \"ovnkube-node-z4h56\" (UID: \"f1c44fe4-da28-4517-a5ce-b47ea2592ba4\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4h56"
Sep 16 14:04:29.397027 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.396848    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f1c44fe4-da28-4517-a5ce-b47ea2592ba4-host-run-netns\") pod \"ovnkube-node-z4h56\" (UID: \"f1c44fe4-da28-4517-a5ce-b47ea2592ba4\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4h56"
Sep 16 14:04:29.397027 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.396863    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/602525a8-0cf7-46fe-b669-1d38f6e15d0e-iptables-alerter-script\") pod \"iptables-alerter-kwf42\" (UID: \"602525a8-0cf7-46fe-b669-1d38f6e15d0e\") " pod="openshift-network-operator/iptables-alerter-kwf42"
Sep 16 14:04:29.397027 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.396886    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rwtn\" (UniqueName: \"kubernetes.io/projected/602525a8-0cf7-46fe-b669-1d38f6e15d0e-kube-api-access-5rwtn\") pod \"iptables-alerter-kwf42\" (UID: \"602525a8-0cf7-46fe-b669-1d38f6e15d0e\") " pod="openshift-network-operator/iptables-alerter-kwf42"
Sep 16 14:04:29.397027 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.396901    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shk9d\" (UniqueName: \"kubernetes.io/projected/7c431872-6b49-42db-829b-4013ef367c3b-kube-api-access-shk9d\") pod \"aws-ebs-csi-driver-node-cbrr5\" (UID: \"7c431872-6b49-42db-829b-4013ef367c3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbrr5"
Sep 16 14:04:29.397027 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.396916    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/16dd3320-b25f-4459-89fa-3a4196939969-etc-sysctl-conf\") pod \"tuned-r62q2\" (UID: \"16dd3320-b25f-4459-89fa-3a4196939969\") " pod="openshift-cluster-node-tuning-operator/tuned-r62q2"
Sep 16 14:04:29.397027 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.396932    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/832cddc0-66b7-45a7-8cad-2a8609131f31-kubelet-config\") pod \"global-pull-secret-syncer-wg92s\" (UID: \"832cddc0-66b7-45a7-8cad-2a8609131f31\") " pod="kube-system/global-pull-secret-syncer-wg92s"
Sep 16 14:04:29.397027 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.396966    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f1c44fe4-da28-4517-a5ce-b47ea2592ba4-host-cni-bin\") pod \"ovnkube-node-z4h56\" (UID: \"f1c44fe4-da28-4517-a5ce-b47ea2592ba4\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4h56"
Sep 16 14:04:29.397027 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.396981    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mxzd\" (UniqueName: \"kubernetes.io/projected/fa3e87c7-91e7-4767-b43d-6f00261f1d56-kube-api-access-6mxzd\") pod \"node-ca-r9k4l\" (UID: \"fa3e87c7-91e7-4767-b43d-6f00261f1d56\") " pod="openshift-image-registry/node-ca-r9k4l"
Sep 16 14:04:29.397027 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.396997    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7c431872-6b49-42db-829b-4013ef367c3b-sys-fs\") pod \"aws-ebs-csi-driver-node-cbrr5\" (UID: \"7c431872-6b49-42db-829b-4013ef367c3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbrr5"
Sep 16 14:04:29.397027 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.397001    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f1c44fe4-da28-4517-a5ce-b47ea2592ba4-host-cni-bin\") pod \"ovnkube-node-z4h56\" (UID: \"f1c44fe4-da28-4517-a5ce-b47ea2592ba4\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4h56"
Sep 16 14:04:29.397027 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.397011    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/16dd3320-b25f-4459-89fa-3a4196939969-etc-tuned\") pod \"tuned-r62q2\" (UID: \"16dd3320-b25f-4459-89fa-3a4196939969\") " pod="openshift-cluster-node-tuning-operator/tuned-r62q2"
Sep 16 14:04:29.397479 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.397053    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d89c07ec-f200-4033-9fe1-c8dab7ef1d48-host-var-lib-cni-multus\") pod \"multus-7xg75\" (UID: \"d89c07ec-f200-4033-9fe1-c8dab7ef1d48\") " pod="openshift-multus/multus-7xg75"
Sep 16 14:04:29.397479 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.397080    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/16dd3320-b25f-4459-89fa-3a4196939969-etc-modprobe-d\") pod \"tuned-r62q2\" (UID: \"16dd3320-b25f-4459-89fa-3a4196939969\") " pod="openshift-cluster-node-tuning-operator/tuned-r62q2"
Sep 16 14:04:29.397479 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.397100    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d89c07ec-f200-4033-9fe1-c8dab7ef1d48-host-var-lib-cni-multus\") pod \"multus-7xg75\" (UID: \"d89c07ec-f200-4033-9fe1-c8dab7ef1d48\") " pod="openshift-multus/multus-7xg75"
Sep 16 14:04:29.397479 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.397103    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/16dd3320-b25f-4459-89fa-3a4196939969-run\") pod \"tuned-r62q2\" (UID: \"16dd3320-b25f-4459-89fa-3a4196939969\") " pod="openshift-cluster-node-tuning-operator/tuned-r62q2"
Sep 16 14:04:29.397479 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.397130    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/16dd3320-b25f-4459-89fa-3a4196939969-sys\") pod \"tuned-r62q2\" (UID: \"16dd3320-b25f-4459-89fa-3a4196939969\") " pod="openshift-cluster-node-tuning-operator/tuned-r62q2"
Sep 16 14:04:29.397479 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.397150    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9840e56c-3a33-49c0-a917-70f641aabf3b-cnibin\") pod \"multus-additional-cni-plugins-c5xjg\" (UID: \"9840e56c-3a33-49c0-a917-70f641aabf3b\") " pod="openshift-multus/multus-additional-cni-plugins-c5xjg"
Sep 16 14:04:29.397479 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.397165    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e046d878-c8a6-4278-bde8-238f63fb96b2-tmp-dir\") pod \"node-resolver-s544c\" (UID: \"e046d878-c8a6-4278-bde8-238f63fb96b2\") " pod="openshift-dns/node-resolver-s544c"
Sep 16 14:04:29.397479 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.397181    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f1c44fe4-da28-4517-a5ce-b47ea2592ba4-log-socket\") pod \"ovnkube-node-z4h56\" (UID: \"f1c44fe4-da28-4517-a5ce-b47ea2592ba4\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4h56"
Sep 16 14:04:29.397479 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.397202    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b586j\" (UniqueName: \"kubernetes.io/projected/9840e56c-3a33-49c0-a917-70f641aabf3b-kube-api-access-b586j\") pod \"multus-additional-cni-plugins-c5xjg\" (UID: \"9840e56c-3a33-49c0-a917-70f641aabf3b\") " pod="openshift-multus/multus-additional-cni-plugins-c5xjg"
Sep 16 14:04:29.397479 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.397232    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f1c44fe4-da28-4517-a5ce-b47ea2592ba4-log-socket\") pod \"ovnkube-node-z4h56\" (UID: \"f1c44fe4-da28-4517-a5ce-b47ea2592ba4\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4h56"
Sep 16 14:04:29.397479 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.397183    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9840e56c-3a33-49c0-a917-70f641aabf3b-cnibin\") pod \"multus-additional-cni-plugins-c5xjg\" (UID: \"9840e56c-3a33-49c0-a917-70f641aabf3b\") " pod="openshift-multus/multus-additional-cni-plugins-c5xjg"
Sep 16 14:04:29.397479 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.397255    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f1c44fe4-da28-4517-a5ce-b47ea2592ba4-host-slash\") pod \"ovnkube-node-z4h56\" (UID: \"f1c44fe4-da28-4517-a5ce-b47ea2592ba4\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4h56"
Sep 16 14:04:29.397479 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.397272    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d89c07ec-f200-4033-9fe1-c8dab7ef1d48-host-var-lib-cni-bin\") pod \"multus-7xg75\" (UID: \"d89c07ec-f200-4033-9fe1-c8dab7ef1d48\") " pod="openshift-multus/multus-7xg75"
Sep 16 14:04:29.397479 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.397288    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d89c07ec-f200-4033-9fe1-c8dab7ef1d48-multus-conf-dir\") pod \"multus-7xg75\" (UID: \"d89c07ec-f200-4033-9fe1-c8dab7ef1d48\") " pod="openshift-multus/multus-7xg75"
Sep 16 14:04:29.397479 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.397305    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ffa9d62b-620c-46d1-ab3a-73a2fdb84817-konnectivity-ca\") pod \"konnectivity-agent-cxcmv\" (UID: \"ffa9d62b-620c-46d1-ab3a-73a2fdb84817\") " pod="kube-system/konnectivity-agent-cxcmv"
Sep 16 14:04:29.397479 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.397320    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7c431872-6b49-42db-829b-4013ef367c3b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cbrr5\" (UID: \"7c431872-6b49-42db-829b-4013ef367c3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbrr5"
Sep 16 14:04:29.397479 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.397334    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7c431872-6b49-42db-829b-4013ef367c3b-device-dir\") pod \"aws-ebs-csi-driver-node-cbrr5\" (UID: \"7c431872-6b49-42db-829b-4013ef367c3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbrr5"
Sep 16 14:04:29.398200 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.397340    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d89c07ec-f200-4033-9fe1-c8dab7ef1d48-multus-conf-dir\") pod \"multus-7xg75\" (UID: \"d89c07ec-f200-4033-9fe1-c8dab7ef1d48\") " pod="openshift-multus/multus-7xg75"
Sep 16 14:04:29.398200 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.397345    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d89c07ec-f200-4033-9fe1-c8dab7ef1d48-host-var-lib-cni-bin\") pod \"multus-7xg75\" (UID: \"d89c07ec-f200-4033-9fe1-c8dab7ef1d48\") " pod="openshift-multus/multus-7xg75"
Sep 16 14:04:29.398200 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.397348    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f1c44fe4-da28-4517-a5ce-b47ea2592ba4-systemd-units\") pod \"ovnkube-node-z4h56\" (UID: \"f1c44fe4-da28-4517-a5ce-b47ea2592ba4\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4h56"
Sep 16 14:04:29.398200 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.397364    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f1c44fe4-da28-4517-a5ce-b47ea2592ba4-host-slash\") pod \"ovnkube-node-z4h56\" (UID: \"f1c44fe4-da28-4517-a5ce-b47ea2592ba4\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4h56"
Sep 16 14:04:29.398200 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.397370    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f1c44fe4-da28-4517-a5ce-b47ea2592ba4-systemd-units\") pod \"ovnkube-node-z4h56\" (UID: \"f1c44fe4-da28-4517-a5ce-b47ea2592ba4\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4h56"
Sep 16 14:04:29.398200 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.397379    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f1c44fe4-da28-4517-a5ce-b47ea2592ba4-var-lib-openvswitch\") pod \"ovnkube-node-z4h56\" (UID: \"f1c44fe4-da28-4517-a5ce-b47ea2592ba4\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4h56"
Sep 16 14:04:29.398200 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.397400    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f1c44fe4-da28-4517-a5ce-b47ea2592ba4-var-lib-openvswitch\") pod \"ovnkube-node-z4h56\" (UID: \"f1c44fe4-da28-4517-a5ce-b47ea2592ba4\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4h56"
Sep 16 14:04:29.398200 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.397408    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d89c07ec-f200-4033-9fe1-c8dab7ef1d48-system-cni-dir\") pod \"multus-7xg75\" (UID: \"d89c07ec-f200-4033-9fe1-c8dab7ef1d48\") " pod="openshift-multus/multus-7xg75"
Sep 16 14:04:29.398200 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.397426    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/16dd3320-b25f-4459-89fa-3a4196939969-etc-sysconfig\") pod \"tuned-r62q2\" (UID: \"16dd3320-b25f-4459-89fa-3a4196939969\") " pod="openshift-cluster-node-tuning-operator/tuned-r62q2"
Sep 16 14:04:29.398200 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.397442    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/832cddc0-66b7-45a7-8cad-2a8609131f31-original-pull-secret\") pod \"global-pull-secret-syncer-wg92s\" (UID: \"832cddc0-66b7-45a7-8cad-2a8609131f31\") " pod="kube-system/global-pull-secret-syncer-wg92s"
Sep 16 14:04:29.398200 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.397466    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f1c44fe4-da28-4517-a5ce-b47ea2592ba4-host-kubelet\") pod \"ovnkube-node-z4h56\" (UID: \"f1c44fe4-da28-4517-a5ce-b47ea2592ba4\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4h56"
Sep 16 14:04:29.398200 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.397466    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e046d878-c8a6-4278-bde8-238f63fb96b2-tmp-dir\") pod \"node-resolver-s544c\" (UID: \"e046d878-c8a6-4278-bde8-238f63fb96b2\") " pod="openshift-dns/node-resolver-s544c"
Sep 16 14:04:29.398200 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.397523    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f1c44fe4-da28-4517-a5ce-b47ea2592ba4-host-kubelet\") pod \"ovnkube-node-z4h56\" (UID: \"f1c44fe4-da28-4517-a5ce-b47ea2592ba4\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4h56"
Sep 16 14:04:29.398200 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.397546    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d89c07ec-f200-4033-9fe1-c8dab7ef1d48-host-run-multus-certs\") pod \"multus-7xg75\" (UID: \"d89c07ec-f200-4033-9fe1-c8dab7ef1d48\") " pod="openshift-multus/multus-7xg75"
Sep 16 14:04:29.398200 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.397554    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d89c07ec-f200-4033-9fe1-c8dab7ef1d48-system-cni-dir\") pod \"multus-7xg75\" (UID: \"d89c07ec-f200-4033-9fe1-c8dab7ef1d48\") " pod="openshift-multus/multus-7xg75"
Sep 16 14:04:29.398200 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.397572    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ww6q\" (UniqueName: \"kubernetes.io/projected/43276d8f-034a-4f4d-ad3d-2ca7f3ddfd38-kube-api-access-7ww6q\") pod \"network-check-target-tgkvk\" (UID: \"43276d8f-034a-4f4d-ad3d-2ca7f3ddfd38\") " pod="openshift-network-diagnostics/network-check-target-tgkvk"
Sep 16 14:04:29.398200 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.397583    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d89c07ec-f200-4033-9fe1-c8dab7ef1d48-host-run-multus-certs\") pod \"multus-7xg75\" (UID: \"d89c07ec-f200-4033-9fe1-c8dab7ef1d48\") " pod="openshift-multus/multus-7xg75"
Sep 16 14:04:29.399006 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.397596    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7c431872-6b49-42db-829b-4013ef367c3b-registration-dir\") pod \"aws-ebs-csi-driver-node-cbrr5\" (UID: \"7c431872-6b49-42db-829b-4013ef367c3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbrr5"
Sep 16 14:04:29.399006 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.397620    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/16dd3320-b25f-4459-89fa-3a4196939969-etc-kubernetes\") pod \"tuned-r62q2\" (UID: \"16dd3320-b25f-4459-89fa-3a4196939969\") " pod="openshift-cluster-node-tuning-operator/tuned-r62q2"
Sep 16 14:04:29.399006 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.397646    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/16dd3320-b25f-4459-89fa-3a4196939969-lib-modules\") pod \"tuned-r62q2\" (UID: \"16dd3320-b25f-4459-89fa-3a4196939969\") " pod="openshift-cluster-node-tuning-operator/tuned-r62q2"
Sep 16 14:04:29.399006 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.397672    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/16dd3320-b25f-4459-89fa-3a4196939969-var-lib-kubelet\") pod \"tuned-r62q2\" (UID: \"16dd3320-b25f-4459-89fa-3a4196939969\") " pod="openshift-cluster-node-tuning-operator/tuned-r62q2"
Sep 16 14:04:29.399006 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.397694    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/16dd3320-b25f-4459-89fa-3a4196939969-tmp\") pod \"tuned-r62q2\" (UID: \"16dd3320-b25f-4459-89fa-3a4196939969\") " pod="openshift-cluster-node-tuning-operator/tuned-r62q2"
Sep 16 14:04:29.399006 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.397722    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9840e56c-3a33-49c0-a917-70f641aabf3b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-c5xjg\" (UID: \"9840e56c-3a33-49c0-a917-70f641aabf3b\") " pod="openshift-multus/multus-additional-cni-plugins-c5xjg"
Sep 16 14:04:29.399006 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.397764    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njl4p\" (UniqueName: \"kubernetes.io/projected/e046d878-c8a6-4278-bde8-238f63fb96b2-kube-api-access-njl4p\") pod \"node-resolver-s544c\" (UID: \"e046d878-c8a6-4278-bde8-238f63fb96b2\") " pod="openshift-dns/node-resolver-s544c"
Sep 16 14:04:29.399006 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.397786    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0346ba8-7f08-4472-933c-7238392887b6-metrics-certs\") pod \"network-metrics-daemon-pxv9d\" (UID: \"f0346ba8-7f08-4472-933c-7238392887b6\") " pod="openshift-multus/network-metrics-daemon-pxv9d"
Sep 16 14:04:29.399006 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.397851    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d89c07ec-f200-4033-9fe1-c8dab7ef1d48-multus-socket-dir-parent\") pod \"multus-7xg75\" (UID: \"d89c07ec-f200-4033-9fe1-c8dab7ef1d48\") " pod="openshift-multus/multus-7xg75"
Sep 16 14:04:29.399006 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.397877    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9840e56c-3a33-49c0-a917-70f641aabf3b-system-cni-dir\") pod \"multus-additional-cni-plugins-c5xjg\" (UID: \"9840e56c-3a33-49c0-a917-70f641aabf3b\") " pod="openshift-multus/multus-additional-cni-plugins-c5xjg"
Sep 16 14:04:29.399006 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:29.397878    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 14:04:29.399006 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.397929    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d89c07ec-f200-4033-9fe1-c8dab7ef1d48-multus-socket-dir-parent\") pod \"multus-7xg75\" (UID: \"d89c07ec-f200-4033-9fe1-c8dab7ef1d48\") " pod="openshift-multus/multus-7xg75"
Sep 16 14:04:29.399006 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:29.397946    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0346ba8-7f08-4472-933c-7238392887b6-metrics-certs podName:f0346ba8-7f08-4472-933c-7238392887b6 nodeName:}" failed. No retries permitted until 2026-09-16 14:04:29.897926609 +0000 UTC m=+2.913036464 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f0346ba8-7f08-4472-933c-7238392887b6-metrics-certs") pod "network-metrics-daemon-pxv9d" (UID: "f0346ba8-7f08-4472-933c-7238392887b6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 14:04:29.399006 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.397949    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9840e56c-3a33-49c0-a917-70f641aabf3b-system-cni-dir\") pod \"multus-additional-cni-plugins-c5xjg\" (UID: \"9840e56c-3a33-49c0-a917-70f641aabf3b\") " pod="openshift-multus/multus-additional-cni-plugins-c5xjg"
Sep 16 14:04:29.399006 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.397963    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f1c44fe4-da28-4517-a5ce-b47ea2592ba4-node-log\") pod \"ovnkube-node-z4h56\" (UID: \"f1c44fe4-da28-4517-a5ce-b47ea2592ba4\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4h56"
Sep 16 14:04:29.399006 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.398168    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f1c44fe4-da28-4517-a5ce-b47ea2592ba4-ovn-node-metrics-cert\") pod \"ovnkube-node-z4h56\" (UID: \"f1c44fe4-da28-4517-a5ce-b47ea2592ba4\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4h56"
Sep 16 14:04:29.399665 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.398194    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsj8s\" (UniqueName: \"kubernetes.io/projected/f1c44fe4-da28-4517-a5ce-b47ea2592ba4-kube-api-access-gsj8s\") pod \"ovnkube-node-z4h56\" (UID: \"f1c44fe4-da28-4517-a5ce-b47ea2592ba4\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4h56"
Sep 16 14:04:29.399665 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.398002    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f1c44fe4-da28-4517-a5ce-b47ea2592ba4-node-log\") pod \"ovnkube-node-z4h56\" (UID: \"f1c44fe4-da28-4517-a5ce-b47ea2592ba4\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4h56"
Sep 16 14:04:29.399665 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.398042    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d89c07ec-f200-4033-9fe1-c8dab7ef1d48-multus-daemon-config\") pod \"multus-7xg75\" (UID: \"d89c07ec-f200-4033-9fe1-c8dab7ef1d48\") " pod="openshift-multus/multus-7xg75"
Sep 16 14:04:29.399665 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.398217    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d89c07ec-f200-4033-9fe1-c8dab7ef1d48-hostroot\") pod \"multus-7xg75\" (UID: \"d89c07ec-f200-4033-9fe1-c8dab7ef1d48\") " pod="openshift-multus/multus-7xg75"
Sep 16 14:04:29.399665 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.398264    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/16dd3320-b25f-4459-89fa-3a4196939969-etc-systemd\") pod \"tuned-r62q2\" (UID: \"16dd3320-b25f-4459-89fa-3a4196939969\") " pod="openshift-cluster-node-tuning-operator/tuned-r62q2"
Sep 16 14:04:29.399665 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.398295    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9840e56c-3a33-49c0-a917-70f641aabf3b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-c5xjg\" (UID: \"9840e56c-3a33-49c0-a917-70f641aabf3b\") " pod="openshift-multus/multus-additional-cni-plugins-c5xjg"
Sep 16 14:04:29.399665 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.398325    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f1c44fe4-da28-4517-a5ce-b47ea2592ba4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-z4h56\" (UID: \"f1c44fe4-da28-4517-a5ce-b47ea2592ba4\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4h56"
Sep 16 14:04:29.399665 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.398351    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f1c44fe4-da28-4517-a5ce-b47ea2592ba4-env-overrides\") pod \"ovnkube-node-z4h56\" (UID: \"f1c44fe4-da28-4517-a5ce-b47ea2592ba4\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4h56"
Sep 16 14:04:29.399665 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.398358    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d89c07ec-f200-4033-9fe1-c8dab7ef1d48-hostroot\") pod \"multus-7xg75\" (UID: \"d89c07ec-f200-4033-9fe1-c8dab7ef1d48\") " pod="openshift-multus/multus-7xg75"
Sep 16 14:04:29.399665 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.398374    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fa3e87c7-91e7-4767-b43d-6f00261f1d56-serviceca\") pod \"node-ca-r9k4l\" (UID: \"fa3e87c7-91e7-4767-b43d-6f00261f1d56\") " pod="openshift-image-registry/node-ca-r9k4l"
Sep 16 14:04:29.399665 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.398396    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d89c07ec-f200-4033-9fe1-c8dab7ef1d48-host-run-netns\") pod \"multus-7xg75\" (UID: \"d89c07ec-f200-4033-9fe1-c8dab7ef1d48\") " pod="openshift-multus/multus-7xg75"
Sep 16 14:04:29.399665 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.398410    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9840e56c-3a33-49c0-a917-70f641aabf3b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-c5xjg\" (UID: \"9840e56c-3a33-49c0-a917-70f641aabf3b\") " pod="openshift-multus/multus-additional-cni-plugins-c5xjg"
Sep 16 14:04:29.399665 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.398422    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/832cddc0-66b7-45a7-8cad-2a8609131f31-dbus\") pod \"global-pull-secret-syncer-wg92s\" (UID: \"832cddc0-66b7-45a7-8cad-2a8609131f31\") " pod="kube-system/global-pull-secret-syncer-wg92s"
Sep 16 14:04:29.399665 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.398454    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f1c44fe4-da28-4517-a5ce-b47ea2592ba4-ovnkube-config\") pod \"ovnkube-node-z4h56\" (UID: \"f1c44fe4-da28-4517-a5ce-b47ea2592ba4\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4h56"
Sep 16 14:04:29.399665 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.398465    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9840e56c-3a33-49c0-a917-70f641aabf3b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-c5xjg\" (UID: \"9840e56c-3a33-49c0-a917-70f641aabf3b\") " pod="openshift-multus/multus-additional-cni-plugins-c5xjg"
Sep 16 14:04:29.399665 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.398478    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d89c07ec-f200-4033-9fe1-c8dab7ef1d48-host-run-netns\") pod \"multus-7xg75\" (UID: \"d89c07ec-f200-4033-9fe1-c8dab7ef1d48\") " pod="openshift-multus/multus-7xg75"
Sep 16 14:04:29.399665 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.398475    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9840e56c-3a33-49c0-a917-70f641aabf3b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-c5xjg\" (UID: \"9840e56c-3a33-49c0-a917-70f641aabf3b\") " pod="openshift-multus/multus-additional-cni-plugins-c5xjg"
Sep 16 14:04:29.400162 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.398479    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f1c44fe4-da28-4517-a5ce-b47ea2592ba4-ovnkube-script-lib\") pod \"ovnkube-node-z4h56\" (UID: \"f1c44fe4-da28-4517-a5ce-b47ea2592ba4\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4h56"
Sep 16 14:04:29.400162 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.398538    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkf9d\" (UniqueName: \"kubernetes.io/projected/d89c07ec-f200-4033-9fe1-c8dab7ef1d48-kube-api-access-kkf9d\") pod \"multus-7xg75\" (UID: \"d89c07ec-f200-4033-9fe1-c8dab7ef1d48\") " pod="openshift-multus/multus-7xg75"
Sep 16 14:04:29.400162 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.398542    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f1c44fe4-da28-4517-a5ce-b47ea2592ba4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-z4h56\" (UID: \"f1c44fe4-da28-4517-a5ce-b47ea2592ba4\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4h56"
Sep 16 14:04:29.400162 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.398562    2580 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 16 14:04:29.400162 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.398584    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fa3e87c7-91e7-4767-b43d-6f00261f1d56-host\") pod \"node-ca-r9k4l\" (UID: \"fa3e87c7-91e7-4767-b43d-6f00261f1d56\") " pod="openshift-image-registry/node-ca-r9k4l"
Sep 16 14:04:29.400162 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.398823    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d89c07ec-f200-4033-9fe1-c8dab7ef1d48-host-run-k8s-cni-cncf-io\") pod \"multus-7xg75\" (UID: \"d89c07ec-f200-4033-9fe1-c8dab7ef1d48\") " pod="openshift-multus/multus-7xg75"
Sep 16 14:04:29.400162 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.398893    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fa3e87c7-91e7-4767-b43d-6f00261f1d56-serviceca\") pod \"node-ca-r9k4l\" (UID: \"fa3e87c7-91e7-4767-b43d-6f00261f1d56\") " pod="openshift-image-registry/node-ca-r9k4l"
Sep 16 14:04:29.400162 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.398902    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7c431872-6b49-42db-829b-4013ef367c3b-socket-dir\") pod \"aws-ebs-csi-driver-node-cbrr5\" (UID: \"7c431872-6b49-42db-829b-4013ef367c3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbrr5"
Sep 16 14:04:29.400162 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.398947    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7c431872-6b49-42db-829b-4013ef367c3b-etc-selinux\") pod \"aws-ebs-csi-driver-node-cbrr5\" (UID: \"7c431872-6b49-42db-829b-4013ef367c3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbrr5"
Sep 16 14:04:29.400162 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.398948    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d89c07ec-f200-4033-9fe1-c8dab7ef1d48-host-run-k8s-cni-cncf-io\") pod \"multus-7xg75\" (UID: \"d89c07ec-f200-4033-9fe1-c8dab7ef1d48\") " pod="openshift-multus/multus-7xg75"
Sep 16 14:04:29.400162 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.398972    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/16dd3320-b25f-4459-89fa-3a4196939969-etc-sysctl-d\") pod \"tuned-r62q2\" (UID: \"16dd3320-b25f-4459-89fa-3a4196939969\") " pod="openshift-cluster-node-tuning-operator/tuned-r62q2"
Sep 16 14:04:29.400162 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.398979    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f1c44fe4-da28-4517-a5ce-b47ea2592ba4-env-overrides\") pod \"ovnkube-node-z4h56\" (UID: \"f1c44fe4-da28-4517-a5ce-b47ea2592ba4\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4h56"
Sep 16 14:04:29.400162 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.398997    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzzq9\" (UniqueName: \"kubernetes.io/projected/16dd3320-b25f-4459-89fa-3a4196939969-kube-api-access-bzzq9\") pod \"tuned-r62q2\" (UID: \"16dd3320-b25f-4459-89fa-3a4196939969\") " pod="openshift-cluster-node-tuning-operator/tuned-r62q2"
Sep 16 14:04:29.400162 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.399026    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fa3e87c7-91e7-4767-b43d-6f00261f1d56-host\") pod \"node-ca-r9k4l\" (UID: \"fa3e87c7-91e7-4767-b43d-6f00261f1d56\") " pod="openshift-image-registry/node-ca-r9k4l"
Sep 16 14:04:29.400162 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.399030    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d89c07ec-f200-4033-9fe1-c8dab7ef1d48-cnibin\") pod \"multus-7xg75\" (UID: \"d89c07ec-f200-4033-9fe1-c8dab7ef1d48\") " pod="openshift-multus/multus-7xg75"
Sep 16 14:04:29.400162 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.399062    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d89c07ec-f200-4033-9fe1-c8dab7ef1d48-cni-binary-copy\") pod \"multus-7xg75\" (UID: \"d89c07ec-f200-4033-9fe1-c8dab7ef1d48\") " pod="openshift-multus/multus-7xg75"
Sep 16 14:04:29.400162 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.399093    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f1c44fe4-da28-4517-a5ce-b47ea2592ba4-ovnkube-config\") pod \"ovnkube-node-z4h56\" (UID: \"f1c44fe4-da28-4517-a5ce-b47ea2592ba4\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4h56"
Sep 16 14:04:29.400162 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.399110    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d89c07ec-f200-4033-9fe1-c8dab7ef1d48-cnibin\") pod \"multus-7xg75\" (UID: \"d89c07ec-f200-4033-9fe1-c8dab7ef1d48\") " pod="openshift-multus/multus-7xg75"
Sep 16 14:04:29.400733 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.399139    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f1c44fe4-da28-4517-a5ce-b47ea2592ba4-etc-openvswitch\") pod \"ovnkube-node-z4h56\" (UID: \"f1c44fe4-da28-4517-a5ce-b47ea2592ba4\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4h56"
Sep 16 14:04:29.400733 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.399138    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f1c44fe4-da28-4517-a5ce-b47ea2592ba4-ovnkube-script-lib\") pod \"ovnkube-node-z4h56\" (UID: \"f1c44fe4-da28-4517-a5ce-b47ea2592ba4\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4h56"
Sep 16 14:04:29.400733 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.399238    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f1c44fe4-da28-4517-a5ce-b47ea2592ba4-run-openvswitch\") pod \"ovnkube-node-z4h56\" (UID: \"f1c44fe4-da28-4517-a5ce-b47ea2592ba4\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4h56"
Sep 16 14:04:29.400733 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.399261    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f1c44fe4-da28-4517-a5ce-b47ea2592ba4-etc-openvswitch\") pod \"ovnkube-node-z4h56\" (UID: \"f1c44fe4-da28-4517-a5ce-b47ea2592ba4\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4h56"
Sep 16 14:04:29.400733 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.399320    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f1c44fe4-da28-4517-a5ce-b47ea2592ba4-run-openvswitch\") pod \"ovnkube-node-z4h56\" (UID: \"f1c44fe4-da28-4517-a5ce-b47ea2592ba4\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4h56"
Sep 16 14:04:29.400733 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.399305    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f1c44fe4-da28-4517-a5ce-b47ea2592ba4-run-ovn\") pod \"ovnkube-node-z4h56\" (UID: \"f1c44fe4-da28-4517-a5ce-b47ea2592ba4\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4h56"
Sep 16 14:04:29.400733 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.399361    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f1c44fe4-da28-4517-a5ce-b47ea2592ba4-run-ovn\") pod \"ovnkube-node-z4h56\" (UID: \"f1c44fe4-da28-4517-a5ce-b47ea2592ba4\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4h56"
Sep 16 14:04:29.400733 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.399410    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47qh2\" (UniqueName: \"kubernetes.io/projected/f0346ba8-7f08-4472-933c-7238392887b6-kube-api-access-47qh2\") pod \"network-metrics-daemon-pxv9d\" (UID: \"f0346ba8-7f08-4472-933c-7238392887b6\") " pod="openshift-multus/network-metrics-daemon-pxv9d"
Sep 16 14:04:29.400733 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.399441    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d89c07ec-f200-4033-9fe1-c8dab7ef1d48-multus-cni-dir\") pod \"multus-7xg75\" (UID: \"d89c07ec-f200-4033-9fe1-c8dab7ef1d48\") " pod="openshift-multus/multus-7xg75"
Sep 16 14:04:29.400733 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.399474    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ffa9d62b-620c-46d1-ab3a-73a2fdb84817-agent-certs\") pod \"konnectivity-agent-cxcmv\" (UID: \"ffa9d62b-620c-46d1-ab3a-73a2fdb84817\") " pod="kube-system/konnectivity-agent-cxcmv"
Sep 16 14:04:29.400733 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.399512    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/16dd3320-b25f-4459-89fa-3a4196939969-host\") pod \"tuned-r62q2\" (UID: \"16dd3320-b25f-4459-89fa-3a4196939969\") " pod="openshift-cluster-node-tuning-operator/tuned-r62q2"
Sep 16 14:04:29.400733 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.399521    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d89c07ec-f200-4033-9fe1-c8dab7ef1d48-cni-binary-copy\") pod \"multus-7xg75\" (UID: \"d89c07ec-f200-4033-9fe1-c8dab7ef1d48\") " pod="openshift-multus/multus-7xg75"
Sep 16 14:04:29.400733 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.399579    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d89c07ec-f200-4033-9fe1-c8dab7ef1d48-multus-cni-dir\") pod \"multus-7xg75\" (UID: \"d89c07ec-f200-4033-9fe1-c8dab7ef1d48\") " pod="openshift-multus/multus-7xg75"
Sep 16 14:04:29.400733 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.399621    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9840e56c-3a33-49c0-a917-70f641aabf3b-cni-binary-copy\") pod \"multus-additional-cni-plugins-c5xjg\" (UID: \"9840e56c-3a33-49c0-a917-70f641aabf3b\") " pod="openshift-multus/multus-additional-cni-plugins-c5xjg"
Sep 16 14:04:29.400733 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.399642    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f1c44fe4-da28-4517-a5ce-b47ea2592ba4-host-run-ovn-kubernetes\") pod \"ovnkube-node-z4h56\" (UID: \"f1c44fe4-da28-4517-a5ce-b47ea2592ba4\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4h56"
Sep 16 14:04:29.400733 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.399657    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f1c44fe4-da28-4517-a5ce-b47ea2592ba4-host-cni-netd\") pod \"ovnkube-node-z4h56\" (UID: \"f1c44fe4-da28-4517-a5ce-b47ea2592ba4\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4h56"
Sep 16 14:04:29.400733 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.399691    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d89c07ec-f200-4033-9fe1-c8dab7ef1d48-host-var-lib-kubelet\") pod \"multus-7xg75\" (UID: \"d89c07ec-f200-4033-9fe1-c8dab7ef1d48\") " pod="openshift-multus/multus-7xg75"
Sep 16 14:04:29.401237 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.399725    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f1c44fe4-da28-4517-a5ce-b47ea2592ba4-host-run-ovn-kubernetes\") pod \"ovnkube-node-z4h56\" (UID: \"f1c44fe4-da28-4517-a5ce-b47ea2592ba4\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4h56"
Sep 16 14:04:29.401237 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.399767    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d89c07ec-f200-4033-9fe1-c8dab7ef1d48-host-var-lib-kubelet\") pod \"multus-7xg75\" (UID: \"d89c07ec-f200-4033-9fe1-c8dab7ef1d48\") " pod="openshift-multus/multus-7xg75"
Sep 16 14:04:29.401237 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.399738    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f1c44fe4-da28-4517-a5ce-b47ea2592ba4-host-cni-netd\") pod \"ovnkube-node-z4h56\" (UID: \"f1c44fe4-da28-4517-a5ce-b47ea2592ba4\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4h56"
Sep 16 14:04:29.401237 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.399989    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9840e56c-3a33-49c0-a917-70f641aabf3b-cni-binary-copy\") pod \"multus-additional-cni-plugins-c5xjg\" (UID: \"9840e56c-3a33-49c0-a917-70f641aabf3b\") " pod="openshift-multus/multus-additional-cni-plugins-c5xjg"
Sep 16 14:04:29.401943 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.401926    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f1c44fe4-da28-4517-a5ce-b47ea2592ba4-ovn-node-metrics-cert\") pod \"ovnkube-node-z4h56\" (UID: \"f1c44fe4-da28-4517-a5ce-b47ea2592ba4\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4h56"
Sep 16 14:04:29.408660 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.408046    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b586j\" (UniqueName: \"kubernetes.io/projected/9840e56c-3a33-49c0-a917-70f641aabf3b-kube-api-access-b586j\") pod \"multus-additional-cni-plugins-c5xjg\" (UID: \"9840e56c-3a33-49c0-a917-70f641aabf3b\") " pod="openshift-multus/multus-additional-cni-plugins-c5xjg"
Sep 16 14:04:29.409067 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.409043    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-njl4p\" (UniqueName: \"kubernetes.io/projected/e046d878-c8a6-4278-bde8-238f63fb96b2-kube-api-access-njl4p\") pod \"node-resolver-s544c\" (UID: \"e046d878-c8a6-4278-bde8-238f63fb96b2\") " pod="openshift-dns/node-resolver-s544c"
Sep 16 14:04:29.409149 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.409136    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mxzd\" (UniqueName: \"kubernetes.io/projected/fa3e87c7-91e7-4767-b43d-6f00261f1d56-kube-api-access-6mxzd\") pod \"node-ca-r9k4l\" (UID: \"fa3e87c7-91e7-4767-b43d-6f00261f1d56\") " pod="openshift-image-registry/node-ca-r9k4l"
Sep 16 14:04:29.409620 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.409603    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkf9d\" (UniqueName: \"kubernetes.io/projected/d89c07ec-f200-4033-9fe1-c8dab7ef1d48-kube-api-access-kkf9d\") pod \"multus-7xg75\" (UID: \"d89c07ec-f200-4033-9fe1-c8dab7ef1d48\") " pod="openshift-multus/multus-7xg75"
Sep 16 14:04:29.409983 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.409964    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gsj8s\" (UniqueName: \"kubernetes.io/projected/f1c44fe4-da28-4517-a5ce-b47ea2592ba4-kube-api-access-gsj8s\") pod \"ovnkube-node-z4h56\" (UID: \"f1c44fe4-da28-4517-a5ce-b47ea2592ba4\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4h56"
Sep 16 14:04:29.410226 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.410206    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-47qh2\" (UniqueName: \"kubernetes.io/projected/f0346ba8-7f08-4472-933c-7238392887b6-kube-api-access-47qh2\") pod \"network-metrics-daemon-pxv9d\" (UID: \"f0346ba8-7f08-4472-933c-7238392887b6\") " pod="openshift-multus/network-metrics-daemon-pxv9d"
Sep 16 14:04:29.500777 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.500736    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7c431872-6b49-42db-829b-4013ef367c3b-device-dir\") pod \"aws-ebs-csi-driver-node-cbrr5\" (UID: \"7c431872-6b49-42db-829b-4013ef367c3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbrr5"
Sep 16 14:04:29.500777 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.500777    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/16dd3320-b25f-4459-89fa-3a4196939969-etc-sysconfig\") pod \"tuned-r62q2\" (UID: \"16dd3320-b25f-4459-89fa-3a4196939969\") " pod="openshift-cluster-node-tuning-operator/tuned-r62q2"
Sep 16 14:04:29.500777 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.500793    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/832cddc0-66b7-45a7-8cad-2a8609131f31-original-pull-secret\") pod \"global-pull-secret-syncer-wg92s\" (UID: \"832cddc0-66b7-45a7-8cad-2a8609131f31\") " pod="kube-system/global-pull-secret-syncer-wg92s"
Sep 16 14:04:29.501055 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.500810    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ww6q\" (UniqueName: \"kubernetes.io/projected/43276d8f-034a-4f4d-ad3d-2ca7f3ddfd38-kube-api-access-7ww6q\") pod \"network-check-target-tgkvk\" (UID: \"43276d8f-034a-4f4d-ad3d-2ca7f3ddfd38\") " pod="openshift-network-diagnostics/network-check-target-tgkvk"
Sep 16 14:04:29.501055 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.500832    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7c431872-6b49-42db-829b-4013ef367c3b-registration-dir\") pod \"aws-ebs-csi-driver-node-cbrr5\" (UID: \"7c431872-6b49-42db-829b-4013ef367c3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbrr5"
Sep 16 14:04:29.501055 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.500853    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/16dd3320-b25f-4459-89fa-3a4196939969-etc-kubernetes\") pod \"tuned-r62q2\" (UID: \"16dd3320-b25f-4459-89fa-3a4196939969\") " pod="openshift-cluster-node-tuning-operator/tuned-r62q2"
Sep 16 14:04:29.501055 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.500862    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/16dd3320-b25f-4459-89fa-3a4196939969-etc-sysconfig\") pod \"tuned-r62q2\" (UID: \"16dd3320-b25f-4459-89fa-3a4196939969\") " pod="openshift-cluster-node-tuning-operator/tuned-r62q2"
Sep 16 14:04:29.501055 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.500862    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7c431872-6b49-42db-829b-4013ef367c3b-device-dir\") pod \"aws-ebs-csi-driver-node-cbrr5\" (UID: \"7c431872-6b49-42db-829b-4013ef367c3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbrr5"
Sep 16 14:04:29.501055 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.500888    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/16dd3320-b25f-4459-89fa-3a4196939969-lib-modules\") pod \"tuned-r62q2\" (UID: \"16dd3320-b25f-4459-89fa-3a4196939969\") " pod="openshift-cluster-node-tuning-operator/tuned-r62q2"
Sep 16 14:04:29.501055 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.500905    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/16dd3320-b25f-4459-89fa-3a4196939969-etc-kubernetes\") pod \"tuned-r62q2\" (UID: \"16dd3320-b25f-4459-89fa-3a4196939969\") " pod="openshift-cluster-node-tuning-operator/tuned-r62q2"
Sep 16 14:04:29.501055 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.500917    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/16dd3320-b25f-4459-89fa-3a4196939969-var-lib-kubelet\") pod \"tuned-r62q2\" (UID: \"16dd3320-b25f-4459-89fa-3a4196939969\") " pod="openshift-cluster-node-tuning-operator/tuned-r62q2"
Sep 16 14:04:29.501055 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.500915    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7c431872-6b49-42db-829b-4013ef367c3b-registration-dir\") pod \"aws-ebs-csi-driver-node-cbrr5\" (UID: \"7c431872-6b49-42db-829b-4013ef367c3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbrr5"
Sep 16 14:04:29.501055 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.500940    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/16dd3320-b25f-4459-89fa-3a4196939969-tmp\") pod \"tuned-r62q2\" (UID: \"16dd3320-b25f-4459-89fa-3a4196939969\") " pod="openshift-cluster-node-tuning-operator/tuned-r62q2"
Sep 16 14:04:29.501055 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.500979    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/16dd3320-b25f-4459-89fa-3a4196939969-var-lib-kubelet\") pod \"tuned-r62q2\" (UID: \"16dd3320-b25f-4459-89fa-3a4196939969\") " pod="openshift-cluster-node-tuning-operator/tuned-r62q2"
Sep 16 14:04:29.501055 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.500985    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/16dd3320-b25f-4459-89fa-3a4196939969-etc-systemd\") pod \"tuned-r62q2\" (UID: \"16dd3320-b25f-4459-89fa-3a4196939969\") " pod="openshift-cluster-node-tuning-operator/tuned-r62q2"
Sep 16 14:04:29.501055 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:29.500924    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 14:04:29.501055 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.501001    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/16dd3320-b25f-4459-89fa-3a4196939969-lib-modules\") pod \"tuned-r62q2\" (UID: \"16dd3320-b25f-4459-89fa-3a4196939969\") " pod="openshift-cluster-node-tuning-operator/tuned-r62q2"
Sep 16 14:04:29.501055 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.501013    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/832cddc0-66b7-45a7-8cad-2a8609131f31-dbus\") pod \"global-pull-secret-syncer-wg92s\" (UID: \"832cddc0-66b7-45a7-8cad-2a8609131f31\") " pod="kube-system/global-pull-secret-syncer-wg92s"
Sep 16 14:04:29.501055 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.501042    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7c431872-6b49-42db-829b-4013ef367c3b-socket-dir\") pod \"aws-ebs-csi-driver-node-cbrr5\" (UID: \"7c431872-6b49-42db-829b-4013ef367c3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbrr5"
Sep 16 14:04:29.501055 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.501049    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/16dd3320-b25f-4459-89fa-3a4196939969-etc-systemd\") pod \"tuned-r62q2\" (UID: \"16dd3320-b25f-4459-89fa-3a4196939969\") " pod="openshift-cluster-node-tuning-operator/tuned-r62q2"
Sep 16 14:04:29.501055 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.501063    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7c431872-6b49-42db-829b-4013ef367c3b-etc-selinux\") pod \"aws-ebs-csi-driver-node-cbrr5\" (UID: \"7c431872-6b49-42db-829b-4013ef367c3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbrr5"
Sep 16 14:04:29.501903 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:29.501085    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/832cddc0-66b7-45a7-8cad-2a8609131f31-original-pull-secret podName:832cddc0-66b7-45a7-8cad-2a8609131f31 nodeName:}" failed. No retries permitted until 2026-09-16 14:04:30.001066584 +0000 UTC m=+3.016176447 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/832cddc0-66b7-45a7-8cad-2a8609131f31-original-pull-secret") pod "global-pull-secret-syncer-wg92s" (UID: "832cddc0-66b7-45a7-8cad-2a8609131f31") : object "kube-system"/"original-pull-secret" not registered
Sep 16 14:04:29.501903 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.501096    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/832cddc0-66b7-45a7-8cad-2a8609131f31-dbus\") pod \"global-pull-secret-syncer-wg92s\" (UID: \"832cddc0-66b7-45a7-8cad-2a8609131f31\") " pod="kube-system/global-pull-secret-syncer-wg92s"
Sep 16 14:04:29.501903 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.501115    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7c431872-6b49-42db-829b-4013ef367c3b-etc-selinux\") pod \"aws-ebs-csi-driver-node-cbrr5\" (UID: \"7c431872-6b49-42db-829b-4013ef367c3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbrr5"
Sep 16 14:04:29.501903 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.501113    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/16dd3320-b25f-4459-89fa-3a4196939969-etc-sysctl-d\") pod \"tuned-r62q2\" (UID: \"16dd3320-b25f-4459-89fa-3a4196939969\") " pod="openshift-cluster-node-tuning-operator/tuned-r62q2"
Sep 16 14:04:29.501903 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.501146    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzzq9\" (UniqueName: \"kubernetes.io/projected/16dd3320-b25f-4459-89fa-3a4196939969-kube-api-access-bzzq9\") pod \"tuned-r62q2\" (UID: \"16dd3320-b25f-4459-89fa-3a4196939969\") " pod="openshift-cluster-node-tuning-operator/tuned-r62q2"
Sep 16 14:04:29.501903 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.501156    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7c431872-6b49-42db-829b-4013ef367c3b-socket-dir\") pod \"aws-ebs-csi-driver-node-cbrr5\" (UID: \"7c431872-6b49-42db-829b-4013ef367c3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbrr5"
Sep 16 14:04:29.501903 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.501179    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ffa9d62b-620c-46d1-ab3a-73a2fdb84817-agent-certs\") pod \"konnectivity-agent-cxcmv\" (UID: \"ffa9d62b-620c-46d1-ab3a-73a2fdb84817\") " pod="kube-system/konnectivity-agent-cxcmv"
Sep 16 14:04:29.501903 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.501202    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/16dd3320-b25f-4459-89fa-3a4196939969-host\") pod \"tuned-r62q2\" (UID: \"16dd3320-b25f-4459-89fa-3a4196939969\") " pod="openshift-cluster-node-tuning-operator/tuned-r62q2"
Sep 16 14:04:29.501903 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.501215    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/16dd3320-b25f-4459-89fa-3a4196939969-etc-sysctl-d\") pod \"tuned-r62q2\" (UID: \"16dd3320-b25f-4459-89fa-3a4196939969\") " pod="openshift-cluster-node-tuning-operator/tuned-r62q2"
Sep 16 14:04:29.501903 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.501230    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/602525a8-0cf7-46fe-b669-1d38f6e15d0e-host-slash\") pod \"iptables-alerter-kwf42\" (UID: \"602525a8-0cf7-46fe-b669-1d38f6e15d0e\") " pod="openshift-network-operator/iptables-alerter-kwf42"
Sep 16 14:04:29.501903 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.501261    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/602525a8-0cf7-46fe-b669-1d38f6e15d0e-iptables-alerter-script\") pod \"iptables-alerter-kwf42\" (UID: \"602525a8-0cf7-46fe-b669-1d38f6e15d0e\") " pod="openshift-network-operator/iptables-alerter-kwf42"
Sep 16 14:04:29.501903 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.501265    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/602525a8-0cf7-46fe-b669-1d38f6e15d0e-host-slash\") pod \"iptables-alerter-kwf42\" (UID: \"602525a8-0cf7-46fe-b669-1d38f6e15d0e\") " pod="openshift-network-operator/iptables-alerter-kwf42"
Sep 16 14:04:29.501903 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.501287    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rwtn\" (UniqueName: \"kubernetes.io/projected/602525a8-0cf7-46fe-b669-1d38f6e15d0e-kube-api-access-5rwtn\") pod \"iptables-alerter-kwf42\" (UID: \"602525a8-0cf7-46fe-b669-1d38f6e15d0e\") " pod="openshift-network-operator/iptables-alerter-kwf42"
Sep 16 14:04:29.501903 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.501292    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/16dd3320-b25f-4459-89fa-3a4196939969-host\") pod \"tuned-r62q2\" (UID: \"16dd3320-b25f-4459-89fa-3a4196939969\") " pod="openshift-cluster-node-tuning-operator/tuned-r62q2"
Sep 16 14:04:29.501903 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.501311    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shk9d\" (UniqueName: \"kubernetes.io/projected/7c431872-6b49-42db-829b-4013ef367c3b-kube-api-access-shk9d\") pod \"aws-ebs-csi-driver-node-cbrr5\" (UID: \"7c431872-6b49-42db-829b-4013ef367c3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbrr5"
Sep 16 14:04:29.501903 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.501365    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/16dd3320-b25f-4459-89fa-3a4196939969-etc-sysctl-conf\") pod \"tuned-r62q2\" (UID: \"16dd3320-b25f-4459-89fa-3a4196939969\") " pod="openshift-cluster-node-tuning-operator/tuned-r62q2"
Sep 16 14:04:29.502409 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.501400    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/832cddc0-66b7-45a7-8cad-2a8609131f31-kubelet-config\") pod \"global-pull-secret-syncer-wg92s\" (UID: \"832cddc0-66b7-45a7-8cad-2a8609131f31\") " pod="kube-system/global-pull-secret-syncer-wg92s"
Sep 16 14:04:29.502409 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.501442    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7c431872-6b49-42db-829b-4013ef367c3b-sys-fs\") pod \"aws-ebs-csi-driver-node-cbrr5\" (UID: \"7c431872-6b49-42db-829b-4013ef367c3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbrr5"
Sep 16 14:04:29.502409 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.501464    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/16dd3320-b25f-4459-89fa-3a4196939969-etc-tuned\") pod \"tuned-r62q2\" (UID: \"16dd3320-b25f-4459-89fa-3a4196939969\") " pod="openshift-cluster-node-tuning-operator/tuned-r62q2"
Sep 16 14:04:29.502409 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.501488    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/16dd3320-b25f-4459-89fa-3a4196939969-etc-modprobe-d\") pod \"tuned-r62q2\" (UID: \"16dd3320-b25f-4459-89fa-3a4196939969\") " pod="openshift-cluster-node-tuning-operator/tuned-r62q2"
Sep 16 14:04:29.502409 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.501543    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/16dd3320-b25f-4459-89fa-3a4196939969-run\") pod \"tuned-r62q2\" (UID: \"16dd3320-b25f-4459-89fa-3a4196939969\") " pod="openshift-cluster-node-tuning-operator/tuned-r62q2"
Sep 16 14:04:29.502409 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.501573    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/16dd3320-b25f-4459-89fa-3a4196939969-sys\") pod \"tuned-r62q2\" (UID: \"16dd3320-b25f-4459-89fa-3a4196939969\") " pod="openshift-cluster-node-tuning-operator/tuned-r62q2"
Sep 16 14:04:29.502409 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.501597    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/16dd3320-b25f-4459-89fa-3a4196939969-etc-modprobe-d\") pod \"tuned-r62q2\" (UID: \"16dd3320-b25f-4459-89fa-3a4196939969\") " pod="openshift-cluster-node-tuning-operator/tuned-r62q2"
Sep 16 14:04:29.502409 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.501510    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7c431872-6b49-42db-829b-4013ef367c3b-sys-fs\") pod \"aws-ebs-csi-driver-node-cbrr5\" (UID: \"7c431872-6b49-42db-829b-4013ef367c3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbrr5"
Sep 16 14:04:29.502409 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.501488    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/832cddc0-66b7-45a7-8cad-2a8609131f31-kubelet-config\") pod \"global-pull-secret-syncer-wg92s\" (UID: \"832cddc0-66b7-45a7-8cad-2a8609131f31\") " pod="kube-system/global-pull-secret-syncer-wg92s"
Sep 16 14:04:29.502409 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.501619    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/16dd3320-b25f-4459-89fa-3a4196939969-run\") pod \"tuned-r62q2\" (UID: \"16dd3320-b25f-4459-89fa-3a4196939969\") " pod="openshift-cluster-node-tuning-operator/tuned-r62q2"
Sep 16 14:04:29.502409 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.501668    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/16dd3320-b25f-4459-89fa-3a4196939969-sys\") pod \"tuned-r62q2\" (UID: \"16dd3320-b25f-4459-89fa-3a4196939969\") " pod="openshift-cluster-node-tuning-operator/tuned-r62q2"
Sep 16 14:04:29.502409 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.501678    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ffa9d62b-620c-46d1-ab3a-73a2fdb84817-konnectivity-ca\") pod \"konnectivity-agent-cxcmv\" (UID: \"ffa9d62b-620c-46d1-ab3a-73a2fdb84817\") " pod="kube-system/konnectivity-agent-cxcmv"
Sep 16 14:04:29.502409 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.501724    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7c431872-6b49-42db-829b-4013ef367c3b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cbrr5\" (UID: \"7c431872-6b49-42db-829b-4013ef367c3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbrr5"
Sep 16 14:04:29.502409 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.501773    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/16dd3320-b25f-4459-89fa-3a4196939969-etc-sysctl-conf\") pod \"tuned-r62q2\" (UID: \"16dd3320-b25f-4459-89fa-3a4196939969\") " pod="openshift-cluster-node-tuning-operator/tuned-r62q2"
Sep 16 14:04:29.502409 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.501817    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7c431872-6b49-42db-829b-4013ef367c3b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cbrr5\" (UID: \"7c431872-6b49-42db-829b-4013ef367c3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbrr5"
Sep 16 14:04:29.502409 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.501830    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/602525a8-0cf7-46fe-b669-1d38f6e15d0e-iptables-alerter-script\") pod \"iptables-alerter-kwf42\" (UID: \"602525a8-0cf7-46fe-b669-1d38f6e15d0e\") " pod="openshift-network-operator/iptables-alerter-kwf42"
Sep 16 14:04:29.502409 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.502243    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ffa9d62b-620c-46d1-ab3a-73a2fdb84817-konnectivity-ca\") pod \"konnectivity-agent-cxcmv\" (UID: \"ffa9d62b-620c-46d1-ab3a-73a2fdb84817\") " pod="kube-system/konnectivity-agent-cxcmv"
Sep 16 14:04:29.503286 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.503269    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/16dd3320-b25f-4459-89fa-3a4196939969-tmp\") pod \"tuned-r62q2\" (UID: \"16dd3320-b25f-4459-89fa-3a4196939969\") " pod="openshift-cluster-node-tuning-operator/tuned-r62q2"
Sep 16 14:04:29.503366 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.503353    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/16dd3320-b25f-4459-89fa-3a4196939969-etc-tuned\") pod \"tuned-r62q2\" (UID: \"16dd3320-b25f-4459-89fa-3a4196939969\") " pod="openshift-cluster-node-tuning-operator/tuned-r62q2"
Sep 16 14:04:29.503520 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.503486    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ffa9d62b-620c-46d1-ab3a-73a2fdb84817-agent-certs\") pod \"konnectivity-agent-cxcmv\" (UID: \"ffa9d62b-620c-46d1-ab3a-73a2fdb84817\") " pod="kube-system/konnectivity-agent-cxcmv"
Sep 16 14:04:29.506964 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:29.506941    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 14:04:29.506964 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:29.506965    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 14:04:29.507129 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:29.506980    2580 projected.go:196] Error preparing data for projected volume kube-api-access-7ww6q for pod openshift-network-diagnostics/network-check-target-tgkvk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 14:04:29.507129 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:29.507044    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/43276d8f-034a-4f4d-ad3d-2ca7f3ddfd38-kube-api-access-7ww6q podName:43276d8f-034a-4f4d-ad3d-2ca7f3ddfd38 nodeName:}" failed. No retries permitted until 2026-09-16 14:04:30.007026752 +0000 UTC m=+3.022136609 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-7ww6q" (UniqueName: "kubernetes.io/projected/43276d8f-034a-4f4d-ad3d-2ca7f3ddfd38-kube-api-access-7ww6q") pod "network-check-target-tgkvk" (UID: "43276d8f-034a-4f4d-ad3d-2ca7f3ddfd38") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 14:04:29.508280 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.508257    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzzq9\" (UniqueName: \"kubernetes.io/projected/16dd3320-b25f-4459-89fa-3a4196939969-kube-api-access-bzzq9\") pod \"tuned-r62q2\" (UID: \"16dd3320-b25f-4459-89fa-3a4196939969\") " pod="openshift-cluster-node-tuning-operator/tuned-r62q2"
Sep 16 14:04:29.508626 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.508601    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rwtn\" (UniqueName: \"kubernetes.io/projected/602525a8-0cf7-46fe-b669-1d38f6e15d0e-kube-api-access-5rwtn\") pod \"iptables-alerter-kwf42\" (UID: \"602525a8-0cf7-46fe-b669-1d38f6e15d0e\") " pod="openshift-network-operator/iptables-alerter-kwf42"
Sep 16 14:04:29.508704 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.508690    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-shk9d\" (UniqueName: \"kubernetes.io/projected/7c431872-6b49-42db-829b-4013ef367c3b-kube-api-access-shk9d\") pod \"aws-ebs-csi-driver-node-cbrr5\" (UID: \"7c431872-6b49-42db-829b-4013ef367c3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbrr5"
Sep 16 14:04:29.609120 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.609031    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-c5xjg"
Sep 16 14:04:29.625013 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.624994    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-s544c"
Sep 16 14:04:29.654561 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.654542    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-z4h56"
Sep 16 14:04:29.668108 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.668089    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-r9k4l"
Sep 16 14:04:29.687653 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.687637    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7xg75"
Sep 16 14:04:29.700179 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.700159    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbrr5"
Sep 16 14:04:29.710740 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.710714    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kwf42"
Sep 16 14:04:29.737054 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.737034    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-cxcmv"
Sep 16 14:04:29.741473 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.741459    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-r62q2"
Sep 16 14:04:29.905846 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:29.905791    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0346ba8-7f08-4472-933c-7238392887b6-metrics-certs\") pod \"network-metrics-daemon-pxv9d\" (UID: \"f0346ba8-7f08-4472-933c-7238392887b6\") " pod="openshift-multus/network-metrics-daemon-pxv9d"
Sep 16 14:04:29.905928 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:29.905875    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 14:04:29.905928 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:29.905916    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0346ba8-7f08-4472-933c-7238392887b6-metrics-certs podName:f0346ba8-7f08-4472-933c-7238392887b6 nodeName:}" failed. No retries permitted until 2026-09-16 14:04:30.905904945 +0000 UTC m=+3.921014782 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f0346ba8-7f08-4472-933c-7238392887b6-metrics-certs") pod "network-metrics-daemon-pxv9d" (UID: "f0346ba8-7f08-4472-933c-7238392887b6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 14:04:30.006646 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:30.006615    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/832cddc0-66b7-45a7-8cad-2a8609131f31-original-pull-secret\") pod \"global-pull-secret-syncer-wg92s\" (UID: \"832cddc0-66b7-45a7-8cad-2a8609131f31\") " pod="kube-system/global-pull-secret-syncer-wg92s"
Sep 16 14:04:30.006801 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:30.006761    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 14:04:30.006853 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:30.006824    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/832cddc0-66b7-45a7-8cad-2a8609131f31-original-pull-secret podName:832cddc0-66b7-45a7-8cad-2a8609131f31 nodeName:}" failed. No retries permitted until 2026-09-16 14:04:31.006810291 +0000 UTC m=+4.021920133 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/832cddc0-66b7-45a7-8cad-2a8609131f31-original-pull-secret") pod "global-pull-secret-syncer-wg92s" (UID: "832cddc0-66b7-45a7-8cad-2a8609131f31") : object "kube-system"/"original-pull-secret" not registered
Sep 16 14:04:30.107077 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:30.107045    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ww6q\" (UniqueName: \"kubernetes.io/projected/43276d8f-034a-4f4d-ad3d-2ca7f3ddfd38-kube-api-access-7ww6q\") pod \"network-check-target-tgkvk\" (UID: \"43276d8f-034a-4f4d-ad3d-2ca7f3ddfd38\") " pod="openshift-network-diagnostics/network-check-target-tgkvk"
Sep 16 14:04:30.107298 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:30.107222    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 14:04:30.107298 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:30.107241    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 14:04:30.107298 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:30.107250    2580 projected.go:196] Error preparing data for projected volume kube-api-access-7ww6q for pod openshift-network-diagnostics/network-check-target-tgkvk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 14:04:30.107407 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:30.107303    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/43276d8f-034a-4f4d-ad3d-2ca7f3ddfd38-kube-api-access-7ww6q podName:43276d8f-034a-4f4d-ad3d-2ca7f3ddfd38 nodeName:}" failed. No retries permitted until 2026-09-16 14:04:31.107288118 +0000 UTC m=+4.122397968 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-7ww6q" (UniqueName: "kubernetes.io/projected/43276d8f-034a-4f4d-ad3d-2ca7f3ddfd38-kube-api-access-7ww6q") pod "network-check-target-tgkvk" (UID: "43276d8f-034a-4f4d-ad3d-2ca7f3ddfd38") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 14:04:30.238545 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:30.238507    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16dd3320_b25f_4459_89fa_3a4196939969.slice/crio-13d83ba2e0f25e502cb21e82932cf7e96cb5c138062385c23dd92ead4c07a388 WatchSource:0}: Error finding container 13d83ba2e0f25e502cb21e82932cf7e96cb5c138062385c23dd92ead4c07a388: Status 404 returned error can't find the container with id 13d83ba2e0f25e502cb21e82932cf7e96cb5c138062385c23dd92ead4c07a388
Sep 16 14:04:30.245211 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:30.245193    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 14:04:30.251915 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:30.251718    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd89c07ec_f200_4033_9fe1_c8dab7ef1d48.slice/crio-9b61646c6faea9be345e709f57e65635a510b580f86a45b00f496904ba7252e4 WatchSource:0}: Error finding container 9b61646c6faea9be345e709f57e65635a510b580f86a45b00f496904ba7252e4: Status 404 returned error can't find the container with id 9b61646c6faea9be345e709f57e65635a510b580f86a45b00f496904ba7252e4
Sep 16 14:04:30.264670 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:30.264650    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c431872_6b49_42db_829b_4013ef367c3b.slice/crio-3b73b6642cac19c1cdfdf957500cdbe264807e27dec54ea0cc35ddc44b70706e WatchSource:0}: Error finding container 3b73b6642cac19c1cdfdf957500cdbe264807e27dec54ea0cc35ddc44b70706e: Status 404 returned error can't find the container with id 3b73b6642cac19c1cdfdf957500cdbe264807e27dec54ea0cc35ddc44b70706e
Sep 16 14:04:30.268684 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:30.268662    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38046ded5fcb5dec29d3f7bce0f86c40.slice/crio-cd4f84801fd4be46eec622d4afb28dd0cdb5cc0081ab8174588b388d9417e0ab WatchSource:0}: Error finding container cd4f84801fd4be46eec622d4afb28dd0cdb5cc0081ab8174588b388d9417e0ab: Status 404 returned error can't find the container with id cd4f84801fd4be46eec622d4afb28dd0cdb5cc0081ab8174588b388d9417e0ab
Sep 16 14:04:30.274153 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:30.274120    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9840e56c_3a33_49c0_a917_70f641aabf3b.slice/crio-cdfc808f3389196498c0fcd52c0804dd67faccdffd092f5cda397aaddbe8e97d WatchSource:0}: Error finding container cdfc808f3389196498c0fcd52c0804dd67faccdffd092f5cda397aaddbe8e97d: Status 404 returned error can't find the container with id cdfc808f3389196498c0fcd52c0804dd67faccdffd092f5cda397aaddbe8e97d
Sep 16 14:04:30.274740 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:30.274716    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode046d878_c8a6_4278_bde8_238f63fb96b2.slice/crio-9ef57b2eaa6ce384feb5c7897dbf13c24e57894a3f7d02e5b9dc852a953b80ca WatchSource:0}: Error finding container 9ef57b2eaa6ce384feb5c7897dbf13c24e57894a3f7d02e5b9dc852a953b80ca: Status 404 returned error can't find the container with id 9ef57b2eaa6ce384feb5c7897dbf13c24e57894a3f7d02e5b9dc852a953b80ca
Sep 16 14:04:30.278412 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:30.278392    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa3e87c7_91e7_4767_b43d_6f00261f1d56.slice/crio-f3169194a1d1e8e447ef2996580c47e8f38769f7955e9dfa5ba095e1360cd6dd WatchSource:0}: Error finding container f3169194a1d1e8e447ef2996580c47e8f38769f7955e9dfa5ba095e1360cd6dd: Status 404 returned error can't find the container with id f3169194a1d1e8e447ef2996580c47e8f38769f7955e9dfa5ba095e1360cd6dd
Sep 16 14:04:30.337642 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:30.337618    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-15 13:59:29 +0000 UTC" deadline="2028-02-28 02:29:16.732907697 +0000 UTC"
Sep 16 14:04:30.337642 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:30.337639    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12708h24m46.395271212s"
Sep 16 14:04:30.424227 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:30.424208    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pxv9d"
Sep 16 14:04:30.424321 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:30.424299    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pxv9d" podUID="f0346ba8-7f08-4472-933c-7238392887b6"
Sep 16 14:04:30.429241 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:30.429208    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7xg75" event={"ID":"d89c07ec-f200-4033-9fe1-c8dab7ef1d48","Type":"ContainerStarted","Data":"9b61646c6faea9be345e709f57e65635a510b580f86a45b00f496904ba7252e4"}
Sep 16 14:04:30.430304 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:30.430282    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-r62q2" event={"ID":"16dd3320-b25f-4459-89fa-3a4196939969","Type":"ContainerStarted","Data":"13d83ba2e0f25e502cb21e82932cf7e96cb5c138062385c23dd92ead4c07a388"}
Sep 16 14:04:30.431430 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:30.431413    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-r9k4l" event={"ID":"fa3e87c7-91e7-4767-b43d-6f00261f1d56","Type":"ContainerStarted","Data":"f3169194a1d1e8e447ef2996580c47e8f38769f7955e9dfa5ba095e1360cd6dd"}
Sep 16 14:04:30.432482 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:30.432465    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-s544c" event={"ID":"e046d878-c8a6-4278-bde8-238f63fb96b2","Type":"ContainerStarted","Data":"9ef57b2eaa6ce384feb5c7897dbf13c24e57894a3f7d02e5b9dc852a953b80ca"}
Sep 16 14:04:30.433541 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:30.433521    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c5xjg" event={"ID":"9840e56c-3a33-49c0-a917-70f641aabf3b","Type":"ContainerStarted","Data":"cdfc808f3389196498c0fcd52c0804dd67faccdffd092f5cda397aaddbe8e97d"}
Sep 16 14:04:30.434638 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:30.434614    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-95.ec2.internal" event={"ID":"38046ded5fcb5dec29d3f7bce0f86c40","Type":"ContainerStarted","Data":"cd4f84801fd4be46eec622d4afb28dd0cdb5cc0081ab8174588b388d9417e0ab"}
Sep 16 14:04:30.435550 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:30.435525    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbrr5" event={"ID":"7c431872-6b49-42db-829b-4013ef367c3b","Type":"ContainerStarted","Data":"3b73b6642cac19c1cdfdf957500cdbe264807e27dec54ea0cc35ddc44b70706e"}
Sep 16 14:04:30.574873 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:30.574837    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffa9d62b_620c_46d1_ab3a_73a2fdb84817.slice/crio-08db88c8dae439970078aa7d7ceef54b4e06a9f2b5689ae13f1079d02b187147 WatchSource:0}: Error finding container 08db88c8dae439970078aa7d7ceef54b4e06a9f2b5689ae13f1079d02b187147: Status 404 returned error can't find the container with id 08db88c8dae439970078aa7d7ceef54b4e06a9f2b5689ae13f1079d02b187147
Sep 16 14:04:30.914817 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:30.914739    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0346ba8-7f08-4472-933c-7238392887b6-metrics-certs\") pod \"network-metrics-daemon-pxv9d\" (UID: \"f0346ba8-7f08-4472-933c-7238392887b6\") " pod="openshift-multus/network-metrics-daemon-pxv9d"
Sep 16 14:04:30.914980 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:30.914921    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 14:04:30.915042 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:30.914989    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0346ba8-7f08-4472-933c-7238392887b6-metrics-certs podName:f0346ba8-7f08-4472-933c-7238392887b6 nodeName:}" failed. No retries permitted until 2026-09-16 14:04:32.914971072 +0000 UTC m=+5.930080926 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f0346ba8-7f08-4472-933c-7238392887b6-metrics-certs") pod "network-metrics-daemon-pxv9d" (UID: "f0346ba8-7f08-4472-933c-7238392887b6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 14:04:31.015124 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:31.015088    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/832cddc0-66b7-45a7-8cad-2a8609131f31-original-pull-secret\") pod \"global-pull-secret-syncer-wg92s\" (UID: \"832cddc0-66b7-45a7-8cad-2a8609131f31\") " pod="kube-system/global-pull-secret-syncer-wg92s"
Sep 16 14:04:31.015313 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:31.015290    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 14:04:31.015386 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:31.015373    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/832cddc0-66b7-45a7-8cad-2a8609131f31-original-pull-secret podName:832cddc0-66b7-45a7-8cad-2a8609131f31 nodeName:}" failed. No retries permitted until 2026-09-16 14:04:33.015353345 +0000 UTC m=+6.030463185 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/832cddc0-66b7-45a7-8cad-2a8609131f31-original-pull-secret") pod "global-pull-secret-syncer-wg92s" (UID: "832cddc0-66b7-45a7-8cad-2a8609131f31") : object "kube-system"/"original-pull-secret" not registered
Sep 16 14:04:31.116234 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:31.116186    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ww6q\" (UniqueName: \"kubernetes.io/projected/43276d8f-034a-4f4d-ad3d-2ca7f3ddfd38-kube-api-access-7ww6q\") pod \"network-check-target-tgkvk\" (UID: \"43276d8f-034a-4f4d-ad3d-2ca7f3ddfd38\") " pod="openshift-network-diagnostics/network-check-target-tgkvk"
Sep 16 14:04:31.116407 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:31.116386    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 14:04:31.116407 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:31.116407    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 14:04:31.116534 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:31.116422    2580 projected.go:196] Error preparing data for projected volume kube-api-access-7ww6q for pod openshift-network-diagnostics/network-check-target-tgkvk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 14:04:31.116534 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:31.116479    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/43276d8f-034a-4f4d-ad3d-2ca7f3ddfd38-kube-api-access-7ww6q podName:43276d8f-034a-4f4d-ad3d-2ca7f3ddfd38 nodeName:}" failed. No retries permitted until 2026-09-16 14:04:33.11646167 +0000 UTC m=+6.131571525 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-7ww6q" (UniqueName: "kubernetes.io/projected/43276d8f-034a-4f4d-ad3d-2ca7f3ddfd38-kube-api-access-7ww6q") pod "network-check-target-tgkvk" (UID: "43276d8f-034a-4f4d-ad3d-2ca7f3ddfd38") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 14:04:31.338410 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:31.338335    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-15 13:59:29 +0000 UTC" deadline="2028-06-25 17:05:07.09112368 +0000 UTC"
Sep 16 14:04:31.338410 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:31.338376    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15555h0m35.752752792s"
Sep 16 14:04:31.426236 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:31.426198    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wg92s"
Sep 16 14:04:31.426402 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:31.426323    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wg92s" podUID="832cddc0-66b7-45a7-8cad-2a8609131f31"
Sep 16 14:04:31.426735 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:31.426712    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tgkvk"
Sep 16 14:04:31.426840 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:31.426809    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tgkvk" podUID="43276d8f-034a-4f4d-ad3d-2ca7f3ddfd38"
Sep 16 14:04:31.439431 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:31.439403    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-cxcmv" event={"ID":"ffa9d62b-620c-46d1-ab3a-73a2fdb84817","Type":"ContainerStarted","Data":"08db88c8dae439970078aa7d7ceef54b4e06a9f2b5689ae13f1079d02b187147"}
Sep 16 14:04:32.424729 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:32.424695    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pxv9d"
Sep 16 14:04:32.425128 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:32.424836    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pxv9d" podUID="f0346ba8-7f08-4472-933c-7238392887b6"
Sep 16 14:04:32.939726 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:32.939624    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0346ba8-7f08-4472-933c-7238392887b6-metrics-certs\") pod \"network-metrics-daemon-pxv9d\" (UID: \"f0346ba8-7f08-4472-933c-7238392887b6\") " pod="openshift-multus/network-metrics-daemon-pxv9d"
Sep 16 14:04:32.939904 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:32.939782    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 14:04:32.939904 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:32.939853    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0346ba8-7f08-4472-933c-7238392887b6-metrics-certs podName:f0346ba8-7f08-4472-933c-7238392887b6 nodeName:}" failed. No retries permitted until 2026-09-16 14:04:36.939833888 +0000 UTC m=+9.954943740 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f0346ba8-7f08-4472-933c-7238392887b6-metrics-certs") pod "network-metrics-daemon-pxv9d" (UID: "f0346ba8-7f08-4472-933c-7238392887b6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 14:04:33.040441 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:33.040391    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/832cddc0-66b7-45a7-8cad-2a8609131f31-original-pull-secret\") pod \"global-pull-secret-syncer-wg92s\" (UID: \"832cddc0-66b7-45a7-8cad-2a8609131f31\") " pod="kube-system/global-pull-secret-syncer-wg92s"
Sep 16 14:04:33.040626 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:33.040558    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 14:04:33.040626 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:33.040618    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/832cddc0-66b7-45a7-8cad-2a8609131f31-original-pull-secret podName:832cddc0-66b7-45a7-8cad-2a8609131f31 nodeName:}" failed. No retries permitted until 2026-09-16 14:04:37.040604442 +0000 UTC m=+10.055714291 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/832cddc0-66b7-45a7-8cad-2a8609131f31-original-pull-secret") pod "global-pull-secret-syncer-wg92s" (UID: "832cddc0-66b7-45a7-8cad-2a8609131f31") : object "kube-system"/"original-pull-secret" not registered
Sep 16 14:04:33.140895 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:33.140861    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ww6q\" (UniqueName: \"kubernetes.io/projected/43276d8f-034a-4f4d-ad3d-2ca7f3ddfd38-kube-api-access-7ww6q\") pod \"network-check-target-tgkvk\" (UID: \"43276d8f-034a-4f4d-ad3d-2ca7f3ddfd38\") " pod="openshift-network-diagnostics/network-check-target-tgkvk"
Sep 16 14:04:33.141064 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:33.141045    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 14:04:33.141123 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:33.141072    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 14:04:33.141123 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:33.141087    2580 projected.go:196] Error preparing data for projected volume kube-api-access-7ww6q for pod openshift-network-diagnostics/network-check-target-tgkvk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 14:04:33.141220 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:33.141154    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/43276d8f-034a-4f4d-ad3d-2ca7f3ddfd38-kube-api-access-7ww6q podName:43276d8f-034a-4f4d-ad3d-2ca7f3ddfd38 nodeName:}" failed. No retries permitted until 2026-09-16 14:04:37.141136425 +0000 UTC m=+10.156246279 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-7ww6q" (UniqueName: "kubernetes.io/projected/43276d8f-034a-4f4d-ad3d-2ca7f3ddfd38-kube-api-access-7ww6q") pod "network-check-target-tgkvk" (UID: "43276d8f-034a-4f4d-ad3d-2ca7f3ddfd38") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 14:04:33.426348 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:33.425609    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wg92s"
Sep 16 14:04:33.426348 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:33.425746    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wg92s" podUID="832cddc0-66b7-45a7-8cad-2a8609131f31"
Sep 16 14:04:33.426348 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:33.426161    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tgkvk"
Sep 16 14:04:33.426348 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:33.426257    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tgkvk" podUID="43276d8f-034a-4f4d-ad3d-2ca7f3ddfd38"
Sep 16 14:04:34.424342 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:34.424277    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pxv9d"
Sep 16 14:04:34.424536 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:34.424416    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pxv9d" podUID="f0346ba8-7f08-4472-933c-7238392887b6"
Sep 16 14:04:35.424713 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:35.424680    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tgkvk"
Sep 16 14:04:35.425247 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:35.424818    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tgkvk" podUID="43276d8f-034a-4f4d-ad3d-2ca7f3ddfd38"
Sep 16 14:04:35.425247 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:35.424683    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wg92s"
Sep 16 14:04:35.425247 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:35.425164    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wg92s" podUID="832cddc0-66b7-45a7-8cad-2a8609131f31"
Sep 16 14:04:36.424309 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:36.424215    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pxv9d"
Sep 16 14:04:36.424459 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:36.424415    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pxv9d" podUID="f0346ba8-7f08-4472-933c-7238392887b6"
Sep 16 14:04:36.968793 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:36.968756    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0346ba8-7f08-4472-933c-7238392887b6-metrics-certs\") pod \"network-metrics-daemon-pxv9d\" (UID: \"f0346ba8-7f08-4472-933c-7238392887b6\") " pod="openshift-multus/network-metrics-daemon-pxv9d"
Sep 16 14:04:36.969359 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:36.968880    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 14:04:36.969359 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:36.968951    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0346ba8-7f08-4472-933c-7238392887b6-metrics-certs podName:f0346ba8-7f08-4472-933c-7238392887b6 nodeName:}" failed. No retries permitted until 2026-09-16 14:04:44.968929375 +0000 UTC m=+17.984039437 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f0346ba8-7f08-4472-933c-7238392887b6-metrics-certs") pod "network-metrics-daemon-pxv9d" (UID: "f0346ba8-7f08-4472-933c-7238392887b6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 14:04:37.069756 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:37.069724    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/832cddc0-66b7-45a7-8cad-2a8609131f31-original-pull-secret\") pod \"global-pull-secret-syncer-wg92s\" (UID: \"832cddc0-66b7-45a7-8cad-2a8609131f31\") " pod="kube-system/global-pull-secret-syncer-wg92s"
Sep 16 14:04:37.069906 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:37.069862    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 14:04:37.069954 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:37.069919    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/832cddc0-66b7-45a7-8cad-2a8609131f31-original-pull-secret podName:832cddc0-66b7-45a7-8cad-2a8609131f31 nodeName:}" failed. No retries permitted until 2026-09-16 14:04:45.06990511 +0000 UTC m=+18.085014971 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/832cddc0-66b7-45a7-8cad-2a8609131f31-original-pull-secret") pod "global-pull-secret-syncer-wg92s" (UID: "832cddc0-66b7-45a7-8cad-2a8609131f31") : object "kube-system"/"original-pull-secret" not registered
Sep 16 14:04:37.170842 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:37.170796    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ww6q\" (UniqueName: \"kubernetes.io/projected/43276d8f-034a-4f4d-ad3d-2ca7f3ddfd38-kube-api-access-7ww6q\") pod \"network-check-target-tgkvk\" (UID: \"43276d8f-034a-4f4d-ad3d-2ca7f3ddfd38\") " pod="openshift-network-diagnostics/network-check-target-tgkvk"
Sep 16 14:04:37.171018 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:37.170949    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 14:04:37.171018 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:37.170970    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 14:04:37.171018 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:37.170980    2580 projected.go:196] Error preparing data for projected volume kube-api-access-7ww6q for pod openshift-network-diagnostics/network-check-target-tgkvk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 14:04:37.171155 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:37.171035    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/43276d8f-034a-4f4d-ad3d-2ca7f3ddfd38-kube-api-access-7ww6q podName:43276d8f-034a-4f4d-ad3d-2ca7f3ddfd38 nodeName:}" failed. No retries permitted until 2026-09-16 14:04:45.171018307 +0000 UTC m=+18.186128166 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-7ww6q" (UniqueName: "kubernetes.io/projected/43276d8f-034a-4f4d-ad3d-2ca7f3ddfd38-kube-api-access-7ww6q") pod "network-check-target-tgkvk" (UID: "43276d8f-034a-4f4d-ad3d-2ca7f3ddfd38") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 14:04:37.425866 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:37.425770    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wg92s"
Sep 16 14:04:37.426023 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:37.425888    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wg92s" podUID="832cddc0-66b7-45a7-8cad-2a8609131f31"
Sep 16 14:04:37.426023 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:37.425935    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tgkvk"
Sep 16 14:04:37.426207 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:37.426045    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tgkvk" podUID="43276d8f-034a-4f4d-ad3d-2ca7f3ddfd38"
Sep 16 14:04:38.214019 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:38.213964    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf3700a3bae4d7dda42c2b6a53ad43be.slice/crio-fe5ad487e302e19bb6f542a5c68a146b5f4c82dc4eb94da24e4508088e98c548 WatchSource:0}: Error finding container fe5ad487e302e19bb6f542a5c68a146b5f4c82dc4eb94da24e4508088e98c548: Status 404 returned error can't find the container with id fe5ad487e302e19bb6f542a5c68a146b5f4c82dc4eb94da24e4508088e98c548
Sep 16 14:04:38.218883 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:04:38.218773    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1c44fe4_da28_4517_a5ce_b47ea2592ba4.slice/crio-3b2b8f3661a6d241dd88a2b5b83f46b272ef7ca29f2885face0cd4908ace57cb WatchSource:0}: Error finding container 3b2b8f3661a6d241dd88a2b5b83f46b272ef7ca29f2885face0cd4908ace57cb: Status 404 returned error can't find the container with id 3b2b8f3661a6d241dd88a2b5b83f46b272ef7ca29f2885face0cd4908ace57cb
Sep 16 14:04:38.425090 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:38.425013    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pxv9d"
Sep 16 14:04:38.425232 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:38.425164    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pxv9d" podUID="f0346ba8-7f08-4472-933c-7238392887b6"
Sep 16 14:04:38.456670 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:38.456626    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-95.ec2.internal" event={"ID":"38046ded5fcb5dec29d3f7bce0f86c40","Type":"ContainerStarted","Data":"41d7f5e649c8e3fce5065d6eb996793ebf37562cc6bf9374f3c9090a7d52c9dc"}
Sep 16 14:04:38.458573 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:38.458466    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z4h56" event={"ID":"f1c44fe4-da28-4517-a5ce-b47ea2592ba4","Type":"ContainerStarted","Data":"3b2b8f3661a6d241dd88a2b5b83f46b272ef7ca29f2885face0cd4908ace57cb"}
Sep 16 14:04:38.462276 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:38.462176    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-95.ec2.internal" event={"ID":"cf3700a3bae4d7dda42c2b6a53ad43be","Type":"ContainerStarted","Data":"fe5ad487e302e19bb6f542a5c68a146b5f4c82dc4eb94da24e4508088e98c548"}
Sep 16 14:04:38.470879 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:38.470852    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kwf42" event={"ID":"602525a8-0cf7-46fe-b669-1d38f6e15d0e","Type":"ContainerStarted","Data":"d16756ba407e81a80d69026493d15ee7f84ad7b3ee439461d6ceefec11155ee6"}
Sep 16 14:04:38.475290 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:38.474964    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-r62q2" event={"ID":"16dd3320-b25f-4459-89fa-3a4196939969","Type":"ContainerStarted","Data":"33c6628115cb6af4e16a7a56b5b341f9ad11e66363c6aa47901f3f32daf33dcd"}
Sep 16 14:04:38.475422 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:38.475280    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-95.ec2.internal" podStartSLOduration=10.475267401 podStartE2EDuration="10.475267401s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 14:04:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:04:38.474409886 +0000 UTC m=+11.489519746" watchObservedRunningTime="2026-09-16 14:04:38.475267401 +0000 UTC m=+11.490377261"
Sep 16 14:04:38.491816 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:38.491697    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-r62q2" podStartSLOduration=3.521199459 podStartE2EDuration="11.491680605s" totalImagesPullingTime="7.970481146s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 14:04:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:04:38.490766535 +0000 UTC m=+11.505876394" watchObservedRunningTime="2026-09-16 14:04:38.491680605 +0000 UTC m=+11.506790468"
Sep 16 14:04:39.424862 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:39.424791    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tgkvk"
Sep 16 14:04:39.425532 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:39.424916    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tgkvk" podUID="43276d8f-034a-4f4d-ad3d-2ca7f3ddfd38"
Sep 16 14:04:39.425532 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:39.425380    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wg92s"
Sep 16 14:04:39.425532 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:39.425479    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wg92s" podUID="832cddc0-66b7-45a7-8cad-2a8609131f31"
Sep 16 14:04:39.479544 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:39.479125    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-r9k4l" event={"ID":"fa3e87c7-91e7-4767-b43d-6f00261f1d56","Type":"ContainerStarted","Data":"e7bd12c8fa9150ee3071291241347975de41fce1f5e060859d2907b4513c040f"}
Sep 16 14:04:39.482327 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:39.482296    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbrr5" event={"ID":"7c431872-6b49-42db-829b-4013ef367c3b","Type":"ContainerStarted","Data":"bb896b2013db84c4c57ec50165d5bda981f76581638c5a25ce3dfa88f36150c3"}
Sep 16 14:04:39.484119 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:39.484088    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-s544c" event={"ID":"e046d878-c8a6-4278-bde8-238f63fb96b2","Type":"ContainerStarted","Data":"120ade0570401a3201fea58911f942edef159438f452637eb65104848b241b4a"}
Sep 16 14:04:39.486655 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:39.486625    2580 generic.go:356] "Generic (PLEG): container finished" podID="9840e56c-3a33-49c0-a917-70f641aabf3b" containerID="29efd5bd7ed93fe21c3807af8f1848b382b1ee5d4227d8a383074b897edf1ddf" exitCode=0
Sep 16 14:04:39.486767 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:39.486710    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c5xjg" event={"ID":"9840e56c-3a33-49c0-a917-70f641aabf3b","Type":"ContainerDied","Data":"29efd5bd7ed93fe21c3807af8f1848b382b1ee5d4227d8a383074b897edf1ddf"}
Sep 16 14:04:39.488699 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:39.488677    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-cxcmv" event={"ID":"ffa9d62b-620c-46d1-ab3a-73a2fdb84817","Type":"ContainerStarted","Data":"25d8ba103197717f023dd4c73ba3e4acad328df48985bf060770cedf63b984db"}
Sep 16 14:04:39.519860 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:39.519809    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-r9k4l" podStartSLOduration=4.592172098 podStartE2EDuration="12.519789395s" totalImagesPullingTime="7.927617297s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 14:04:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:04:39.495940374 +0000 UTC m=+12.511050236" watchObservedRunningTime="2026-09-16 14:04:39.519789395 +0000 UTC m=+12.534899259"
Sep 16 14:04:39.535281 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:39.535230    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-s544c" podStartSLOduration=4.6026880420000005 podStartE2EDuration="12.535209948s" totalImagesPullingTime="7.932521906s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 14:04:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:04:39.534530046 +0000 UTC m=+12.549639908" watchObservedRunningTime="2026-09-16 14:04:39.535209948 +0000 UTC m=+12.550319810"
Sep 16 14:04:39.552135 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:39.552090    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-cxcmv" podStartSLOduration=4.964743824 podStartE2EDuration="12.552071682s" totalImagesPullingTime="7.587327858s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:04:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:04:39.551775197 +0000 UTC m=+12.566885093" watchObservedRunningTime="2026-09-16 14:04:39.552071682 +0000 UTC m=+12.567181543"
Sep 16 14:04:40.134353 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:40.134324    2580 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 16 14:04:40.343252 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:40.343069    2580 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-16T14:04:40.134347994Z","UUID":"5e9f875f-0639-460b-bdad-958690508680","Handler":null,"Name":"","Endpoint":""}
Sep 16 14:04:40.344832 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:40.344813    2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 16 14:04:40.344962 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:40.344850    2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 16 14:04:40.424696 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:40.424667    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pxv9d"
Sep 16 14:04:40.424849 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:40.424794    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pxv9d" podUID="f0346ba8-7f08-4472-933c-7238392887b6"
Sep 16 14:04:40.493182 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:40.493146    2580 generic.go:356] "Generic (PLEG): container finished" podID="cf3700a3bae4d7dda42c2b6a53ad43be" containerID="aa336d6ac60c75f6ec0ce8a38fa932ce88e457e0cdbd9212992557f59f919475" exitCode=0
Sep 16 14:04:40.493673 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:40.493243    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-95.ec2.internal" event={"ID":"cf3700a3bae4d7dda42c2b6a53ad43be","Type":"ContainerDied","Data":"aa336d6ac60c75f6ec0ce8a38fa932ce88e457e0cdbd9212992557f59f919475"}
Sep 16 14:04:40.495154 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:40.495110    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kwf42" event={"ID":"602525a8-0cf7-46fe-b669-1d38f6e15d0e","Type":"ContainerStarted","Data":"e3c78efd3108ce35d5ec3c8290de8f46887e2b76d15e59b2910fac4ddc61328e"}
Sep 16 14:04:40.497267 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:40.497242    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbrr5" event={"ID":"7c431872-6b49-42db-829b-4013ef367c3b","Type":"ContainerStarted","Data":"f218a3687251ce42739eaefd0063820313ffe09c74eb45064a837fae926ce350"}
Sep 16 14:04:40.527905 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:40.527858    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-kwf42" podStartSLOduration=13.527839902 podStartE2EDuration="13.527839902s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 14:04:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:04:40.527731261 +0000 UTC m=+13.542841121" watchObservedRunningTime="2026-09-16 14:04:40.527839902 +0000 UTC m=+13.542949764"
Sep 16 14:04:40.542901 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:40.542874    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-cxcmv"
Sep 16 14:04:40.543469 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:40.543450    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-cxcmv"
Sep 16 14:04:41.424462 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:41.424423    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tgkvk"
Sep 16 14:04:41.424661 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:41.424441    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wg92s"
Sep 16 14:04:41.424661 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:41.424580    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tgkvk" podUID="43276d8f-034a-4f4d-ad3d-2ca7f3ddfd38"
Sep 16 14:04:41.424661 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:41.424591    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wg92s" podUID="832cddc0-66b7-45a7-8cad-2a8609131f31"
Sep 16 14:04:41.499013 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:41.498952    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-cxcmv"
Sep 16 14:04:41.499531 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:41.499514    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-cxcmv"
Sep 16 14:04:42.424745 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:42.424713    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pxv9d"
Sep 16 14:04:42.424929 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:42.424900    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pxv9d" podUID="f0346ba8-7f08-4472-933c-7238392887b6"
Sep 16 14:04:43.424717 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:43.424679    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tgkvk"
Sep 16 14:04:43.424717 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:43.424724    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wg92s"
Sep 16 14:04:43.425171 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:43.424804    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tgkvk" podUID="43276d8f-034a-4f4d-ad3d-2ca7f3ddfd38"
Sep 16 14:04:43.425171 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:43.424916    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wg92s" podUID="832cddc0-66b7-45a7-8cad-2a8609131f31"
Sep 16 14:04:44.424657 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:44.424430    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pxv9d"
Sep 16 14:04:44.424844 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:44.424785    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pxv9d" podUID="f0346ba8-7f08-4472-933c-7238392887b6"
Sep 16 14:04:44.506174 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:44.506088    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7xg75" event={"ID":"d89c07ec-f200-4033-9fe1-c8dab7ef1d48","Type":"ContainerStarted","Data":"a83f6689b99979a2e4886869537c919eaa8a8ca5519d081e0ced19eb1068efe6"}
Sep 16 14:04:44.508843 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:44.508815    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-95.ec2.internal" event={"ID":"cf3700a3bae4d7dda42c2b6a53ad43be","Type":"ContainerStarted","Data":"e2d58e93c2ace39056ebddb1c9164220c780eb5f83c18439d903608bea372235"}
Sep 16 14:04:44.541384 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:44.541330    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-7xg75" podStartSLOduration=3.856054078 podStartE2EDuration="17.541311683s" totalImagesPullingTime="13.685257605s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 14:04:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:04:44.540740244 +0000 UTC m=+17.555850103" watchObservedRunningTime="2026-09-16 14:04:44.541311683 +0000 UTC m=+17.556421545"
Sep 16 14:04:44.556100 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:44.556048    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-95.ec2.internal" podStartSLOduration=16.556031315 podStartE2EDuration="16.556031315s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 14:04:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:04:44.555681716 +0000 UTC m=+17.570791577" watchObservedRunningTime="2026-09-16 14:04:44.556031315 +0000 UTC m=+17.571141175"
Sep 16 14:04:45.028874 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:45.028837    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0346ba8-7f08-4472-933c-7238392887b6-metrics-certs\") pod \"network-metrics-daemon-pxv9d\" (UID: \"f0346ba8-7f08-4472-933c-7238392887b6\") " pod="openshift-multus/network-metrics-daemon-pxv9d"
Sep 16 14:04:45.029036 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:45.028961    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 14:04:45.029036 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:45.029017    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0346ba8-7f08-4472-933c-7238392887b6-metrics-certs podName:f0346ba8-7f08-4472-933c-7238392887b6 nodeName:}" failed. No retries permitted until 2026-09-16 14:05:01.029001939 +0000 UTC m=+34.044111800 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f0346ba8-7f08-4472-933c-7238392887b6-metrics-certs") pod "network-metrics-daemon-pxv9d" (UID: "f0346ba8-7f08-4472-933c-7238392887b6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 14:04:45.129692 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:45.129659    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/832cddc0-66b7-45a7-8cad-2a8609131f31-original-pull-secret\") pod \"global-pull-secret-syncer-wg92s\" (UID: \"832cddc0-66b7-45a7-8cad-2a8609131f31\") " pod="kube-system/global-pull-secret-syncer-wg92s"
Sep 16 14:04:45.129888 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:45.129797    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 14:04:45.129888 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:45.129869    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/832cddc0-66b7-45a7-8cad-2a8609131f31-original-pull-secret podName:832cddc0-66b7-45a7-8cad-2a8609131f31 nodeName:}" failed. No retries permitted until 2026-09-16 14:05:01.129851121 +0000 UTC m=+34.144960965 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/832cddc0-66b7-45a7-8cad-2a8609131f31-original-pull-secret") pod "global-pull-secret-syncer-wg92s" (UID: "832cddc0-66b7-45a7-8cad-2a8609131f31") : object "kube-system"/"original-pull-secret" not registered
Sep 16 14:04:45.230523 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:45.230481    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ww6q\" (UniqueName: \"kubernetes.io/projected/43276d8f-034a-4f4d-ad3d-2ca7f3ddfd38-kube-api-access-7ww6q\") pod \"network-check-target-tgkvk\" (UID: \"43276d8f-034a-4f4d-ad3d-2ca7f3ddfd38\") " pod="openshift-network-diagnostics/network-check-target-tgkvk"
Sep 16 14:04:45.230695 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:45.230612    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 14:04:45.230695 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:45.230628    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 14:04:45.230695 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:45.230636    2580 projected.go:196] Error preparing data for projected volume kube-api-access-7ww6q for pod openshift-network-diagnostics/network-check-target-tgkvk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 14:04:45.230695 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:45.230695    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/43276d8f-034a-4f4d-ad3d-2ca7f3ddfd38-kube-api-access-7ww6q podName:43276d8f-034a-4f4d-ad3d-2ca7f3ddfd38 nodeName:}" failed. No retries permitted until 2026-09-16 14:05:01.230682296 +0000 UTC m=+34.245792147 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-7ww6q" (UniqueName: "kubernetes.io/projected/43276d8f-034a-4f4d-ad3d-2ca7f3ddfd38-kube-api-access-7ww6q") pod "network-check-target-tgkvk" (UID: "43276d8f-034a-4f4d-ad3d-2ca7f3ddfd38") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 14:04:45.424306 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:45.424233    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tgkvk"
Sep 16 14:04:45.424525 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:45.424233    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wg92s"
Sep 16 14:04:45.424525 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:45.424374    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tgkvk" podUID="43276d8f-034a-4f4d-ad3d-2ca7f3ddfd38"
Sep 16 14:04:45.424525 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:45.424452    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wg92s" podUID="832cddc0-66b7-45a7-8cad-2a8609131f31"
Sep 16 14:04:46.424432 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:46.424385    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pxv9d"
Sep 16 14:04:46.425026 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:46.424544    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pxv9d" podUID="f0346ba8-7f08-4472-933c-7238392887b6"
Sep 16 14:04:46.516833 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:46.516760    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbrr5" event={"ID":"7c431872-6b49-42db-829b-4013ef367c3b","Type":"ContainerStarted","Data":"626b34d3e48152f4a9ec5923d35b4b42dc458c437bf27aade5201778fbc8dcca"}
Sep 16 14:04:46.536843 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:46.536796    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cbrr5" podStartSLOduration=4.376477078 podStartE2EDuration="19.536776268s" totalImagesPullingTime="15.16029919s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 14:04:27 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:04:46.536054801 +0000 UTC m=+19.551164664" watchObservedRunningTime="2026-09-16 14:04:46.536776268 +0000 UTC m=+19.551886186"
Sep 16 14:04:47.425818 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:47.425083    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wg92s"
Sep 16 14:04:47.425818 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:47.425209    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wg92s" podUID="832cddc0-66b7-45a7-8cad-2a8609131f31"
Sep 16 14:04:47.425818 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:47.425246    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tgkvk"
Sep 16 14:04:47.425818 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:47.425313    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tgkvk" podUID="43276d8f-034a-4f4d-ad3d-2ca7f3ddfd38"
Sep 16 14:04:48.424073 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:48.424038    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pxv9d"
Sep 16 14:04:48.424244 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:48.424167    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pxv9d" podUID="f0346ba8-7f08-4472-933c-7238392887b6"
Sep 16 14:04:49.424005 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:49.423968    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wg92s"
Sep 16 14:04:49.424446 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:49.423974    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tgkvk"
Sep 16 14:04:49.424446 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:49.424114    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wg92s" podUID="832cddc0-66b7-45a7-8cad-2a8609131f31"
Sep 16 14:04:49.424446 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:49.424187    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tgkvk" podUID="43276d8f-034a-4f4d-ad3d-2ca7f3ddfd38"
Sep 16 14:04:50.424634 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:50.424602    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pxv9d"
Sep 16 14:04:50.425029 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:50.424717    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pxv9d" podUID="f0346ba8-7f08-4472-933c-7238392887b6"
Sep 16 14:04:51.424153 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:51.424116    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tgkvk"
Sep 16 14:04:51.424422 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:51.424248    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tgkvk" podUID="43276d8f-034a-4f4d-ad3d-2ca7f3ddfd38"
Sep 16 14:04:51.424422 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:51.424290    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wg92s"
Sep 16 14:04:51.424422 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:51.424366    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wg92s" podUID="832cddc0-66b7-45a7-8cad-2a8609131f31"
Sep 16 14:04:52.424705 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:52.424678    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pxv9d"
Sep 16 14:04:52.425094 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:52.424785    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pxv9d" podUID="f0346ba8-7f08-4472-933c-7238392887b6"
Sep 16 14:04:53.424665 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:53.424633    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wg92s"
Sep 16 14:04:53.424804 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:53.424741    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wg92s" podUID="832cddc0-66b7-45a7-8cad-2a8609131f31"
Sep 16 14:04:53.425062 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:53.424815    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tgkvk"
Sep 16 14:04:53.425062 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:53.424902    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tgkvk" podUID="43276d8f-034a-4f4d-ad3d-2ca7f3ddfd38"
Sep 16 14:04:54.424187 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:54.423986    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pxv9d"
Sep 16 14:04:54.424320 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:54.424233    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pxv9d" podUID="f0346ba8-7f08-4472-933c-7238392887b6"
Sep 16 14:04:54.536481 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:54.536451    2580 generic.go:356] "Generic (PLEG): container finished" podID="9840e56c-3a33-49c0-a917-70f641aabf3b" containerID="a64dba39c8d699c9f53e00daff3c207ad76f55de88a3aa5240eaf917cf05c4c8" exitCode=0
Sep 16 14:04:54.537205 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:54.536552    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c5xjg" event={"ID":"9840e56c-3a33-49c0-a917-70f641aabf3b","Type":"ContainerDied","Data":"a64dba39c8d699c9f53e00daff3c207ad76f55de88a3aa5240eaf917cf05c4c8"}
Sep 16 14:04:54.539738 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:54.539714    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z4h56" event={"ID":"f1c44fe4-da28-4517-a5ce-b47ea2592ba4","Type":"ContainerStarted","Data":"801cc3b7529db5bcf4f8a4019a244f0d380d65a9c60ed92676230f40fe869b81"}
Sep 16 14:04:54.539828 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:54.539781    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z4h56" event={"ID":"f1c44fe4-da28-4517-a5ce-b47ea2592ba4","Type":"ContainerStarted","Data":"0957aa3e46f40cc386a94aa3d5b33d9278dcec089b1724bbc262139cd0eb8d46"}
Sep 16 14:04:54.539828 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:54.539814    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z4h56" event={"ID":"f1c44fe4-da28-4517-a5ce-b47ea2592ba4","Type":"ContainerStarted","Data":"e8b03c5667188b473d5b2b7f38c7613d65d63c5b53e8742925ecf16c80853deb"}
Sep 16 14:04:54.539913 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:54.539843    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z4h56" event={"ID":"f1c44fe4-da28-4517-a5ce-b47ea2592ba4","Type":"ContainerStarted","Data":"eceba4ba371cbc30770eb210097bf049c5ca248363e7a3c560c01bd5141e2c49"}
Sep 16 14:04:54.539913 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:54.539871    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z4h56" event={"ID":"f1c44fe4-da28-4517-a5ce-b47ea2592ba4","Type":"ContainerStarted","Data":"b416831f15208005890b6f528cc58607c9ab0b8c8f48f09d9250da5177f28477"}
Sep 16 14:04:54.539913 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:54.539899    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z4h56" event={"ID":"f1c44fe4-da28-4517-a5ce-b47ea2592ba4","Type":"ContainerStarted","Data":"c8cd2aa717771f0161fe38435c057e7ddfd832216e28b97f0a1c6fe839381398"}
Sep 16 14:04:55.424509 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:55.424481    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tgkvk"
Sep 16 14:04:55.424685 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:55.424640    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tgkvk" podUID="43276d8f-034a-4f4d-ad3d-2ca7f3ddfd38"
Sep 16 14:04:55.424744 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:55.424684    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wg92s"
Sep 16 14:04:55.424810 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:55.424791    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wg92s" podUID="832cddc0-66b7-45a7-8cad-2a8609131f31"
Sep 16 14:04:56.424653 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:56.424511    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pxv9d"
Sep 16 14:04:56.424951 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:56.424729    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pxv9d" podUID="f0346ba8-7f08-4472-933c-7238392887b6"
Sep 16 14:04:56.545288 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:56.545258    2580 generic.go:356] "Generic (PLEG): container finished" podID="9840e56c-3a33-49c0-a917-70f641aabf3b" containerID="2939509fbf6f3ca02bed3234dda930686a0679a249e541b1823d85b00115774a" exitCode=0
Sep 16 14:04:56.545419 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:56.545348    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c5xjg" event={"ID":"9840e56c-3a33-49c0-a917-70f641aabf3b","Type":"ContainerDied","Data":"2939509fbf6f3ca02bed3234dda930686a0679a249e541b1823d85b00115774a"}
Sep 16 14:04:56.548787 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:56.548767    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z4h56" event={"ID":"f1c44fe4-da28-4517-a5ce-b47ea2592ba4","Type":"ContainerStarted","Data":"4836e7783fd0bebde3e674777c3c067e616e655474b81b5d0bd2a89b6e74c308"}
Sep 16 14:04:57.424908 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:57.424879    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tgkvk"
Sep 16 14:04:57.425281 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:57.424964    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tgkvk" podUID="43276d8f-034a-4f4d-ad3d-2ca7f3ddfd38"
Sep 16 14:04:57.425281 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:57.425053    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wg92s"
Sep 16 14:04:57.425281 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:57.425139    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wg92s" podUID="832cddc0-66b7-45a7-8cad-2a8609131f31"
Sep 16 14:04:58.424592 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:58.424565    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pxv9d"
Sep 16 14:04:58.424754 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:58.424697    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pxv9d" podUID="f0346ba8-7f08-4472-933c-7238392887b6"
Sep 16 14:04:58.558112 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:58.557973    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z4h56" event={"ID":"f1c44fe4-da28-4517-a5ce-b47ea2592ba4","Type":"ContainerStarted","Data":"18cf702613de29c971b4c105b7663483f503480e395c80d6c1282c7b9dee774d"}
Sep 16 14:04:58.558849 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:58.558260    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-z4h56"
Sep 16 14:04:58.558849 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:58.558342    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-z4h56"
Sep 16 14:04:58.558849 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:58.558394    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-z4h56"
Sep 16 14:04:58.575789 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:58.575629    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-z4h56"
Sep 16 14:04:58.576975 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:58.576900    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-z4h56"
Sep 16 14:04:58.594836 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:58.594775    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-z4h56" podStartSLOduration=15.959947976 podStartE2EDuration="31.594760818s" totalImagesPullingTime="15.634812842s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 14:04:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:04:58.593560611 +0000 UTC m=+31.608670470" watchObservedRunningTime="2026-09-16 14:04:58.594760818 +0000 UTC m=+31.609870678"
Sep 16 14:04:59.424546 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:59.424519    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tgkvk"
Sep 16 14:04:59.424708 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:59.424644    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tgkvk" podUID="43276d8f-034a-4f4d-ad3d-2ca7f3ddfd38"
Sep 16 14:04:59.424708 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:04:59.424699    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wg92s"
Sep 16 14:04:59.424831 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:04:59.424785    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wg92s" podUID="832cddc0-66b7-45a7-8cad-2a8609131f31"
Sep 16 14:05:00.389178 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:00.389006    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-tgkvk"]
Sep 16 14:05:00.389742 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:00.389259    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tgkvk"
Sep 16 14:05:00.389742 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:05:00.389355    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tgkvk" podUID="43276d8f-034a-4f4d-ad3d-2ca7f3ddfd38"
Sep 16 14:05:00.392023 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:00.391993    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-pxv9d"]
Sep 16 14:05:00.392124 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:00.392089    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pxv9d"
Sep 16 14:05:00.392180 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:05:00.392165    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pxv9d" podUID="f0346ba8-7f08-4472-933c-7238392887b6"
Sep 16 14:05:00.403622 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:00.403523    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wg92s"]
Sep 16 14:05:00.403622 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:00.403608    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wg92s"
Sep 16 14:05:00.403803 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:05:00.403709    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wg92s" podUID="832cddc0-66b7-45a7-8cad-2a8609131f31"
Sep 16 14:05:00.564669 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:00.564637    2580 generic.go:356] "Generic (PLEG): container finished" podID="9840e56c-3a33-49c0-a917-70f641aabf3b" containerID="59404dd6231b35e920e74ce2850eed4396ebdf8d8dd1cb1a94ec4bab0409efaa" exitCode=0
Sep 16 14:05:00.564825 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:00.564712    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c5xjg" event={"ID":"9840e56c-3a33-49c0-a917-70f641aabf3b","Type":"ContainerDied","Data":"59404dd6231b35e920e74ce2850eed4396ebdf8d8dd1cb1a94ec4bab0409efaa"}
Sep 16 14:05:01.045417 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:01.045384    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0346ba8-7f08-4472-933c-7238392887b6-metrics-certs\") pod \"network-metrics-daemon-pxv9d\" (UID: \"f0346ba8-7f08-4472-933c-7238392887b6\") " pod="openshift-multus/network-metrics-daemon-pxv9d"
Sep 16 14:05:01.045593 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:05:01.045494    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 14:05:01.045593 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:05:01.045557    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0346ba8-7f08-4472-933c-7238392887b6-metrics-certs podName:f0346ba8-7f08-4472-933c-7238392887b6 nodeName:}" failed. No retries permitted until 2026-09-16 14:05:33.045543359 +0000 UTC m=+66.060653197 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f0346ba8-7f08-4472-933c-7238392887b6-metrics-certs") pod "network-metrics-daemon-pxv9d" (UID: "f0346ba8-7f08-4472-933c-7238392887b6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 14:05:01.146615 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:01.146585    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/832cddc0-66b7-45a7-8cad-2a8609131f31-original-pull-secret\") pod \"global-pull-secret-syncer-wg92s\" (UID: \"832cddc0-66b7-45a7-8cad-2a8609131f31\") " pod="kube-system/global-pull-secret-syncer-wg92s"
Sep 16 14:05:01.146761 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:05:01.146728    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 14:05:01.146796 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:05:01.146786    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/832cddc0-66b7-45a7-8cad-2a8609131f31-original-pull-secret podName:832cddc0-66b7-45a7-8cad-2a8609131f31 nodeName:}" failed. No retries permitted until 2026-09-16 14:05:33.146768409 +0000 UTC m=+66.161878267 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/832cddc0-66b7-45a7-8cad-2a8609131f31-original-pull-secret") pod "global-pull-secret-syncer-wg92s" (UID: "832cddc0-66b7-45a7-8cad-2a8609131f31") : object "kube-system"/"original-pull-secret" not registered
Sep 16 14:05:01.247376 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:01.247346    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ww6q\" (UniqueName: \"kubernetes.io/projected/43276d8f-034a-4f4d-ad3d-2ca7f3ddfd38-kube-api-access-7ww6q\") pod \"network-check-target-tgkvk\" (UID: \"43276d8f-034a-4f4d-ad3d-2ca7f3ddfd38\") " pod="openshift-network-diagnostics/network-check-target-tgkvk"
Sep 16 14:05:01.247559 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:05:01.247485    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 14:05:01.247559 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:05:01.247510    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 14:05:01.247559 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:05:01.247520    2580 projected.go:196] Error preparing data for projected volume kube-api-access-7ww6q for pod openshift-network-diagnostics/network-check-target-tgkvk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 14:05:01.247662 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:05:01.247569    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/43276d8f-034a-4f4d-ad3d-2ca7f3ddfd38-kube-api-access-7ww6q podName:43276d8f-034a-4f4d-ad3d-2ca7f3ddfd38 nodeName:}" failed. No retries permitted until 2026-09-16 14:05:33.247553238 +0000 UTC m=+66.262663094 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-7ww6q" (UniqueName: "kubernetes.io/projected/43276d8f-034a-4f4d-ad3d-2ca7f3ddfd38-kube-api-access-7ww6q") pod "network-check-target-tgkvk" (UID: "43276d8f-034a-4f4d-ad3d-2ca7f3ddfd38") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 14:05:02.424415 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:02.424385    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pxv9d"
Sep 16 14:05:02.424930 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:02.424385    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wg92s"
Sep 16 14:05:02.424930 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:02.424385    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tgkvk"
Sep 16 14:05:02.424930 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:05:02.424515    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pxv9d" podUID="f0346ba8-7f08-4472-933c-7238392887b6"
Sep 16 14:05:02.424930 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:05:02.424584    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wg92s" podUID="832cddc0-66b7-45a7-8cad-2a8609131f31"
Sep 16 14:05:02.424930 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:05:02.424642    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tgkvk" podUID="43276d8f-034a-4f4d-ad3d-2ca7f3ddfd38"
Sep 16 14:05:04.424403 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:04.424355    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wg92s"
Sep 16 14:05:04.424403 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:04.424394    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pxv9d"
Sep 16 14:05:04.424946 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:04.424356    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tgkvk"
Sep 16 14:05:04.424946 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:05:04.424477    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wg92s" podUID="832cddc0-66b7-45a7-8cad-2a8609131f31"
Sep 16 14:05:04.424946 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:05:04.424570    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tgkvk" podUID="43276d8f-034a-4f4d-ad3d-2ca7f3ddfd38"
Sep 16 14:05:04.424946 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:05:04.424641    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pxv9d" podUID="f0346ba8-7f08-4472-933c-7238392887b6"
Sep 16 14:05:05.581104 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:05.581047    2580 generic.go:356] "Generic (PLEG): container finished" podID="9840e56c-3a33-49c0-a917-70f641aabf3b" containerID="3b945753d4f68b96e88b28c049b7b02d2988a6fd3b374c7ab85f1acc6e8410d5" exitCode=0
Sep 16 14:05:05.581104 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:05.581102    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c5xjg" event={"ID":"9840e56c-3a33-49c0-a917-70f641aabf3b","Type":"ContainerDied","Data":"3b945753d4f68b96e88b28c049b7b02d2988a6fd3b374c7ab85f1acc6e8410d5"}
Sep 16 14:05:06.423945 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:06.423919    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wg92s"
Sep 16 14:05:06.423945 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:06.423947    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tgkvk"
Sep 16 14:05:06.424156 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:06.423919    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pxv9d"
Sep 16 14:05:06.424156 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:05:06.424028    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wg92s" podUID="832cddc0-66b7-45a7-8cad-2a8609131f31"
Sep 16 14:05:06.424156 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:05:06.424097    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tgkvk" podUID="43276d8f-034a-4f4d-ad3d-2ca7f3ddfd38"
Sep 16 14:05:06.424279 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:05:06.424168    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pxv9d" podUID="f0346ba8-7f08-4472-933c-7238392887b6"
Sep 16 14:05:06.586113 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:06.586082    2580 generic.go:356] "Generic (PLEG): container finished" podID="9840e56c-3a33-49c0-a917-70f641aabf3b" containerID="0b54bcac9d9990e6d60304332b83c171aa6d44d4de2aacb1e294408186f4fd91" exitCode=0
Sep 16 14:05:06.586479 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:06.586137    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c5xjg" event={"ID":"9840e56c-3a33-49c0-a917-70f641aabf3b","Type":"ContainerDied","Data":"0b54bcac9d9990e6d60304332b83c171aa6d44d4de2aacb1e294408186f4fd91"}
Sep 16 14:05:07.591620 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:07.591430    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c5xjg" event={"ID":"9840e56c-3a33-49c0-a917-70f641aabf3b","Type":"ContainerStarted","Data":"a7b4adf16051e02e76c573a01563107e5c64849066a118010ee3897b45bb6831"}
Sep 16 14:05:07.621150 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:07.621105    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-c5xjg" podStartSLOduration=9.532409768 podStartE2EDuration="40.621092388s" totalImagesPullingTime="31.08868262s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 14:04:27 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:05:07.620799316 +0000 UTC m=+40.635909175" watchObservedRunningTime="2026-09-16 14:05:07.621092388 +0000 UTC m=+40.636202248"
Sep 16 14:05:07.742719 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:07.742698    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-95.ec2.internal" event="NodeReady"
Sep 16 14:05:07.742829 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:07.742778    2580 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 16 14:05:07.825356 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:07.825334    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-5zzjr"]
Sep 16 14:05:07.856865 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:07.856848    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-szx9c"]
Sep 16 14:05:07.856989 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:07.856976    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5zzjr"
Sep 16 14:05:07.860629 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:07.860610    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 16 14:05:07.860725 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:07.860632    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ss8nr\""
Sep 16 14:05:07.861020 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:07.861003    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 16 14:05:07.886645 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:07.886628    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-szx9c"]
Sep 16 14:05:07.886747 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:07.886654    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5zzjr"]
Sep 16 14:05:07.886747 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:07.886719    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-szx9c"
Sep 16 14:05:07.893219 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:07.893198    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 16 14:05:07.893324 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:07.893247    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 16 14:05:07.893467 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:07.893453    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-jrwbv\""
Sep 16 14:05:07.893560 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:07.893467    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 16 14:05:07.977452 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:07.977425    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-lhttb"]
Sep 16 14:05:07.996867 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:07.996821    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/39a375e4-748f-4cc1-b445-f5c5a32a2652-metrics-tls\") pod \"dns-default-5zzjr\" (UID: \"39a375e4-748f-4cc1-b445-f5c5a32a2652\") " pod="openshift-dns/dns-default-5zzjr"
Sep 16 14:05:07.996867 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:07.996844    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39a375e4-748f-4cc1-b445-f5c5a32a2652-tmp-dir\") pod \"dns-default-5zzjr\" (UID: \"39a375e4-748f-4cc1-b445-f5c5a32a2652\") " pod="openshift-dns/dns-default-5zzjr"
Sep 16 14:05:07.996867 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:07.996860    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fm9pq\" (UniqueName: \"kubernetes.io/projected/39a375e4-748f-4cc1-b445-f5c5a32a2652-kube-api-access-fm9pq\") pod \"dns-default-5zzjr\" (UID: \"39a375e4-748f-4cc1-b445-f5c5a32a2652\") " pod="openshift-dns/dns-default-5zzjr"
Sep 16 14:05:07.997006 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:07.996928    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4b7b253f-9622-4e91-8def-d4c654bb1f14-cert\") pod \"ingress-canary-szx9c\" (UID: \"4b7b253f-9622-4e91-8def-d4c654bb1f14\") " pod="openshift-ingress-canary/ingress-canary-szx9c"
Sep 16 14:05:07.997006 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:07.996954    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vss77\" (UniqueName: \"kubernetes.io/projected/4b7b253f-9622-4e91-8def-d4c654bb1f14-kube-api-access-vss77\") pod \"ingress-canary-szx9c\" (UID: \"4b7b253f-9622-4e91-8def-d4c654bb1f14\") " pod="openshift-ingress-canary/ingress-canary-szx9c"
Sep 16 14:05:07.997006 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:07.996972    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/39a375e4-748f-4cc1-b445-f5c5a32a2652-config-volume\") pod \"dns-default-5zzjr\" (UID: \"39a375e4-748f-4cc1-b445-f5c5a32a2652\") " pod="openshift-dns/dns-default-5zzjr"
Sep 16 14:05:07.998678 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:07.998643    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lhttb"
Sep 16 14:05:07.999837 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:07.999815    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lhttb"]
Sep 16 14:05:08.000670 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:08.000654    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 16 14:05:08.001867 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:08.001851    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 16 14:05:08.001961 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:08.001878    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-zzz2w\""
Sep 16 14:05:08.001961 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:08.001886    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 16 14:05:08.001961 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:08.001851    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 16 14:05:08.097978 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:08.097954    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7k726\" (UniqueName: \"kubernetes.io/projected/ef29ef06-9374-482a-a3fc-77032fbf6537-kube-api-access-7k726\") pod \"insights-runtime-extractor-lhttb\" (UID: \"ef29ef06-9374-482a-a3fc-77032fbf6537\") " pod="openshift-insights/insights-runtime-extractor-lhttb"
Sep 16 14:05:08.098089 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:08.097987    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/39a375e4-748f-4cc1-b445-f5c5a32a2652-config-volume\") pod \"dns-default-5zzjr\" (UID: \"39a375e4-748f-4cc1-b445-f5c5a32a2652\") " pod="openshift-dns/dns-default-5zzjr"
Sep 16 14:05:08.098089 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:08.098009    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ef29ef06-9374-482a-a3fc-77032fbf6537-crio-socket\") pod \"insights-runtime-extractor-lhttb\" (UID: \"ef29ef06-9374-482a-a3fc-77032fbf6537\") " pod="openshift-insights/insights-runtime-extractor-lhttb"
Sep 16 14:05:08.098089 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:08.098027    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/39a375e4-748f-4cc1-b445-f5c5a32a2652-metrics-tls\") pod \"dns-default-5zzjr\" (UID: \"39a375e4-748f-4cc1-b445-f5c5a32a2652\") " pod="openshift-dns/dns-default-5zzjr"
Sep 16 14:05:08.098089 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:08.098044    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39a375e4-748f-4cc1-b445-f5c5a32a2652-tmp-dir\") pod \"dns-default-5zzjr\" (UID: \"39a375e4-748f-4cc1-b445-f5c5a32a2652\") " pod="openshift-dns/dns-default-5zzjr"
Sep 16 14:05:08.098295 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:05:08.098134    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 14:05:08.098295 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:05:08.098202    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39a375e4-748f-4cc1-b445-f5c5a32a2652-metrics-tls podName:39a375e4-748f-4cc1-b445-f5c5a32a2652 nodeName:}" failed. No retries permitted until 2026-09-16 14:05:08.598182342 +0000 UTC m=+41.613292183 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/39a375e4-748f-4cc1-b445-f5c5a32a2652-metrics-tls") pod "dns-default-5zzjr" (UID: "39a375e4-748f-4cc1-b445-f5c5a32a2652") : secret "dns-default-metrics-tls" not found
Sep 16 14:05:08.098295 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:08.098225    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fm9pq\" (UniqueName: \"kubernetes.io/projected/39a375e4-748f-4cc1-b445-f5c5a32a2652-kube-api-access-fm9pq\") pod \"dns-default-5zzjr\" (UID: \"39a375e4-748f-4cc1-b445-f5c5a32a2652\") " pod="openshift-dns/dns-default-5zzjr"
Sep 16 14:05:08.098295 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:08.098274    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ef29ef06-9374-482a-a3fc-77032fbf6537-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lhttb\" (UID: \"ef29ef06-9374-482a-a3fc-77032fbf6537\") " pod="openshift-insights/insights-runtime-extractor-lhttb"
Sep 16 14:05:08.098460 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:08.098304    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ef29ef06-9374-482a-a3fc-77032fbf6537-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lhttb\" (UID: \"ef29ef06-9374-482a-a3fc-77032fbf6537\") " pod="openshift-insights/insights-runtime-extractor-lhttb"
Sep 16 14:05:08.098460 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:08.098280    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39a375e4-748f-4cc1-b445-f5c5a32a2652-tmp-dir\") pod \"dns-default-5zzjr\" (UID: \"39a375e4-748f-4cc1-b445-f5c5a32a2652\") " pod="openshift-dns/dns-default-5zzjr"
Sep 16 14:05:08.098460 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:08.098399    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ef29ef06-9374-482a-a3fc-77032fbf6537-data-volume\") pod \"insights-runtime-extractor-lhttb\" (UID: \"ef29ef06-9374-482a-a3fc-77032fbf6537\") " pod="openshift-insights/insights-runtime-extractor-lhttb"
Sep 16 14:05:08.098460 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:08.098440    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4b7b253f-9622-4e91-8def-d4c654bb1f14-cert\") pod \"ingress-canary-szx9c\" (UID: \"4b7b253f-9622-4e91-8def-d4c654bb1f14\") " pod="openshift-ingress-canary/ingress-canary-szx9c"
Sep 16 14:05:08.098614 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:08.098468    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vss77\" (UniqueName: \"kubernetes.io/projected/4b7b253f-9622-4e91-8def-d4c654bb1f14-kube-api-access-vss77\") pod \"ingress-canary-szx9c\" (UID: \"4b7b253f-9622-4e91-8def-d4c654bb1f14\") " pod="openshift-ingress-canary/ingress-canary-szx9c"
Sep 16 14:05:08.098614 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:08.098538    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/39a375e4-748f-4cc1-b445-f5c5a32a2652-config-volume\") pod \"dns-default-5zzjr\" (UID: \"39a375e4-748f-4cc1-b445-f5c5a32a2652\") " pod="openshift-dns/dns-default-5zzjr"
Sep 16 14:05:08.102443 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:08.102424    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4b7b253f-9622-4e91-8def-d4c654bb1f14-cert\") pod \"ingress-canary-szx9c\" (UID: \"4b7b253f-9622-4e91-8def-d4c654bb1f14\") " pod="openshift-ingress-canary/ingress-canary-szx9c"
Sep 16 14:05:08.106801 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:08.106781    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fm9pq\" (UniqueName: \"kubernetes.io/projected/39a375e4-748f-4cc1-b445-f5c5a32a2652-kube-api-access-fm9pq\") pod \"dns-default-5zzjr\" (UID: \"39a375e4-748f-4cc1-b445-f5c5a32a2652\") " pod="openshift-dns/dns-default-5zzjr"
Sep 16 14:05:08.107273 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:08.107253    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vss77\" (UniqueName: \"kubernetes.io/projected/4b7b253f-9622-4e91-8def-d4c654bb1f14-kube-api-access-vss77\") pod \"ingress-canary-szx9c\" (UID: \"4b7b253f-9622-4e91-8def-d4c654bb1f14\") " pod="openshift-ingress-canary/ingress-canary-szx9c"
Sep 16 14:05:08.195140 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:08.195116    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-szx9c"
Sep 16 14:05:08.199356 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:08.199339    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ef29ef06-9374-482a-a3fc-77032fbf6537-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lhttb\" (UID: \"ef29ef06-9374-482a-a3fc-77032fbf6537\") " pod="openshift-insights/insights-runtime-extractor-lhttb"
Sep 16 14:05:08.199449 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:08.199364    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ef29ef06-9374-482a-a3fc-77032fbf6537-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lhttb\" (UID: \"ef29ef06-9374-482a-a3fc-77032fbf6537\") " pod="openshift-insights/insights-runtime-extractor-lhttb"
Sep 16 14:05:08.199449 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:08.199407    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ef29ef06-9374-482a-a3fc-77032fbf6537-data-volume\") pod \"insights-runtime-extractor-lhttb\" (UID: \"ef29ef06-9374-482a-a3fc-77032fbf6537\") " pod="openshift-insights/insights-runtime-extractor-lhttb"
Sep 16 14:05:08.199449 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:08.199446    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7k726\" (UniqueName: \"kubernetes.io/projected/ef29ef06-9374-482a-a3fc-77032fbf6537-kube-api-access-7k726\") pod \"insights-runtime-extractor-lhttb\" (UID: \"ef29ef06-9374-482a-a3fc-77032fbf6537\") " pod="openshift-insights/insights-runtime-extractor-lhttb"
Sep 16 14:05:08.199625 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:05:08.199449    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 14:05:08.199625 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:08.199464    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ef29ef06-9374-482a-a3fc-77032fbf6537-crio-socket\") pod \"insights-runtime-extractor-lhttb\" (UID: \"ef29ef06-9374-482a-a3fc-77032fbf6537\") " pod="openshift-insights/insights-runtime-extractor-lhttb"
Sep 16 14:05:08.199625 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:05:08.199492    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 14:05:08.199625 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:05:08.199521    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ef29ef06-9374-482a-a3fc-77032fbf6537-kube-rbac-proxy-cm podName:ef29ef06-9374-482a-a3fc-77032fbf6537 nodeName:}" failed. No retries permitted until 2026-09-16 14:05:08.699486647 +0000 UTC m=+41.714596509 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ef29ef06-9374-482a-a3fc-77032fbf6537-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lhttb" (UID: "ef29ef06-9374-482a-a3fc-77032fbf6537") : configmap "kube-rbac-proxy" not found
Sep 16 14:05:08.199625 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:05:08.199592    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef29ef06-9374-482a-a3fc-77032fbf6537-insights-runtime-extractor-tls podName:ef29ef06-9374-482a-a3fc-77032fbf6537 nodeName:}" failed. No retries permitted until 2026-09-16 14:05:08.69957519 +0000 UTC m=+41.714685034 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ef29ef06-9374-482a-a3fc-77032fbf6537-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lhttb" (UID: "ef29ef06-9374-482a-a3fc-77032fbf6537") : secret "insights-runtime-extractor-tls" not found
Sep 16 14:05:08.199793 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:08.199639    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ef29ef06-9374-482a-a3fc-77032fbf6537-crio-socket\") pod \"insights-runtime-extractor-lhttb\" (UID: \"ef29ef06-9374-482a-a3fc-77032fbf6537\") " pod="openshift-insights/insights-runtime-extractor-lhttb"
Sep 16 14:05:08.199793 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:08.199711    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ef29ef06-9374-482a-a3fc-77032fbf6537-data-volume\") pod \"insights-runtime-extractor-lhttb\" (UID: \"ef29ef06-9374-482a-a3fc-77032fbf6537\") " pod="openshift-insights/insights-runtime-extractor-lhttb"
Sep 16 14:05:08.207044 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:08.207028    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7k726\" (UniqueName: \"kubernetes.io/projected/ef29ef06-9374-482a-a3fc-77032fbf6537-kube-api-access-7k726\") pod \"insights-runtime-extractor-lhttb\" (UID: \"ef29ef06-9374-482a-a3fc-77032fbf6537\") " pod="openshift-insights/insights-runtime-extractor-lhttb"
Sep 16 14:05:08.424854 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:08.424826    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wg92s"
Sep 16 14:05:08.424985 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:08.424920    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tgkvk"
Sep 16 14:05:08.424985 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:08.424948    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pxv9d"
Sep 16 14:05:08.427240 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:08.427218    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 16 14:05:08.427336 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:08.427258    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 16 14:05:08.427336 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:08.427265    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-j74pt\""
Sep 16 14:05:08.427571 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:08.427555    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-df78v\""
Sep 16 14:05:08.427653 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:08.427637    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 16 14:05:08.427734 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:08.427721    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 16 14:05:08.515131 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:08.515109    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-szx9c"]
Sep 16 14:05:08.518492 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:05:08.518471    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b7b253f_9622_4e91_8def_d4c654bb1f14.slice/crio-35edf716f3d5eb088c7e6cdd2aa4dde7932d81ca84474b1269a2f5802cc47e3c WatchSource:0}: Error finding container 35edf716f3d5eb088c7e6cdd2aa4dde7932d81ca84474b1269a2f5802cc47e3c: Status 404 returned error can't find the container with id 35edf716f3d5eb088c7e6cdd2aa4dde7932d81ca84474b1269a2f5802cc47e3c
Sep 16 14:05:08.594875 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:08.594798    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-szx9c" event={"ID":"4b7b253f-9622-4e91-8def-d4c654bb1f14","Type":"ContainerStarted","Data":"35edf716f3d5eb088c7e6cdd2aa4dde7932d81ca84474b1269a2f5802cc47e3c"}
Sep 16 14:05:08.602385 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:08.602367    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/39a375e4-748f-4cc1-b445-f5c5a32a2652-metrics-tls\") pod \"dns-default-5zzjr\" (UID: \"39a375e4-748f-4cc1-b445-f5c5a32a2652\") " pod="openshift-dns/dns-default-5zzjr"
Sep 16 14:05:08.602527 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:05:08.602515    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 14:05:08.602577 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:05:08.602563    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39a375e4-748f-4cc1-b445-f5c5a32a2652-metrics-tls podName:39a375e4-748f-4cc1-b445-f5c5a32a2652 nodeName:}" failed. No retries permitted until 2026-09-16 14:05:09.602550689 +0000 UTC m=+42.617660527 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/39a375e4-748f-4cc1-b445-f5c5a32a2652-metrics-tls") pod "dns-default-5zzjr" (UID: "39a375e4-748f-4cc1-b445-f5c5a32a2652") : secret "dns-default-metrics-tls" not found
Sep 16 14:05:08.703591 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:08.703564    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ef29ef06-9374-482a-a3fc-77032fbf6537-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lhttb\" (UID: \"ef29ef06-9374-482a-a3fc-77032fbf6537\") " pod="openshift-insights/insights-runtime-extractor-lhttb"
Sep 16 14:05:08.703591 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:08.703592    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ef29ef06-9374-482a-a3fc-77032fbf6537-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lhttb\" (UID: \"ef29ef06-9374-482a-a3fc-77032fbf6537\") " pod="openshift-insights/insights-runtime-extractor-lhttb"
Sep 16 14:05:08.703736 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:05:08.703677    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 14:05:08.703736 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:05:08.703730    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ef29ef06-9374-482a-a3fc-77032fbf6537-kube-rbac-proxy-cm podName:ef29ef06-9374-482a-a3fc-77032fbf6537 nodeName:}" failed. No retries permitted until 2026-09-16 14:05:09.703716834 +0000 UTC m=+42.718826673 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ef29ef06-9374-482a-a3fc-77032fbf6537-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lhttb" (UID: "ef29ef06-9374-482a-a3fc-77032fbf6537") : configmap "kube-rbac-proxy" not found
Sep 16 14:05:08.703736 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:05:08.703729    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 14:05:08.703891 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:05:08.703766    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef29ef06-9374-482a-a3fc-77032fbf6537-insights-runtime-extractor-tls podName:ef29ef06-9374-482a-a3fc-77032fbf6537 nodeName:}" failed. No retries permitted until 2026-09-16 14:05:09.703757363 +0000 UTC m=+42.718867204 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ef29ef06-9374-482a-a3fc-77032fbf6537-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lhttb" (UID: "ef29ef06-9374-482a-a3fc-77032fbf6537") : secret "insights-runtime-extractor-tls" not found
Sep 16 14:05:09.611281 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:09.611034    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/39a375e4-748f-4cc1-b445-f5c5a32a2652-metrics-tls\") pod \"dns-default-5zzjr\" (UID: \"39a375e4-748f-4cc1-b445-f5c5a32a2652\") " pod="openshift-dns/dns-default-5zzjr"
Sep 16 14:05:09.611747 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:05:09.611159    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 14:05:09.611747 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:05:09.611376    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39a375e4-748f-4cc1-b445-f5c5a32a2652-metrics-tls podName:39a375e4-748f-4cc1-b445-f5c5a32a2652 nodeName:}" failed. No retries permitted until 2026-09-16 14:05:11.611354934 +0000 UTC m=+44.626464788 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/39a375e4-748f-4cc1-b445-f5c5a32a2652-metrics-tls") pod "dns-default-5zzjr" (UID: "39a375e4-748f-4cc1-b445-f5c5a32a2652") : secret "dns-default-metrics-tls" not found
Sep 16 14:05:09.712138 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:09.712100    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ef29ef06-9374-482a-a3fc-77032fbf6537-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lhttb\" (UID: \"ef29ef06-9374-482a-a3fc-77032fbf6537\") " pod="openshift-insights/insights-runtime-extractor-lhttb"
Sep 16 14:05:09.712315 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:09.712148    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ef29ef06-9374-482a-a3fc-77032fbf6537-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lhttb\" (UID: \"ef29ef06-9374-482a-a3fc-77032fbf6537\") " pod="openshift-insights/insights-runtime-extractor-lhttb"
Sep 16 14:05:09.712315 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:05:09.712225    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 14:05:09.712315 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:05:09.712302    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ef29ef06-9374-482a-a3fc-77032fbf6537-kube-rbac-proxy-cm podName:ef29ef06-9374-482a-a3fc-77032fbf6537 nodeName:}" failed. No retries permitted until 2026-09-16 14:05:11.712280692 +0000 UTC m=+44.727390550 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ef29ef06-9374-482a-a3fc-77032fbf6537-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lhttb" (UID: "ef29ef06-9374-482a-a3fc-77032fbf6537") : configmap "kube-rbac-proxy" not found
Sep 16 14:05:09.712315 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:05:09.712307    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 14:05:09.712644 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:05:09.712368    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef29ef06-9374-482a-a3fc-77032fbf6537-insights-runtime-extractor-tls podName:ef29ef06-9374-482a-a3fc-77032fbf6537 nodeName:}" failed. No retries permitted until 2026-09-16 14:05:11.712352769 +0000 UTC m=+44.727462621 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ef29ef06-9374-482a-a3fc-77032fbf6537-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lhttb" (UID: "ef29ef06-9374-482a-a3fc-77032fbf6537") : secret "insights-runtime-extractor-tls" not found
Sep 16 14:05:10.419743 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:10.419651    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-zp2tn"]
Sep 16 14:05:10.433377 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:10.433352    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-zp2tn"]
Sep 16 14:05:10.433537 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:10.433478    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-zp2tn"
Sep 16 14:05:10.435612 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:10.435587    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 16 14:05:10.436294 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:10.436191    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 16 14:05:10.436294 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:10.436224    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 16 14:05:10.436458 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:10.436224    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-wwm75\""
Sep 16 14:05:10.436458 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:10.436343    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 16 14:05:10.519387 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:10.519343    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a776e142-f9d2-45c5-8e0e-b01fcadd39a8-signing-key\") pod \"service-ca-568869fdc5-zp2tn\" (UID: \"a776e142-f9d2-45c5-8e0e-b01fcadd39a8\") " pod="openshift-service-ca/service-ca-568869fdc5-zp2tn"
Sep 16 14:05:10.519387 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:10.519390    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lg5v8\" (UniqueName: \"kubernetes.io/projected/a776e142-f9d2-45c5-8e0e-b01fcadd39a8-kube-api-access-lg5v8\") pod \"service-ca-568869fdc5-zp2tn\" (UID: \"a776e142-f9d2-45c5-8e0e-b01fcadd39a8\") " pod="openshift-service-ca/service-ca-568869fdc5-zp2tn"
Sep 16 14:05:10.519623 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:10.519460    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a776e142-f9d2-45c5-8e0e-b01fcadd39a8-signing-cabundle\") pod \"service-ca-568869fdc5-zp2tn\" (UID: \"a776e142-f9d2-45c5-8e0e-b01fcadd39a8\") " pod="openshift-service-ca/service-ca-568869fdc5-zp2tn"
Sep 16 14:05:10.619843 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:10.619811    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lg5v8\" (UniqueName: \"kubernetes.io/projected/a776e142-f9d2-45c5-8e0e-b01fcadd39a8-kube-api-access-lg5v8\") pod \"service-ca-568869fdc5-zp2tn\" (UID: \"a776e142-f9d2-45c5-8e0e-b01fcadd39a8\") " pod="openshift-service-ca/service-ca-568869fdc5-zp2tn"
Sep 16 14:05:10.620138 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:10.619855    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a776e142-f9d2-45c5-8e0e-b01fcadd39a8-signing-cabundle\") pod \"service-ca-568869fdc5-zp2tn\" (UID: \"a776e142-f9d2-45c5-8e0e-b01fcadd39a8\") " pod="openshift-service-ca/service-ca-568869fdc5-zp2tn"
Sep 16 14:05:10.620138 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:10.620083    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a776e142-f9d2-45c5-8e0e-b01fcadd39a8-signing-key\") pod \"service-ca-568869fdc5-zp2tn\" (UID: \"a776e142-f9d2-45c5-8e0e-b01fcadd39a8\") " pod="openshift-service-ca/service-ca-568869fdc5-zp2tn"
Sep 16 14:05:10.620658 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:10.620642    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a776e142-f9d2-45c5-8e0e-b01fcadd39a8-signing-cabundle\") pod \"service-ca-568869fdc5-zp2tn\" (UID: \"a776e142-f9d2-45c5-8e0e-b01fcadd39a8\") " pod="openshift-service-ca/service-ca-568869fdc5-zp2tn"
Sep 16 14:05:10.623294 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:10.623273    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a776e142-f9d2-45c5-8e0e-b01fcadd39a8-signing-key\") pod \"service-ca-568869fdc5-zp2tn\" (UID: \"a776e142-f9d2-45c5-8e0e-b01fcadd39a8\") " pod="openshift-service-ca/service-ca-568869fdc5-zp2tn"
Sep 16 14:05:10.629525 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:10.629490    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lg5v8\" (UniqueName: \"kubernetes.io/projected/a776e142-f9d2-45c5-8e0e-b01fcadd39a8-kube-api-access-lg5v8\") pod \"service-ca-568869fdc5-zp2tn\" (UID: \"a776e142-f9d2-45c5-8e0e-b01fcadd39a8\") " pod="openshift-service-ca/service-ca-568869fdc5-zp2tn"
Sep 16 14:05:10.745474 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:10.745454    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-zp2tn"
Sep 16 14:05:10.903307 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:10.903289    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-s544c_e046d878-c8a6-4278-bde8-238f63fb96b2/dns-node-resolver/0.log"
Sep 16 14:05:11.190308 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:11.190279    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-zp2tn"]
Sep 16 14:05:11.194036 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:05:11.194010    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda776e142_f9d2_45c5_8e0e_b01fcadd39a8.slice/crio-9872b761b90c07365ab8e61417edbfd0110729db3e83272805deadf9be8d834a WatchSource:0}: Error finding container 9872b761b90c07365ab8e61417edbfd0110729db3e83272805deadf9be8d834a: Status 404 returned error can't find the container with id 9872b761b90c07365ab8e61417edbfd0110729db3e83272805deadf9be8d834a
Sep 16 14:05:11.603434 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:11.603402    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-zp2tn" event={"ID":"a776e142-f9d2-45c5-8e0e-b01fcadd39a8","Type":"ContainerStarted","Data":"9872b761b90c07365ab8e61417edbfd0110729db3e83272805deadf9be8d834a"}
Sep 16 14:05:11.604723 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:11.604697    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-szx9c" event={"ID":"4b7b253f-9622-4e91-8def-d4c654bb1f14","Type":"ContainerStarted","Data":"a4e45f451730ffa108c670f9ae80080bcf502051e9e32cce8cb66a327dec179d"}
Sep 16 14:05:11.623017 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:11.622963    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-szx9c" podStartSLOduration=2.212308193 podStartE2EDuration="4.622945617s" totalImagesPullingTime="2.410637424s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:05:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:05:11.621681105 +0000 UTC m=+44.636790966" watchObservedRunningTime="2026-09-16 14:05:11.622945617 +0000 UTC m=+44.638055478"
Sep 16 14:05:11.628568 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:11.628547    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/39a375e4-748f-4cc1-b445-f5c5a32a2652-metrics-tls\") pod \"dns-default-5zzjr\" (UID: \"39a375e4-748f-4cc1-b445-f5c5a32a2652\") " pod="openshift-dns/dns-default-5zzjr"
Sep 16 14:05:11.628701 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:05:11.628686    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 14:05:11.628746 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:05:11.628737    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39a375e4-748f-4cc1-b445-f5c5a32a2652-metrics-tls podName:39a375e4-748f-4cc1-b445-f5c5a32a2652 nodeName:}" failed. No retries permitted until 2026-09-16 14:05:15.628725151 +0000 UTC m=+48.643834989 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/39a375e4-748f-4cc1-b445-f5c5a32a2652-metrics-tls") pod "dns-default-5zzjr" (UID: "39a375e4-748f-4cc1-b445-f5c5a32a2652") : secret "dns-default-metrics-tls" not found
Sep 16 14:05:11.709552 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:11.709528    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-r9k4l_fa3e87c7-91e7-4767-b43d-6f00261f1d56/node-ca/0.log"
Sep 16 14:05:11.729904 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:11.729873    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ef29ef06-9374-482a-a3fc-77032fbf6537-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lhttb\" (UID: \"ef29ef06-9374-482a-a3fc-77032fbf6537\") " pod="openshift-insights/insights-runtime-extractor-lhttb"
Sep 16 14:05:11.730049 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:11.729923    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ef29ef06-9374-482a-a3fc-77032fbf6537-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lhttb\" (UID: \"ef29ef06-9374-482a-a3fc-77032fbf6537\") " pod="openshift-insights/insights-runtime-extractor-lhttb"
Sep 16 14:05:11.730049 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:05:11.729953    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 14:05:11.730049 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:05:11.730014    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ef29ef06-9374-482a-a3fc-77032fbf6537-kube-rbac-proxy-cm podName:ef29ef06-9374-482a-a3fc-77032fbf6537 nodeName:}" failed. No retries permitted until 2026-09-16 14:05:15.729997203 +0000 UTC m=+48.745107042 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ef29ef06-9374-482a-a3fc-77032fbf6537-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lhttb" (UID: "ef29ef06-9374-482a-a3fc-77032fbf6537") : configmap "kube-rbac-proxy" not found
Sep 16 14:05:11.730197 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:05:11.730080    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 14:05:11.730197 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:05:11.730132    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef29ef06-9374-482a-a3fc-77032fbf6537-insights-runtime-extractor-tls podName:ef29ef06-9374-482a-a3fc-77032fbf6537 nodeName:}" failed. No retries permitted until 2026-09-16 14:05:15.730115833 +0000 UTC m=+48.745225704 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ef29ef06-9374-482a-a3fc-77032fbf6537-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lhttb" (UID: "ef29ef06-9374-482a-a3fc-77032fbf6537") : secret "insights-runtime-extractor-tls" not found
Sep 16 14:05:12.506091 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:12.506065    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-szx9c_4b7b253f-9622-4e91-8def-d4c654bb1f14/serve-healthcheck-canary/0.log"
Sep 16 14:05:13.611891 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:13.611795    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-zp2tn" event={"ID":"a776e142-f9d2-45c5-8e0e-b01fcadd39a8","Type":"ContainerStarted","Data":"bf467041c444213513a244b984cf51cd42868b789cdb686a55464ebb845773ad"}
Sep 16 14:05:13.628747 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:13.628697    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-568869fdc5-zp2tn" podStartSLOduration=1.629240118 podStartE2EDuration="3.628678503s" totalImagesPullingTime="1.999438385s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:05:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:05:13.628004841 +0000 UTC m=+46.643114700" watchObservedRunningTime="2026-09-16 14:05:13.628678503 +0000 UTC m=+46.643788365"
Sep 16 14:05:15.663607 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:15.663577    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/39a375e4-748f-4cc1-b445-f5c5a32a2652-metrics-tls\") pod \"dns-default-5zzjr\" (UID: \"39a375e4-748f-4cc1-b445-f5c5a32a2652\") " pod="openshift-dns/dns-default-5zzjr"
Sep 16 14:05:15.663970 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:05:15.663683    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 14:05:15.663970 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:05:15.663732    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39a375e4-748f-4cc1-b445-f5c5a32a2652-metrics-tls podName:39a375e4-748f-4cc1-b445-f5c5a32a2652 nodeName:}" failed. No retries permitted until 2026-09-16 14:05:23.663719413 +0000 UTC m=+56.678829264 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/39a375e4-748f-4cc1-b445-f5c5a32a2652-metrics-tls") pod "dns-default-5zzjr" (UID: "39a375e4-748f-4cc1-b445-f5c5a32a2652") : secret "dns-default-metrics-tls" not found
Sep 16 14:05:15.764760 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:15.764722    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ef29ef06-9374-482a-a3fc-77032fbf6537-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lhttb\" (UID: \"ef29ef06-9374-482a-a3fc-77032fbf6537\") " pod="openshift-insights/insights-runtime-extractor-lhttb"
Sep 16 14:05:15.764931 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:15.764767    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ef29ef06-9374-482a-a3fc-77032fbf6537-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lhttb\" (UID: \"ef29ef06-9374-482a-a3fc-77032fbf6537\") " pod="openshift-insights/insights-runtime-extractor-lhttb"
Sep 16 14:05:15.764931 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:05:15.764847    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 14:05:15.764931 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:05:15.764871    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 14:05:15.764931 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:05:15.764914    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ef29ef06-9374-482a-a3fc-77032fbf6537-kube-rbac-proxy-cm podName:ef29ef06-9374-482a-a3fc-77032fbf6537 nodeName:}" failed. No retries permitted until 2026-09-16 14:05:23.764898515 +0000 UTC m=+56.780008382 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ef29ef06-9374-482a-a3fc-77032fbf6537-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lhttb" (UID: "ef29ef06-9374-482a-a3fc-77032fbf6537") : configmap "kube-rbac-proxy" not found
Sep 16 14:05:15.765084 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:05:15.764978    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef29ef06-9374-482a-a3fc-77032fbf6537-insights-runtime-extractor-tls podName:ef29ef06-9374-482a-a3fc-77032fbf6537 nodeName:}" failed. No retries permitted until 2026-09-16 14:05:23.764963118 +0000 UTC m=+56.780072957 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ef29ef06-9374-482a-a3fc-77032fbf6537-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lhttb" (UID: "ef29ef06-9374-482a-a3fc-77032fbf6537") : secret "insights-runtime-extractor-tls" not found
Sep 16 14:05:23.723451 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:23.723416    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/39a375e4-748f-4cc1-b445-f5c5a32a2652-metrics-tls\") pod \"dns-default-5zzjr\" (UID: \"39a375e4-748f-4cc1-b445-f5c5a32a2652\") " pod="openshift-dns/dns-default-5zzjr"
Sep 16 14:05:23.726271 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:23.726250    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/39a375e4-748f-4cc1-b445-f5c5a32a2652-metrics-tls\") pod \"dns-default-5zzjr\" (UID: \"39a375e4-748f-4cc1-b445-f5c5a32a2652\") " pod="openshift-dns/dns-default-5zzjr"
Sep 16 14:05:23.765707 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:23.765684    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5zzjr"
Sep 16 14:05:23.824140 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:23.824108    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ef29ef06-9374-482a-a3fc-77032fbf6537-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lhttb\" (UID: \"ef29ef06-9374-482a-a3fc-77032fbf6537\") " pod="openshift-insights/insights-runtime-extractor-lhttb"
Sep 16 14:05:23.824265 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:23.824149    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ef29ef06-9374-482a-a3fc-77032fbf6537-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lhttb\" (UID: \"ef29ef06-9374-482a-a3fc-77032fbf6537\") " pod="openshift-insights/insights-runtime-extractor-lhttb"
Sep 16 14:05:23.824328 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:05:23.824278    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 14:05:23.824379 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:05:23.824344    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef29ef06-9374-482a-a3fc-77032fbf6537-insights-runtime-extractor-tls podName:ef29ef06-9374-482a-a3fc-77032fbf6537 nodeName:}" failed. No retries permitted until 2026-09-16 14:05:39.824323801 +0000 UTC m=+72.839433661 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ef29ef06-9374-482a-a3fc-77032fbf6537-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lhttb" (UID: "ef29ef06-9374-482a-a3fc-77032fbf6537") : secret "insights-runtime-extractor-tls" not found
Sep 16 14:05:23.824751 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:23.824716    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ef29ef06-9374-482a-a3fc-77032fbf6537-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lhttb\" (UID: \"ef29ef06-9374-482a-a3fc-77032fbf6537\") " pod="openshift-insights/insights-runtime-extractor-lhttb"
Sep 16 14:05:24.034332 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:24.034266    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5zzjr"]
Sep 16 14:05:24.038255 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:05:24.038229    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39a375e4_748f_4cc1_b445_f5c5a32a2652.slice/crio-2e3d0741b1b3a32d5bd773f605cf34ca41ffb6f65b22f399bd5f49859a7c2d03 WatchSource:0}: Error finding container 2e3d0741b1b3a32d5bd773f605cf34ca41ffb6f65b22f399bd5f49859a7c2d03: Status 404 returned error can't find the container with id 2e3d0741b1b3a32d5bd773f605cf34ca41ffb6f65b22f399bd5f49859a7c2d03
Sep 16 14:05:24.638597 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:24.638563    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5zzjr" event={"ID":"39a375e4-748f-4cc1-b445-f5c5a32a2652","Type":"ContainerStarted","Data":"2e3d0741b1b3a32d5bd773f605cf34ca41ffb6f65b22f399bd5f49859a7c2d03"}
Sep 16 14:05:27.647936 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:27.647897    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5zzjr" event={"ID":"39a375e4-748f-4cc1-b445-f5c5a32a2652","Type":"ContainerStarted","Data":"ed9531ae705e9881f508d1086f1ff7c8273b6b2615947671005ca4b95ad2eed8"}
Sep 16 14:05:27.648324 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:27.647948    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5zzjr" event={"ID":"39a375e4-748f-4cc1-b445-f5c5a32a2652","Type":"ContainerStarted","Data":"6f01e929514d4a62d607cb9ecbc039ae7427b2c8a420c97506208f4a1e8c21bc"}
Sep 16 14:05:27.648324 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:27.648043    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-5zzjr"
Sep 16 14:05:27.670517 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:27.670454    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-5zzjr" podStartSLOduration=18.118134917 podStartE2EDuration="20.670439698s" totalImagesPullingTime="2.552304781s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:05:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:05:27.67031424 +0000 UTC m=+60.685424111" watchObservedRunningTime="2026-09-16 14:05:27.670439698 +0000 UTC m=+60.685549557"
Sep 16 14:05:30.313930 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:30.313852    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-568d556798-dsrhx"]
Sep 16 14:05:30.318395 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:30.318372    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-568d556798-dsrhx"
Sep 16 14:05:30.319998 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:30.319961    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bc8b4d658-zx26l"]
Sep 16 14:05:30.321480 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:30.321451    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 16 14:05:30.321793 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:30.321773    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Sep 16 14:05:30.322035 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:30.322014    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 16 14:05:30.322533 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:30.322281    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-gst4t\""
Sep 16 14:05:30.322533 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:30.322375    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 16 14:05:30.323987 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:30.323972    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bc8b4d658-zx26l"
Sep 16 14:05:30.327010 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:30.326992    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Sep 16 14:05:30.327102 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:30.327021    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Sep 16 14:05:30.327102 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:30.327001    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Sep 16 14:05:30.327102 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:30.327082    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Sep 16 14:05:30.334609 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:30.334589    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-568d556798-dsrhx"]
Sep 16 14:05:30.337038 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:30.337016    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bc8b4d658-zx26l"]
Sep 16 14:05:30.369925 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:30.369896    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e5e56611-efe3-4e6a-92f8-e71c2d6c4f31-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-bc8b4d658-zx26l\" (UID: \"e5e56611-efe3-4e6a-92f8-e71c2d6c4f31\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bc8b4d658-zx26l"
Sep 16 14:05:30.370090 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:30.369930    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/e5e56611-efe3-4e6a-92f8-e71c2d6c4f31-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-bc8b4d658-zx26l\" (UID: \"e5e56611-efe3-4e6a-92f8-e71c2d6c4f31\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bc8b4d658-zx26l"
Sep 16 14:05:30.370090 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:30.369953    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/e5e56611-efe3-4e6a-92f8-e71c2d6c4f31-hub\") pod \"cluster-proxy-proxy-agent-bc8b4d658-zx26l\" (UID: \"e5e56611-efe3-4e6a-92f8-e71c2d6c4f31\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bc8b4d658-zx26l"
Sep 16 14:05:30.370090 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:30.370009    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mfnf\" (UniqueName: \"kubernetes.io/projected/6ad0c930-a19b-4565-8db2-f54a8004eb44-kube-api-access-9mfnf\") pod \"managed-serviceaccount-addon-agent-568d556798-dsrhx\" (UID: \"6ad0c930-a19b-4565-8db2-f54a8004eb44\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-568d556798-dsrhx"
Sep 16 14:05:30.370090 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:30.370047    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6cw2\" (UniqueName: \"kubernetes.io/projected/e5e56611-efe3-4e6a-92f8-e71c2d6c4f31-kube-api-access-n6cw2\") pod \"cluster-proxy-proxy-agent-bc8b4d658-zx26l\" (UID: \"e5e56611-efe3-4e6a-92f8-e71c2d6c4f31\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bc8b4d658-zx26l"
Sep 16 14:05:30.370291 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:30.370107    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6ad0c930-a19b-4565-8db2-f54a8004eb44-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-568d556798-dsrhx\" (UID: \"6ad0c930-a19b-4565-8db2-f54a8004eb44\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-568d556798-dsrhx"
Sep 16 14:05:30.370291 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:30.370144    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/e5e56611-efe3-4e6a-92f8-e71c2d6c4f31-ca\") pod \"cluster-proxy-proxy-agent-bc8b4d658-zx26l\" (UID: \"e5e56611-efe3-4e6a-92f8-e71c2d6c4f31\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bc8b4d658-zx26l"
Sep 16 14:05:30.370291 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:30.370188    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5e56611-efe3-4e6a-92f8-e71c2d6c4f31-additional-service-ca\") pod \"cluster-proxy-proxy-agent-bc8b4d658-zx26l\" (UID: \"e5e56611-efe3-4e6a-92f8-e71c2d6c4f31\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bc8b4d658-zx26l"
Sep 16 14:05:30.470730 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:30.470693    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/e5e56611-efe3-4e6a-92f8-e71c2d6c4f31-ca\") pod \"cluster-proxy-proxy-agent-bc8b4d658-zx26l\" (UID: \"e5e56611-efe3-4e6a-92f8-e71c2d6c4f31\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bc8b4d658-zx26l"
Sep 16 14:05:30.470928 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:30.470740    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5e56611-efe3-4e6a-92f8-e71c2d6c4f31-additional-service-ca\") pod \"cluster-proxy-proxy-agent-bc8b4d658-zx26l\" (UID: \"e5e56611-efe3-4e6a-92f8-e71c2d6c4f31\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bc8b4d658-zx26l"
Sep 16 14:05:30.470928 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:30.470809    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e5e56611-efe3-4e6a-92f8-e71c2d6c4f31-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-bc8b4d658-zx26l\" (UID: \"e5e56611-efe3-4e6a-92f8-e71c2d6c4f31\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bc8b4d658-zx26l"
Sep 16 14:05:30.470928 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:30.470837    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/e5e56611-efe3-4e6a-92f8-e71c2d6c4f31-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-bc8b4d658-zx26l\" (UID: \"e5e56611-efe3-4e6a-92f8-e71c2d6c4f31\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bc8b4d658-zx26l"
Sep 16 14:05:30.470928 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:30.470859    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/e5e56611-efe3-4e6a-92f8-e71c2d6c4f31-hub\") pod \"cluster-proxy-proxy-agent-bc8b4d658-zx26l\" (UID: \"e5e56611-efe3-4e6a-92f8-e71c2d6c4f31\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bc8b4d658-zx26l"
Sep 16 14:05:30.471151 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:30.470994    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mfnf\" (UniqueName: \"kubernetes.io/projected/6ad0c930-a19b-4565-8db2-f54a8004eb44-kube-api-access-9mfnf\") pod \"managed-serviceaccount-addon-agent-568d556798-dsrhx\" (UID: \"6ad0c930-a19b-4565-8db2-f54a8004eb44\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-568d556798-dsrhx"
Sep 16 14:05:30.471151 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:30.471077    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6cw2\" (UniqueName: \"kubernetes.io/projected/e5e56611-efe3-4e6a-92f8-e71c2d6c4f31-kube-api-access-n6cw2\") pod \"cluster-proxy-proxy-agent-bc8b4d658-zx26l\" (UID: \"e5e56611-efe3-4e6a-92f8-e71c2d6c4f31\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bc8b4d658-zx26l"
Sep 16 14:05:30.471151 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:30.471134    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6ad0c930-a19b-4565-8db2-f54a8004eb44-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-568d556798-dsrhx\" (UID: \"6ad0c930-a19b-4565-8db2-f54a8004eb44\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-568d556798-dsrhx"
Sep 16 14:05:30.471741 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:30.471692    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5e56611-efe3-4e6a-92f8-e71c2d6c4f31-additional-service-ca\") pod \"cluster-proxy-proxy-agent-bc8b4d658-zx26l\" (UID: \"e5e56611-efe3-4e6a-92f8-e71c2d6c4f31\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bc8b4d658-zx26l"
Sep 16 14:05:30.473360 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:30.473335    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/e5e56611-efe3-4e6a-92f8-e71c2d6c4f31-ca\") pod \"cluster-proxy-proxy-agent-bc8b4d658-zx26l\" (UID: \"e5e56611-efe3-4e6a-92f8-e71c2d6c4f31\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bc8b4d658-zx26l"
Sep 16 14:05:30.473534 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:30.473517    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e5e56611-efe3-4e6a-92f8-e71c2d6c4f31-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-bc8b4d658-zx26l\" (UID: \"e5e56611-efe3-4e6a-92f8-e71c2d6c4f31\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bc8b4d658-zx26l"
Sep 16 14:05:30.473674 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:30.473658    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/e5e56611-efe3-4e6a-92f8-e71c2d6c4f31-hub\") pod \"cluster-proxy-proxy-agent-bc8b4d658-zx26l\" (UID: \"e5e56611-efe3-4e6a-92f8-e71c2d6c4f31\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bc8b4d658-zx26l"
Sep 16 14:05:30.473778 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:30.473757    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6ad0c930-a19b-4565-8db2-f54a8004eb44-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-568d556798-dsrhx\" (UID: \"6ad0c930-a19b-4565-8db2-f54a8004eb44\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-568d556798-dsrhx"
Sep 16 14:05:30.473813 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:30.473764    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/e5e56611-efe3-4e6a-92f8-e71c2d6c4f31-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-bc8b4d658-zx26l\" (UID: \"e5e56611-efe3-4e6a-92f8-e71c2d6c4f31\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bc8b4d658-zx26l"
Sep 16 14:05:30.486623 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:30.486596    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mfnf\" (UniqueName: \"kubernetes.io/projected/6ad0c930-a19b-4565-8db2-f54a8004eb44-kube-api-access-9mfnf\") pod \"managed-serviceaccount-addon-agent-568d556798-dsrhx\" (UID: \"6ad0c930-a19b-4565-8db2-f54a8004eb44\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-568d556798-dsrhx"
Sep 16 14:05:30.487603 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:30.487587    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6cw2\" (UniqueName: \"kubernetes.io/projected/e5e56611-efe3-4e6a-92f8-e71c2d6c4f31-kube-api-access-n6cw2\") pod \"cluster-proxy-proxy-agent-bc8b4d658-zx26l\" (UID: \"e5e56611-efe3-4e6a-92f8-e71c2d6c4f31\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bc8b4d658-zx26l"
Sep 16 14:05:30.575176 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:30.575112    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-z4h56"
Sep 16 14:05:30.635834 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:30.635808    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-568d556798-dsrhx"
Sep 16 14:05:30.642058 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:30.642039    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bc8b4d658-zx26l"
Sep 16 14:05:30.912083 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:30.912008    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-568d556798-dsrhx"]
Sep 16 14:05:30.916681 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:05:30.916194    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6ad0c930_a19b_4565_8db2_f54a8004eb44.slice/crio-47dbbcf3078ccb53dc0320f24f8b953b5949e115bef665306e106256745b7c45 WatchSource:0}: Error finding container 47dbbcf3078ccb53dc0320f24f8b953b5949e115bef665306e106256745b7c45: Status 404 returned error can't find the container with id 47dbbcf3078ccb53dc0320f24f8b953b5949e115bef665306e106256745b7c45
Sep 16 14:05:30.917434 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:30.917405    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bc8b4d658-zx26l"]
Sep 16 14:05:30.919779 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:05:30.919752    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode5e56611_efe3_4e6a_92f8_e71c2d6c4f31.slice/crio-05eb47749cb35fc802a5dbba722d73911575dd9785bfa46e26ab01c98f9b3d15 WatchSource:0}: Error finding container 05eb47749cb35fc802a5dbba722d73911575dd9785bfa46e26ab01c98f9b3d15: Status 404 returned error can't find the container with id 05eb47749cb35fc802a5dbba722d73911575dd9785bfa46e26ab01c98f9b3d15
Sep 16 14:05:31.659356 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:31.659320    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bc8b4d658-zx26l" event={"ID":"e5e56611-efe3-4e6a-92f8-e71c2d6c4f31","Type":"ContainerStarted","Data":"05eb47749cb35fc802a5dbba722d73911575dd9785bfa46e26ab01c98f9b3d15"}
Sep 16 14:05:31.660462 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:31.660431    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-568d556798-dsrhx" event={"ID":"6ad0c930-a19b-4565-8db2-f54a8004eb44","Type":"ContainerStarted","Data":"47dbbcf3078ccb53dc0320f24f8b953b5949e115bef665306e106256745b7c45"}
Sep 16 14:05:33.090592 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:33.090553    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0346ba8-7f08-4472-933c-7238392887b6-metrics-certs\") pod \"network-metrics-daemon-pxv9d\" (UID: \"f0346ba8-7f08-4472-933c-7238392887b6\") " pod="openshift-multus/network-metrics-daemon-pxv9d"
Sep 16 14:05:33.092910 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:33.092892    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 16 14:05:33.102972 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:33.102954    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0346ba8-7f08-4472-933c-7238392887b6-metrics-certs\") pod \"network-metrics-daemon-pxv9d\" (UID: \"f0346ba8-7f08-4472-933c-7238392887b6\") " pod="openshift-multus/network-metrics-daemon-pxv9d"
Sep 16 14:05:33.191824 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:33.191787    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/832cddc0-66b7-45a7-8cad-2a8609131f31-original-pull-secret\") pod \"global-pull-secret-syncer-wg92s\" (UID: \"832cddc0-66b7-45a7-8cad-2a8609131f31\") " pod="kube-system/global-pull-secret-syncer-wg92s"
Sep 16 14:05:33.195226 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:33.195204    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 16 14:05:33.204184 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:33.204162    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/832cddc0-66b7-45a7-8cad-2a8609131f31-original-pull-secret\") pod \"global-pull-secret-syncer-wg92s\" (UID: \"832cddc0-66b7-45a7-8cad-2a8609131f31\") " pod="kube-system/global-pull-secret-syncer-wg92s"
Sep 16 14:05:33.292634 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:33.292592    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ww6q\" (UniqueName: \"kubernetes.io/projected/43276d8f-034a-4f4d-ad3d-2ca7f3ddfd38-kube-api-access-7ww6q\") pod \"network-check-target-tgkvk\" (UID: \"43276d8f-034a-4f4d-ad3d-2ca7f3ddfd38\") " pod="openshift-network-diagnostics/network-check-target-tgkvk"
Sep 16 14:05:33.297707 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:33.297677    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 16 14:05:33.305880 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:33.305850    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 16 14:05:33.317555 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:33.317530    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ww6q\" (UniqueName: \"kubernetes.io/projected/43276d8f-034a-4f4d-ad3d-2ca7f3ddfd38-kube-api-access-7ww6q\") pod \"network-check-target-tgkvk\" (UID: \"43276d8f-034a-4f4d-ad3d-2ca7f3ddfd38\") " pod="openshift-network-diagnostics/network-check-target-tgkvk"
Sep 16 14:05:33.335405 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:33.335378    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wg92s"
Sep 16 14:05:33.345543 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:33.345521    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-j74pt\""
Sep 16 14:05:33.348443 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:33.348424    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-df78v\""
Sep 16 14:05:33.352269 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:33.352249    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tgkvk"
Sep 16 14:05:33.356688 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:33.356666    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pxv9d"
Sep 16 14:05:33.636717 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:33.635625    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wg92s"]
Sep 16 14:05:33.640236 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:05:33.640209    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod832cddc0_66b7_45a7_8cad_2a8609131f31.slice/crio-086852cc348d70f1ff734f3c54644991ee41b4d02e9ed764e54146db3ead1e4b WatchSource:0}: Error finding container 086852cc348d70f1ff734f3c54644991ee41b4d02e9ed764e54146db3ead1e4b: Status 404 returned error can't find the container with id 086852cc348d70f1ff734f3c54644991ee41b4d02e9ed764e54146db3ead1e4b
Sep 16 14:05:33.661518 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:33.661435    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-pxv9d"]
Sep 16 14:05:33.665340 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:33.665282    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-tgkvk"]
Sep 16 14:05:33.667948 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:33.667921    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wg92s" event={"ID":"832cddc0-66b7-45a7-8cad-2a8609131f31","Type":"ContainerStarted","Data":"086852cc348d70f1ff734f3c54644991ee41b4d02e9ed764e54146db3ead1e4b"}
Sep 16 14:05:33.669133 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:05:33.669109    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0346ba8_7f08_4472_933c_7238392887b6.slice/crio-02202f360dae795bd28dae554818e1585b4d4cfc37dbda950c7dff6f96be8af3 WatchSource:0}: Error finding container 02202f360dae795bd28dae554818e1585b4d4cfc37dbda950c7dff6f96be8af3: Status 404 returned error can't find the container with id 02202f360dae795bd28dae554818e1585b4d4cfc37dbda950c7dff6f96be8af3
Sep 16 14:05:33.672342 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:05:33.672313    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43276d8f_034a_4f4d_ad3d_2ca7f3ddfd38.slice/crio-f83866491a0b0cd0c379b6c29eec72719352b2e8bf33f8849dfd223939f0752c WatchSource:0}: Error finding container f83866491a0b0cd0c379b6c29eec72719352b2e8bf33f8849dfd223939f0752c: Status 404 returned error can't find the container with id f83866491a0b0cd0c379b6c29eec72719352b2e8bf33f8849dfd223939f0752c
Sep 16 14:05:34.676598 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:34.676525    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pxv9d" event={"ID":"f0346ba8-7f08-4472-933c-7238392887b6","Type":"ContainerStarted","Data":"02202f360dae795bd28dae554818e1585b4d4cfc37dbda950c7dff6f96be8af3"}
Sep 16 14:05:34.678896 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:34.678870    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-tgkvk" event={"ID":"43276d8f-034a-4f4d-ad3d-2ca7f3ddfd38","Type":"ContainerStarted","Data":"f83866491a0b0cd0c379b6c29eec72719352b2e8bf33f8849dfd223939f0752c"}
Sep 16 14:05:37.653733 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:37.653702    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-5zzjr"
Sep 16 14:05:39.851285 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:39.851238    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ef29ef06-9374-482a-a3fc-77032fbf6537-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lhttb\" (UID: \"ef29ef06-9374-482a-a3fc-77032fbf6537\") " pod="openshift-insights/insights-runtime-extractor-lhttb"
Sep 16 14:05:39.853844 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:39.853816    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ef29ef06-9374-482a-a3fc-77032fbf6537-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lhttb\" (UID: \"ef29ef06-9374-482a-a3fc-77032fbf6537\") " pod="openshift-insights/insights-runtime-extractor-lhttb"
Sep 16 14:05:40.112000 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:40.111931    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-zzz2w\""
Sep 16 14:05:40.118790 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:40.118768    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lhttb"
Sep 16 14:05:40.703586 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:40.703007    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-568d556798-dsrhx" event={"ID":"6ad0c930-a19b-4565-8db2-f54a8004eb44","Type":"ContainerStarted","Data":"71091d95d357d695707255b55e9386e1d413625b9369f6c13a6f57da1eb5fdc4"}
Sep 16 14:05:40.709257 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:40.709036    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bc8b4d658-zx26l" event={"ID":"e5e56611-efe3-4e6a-92f8-e71c2d6c4f31","Type":"ContainerStarted","Data":"267b0e0d15ef905882a4c043e0443ffb0a7696159f5d52ad5af619fdb41ecd4b"}
Sep 16 14:05:40.720829 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:40.720785    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-568d556798-dsrhx" podStartSLOduration=1.224177875 podStartE2EDuration="10.720769691s" totalImagesPullingTime="9.496591816s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:05:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:05:40.720186879 +0000 UTC m=+73.735296945" watchObservedRunningTime="2026-09-16 14:05:40.720769691 +0000 UTC m=+73.735879550"
Sep 16 14:05:40.762729 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:40.762683    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lhttb"]
Sep 16 14:05:41.716073 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:41.716042    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bc8b4d658-zx26l" event={"ID":"e5e56611-efe3-4e6a-92f8-e71c2d6c4f31","Type":"ContainerStarted","Data":"f57816e317b3b87cb604bc58a434d44f19e3d923444dff35478ed11152c58626"}
Sep 16 14:05:41.716467 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:41.716086    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bc8b4d658-zx26l" event={"ID":"e5e56611-efe3-4e6a-92f8-e71c2d6c4f31","Type":"ContainerStarted","Data":"a8f11b560563e147f742f29c35dbdfa98623a3638988dd50bbb30ef2291760f3"}
Sep 16 14:05:41.717471 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:41.717447    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wg92s" event={"ID":"832cddc0-66b7-45a7-8cad-2a8609131f31","Type":"ContainerStarted","Data":"95f7882cd88e349032834df78dec2bed0e16661752fd6423b5f7bf4a1321f281"}
Sep 16 14:05:41.719117 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:41.719096    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pxv9d" event={"ID":"f0346ba8-7f08-4472-933c-7238392887b6","Type":"ContainerStarted","Data":"8bef903fab01868cf74a3f1abce6ed6ae8e01becf9ffa14427dd59a923d8148a"}
Sep 16 14:05:41.719209 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:41.719133    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pxv9d" event={"ID":"f0346ba8-7f08-4472-933c-7238392887b6","Type":"ContainerStarted","Data":"8a3e6e32cf9402b04098dc7e3e1cb59b3acdc348c2f6ab8b6d63664a3829e31b"}
Sep 16 14:05:41.720733 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:41.720712    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lhttb" event={"ID":"ef29ef06-9374-482a-a3fc-77032fbf6537","Type":"ContainerStarted","Data":"0eeea4f4bc5f4f4d7297fb3eb902fc19c96cd373ebf0a92236486e9acf715012"}
Sep 16 14:05:41.720814 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:41.720753    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lhttb" event={"ID":"ef29ef06-9374-482a-a3fc-77032fbf6537","Type":"ContainerStarted","Data":"fad2648fc0c4c3c4989379048086b33dfc92d1cca28cbce282f6406e6b782825"}
Sep 16 14:05:41.720814 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:41.720770    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lhttb" event={"ID":"ef29ef06-9374-482a-a3fc-77032fbf6537","Type":"ContainerStarted","Data":"e2f155d90b6ee03e0b8681611761270ea9c3f27f4b0ae75e636cb92c21cf16b4"}
Sep 16 14:05:41.721941 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:41.721923    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-tgkvk" event={"ID":"43276d8f-034a-4f4d-ad3d-2ca7f3ddfd38","Type":"ContainerStarted","Data":"c7bc08dba8c2bf97a7437fab809cc2fd62ee1d7f4691f74a7a26602bffe5cbbc"}
Sep 16 14:05:41.748239 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:41.748196    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bc8b4d658-zx26l" podStartSLOduration=2.177695271 podStartE2EDuration="11.74818507s" totalImagesPullingTime="9.570489799s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:05:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:05:41.747423694 +0000 UTC m=+74.762533568" watchObservedRunningTime="2026-09-16 14:05:41.74818507 +0000 UTC m=+74.763294931"
Sep 16 14:05:41.786848 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:41.786765    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-tgkvk" podStartSLOduration=67.963202467 podStartE2EDuration="1m14.786752037s" totalImagesPullingTime="6.82354957s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:04:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:05:41.786138882 +0000 UTC m=+74.801248766" watchObservedRunningTime="2026-09-16 14:05:41.786752037 +0000 UTC m=+74.801861897"
Sep 16 14:05:41.786990 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:41.786964    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-pxv9d" podStartSLOduration=67.96813698 podStartE2EDuration="1m14.786953886s" totalImagesPullingTime="6.818816906s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:04:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:05:41.770146213 +0000 UTC m=+74.785256072" watchObservedRunningTime="2026-09-16 14:05:41.786953886 +0000 UTC m=+74.802063746"
Sep 16 14:05:41.803662 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:41.803626    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-wg92s" podStartSLOduration=67.956027953 podStartE2EDuration="1m14.803619488s" totalImagesPullingTime="6.847591535s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 14:04:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:05:41.803193201 +0000 UTC m=+74.818303065" watchObservedRunningTime="2026-09-16 14:05:41.803619488 +0000 UTC m=+74.818729347"
Sep 16 14:05:42.726190 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:42.726155    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-tgkvk"
Sep 16 14:05:43.730701 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:43.730660    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lhttb" event={"ID":"ef29ef06-9374-482a-a3fc-77032fbf6537","Type":"ContainerStarted","Data":"5e96d81e10345907c0e90d9c5157dea04f9471649b2471782c075840475396ec"}
Sep 16 14:05:43.750362 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:43.750195    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-lhttb" podStartSLOduration=34.808921237 podStartE2EDuration="36.750179472s" totalImagesPullingTime="1.941258235s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 14:05:07 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:05:43.749553862 +0000 UTC m=+76.764663721" watchObservedRunningTime="2026-09-16 14:05:43.750179472 +0000 UTC m=+76.765289332"
Sep 16 14:05:44.599702 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:44.599663    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-v5khd"]
Sep 16 14:05:44.603374 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:44.603350    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-v5khd"
Sep 16 14:05:44.606413 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:44.606385    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-dpfk2\""
Sep 16 14:05:44.607160 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:44.607136    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 16 14:05:44.607444 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:44.607428    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 16 14:05:44.607660 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:44.607637    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 16 14:05:44.607833 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:44.607816    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 16 14:05:44.608146 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:44.608126    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 16 14:05:44.608579 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:44.608562    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 16 14:05:44.689476 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:44.689447    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/375147d4-f89a-4678-9f71-70a295e903a1-root\") pod \"node-exporter-v5khd\" (UID: \"375147d4-f89a-4678-9f71-70a295e903a1\") " pod="openshift-monitoring/node-exporter-v5khd"
Sep 16 14:05:44.689633 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:44.689481    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/375147d4-f89a-4678-9f71-70a295e903a1-node-exporter-textfile\") pod \"node-exporter-v5khd\" (UID: \"375147d4-f89a-4678-9f71-70a295e903a1\") " pod="openshift-monitoring/node-exporter-v5khd"
Sep 16 14:05:44.689633 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:44.689519    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/375147d4-f89a-4678-9f71-70a295e903a1-node-exporter-accelerators-collector-config\") pod \"node-exporter-v5khd\" (UID: \"375147d4-f89a-4678-9f71-70a295e903a1\") " pod="openshift-monitoring/node-exporter-v5khd"
Sep 16 14:05:44.689633 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:44.689573    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/375147d4-f89a-4678-9f71-70a295e903a1-node-exporter-wtmp\") pod \"node-exporter-v5khd\" (UID: \"375147d4-f89a-4678-9f71-70a295e903a1\") " pod="openshift-monitoring/node-exporter-v5khd"
Sep 16 14:05:44.689633 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:44.689623    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/375147d4-f89a-4678-9f71-70a295e903a1-node-exporter-tls\") pod \"node-exporter-v5khd\" (UID: \"375147d4-f89a-4678-9f71-70a295e903a1\") " pod="openshift-monitoring/node-exporter-v5khd"
Sep 16 14:05:44.689799 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:44.689658    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vsbng\" (UniqueName: \"kubernetes.io/projected/375147d4-f89a-4678-9f71-70a295e903a1-kube-api-access-vsbng\") pod \"node-exporter-v5khd\" (UID: \"375147d4-f89a-4678-9f71-70a295e903a1\") " pod="openshift-monitoring/node-exporter-v5khd"
Sep 16 14:05:44.689799 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:44.689682    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/375147d4-f89a-4678-9f71-70a295e903a1-metrics-client-ca\") pod \"node-exporter-v5khd\" (UID: \"375147d4-f89a-4678-9f71-70a295e903a1\") " pod="openshift-monitoring/node-exporter-v5khd"
Sep 16 14:05:44.689799 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:44.689706    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/375147d4-f89a-4678-9f71-70a295e903a1-sys\") pod \"node-exporter-v5khd\" (UID: \"375147d4-f89a-4678-9f71-70a295e903a1\") " pod="openshift-monitoring/node-exporter-v5khd"
Sep 16 14:05:44.689799 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:44.689736    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/375147d4-f89a-4678-9f71-70a295e903a1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-v5khd\" (UID: \"375147d4-f89a-4678-9f71-70a295e903a1\") " pod="openshift-monitoring/node-exporter-v5khd"
Sep 16 14:05:44.790792 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:44.790763    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/375147d4-f89a-4678-9f71-70a295e903a1-node-exporter-textfile\") pod \"node-exporter-v5khd\" (UID: \"375147d4-f89a-4678-9f71-70a295e903a1\") " pod="openshift-monitoring/node-exporter-v5khd"
Sep 16 14:05:44.791219 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:44.790796    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/375147d4-f89a-4678-9f71-70a295e903a1-node-exporter-accelerators-collector-config\") pod \"node-exporter-v5khd\" (UID: \"375147d4-f89a-4678-9f71-70a295e903a1\") " pod="openshift-monitoring/node-exporter-v5khd"
Sep 16 14:05:44.791959 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:44.791383    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/375147d4-f89a-4678-9f71-70a295e903a1-node-exporter-wtmp\") pod \"node-exporter-v5khd\" (UID: \"375147d4-f89a-4678-9f71-70a295e903a1\") " pod="openshift-monitoring/node-exporter-v5khd"
Sep 16 14:05:44.791959 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:44.791477    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/375147d4-f89a-4678-9f71-70a295e903a1-node-exporter-tls\") pod \"node-exporter-v5khd\" (UID: \"375147d4-f89a-4678-9f71-70a295e903a1\") " pod="openshift-monitoring/node-exporter-v5khd"
Sep 16 14:05:44.791959 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:44.791527    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/375147d4-f89a-4678-9f71-70a295e903a1-node-exporter-textfile\") pod \"node-exporter-v5khd\" (UID: \"375147d4-f89a-4678-9f71-70a295e903a1\") " pod="openshift-monitoring/node-exporter-v5khd"
Sep 16 14:05:44.791959 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:44.791647    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/375147d4-f89a-4678-9f71-70a295e903a1-node-exporter-wtmp\") pod \"node-exporter-v5khd\" (UID: \"375147d4-f89a-4678-9f71-70a295e903a1\") " pod="openshift-monitoring/node-exporter-v5khd"
Sep 16 14:05:44.791959 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:44.791746    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsbng\" (UniqueName: \"kubernetes.io/projected/375147d4-f89a-4678-9f71-70a295e903a1-kube-api-access-vsbng\") pod \"node-exporter-v5khd\" (UID: \"375147d4-f89a-4678-9f71-70a295e903a1\") " pod="openshift-monitoring/node-exporter-v5khd"
Sep 16 14:05:44.791959 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:44.791799    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/375147d4-f89a-4678-9f71-70a295e903a1-metrics-client-ca\") pod \"node-exporter-v5khd\" (UID: \"375147d4-f89a-4678-9f71-70a295e903a1\") " pod="openshift-monitoring/node-exporter-v5khd"
Sep 16 14:05:44.791959 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:44.791833    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/375147d4-f89a-4678-9f71-70a295e903a1-node-exporter-accelerators-collector-config\") pod \"node-exporter-v5khd\" (UID: \"375147d4-f89a-4678-9f71-70a295e903a1\") " pod="openshift-monitoring/node-exporter-v5khd"
Sep 16 14:05:44.791959 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:44.791851    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/375147d4-f89a-4678-9f71-70a295e903a1-sys\") pod \"node-exporter-v5khd\" (UID: \"375147d4-f89a-4678-9f71-70a295e903a1\") " pod="openshift-monitoring/node-exporter-v5khd"
Sep 16 14:05:44.792440 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:44.792189    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/375147d4-f89a-4678-9f71-70a295e903a1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-v5khd\" (UID: \"375147d4-f89a-4678-9f71-70a295e903a1\") " pod="openshift-monitoring/node-exporter-v5khd"
Sep 16 14:05:44.792440 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:44.792219    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/375147d4-f89a-4678-9f71-70a295e903a1-sys\") pod \"node-exporter-v5khd\" (UID: \"375147d4-f89a-4678-9f71-70a295e903a1\") " pod="openshift-monitoring/node-exporter-v5khd"
Sep 16 14:05:44.792440 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:44.792273    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/375147d4-f89a-4678-9f71-70a295e903a1-root\") pod \"node-exporter-v5khd\" (UID: \"375147d4-f89a-4678-9f71-70a295e903a1\") " pod="openshift-monitoring/node-exporter-v5khd"
Sep 16 14:05:44.792440 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:44.792367    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/375147d4-f89a-4678-9f71-70a295e903a1-root\") pod \"node-exporter-v5khd\" (UID: \"375147d4-f89a-4678-9f71-70a295e903a1\") " pod="openshift-monitoring/node-exporter-v5khd"
Sep 16 14:05:44.792647 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:44.792491    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/375147d4-f89a-4678-9f71-70a295e903a1-metrics-client-ca\") pod \"node-exporter-v5khd\" (UID: \"375147d4-f89a-4678-9f71-70a295e903a1\") " pod="openshift-monitoring/node-exporter-v5khd"
Sep 16 14:05:44.798493 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:44.798466    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/375147d4-f89a-4678-9f71-70a295e903a1-node-exporter-tls\") pod \"node-exporter-v5khd\" (UID: \"375147d4-f89a-4678-9f71-70a295e903a1\") " pod="openshift-monitoring/node-exporter-v5khd"
Sep 16 14:05:44.798715 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:44.798693    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/375147d4-f89a-4678-9f71-70a295e903a1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-v5khd\" (UID: \"375147d4-f89a-4678-9f71-70a295e903a1\") " pod="openshift-monitoring/node-exporter-v5khd"
Sep 16 14:05:44.800223 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:44.800200    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vsbng\" (UniqueName: \"kubernetes.io/projected/375147d4-f89a-4678-9f71-70a295e903a1-kube-api-access-vsbng\") pod \"node-exporter-v5khd\" (UID: \"375147d4-f89a-4678-9f71-70a295e903a1\") " pod="openshift-monitoring/node-exporter-v5khd"
Sep 16 14:05:44.917297 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:44.917231    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-v5khd"
Sep 16 14:05:44.929113 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:05:44.929087    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod375147d4_f89a_4678_9f71_70a295e903a1.slice/crio-13fdce61b61234f13454d82658e7efd977f0c72bcad6f23d14a212d32228b7ad WatchSource:0}: Error finding container 13fdce61b61234f13454d82658e7efd977f0c72bcad6f23d14a212d32228b7ad: Status 404 returned error can't find the container with id 13fdce61b61234f13454d82658e7efd977f0c72bcad6f23d14a212d32228b7ad
Sep 16 14:05:45.739373 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:45.739335    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-v5khd" event={"ID":"375147d4-f89a-4678-9f71-70a295e903a1","Type":"ContainerStarted","Data":"13fdce61b61234f13454d82658e7efd977f0c72bcad6f23d14a212d32228b7ad"}
Sep 16 14:05:46.743701 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:46.743670    2580 generic.go:356] "Generic (PLEG): container finished" podID="375147d4-f89a-4678-9f71-70a295e903a1" containerID="519be3471fe485d0a8ee2076921d05fcc650b915d2646f6bdbf038d30e2db3f6" exitCode=0
Sep 16 14:05:46.744060 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:46.743734    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-v5khd" event={"ID":"375147d4-f89a-4678-9f71-70a295e903a1","Type":"ContainerDied","Data":"519be3471fe485d0a8ee2076921d05fcc650b915d2646f6bdbf038d30e2db3f6"}
Sep 16 14:05:47.749291 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:47.749257    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-v5khd" event={"ID":"375147d4-f89a-4678-9f71-70a295e903a1","Type":"ContainerStarted","Data":"0c8d3e1305ff96ea915b5033370d39b6c211bc7be681f6dc13878a8470789240"}
Sep 16 14:05:47.749680 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:47.749307    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-v5khd" event={"ID":"375147d4-f89a-4678-9f71-70a295e903a1","Type":"ContainerStarted","Data":"95601f95b499214e64f708cd5005368977869f8cd028ee260c912af7baeeb9ec"}
Sep 16 14:05:47.770513 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:05:47.770452    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-v5khd" podStartSLOduration=2.552747041 podStartE2EDuration="3.770437484s" totalImagesPullingTime="1.158335146s" totalInitContainerRuntime="59.355297ms" isStatefulPod=true podCreationTimestamp="2026-09-16 14:05:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:05:47.768648682 +0000 UTC m=+80.783758541" watchObservedRunningTime="2026-09-16 14:05:47.770437484 +0000 UTC m=+80.785547343"
Sep 16 14:06:13.733770 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:06:13.733741    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-tgkvk"
Sep 16 14:06:28.444440 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:06:28.444405    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-szx9c_4b7b253f-9622-4e91-8def-d4c654bb1f14/serve-healthcheck-canary/0.log"
Sep 16 14:07:08.986769 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:07:08.986731    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-6c77cb6fc8-vg979"]
Sep 16 14:07:08.990088 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:07:08.990066    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6c77cb6fc8-vg979"
Sep 16 14:07:08.993307 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:07:08.993275    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Sep 16 14:07:08.993797 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:07:08.993764    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Sep 16 14:07:08.993797 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:07:08.993771    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-pf2c4\""
Sep 16 14:07:08.993973 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:07:08.993799    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Sep 16 14:07:08.994057 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:07:08.994044    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Sep 16 14:07:08.994158 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:07:08.994142    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Sep 16 14:07:08.999795 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:07:08.999779    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Sep 16 14:07:09.009567 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:07:09.009541    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6c77cb6fc8-vg979"]
Sep 16 14:07:09.135036 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:07:09.134992    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5f86q\" (UniqueName: \"kubernetes.io/projected/25d3799f-6859-473b-af72-56716dfa3dbb-kube-api-access-5f86q\") pod \"telemeter-client-6c77cb6fc8-vg979\" (UID: \"25d3799f-6859-473b-af72-56716dfa3dbb\") " pod="openshift-monitoring/telemeter-client-6c77cb6fc8-vg979"
Sep 16 14:07:09.135036 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:07:09.135037    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/25d3799f-6859-473b-af72-56716dfa3dbb-federate-client-tls\") pod \"telemeter-client-6c77cb6fc8-vg979\" (UID: \"25d3799f-6859-473b-af72-56716dfa3dbb\") " pod="openshift-monitoring/telemeter-client-6c77cb6fc8-vg979"
Sep 16 14:07:09.135237 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:07:09.135056    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25d3799f-6859-473b-af72-56716dfa3dbb-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6c77cb6fc8-vg979\" (UID: \"25d3799f-6859-473b-af72-56716dfa3dbb\") " pod="openshift-monitoring/telemeter-client-6c77cb6fc8-vg979"
Sep 16 14:07:09.135237 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:07:09.135088    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/25d3799f-6859-473b-af72-56716dfa3dbb-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6c77cb6fc8-vg979\" (UID: \"25d3799f-6859-473b-af72-56716dfa3dbb\") " pod="openshift-monitoring/telemeter-client-6c77cb6fc8-vg979"
Sep 16 14:07:09.135237 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:07:09.135107    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/25d3799f-6859-473b-af72-56716dfa3dbb-secret-telemeter-client\") pod \"telemeter-client-6c77cb6fc8-vg979\" (UID: \"25d3799f-6859-473b-af72-56716dfa3dbb\") " pod="openshift-monitoring/telemeter-client-6c77cb6fc8-vg979"
Sep 16 14:07:09.135237 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:07:09.135131    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/25d3799f-6859-473b-af72-56716dfa3dbb-metrics-client-ca\") pod \"telemeter-client-6c77cb6fc8-vg979\" (UID: \"25d3799f-6859-473b-af72-56716dfa3dbb\") " pod="openshift-monitoring/telemeter-client-6c77cb6fc8-vg979"
Sep 16 14:07:09.135237 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:07:09.135160    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/25d3799f-6859-473b-af72-56716dfa3dbb-telemeter-client-tls\") pod \"telemeter-client-6c77cb6fc8-vg979\" (UID: \"25d3799f-6859-473b-af72-56716dfa3dbb\") " pod="openshift-monitoring/telemeter-client-6c77cb6fc8-vg979"
Sep 16 14:07:09.135387 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:07:09.135233    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25d3799f-6859-473b-af72-56716dfa3dbb-serving-certs-ca-bundle\") pod \"telemeter-client-6c77cb6fc8-vg979\" (UID: \"25d3799f-6859-473b-af72-56716dfa3dbb\") " pod="openshift-monitoring/telemeter-client-6c77cb6fc8-vg979"
Sep 16 14:07:09.236128 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:07:09.236095    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5f86q\" (UniqueName: \"kubernetes.io/projected/25d3799f-6859-473b-af72-56716dfa3dbb-kube-api-access-5f86q\") pod \"telemeter-client-6c77cb6fc8-vg979\" (UID: \"25d3799f-6859-473b-af72-56716dfa3dbb\") " pod="openshift-monitoring/telemeter-client-6c77cb6fc8-vg979"
Sep 16 14:07:09.236292 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:07:09.236137    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/25d3799f-6859-473b-af72-56716dfa3dbb-federate-client-tls\") pod \"telemeter-client-6c77cb6fc8-vg979\" (UID: \"25d3799f-6859-473b-af72-56716dfa3dbb\") " pod="openshift-monitoring/telemeter-client-6c77cb6fc8-vg979"
Sep 16 14:07:09.236292 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:07:09.236155    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25d3799f-6859-473b-af72-56716dfa3dbb-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6c77cb6fc8-vg979\" (UID: \"25d3799f-6859-473b-af72-56716dfa3dbb\") " pod="openshift-monitoring/telemeter-client-6c77cb6fc8-vg979"
Sep 16 14:07:09.236292 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:07:09.236176    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/25d3799f-6859-473b-af72-56716dfa3dbb-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6c77cb6fc8-vg979\" (UID: \"25d3799f-6859-473b-af72-56716dfa3dbb\") " pod="openshift-monitoring/telemeter-client-6c77cb6fc8-vg979"
Sep 16 14:07:09.236292 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:07:09.236194    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/25d3799f-6859-473b-af72-56716dfa3dbb-secret-telemeter-client\") pod \"telemeter-client-6c77cb6fc8-vg979\" (UID: \"25d3799f-6859-473b-af72-56716dfa3dbb\") " pod="openshift-monitoring/telemeter-client-6c77cb6fc8-vg979"
Sep 16 14:07:09.236492 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:07:09.236321    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/25d3799f-6859-473b-af72-56716dfa3dbb-metrics-client-ca\") pod \"telemeter-client-6c77cb6fc8-vg979\" (UID: \"25d3799f-6859-473b-af72-56716dfa3dbb\") " pod="openshift-monitoring/telemeter-client-6c77cb6fc8-vg979"
Sep 16 14:07:09.236492 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:07:09.236369    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/25d3799f-6859-473b-af72-56716dfa3dbb-telemeter-client-tls\") pod \"telemeter-client-6c77cb6fc8-vg979\" (UID: \"25d3799f-6859-473b-af72-56716dfa3dbb\") " pod="openshift-monitoring/telemeter-client-6c77cb6fc8-vg979"
Sep 16 14:07:09.236492 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:07:09.236414    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25d3799f-6859-473b-af72-56716dfa3dbb-serving-certs-ca-bundle\") pod \"telemeter-client-6c77cb6fc8-vg979\" (UID: \"25d3799f-6859-473b-af72-56716dfa3dbb\") " pod="openshift-monitoring/telemeter-client-6c77cb6fc8-vg979"
Sep 16 14:07:09.237240 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:07:09.237175    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25d3799f-6859-473b-af72-56716dfa3dbb-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6c77cb6fc8-vg979\" (UID: \"25d3799f-6859-473b-af72-56716dfa3dbb\") " pod="openshift-monitoring/telemeter-client-6c77cb6fc8-vg979"
Sep 16 14:07:09.237240 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:07:09.237233    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/25d3799f-6859-473b-af72-56716dfa3dbb-metrics-client-ca\") pod \"telemeter-client-6c77cb6fc8-vg979\" (UID: \"25d3799f-6859-473b-af72-56716dfa3dbb\") " pod="openshift-monitoring/telemeter-client-6c77cb6fc8-vg979"
Sep 16 14:07:09.237410 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:07:09.237260    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25d3799f-6859-473b-af72-56716dfa3dbb-serving-certs-ca-bundle\") pod \"telemeter-client-6c77cb6fc8-vg979\" (UID: \"25d3799f-6859-473b-af72-56716dfa3dbb\") " pod="openshift-monitoring/telemeter-client-6c77cb6fc8-vg979"
Sep 16 14:07:09.238899 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:07:09.238874    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/25d3799f-6859-473b-af72-56716dfa3dbb-telemeter-client-tls\") pod \"telemeter-client-6c77cb6fc8-vg979\" (UID: \"25d3799f-6859-473b-af72-56716dfa3dbb\") " pod="openshift-monitoring/telemeter-client-6c77cb6fc8-vg979"
Sep 16 14:07:09.239008 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:07:09.238924    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/25d3799f-6859-473b-af72-56716dfa3dbb-federate-client-tls\") pod \"telemeter-client-6c77cb6fc8-vg979\" (UID: \"25d3799f-6859-473b-af72-56716dfa3dbb\") " pod="openshift-monitoring/telemeter-client-6c77cb6fc8-vg979"
Sep 16 14:07:09.239180 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:07:09.239164    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/25d3799f-6859-473b-af72-56716dfa3dbb-secret-telemeter-client\") pod \"telemeter-client-6c77cb6fc8-vg979\" (UID: \"25d3799f-6859-473b-af72-56716dfa3dbb\") " pod="openshift-monitoring/telemeter-client-6c77cb6fc8-vg979"
Sep 16 14:07:09.239252 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:07:09.239230    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/25d3799f-6859-473b-af72-56716dfa3dbb-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6c77cb6fc8-vg979\" (UID: \"25d3799f-6859-473b-af72-56716dfa3dbb\") " pod="openshift-monitoring/telemeter-client-6c77cb6fc8-vg979"
Sep 16 14:07:09.246935 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:07:09.246903    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5f86q\" (UniqueName: \"kubernetes.io/projected/25d3799f-6859-473b-af72-56716dfa3dbb-kube-api-access-5f86q\") pod \"telemeter-client-6c77cb6fc8-vg979\" (UID: \"25d3799f-6859-473b-af72-56716dfa3dbb\") " pod="openshift-monitoring/telemeter-client-6c77cb6fc8-vg979"
Sep 16 14:07:09.301068 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:07:09.301041    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6c77cb6fc8-vg979"
Sep 16 14:07:09.582415 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:07:09.582391    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6c77cb6fc8-vg979"]
Sep 16 14:07:09.584723 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:07:09.584698    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25d3799f_6859_473b_af72_56716dfa3dbb.slice/crio-5b83f25210d5de167e5818324042f56bc29726d40ddaae44aa3acb1f986d7547 WatchSource:0}: Error finding container 5b83f25210d5de167e5818324042f56bc29726d40ddaae44aa3acb1f986d7547: Status 404 returned error can't find the container with id 5b83f25210d5de167e5818324042f56bc29726d40ddaae44aa3acb1f986d7547
Sep 16 14:07:10.014324 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:07:10.014287    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6c77cb6fc8-vg979" event={"ID":"25d3799f-6859-473b-af72-56716dfa3dbb","Type":"ContainerStarted","Data":"5b83f25210d5de167e5818324042f56bc29726d40ddaae44aa3acb1f986d7547"}
Sep 16 14:07:14.028746 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:07:14.028712    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6c77cb6fc8-vg979" event={"ID":"25d3799f-6859-473b-af72-56716dfa3dbb","Type":"ContainerStarted","Data":"9f3771adeb9d5d292071ee418cb721034fb873f06271aed2e8f6edadfcef8075"}
Sep 16 14:07:16.036623 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:07:16.036587    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6c77cb6fc8-vg979" event={"ID":"25d3799f-6859-473b-af72-56716dfa3dbb","Type":"ContainerStarted","Data":"5dc3d62427e531235f3b229f0b3b1c2614130eaff9e1374636a6bbf7938f69d1"}
Sep 16 14:07:16.037021 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:07:16.036650    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6c77cb6fc8-vg979" event={"ID":"25d3799f-6859-473b-af72-56716dfa3dbb","Type":"ContainerStarted","Data":"27637bb6193509d42abc9832449c85655c1eda0af38552aea6bee3b1be417fa2"}
Sep 16 14:09:27.307618 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:09:27.307595    2580 kubelet.go:1648] "Image garbage collection succeeded"
Sep 16 14:09:50.249001 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:09:50.248937    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-6c77cb6fc8-vg979" podStartSLOduration=156.89914836 podStartE2EDuration="2m42.248915289s" totalImagesPullingTime="5.349766929s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:07:08 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:07:16.079933538 +0000 UTC m=+169.095043398" watchObservedRunningTime="2026-09-16 14:09:50.248915289 +0000 UTC m=+323.264025151"
Sep 16 14:09:50.251519 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:09:50.250030    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgtcns"]
Sep 16 14:09:50.252908 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:09:50.252889    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgtcns"
Sep 16 14:09:50.259292 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:09:50.259268    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 16 14:09:50.259369 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:09:50.259273    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 16 14:09:50.259945 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:09:50.259926    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-gdgch\""
Sep 16 14:09:50.273484 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:09:50.273459    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgtcns"]
Sep 16 14:09:50.371047 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:09:50.371012    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grjnh\" (UniqueName: \"kubernetes.io/projected/dda6b0b3-2b57-471d-8e7d-9b8f092bd4e1-kube-api-access-grjnh\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgtcns\" (UID: \"dda6b0b3-2b57-471d-8e7d-9b8f092bd4e1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgtcns"
Sep 16 14:09:50.371216 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:09:50.371059    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dda6b0b3-2b57-471d-8e7d-9b8f092bd4e1-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgtcns\" (UID: \"dda6b0b3-2b57-471d-8e7d-9b8f092bd4e1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgtcns"
Sep 16 14:09:50.371216 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:09:50.371121    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dda6b0b3-2b57-471d-8e7d-9b8f092bd4e1-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgtcns\" (UID: \"dda6b0b3-2b57-471d-8e7d-9b8f092bd4e1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgtcns"
Sep 16 14:09:50.472465 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:09:50.472423    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grjnh\" (UniqueName: \"kubernetes.io/projected/dda6b0b3-2b57-471d-8e7d-9b8f092bd4e1-kube-api-access-grjnh\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgtcns\" (UID: \"dda6b0b3-2b57-471d-8e7d-9b8f092bd4e1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgtcns"
Sep 16 14:09:50.472672 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:09:50.472478    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dda6b0b3-2b57-471d-8e7d-9b8f092bd4e1-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgtcns\" (UID: \"dda6b0b3-2b57-471d-8e7d-9b8f092bd4e1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgtcns"
Sep 16 14:09:50.472672 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:09:50.472527    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dda6b0b3-2b57-471d-8e7d-9b8f092bd4e1-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgtcns\" (UID: \"dda6b0b3-2b57-471d-8e7d-9b8f092bd4e1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgtcns"
Sep 16 14:09:50.472847 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:09:50.472825    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dda6b0b3-2b57-471d-8e7d-9b8f092bd4e1-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgtcns\" (UID: \"dda6b0b3-2b57-471d-8e7d-9b8f092bd4e1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgtcns"
Sep 16 14:09:50.472927 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:09:50.472907    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dda6b0b3-2b57-471d-8e7d-9b8f092bd4e1-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgtcns\" (UID: \"dda6b0b3-2b57-471d-8e7d-9b8f092bd4e1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgtcns"
Sep 16 14:09:50.491647 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:09:50.491616    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-grjnh\" (UniqueName: \"kubernetes.io/projected/dda6b0b3-2b57-471d-8e7d-9b8f092bd4e1-kube-api-access-grjnh\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgtcns\" (UID: \"dda6b0b3-2b57-471d-8e7d-9b8f092bd4e1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgtcns"
Sep 16 14:09:50.561914 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:09:50.561822    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgtcns"
Sep 16 14:09:50.837236 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:09:50.837159    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgtcns"]
Sep 16 14:09:50.840607 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:09:50.840579    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddda6b0b3_2b57_471d_8e7d_9b8f092bd4e1.slice/crio-fd30b8b0f9836c91c3342f5bfacb61a5db61f8938f77ec9b5db3893659652cf7 WatchSource:0}: Error finding container fd30b8b0f9836c91c3342f5bfacb61a5db61f8938f77ec9b5db3893659652cf7: Status 404 returned error can't find the container with id fd30b8b0f9836c91c3342f5bfacb61a5db61f8938f77ec9b5db3893659652cf7
Sep 16 14:09:50.842526 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:09:50.842493    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 14:09:51.520323 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:09:51.520285    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgtcns" event={"ID":"dda6b0b3-2b57-471d-8e7d-9b8f092bd4e1","Type":"ContainerStarted","Data":"fd30b8b0f9836c91c3342f5bfacb61a5db61f8938f77ec9b5db3893659652cf7"}
Sep 16 14:09:59.550830 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:09:59.550798    2580 generic.go:356] "Generic (PLEG): container finished" podID="dda6b0b3-2b57-471d-8e7d-9b8f092bd4e1" containerID="0aaecee84138a51bd7e17a291bcb8260519fe8088c4c62fc6f9acac33a944c92" exitCode=0
Sep 16 14:09:59.551196 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:09:59.550884    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgtcns" event={"ID":"dda6b0b3-2b57-471d-8e7d-9b8f092bd4e1","Type":"ContainerDied","Data":"0aaecee84138a51bd7e17a291bcb8260519fe8088c4c62fc6f9acac33a944c92"}
Sep 16 14:10:00.842649 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:00.842608    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-d4cmd"]
Sep 16 14:10:00.845931 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:00.845905    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-d4cmd"
Sep 16 14:10:00.847965 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:00.847935    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 16 14:10:00.850523 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:00.850489    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 16 14:10:00.851879 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:00.851860    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-rwtmt\""
Sep 16 14:10:00.860401 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:00.860379    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-d4cmd"]
Sep 16 14:10:00.952458 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:00.952421    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/247c16df-0d6c-40ab-a71e-ff8c2cb09224-bound-sa-token\") pod \"cert-manager-55b4d954d5-d4cmd\" (UID: \"247c16df-0d6c-40ab-a71e-ff8c2cb09224\") " pod="cert-manager/cert-manager-55b4d954d5-d4cmd"
Sep 16 14:10:00.952667 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:00.952468    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pj2cm\" (UniqueName: \"kubernetes.io/projected/247c16df-0d6c-40ab-a71e-ff8c2cb09224-kube-api-access-pj2cm\") pod \"cert-manager-55b4d954d5-d4cmd\" (UID: \"247c16df-0d6c-40ab-a71e-ff8c2cb09224\") " pod="cert-manager/cert-manager-55b4d954d5-d4cmd"
Sep 16 14:10:01.053733 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:01.053686    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/247c16df-0d6c-40ab-a71e-ff8c2cb09224-bound-sa-token\") pod \"cert-manager-55b4d954d5-d4cmd\" (UID: \"247c16df-0d6c-40ab-a71e-ff8c2cb09224\") " pod="cert-manager/cert-manager-55b4d954d5-d4cmd"
Sep 16 14:10:01.053733 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:01.053737    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pj2cm\" (UniqueName: \"kubernetes.io/projected/247c16df-0d6c-40ab-a71e-ff8c2cb09224-kube-api-access-pj2cm\") pod \"cert-manager-55b4d954d5-d4cmd\" (UID: \"247c16df-0d6c-40ab-a71e-ff8c2cb09224\") " pod="cert-manager/cert-manager-55b4d954d5-d4cmd"
Sep 16 14:10:01.062869 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:01.062809    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pj2cm\" (UniqueName: \"kubernetes.io/projected/247c16df-0d6c-40ab-a71e-ff8c2cb09224-kube-api-access-pj2cm\") pod \"cert-manager-55b4d954d5-d4cmd\" (UID: \"247c16df-0d6c-40ab-a71e-ff8c2cb09224\") " pod="cert-manager/cert-manager-55b4d954d5-d4cmd"
Sep 16 14:10:01.065007 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:01.064982    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/247c16df-0d6c-40ab-a71e-ff8c2cb09224-bound-sa-token\") pod \"cert-manager-55b4d954d5-d4cmd\" (UID: \"247c16df-0d6c-40ab-a71e-ff8c2cb09224\") " pod="cert-manager/cert-manager-55b4d954d5-d4cmd"
Sep 16 14:10:01.155211 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:01.155118    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-d4cmd"
Sep 16 14:10:01.436932 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:01.436901    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-d4cmd"]
Sep 16 14:10:02.112855 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:10:02.112825    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod247c16df_0d6c_40ab_a71e_ff8c2cb09224.slice/crio-280349feb7ea60d7a93e93f0880ff8c75b88f00ce5f19215c855cafeef161e54 WatchSource:0}: Error finding container 280349feb7ea60d7a93e93f0880ff8c75b88f00ce5f19215c855cafeef161e54: Status 404 returned error can't find the container with id 280349feb7ea60d7a93e93f0880ff8c75b88f00ce5f19215c855cafeef161e54
Sep 16 14:10:02.562556 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:02.562521    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-d4cmd" event={"ID":"247c16df-0d6c-40ab-a71e-ff8c2cb09224","Type":"ContainerStarted","Data":"280349feb7ea60d7a93e93f0880ff8c75b88f00ce5f19215c855cafeef161e54"}
Sep 16 14:10:02.564312 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:02.564280    2580 generic.go:356] "Generic (PLEG): container finished" podID="dda6b0b3-2b57-471d-8e7d-9b8f092bd4e1" containerID="d5abbbccbd58d020adcf56b9e0d0daadf32c76ce8b4c9a3ee797916f56911e0a" exitCode=0
Sep 16 14:10:02.564431 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:02.564321    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgtcns" event={"ID":"dda6b0b3-2b57-471d-8e7d-9b8f092bd4e1","Type":"ContainerDied","Data":"d5abbbccbd58d020adcf56b9e0d0daadf32c76ce8b4c9a3ee797916f56911e0a"}
Sep 16 14:10:10.598956 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:10.598919    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-d4cmd" event={"ID":"247c16df-0d6c-40ab-a71e-ff8c2cb09224","Type":"ContainerStarted","Data":"042c2ee540e66835605ae88f6aaa14ab1aa15fa63562dcaeac43373dc9bf59cb"}
Sep 16 14:10:10.601058 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:10.601035    2580 generic.go:356] "Generic (PLEG): container finished" podID="dda6b0b3-2b57-471d-8e7d-9b8f092bd4e1" containerID="d5c1bfec8cc2f4ad09853fd21ce54e8bb20be8990d1eb56cd7719e4c0d8c68ef" exitCode=0
Sep 16 14:10:10.601174 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:10.601084    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgtcns" event={"ID":"dda6b0b3-2b57-471d-8e7d-9b8f092bd4e1","Type":"ContainerDied","Data":"d5c1bfec8cc2f4ad09853fd21ce54e8bb20be8990d1eb56cd7719e4c0d8c68ef"}
Sep 16 14:10:10.616527 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:10.616468    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-d4cmd" podStartSLOduration=2.94814519 podStartE2EDuration="10.616452667s" totalImagesPullingTime="7.668307477s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:10:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:10:10.615246518 +0000 UTC m=+343.630356390" watchObservedRunningTime="2026-09-16 14:10:10.616452667 +0000 UTC m=+343.631562528"
Sep 16 14:10:11.720047 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:11.720022    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgtcns"
Sep 16 14:10:11.722120 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:11.722101    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dda6b0b3-2b57-471d-8e7d-9b8f092bd4e1-util\") pod \"dda6b0b3-2b57-471d-8e7d-9b8f092bd4e1\" (UID: \"dda6b0b3-2b57-471d-8e7d-9b8f092bd4e1\") "
Sep 16 14:10:11.722188 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:11.722141    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-grjnh\" (UniqueName: \"kubernetes.io/projected/dda6b0b3-2b57-471d-8e7d-9b8f092bd4e1-kube-api-access-grjnh\") pod \"dda6b0b3-2b57-471d-8e7d-9b8f092bd4e1\" (UID: \"dda6b0b3-2b57-471d-8e7d-9b8f092bd4e1\") "
Sep 16 14:10:11.722188 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:11.722162    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dda6b0b3-2b57-471d-8e7d-9b8f092bd4e1-bundle\") pod \"dda6b0b3-2b57-471d-8e7d-9b8f092bd4e1\" (UID: \"dda6b0b3-2b57-471d-8e7d-9b8f092bd4e1\") "
Sep 16 14:10:11.722515 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:11.722480    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dda6b0b3-2b57-471d-8e7d-9b8f092bd4e1-bundle" (OuterVolumeSpecName: "bundle") pod "dda6b0b3-2b57-471d-8e7d-9b8f092bd4e1" (UID: "dda6b0b3-2b57-471d-8e7d-9b8f092bd4e1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:10:11.724199 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:11.724179    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dda6b0b3-2b57-471d-8e7d-9b8f092bd4e1-kube-api-access-grjnh" (OuterVolumeSpecName: "kube-api-access-grjnh") pod "dda6b0b3-2b57-471d-8e7d-9b8f092bd4e1" (UID: "dda6b0b3-2b57-471d-8e7d-9b8f092bd4e1"). InnerVolumeSpecName "kube-api-access-grjnh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:10:11.726179 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:11.726157    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dda6b0b3-2b57-471d-8e7d-9b8f092bd4e1-util" (OuterVolumeSpecName: "util") pod "dda6b0b3-2b57-471d-8e7d-9b8f092bd4e1" (UID: "dda6b0b3-2b57-471d-8e7d-9b8f092bd4e1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:10:11.822681 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:11.822650    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-grjnh\" (UniqueName: \"kubernetes.io/projected/dda6b0b3-2b57-471d-8e7d-9b8f092bd4e1-kube-api-access-grjnh\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:10:11.822681 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:11.822677    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dda6b0b3-2b57-471d-8e7d-9b8f092bd4e1-bundle\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:10:11.822681 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:11.822685    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dda6b0b3-2b57-471d-8e7d-9b8f092bd4e1-util\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:10:12.610075 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:12.610039    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgtcns" event={"ID":"dda6b0b3-2b57-471d-8e7d-9b8f092bd4e1","Type":"ContainerDied","Data":"fd30b8b0f9836c91c3342f5bfacb61a5db61f8938f77ec9b5db3893659652cf7"}
Sep 16 14:10:12.610238 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:12.610087    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fd30b8b0f9836c91c3342f5bfacb61a5db61f8938f77ec9b5db3893659652cf7"
Sep 16 14:10:12.610238 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:12.610046    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgtcns"
Sep 16 14:10:35.310206 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:35.310130    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02g5qm"]
Sep 16 14:10:35.310673 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:35.310382    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dda6b0b3-2b57-471d-8e7d-9b8f092bd4e1" containerName="util"
Sep 16 14:10:35.310673 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:35.310393    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="dda6b0b3-2b57-471d-8e7d-9b8f092bd4e1" containerName="util"
Sep 16 14:10:35.310673 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:35.310408    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dda6b0b3-2b57-471d-8e7d-9b8f092bd4e1" containerName="pull"
Sep 16 14:10:35.310673 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:35.310416    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="dda6b0b3-2b57-471d-8e7d-9b8f092bd4e1" containerName="pull"
Sep 16 14:10:35.310673 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:35.310428    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dda6b0b3-2b57-471d-8e7d-9b8f092bd4e1" containerName="extract"
Sep 16 14:10:35.310673 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:35.310435    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="dda6b0b3-2b57-471d-8e7d-9b8f092bd4e1" containerName="extract"
Sep 16 14:10:35.310673 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:35.310482    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="dda6b0b3-2b57-471d-8e7d-9b8f092bd4e1" containerName="extract"
Sep 16 14:10:35.313540 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:35.313523    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02g5qm"
Sep 16 14:10:35.316043 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:35.316015    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 16 14:10:35.317924 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:35.317904    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 16 14:10:35.318047 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:35.317944    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-gdgch\""
Sep 16 14:10:35.323856 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:35.323830    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02g5qm"]
Sep 16 14:10:35.363798 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:35.363772    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qddm5\" (UniqueName: \"kubernetes.io/projected/175c9622-895f-4467-bb33-6462359de0c1-kube-api-access-qddm5\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02g5qm\" (UID: \"175c9622-895f-4467-bb33-6462359de0c1\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02g5qm"
Sep 16 14:10:35.363928 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:35.363812    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/175c9622-895f-4467-bb33-6462359de0c1-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02g5qm\" (UID: \"175c9622-895f-4467-bb33-6462359de0c1\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02g5qm"
Sep 16 14:10:35.363928 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:35.363835    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/175c9622-895f-4467-bb33-6462359de0c1-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02g5qm\" (UID: \"175c9622-895f-4467-bb33-6462359de0c1\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02g5qm"
Sep 16 14:10:35.464914 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:35.464875    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/175c9622-895f-4467-bb33-6462359de0c1-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02g5qm\" (UID: \"175c9622-895f-4467-bb33-6462359de0c1\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02g5qm"
Sep 16 14:10:35.465107 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:35.464934    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qddm5\" (UniqueName: \"kubernetes.io/projected/175c9622-895f-4467-bb33-6462359de0c1-kube-api-access-qddm5\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02g5qm\" (UID: \"175c9622-895f-4467-bb33-6462359de0c1\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02g5qm"
Sep 16 14:10:35.465107 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:35.465044    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/175c9622-895f-4467-bb33-6462359de0c1-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02g5qm\" (UID: \"175c9622-895f-4467-bb33-6462359de0c1\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02g5qm"
Sep 16 14:10:35.465323 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:35.465299    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/175c9622-895f-4467-bb33-6462359de0c1-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02g5qm\" (UID: \"175c9622-895f-4467-bb33-6462359de0c1\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02g5qm"
Sep 16 14:10:35.465387 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:35.465342    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/175c9622-895f-4467-bb33-6462359de0c1-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02g5qm\" (UID: \"175c9622-895f-4467-bb33-6462359de0c1\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02g5qm"
Sep 16 14:10:35.472789 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:35.472765    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qddm5\" (UniqueName: \"kubernetes.io/projected/175c9622-895f-4467-bb33-6462359de0c1-kube-api-access-qddm5\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02g5qm\" (UID: \"175c9622-895f-4467-bb33-6462359de0c1\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02g5qm"
Sep 16 14:10:35.622095 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:35.622012    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02g5qm"
Sep 16 14:10:35.899434 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:35.899409    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02g5qm"]
Sep 16 14:10:35.901759 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:10:35.901728    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod175c9622_895f_4467_bb33_6462359de0c1.slice/crio-3c8dcfa6664eca53e7a838ece753ccffdbef86c4f53f7a2ca0e0057d6d92495a WatchSource:0}: Error finding container 3c8dcfa6664eca53e7a838ece753ccffdbef86c4f53f7a2ca0e0057d6d92495a: Status 404 returned error can't find the container with id 3c8dcfa6664eca53e7a838ece753ccffdbef86c4f53f7a2ca0e0057d6d92495a
Sep 16 14:10:36.695516 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:36.695466    2580 generic.go:356] "Generic (PLEG): container finished" podID="175c9622-895f-4467-bb33-6462359de0c1" containerID="1c6299c2bb8d1f93d711f5bba0b0d62be9c093264a10b2b83e24e9aa2126d6ef" exitCode=0
Sep 16 14:10:36.695893 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:36.695554    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02g5qm" event={"ID":"175c9622-895f-4467-bb33-6462359de0c1","Type":"ContainerDied","Data":"1c6299c2bb8d1f93d711f5bba0b0d62be9c093264a10b2b83e24e9aa2126d6ef"}
Sep 16 14:10:36.695893 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:36.695602    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02g5qm" event={"ID":"175c9622-895f-4467-bb33-6462359de0c1","Type":"ContainerStarted","Data":"3c8dcfa6664eca53e7a838ece753ccffdbef86c4f53f7a2ca0e0057d6d92495a"}
Sep 16 14:10:36.979163 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:36.979135    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-77f6d9877-g8lwc"]
Sep 16 14:10:36.982310 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:36.982293    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-77f6d9877-g8lwc"
Sep 16 14:10:36.987686 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:36.987662    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 16 14:10:36.987901 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:36.987886    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 16 14:10:36.988001 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:36.987985    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-kkfv7\""
Sep 16 14:10:36.988082 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:36.987983    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 16 14:10:36.988082 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:36.988065    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 16 14:10:37.010150 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:37.010125    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-77f6d9877-g8lwc"]
Sep 16 14:10:37.078380 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:37.078343    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/52f3de3b-2520-41d1-9d34-0ad44c966111-apiservice-cert\") pod \"opendatahub-operator-controller-manager-77f6d9877-g8lwc\" (UID: \"52f3de3b-2520-41d1-9d34-0ad44c966111\") " pod="opendatahub/opendatahub-operator-controller-manager-77f6d9877-g8lwc"
Sep 16 14:10:37.078563 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:37.078385    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8xbc\" (UniqueName: \"kubernetes.io/projected/52f3de3b-2520-41d1-9d34-0ad44c966111-kube-api-access-f8xbc\") pod \"opendatahub-operator-controller-manager-77f6d9877-g8lwc\" (UID: \"52f3de3b-2520-41d1-9d34-0ad44c966111\") " pod="opendatahub/opendatahub-operator-controller-manager-77f6d9877-g8lwc"
Sep 16 14:10:37.078563 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:37.078475    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/52f3de3b-2520-41d1-9d34-0ad44c966111-webhook-cert\") pod \"opendatahub-operator-controller-manager-77f6d9877-g8lwc\" (UID: \"52f3de3b-2520-41d1-9d34-0ad44c966111\") " pod="opendatahub/opendatahub-operator-controller-manager-77f6d9877-g8lwc"
Sep 16 14:10:37.078563 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:37.078543    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/52f3de3b-2520-41d1-9d34-0ad44c966111-manifests\") pod \"opendatahub-operator-controller-manager-77f6d9877-g8lwc\" (UID: \"52f3de3b-2520-41d1-9d34-0ad44c966111\") " pod="opendatahub/opendatahub-operator-controller-manager-77f6d9877-g8lwc"
Sep 16 14:10:37.078684 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:37.078576    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/52f3de3b-2520-41d1-9d34-0ad44c966111-tmp\") pod \"opendatahub-operator-controller-manager-77f6d9877-g8lwc\" (UID: \"52f3de3b-2520-41d1-9d34-0ad44c966111\") " pod="opendatahub/opendatahub-operator-controller-manager-77f6d9877-g8lwc"
Sep 16 14:10:37.179725 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:37.179689    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/52f3de3b-2520-41d1-9d34-0ad44c966111-tmp\") pod \"opendatahub-operator-controller-manager-77f6d9877-g8lwc\" (UID: \"52f3de3b-2520-41d1-9d34-0ad44c966111\") " pod="opendatahub/opendatahub-operator-controller-manager-77f6d9877-g8lwc"
Sep 16 14:10:37.179907 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:37.179742    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/52f3de3b-2520-41d1-9d34-0ad44c966111-apiservice-cert\") pod \"opendatahub-operator-controller-manager-77f6d9877-g8lwc\" (UID: \"52f3de3b-2520-41d1-9d34-0ad44c966111\") " pod="opendatahub/opendatahub-operator-controller-manager-77f6d9877-g8lwc"
Sep 16 14:10:37.179907 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:37.179772    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8xbc\" (UniqueName: \"kubernetes.io/projected/52f3de3b-2520-41d1-9d34-0ad44c966111-kube-api-access-f8xbc\") pod \"opendatahub-operator-controller-manager-77f6d9877-g8lwc\" (UID: \"52f3de3b-2520-41d1-9d34-0ad44c966111\") " pod="opendatahub/opendatahub-operator-controller-manager-77f6d9877-g8lwc"
Sep 16 14:10:37.179907 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:37.179815    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/52f3de3b-2520-41d1-9d34-0ad44c966111-webhook-cert\") pod \"opendatahub-operator-controller-manager-77f6d9877-g8lwc\" (UID: \"52f3de3b-2520-41d1-9d34-0ad44c966111\") " pod="opendatahub/opendatahub-operator-controller-manager-77f6d9877-g8lwc"
Sep 16 14:10:37.179907 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:37.179840    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/52f3de3b-2520-41d1-9d34-0ad44c966111-manifests\") pod \"opendatahub-operator-controller-manager-77f6d9877-g8lwc\" (UID: \"52f3de3b-2520-41d1-9d34-0ad44c966111\") " pod="opendatahub/opendatahub-operator-controller-manager-77f6d9877-g8lwc"
Sep 16 14:10:37.180218 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:37.180196    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/52f3de3b-2520-41d1-9d34-0ad44c966111-manifests\") pod \"opendatahub-operator-controller-manager-77f6d9877-g8lwc\" (UID: \"52f3de3b-2520-41d1-9d34-0ad44c966111\") " pod="opendatahub/opendatahub-operator-controller-manager-77f6d9877-g8lwc"
Sep 16 14:10:37.182044 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:37.182006    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/52f3de3b-2520-41d1-9d34-0ad44c966111-tmp\") pod \"opendatahub-operator-controller-manager-77f6d9877-g8lwc\" (UID: \"52f3de3b-2520-41d1-9d34-0ad44c966111\") " pod="opendatahub/opendatahub-operator-controller-manager-77f6d9877-g8lwc"
Sep 16 14:10:37.182183 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:37.182167    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/52f3de3b-2520-41d1-9d34-0ad44c966111-apiservice-cert\") pod \"opendatahub-operator-controller-manager-77f6d9877-g8lwc\" (UID: \"52f3de3b-2520-41d1-9d34-0ad44c966111\") " pod="opendatahub/opendatahub-operator-controller-manager-77f6d9877-g8lwc"
Sep 16 14:10:37.182369 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:37.182350    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/52f3de3b-2520-41d1-9d34-0ad44c966111-webhook-cert\") pod \"opendatahub-operator-controller-manager-77f6d9877-g8lwc\" (UID: \"52f3de3b-2520-41d1-9d34-0ad44c966111\") " pod="opendatahub/opendatahub-operator-controller-manager-77f6d9877-g8lwc"
Sep 16 14:10:37.190565 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:37.190546    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8xbc\" (UniqueName: \"kubernetes.io/projected/52f3de3b-2520-41d1-9d34-0ad44c966111-kube-api-access-f8xbc\") pod \"opendatahub-operator-controller-manager-77f6d9877-g8lwc\" (UID: \"52f3de3b-2520-41d1-9d34-0ad44c966111\") " pod="opendatahub/opendatahub-operator-controller-manager-77f6d9877-g8lwc"
Sep 16 14:10:37.292288 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:37.292196    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-77f6d9877-g8lwc"
Sep 16 14:10:37.570912 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:37.570879    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-77f6d9877-g8lwc"]
Sep 16 14:10:37.573187 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:10:37.573155    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52f3de3b_2520_41d1_9d34_0ad44c966111.slice/crio-b51919c29924da80dbea3073bec288de8dc3e32ff39d5830d35fa3043abe5596 WatchSource:0}: Error finding container b51919c29924da80dbea3073bec288de8dc3e32ff39d5830d35fa3043abe5596: Status 404 returned error can't find the container with id b51919c29924da80dbea3073bec288de8dc3e32ff39d5830d35fa3043abe5596
Sep 16 14:10:37.699646 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:37.699612    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-77f6d9877-g8lwc" event={"ID":"52f3de3b-2520-41d1-9d34-0ad44c966111","Type":"ContainerStarted","Data":"b51919c29924da80dbea3073bec288de8dc3e32ff39d5830d35fa3043abe5596"}
Sep 16 14:10:38.707309 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:38.707234    2580 generic.go:356] "Generic (PLEG): container finished" podID="175c9622-895f-4467-bb33-6462359de0c1" containerID="bae338b94084a18d78819aadc4508458742ccfa57de815d787a705b1043ec5eb" exitCode=0
Sep 16 14:10:38.707793 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:38.707343    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02g5qm" event={"ID":"175c9622-895f-4467-bb33-6462359de0c1","Type":"ContainerDied","Data":"bae338b94084a18d78819aadc4508458742ccfa57de815d787a705b1043ec5eb"}
Sep 16 14:10:39.714476 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:39.714440    2580 generic.go:356] "Generic (PLEG): container finished" podID="175c9622-895f-4467-bb33-6462359de0c1" containerID="5923b36c35ce0c8297218217f003149390e501f1262a87c5bba6c701bda54f71" exitCode=0
Sep 16 14:10:39.714952 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:39.714536    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02g5qm" event={"ID":"175c9622-895f-4467-bb33-6462359de0c1","Type":"ContainerDied","Data":"5923b36c35ce0c8297218217f003149390e501f1262a87c5bba6c701bda54f71"}
Sep 16 14:10:40.842888 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:40.842867    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02g5qm"
Sep 16 14:10:40.907877 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:40.907840    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qddm5\" (UniqueName: \"kubernetes.io/projected/175c9622-895f-4467-bb33-6462359de0c1-kube-api-access-qddm5\") pod \"175c9622-895f-4467-bb33-6462359de0c1\" (UID: \"175c9622-895f-4467-bb33-6462359de0c1\") "
Sep 16 14:10:40.907877 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:40.907882    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/175c9622-895f-4467-bb33-6462359de0c1-bundle\") pod \"175c9622-895f-4467-bb33-6462359de0c1\" (UID: \"175c9622-895f-4467-bb33-6462359de0c1\") "
Sep 16 14:10:40.908117 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:40.907911    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/175c9622-895f-4467-bb33-6462359de0c1-util\") pod \"175c9622-895f-4467-bb33-6462359de0c1\" (UID: \"175c9622-895f-4467-bb33-6462359de0c1\") "
Sep 16 14:10:40.908619 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:40.908595    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/175c9622-895f-4467-bb33-6462359de0c1-bundle" (OuterVolumeSpecName: "bundle") pod "175c9622-895f-4467-bb33-6462359de0c1" (UID: "175c9622-895f-4467-bb33-6462359de0c1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:10:40.910087 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:40.910061    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/175c9622-895f-4467-bb33-6462359de0c1-kube-api-access-qddm5" (OuterVolumeSpecName: "kube-api-access-qddm5") pod "175c9622-895f-4467-bb33-6462359de0c1" (UID: "175c9622-895f-4467-bb33-6462359de0c1"). InnerVolumeSpecName "kube-api-access-qddm5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:10:40.914057 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:40.914018    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/175c9622-895f-4467-bb33-6462359de0c1-util" (OuterVolumeSpecName: "util") pod "175c9622-895f-4467-bb33-6462359de0c1" (UID: "175c9622-895f-4467-bb33-6462359de0c1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:10:41.009392 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:41.009314    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/175c9622-895f-4467-bb33-6462359de0c1-bundle\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:10:41.009392 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:41.009342    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/175c9622-895f-4467-bb33-6462359de0c1-util\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:10:41.009392 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:41.009351    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qddm5\" (UniqueName: \"kubernetes.io/projected/175c9622-895f-4467-bb33-6462359de0c1-kube-api-access-qddm5\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:10:41.726164 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:41.726065    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-77f6d9877-g8lwc" event={"ID":"52f3de3b-2520-41d1-9d34-0ad44c966111","Type":"ContainerStarted","Data":"206cb277b30c9f9e9a699ad483c7ec5ec4f47548b48e85a8d8a35a4c2791255d"}
Sep 16 14:10:41.728139 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:41.728116    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02g5qm" event={"ID":"175c9622-895f-4467-bb33-6462359de0c1","Type":"ContainerDied","Data":"3c8dcfa6664eca53e7a838ece753ccffdbef86c4f53f7a2ca0e0057d6d92495a"}
Sep 16 14:10:41.728251 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:41.728150    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3c8dcfa6664eca53e7a838ece753ccffdbef86c4f53f7a2ca0e0057d6d92495a"
Sep 16 14:10:41.728251 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:41.728149    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02g5qm"
Sep 16 14:10:42.733003 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:42.732970    2580 generic.go:356] "Generic (PLEG): container finished" podID="52f3de3b-2520-41d1-9d34-0ad44c966111" containerID="206cb277b30c9f9e9a699ad483c7ec5ec4f47548b48e85a8d8a35a4c2791255d" exitCode=0
Sep 16 14:10:42.733451 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:42.733061    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-77f6d9877-g8lwc" event={"ID":"52f3de3b-2520-41d1-9d34-0ad44c966111","Type":"ContainerDied","Data":"206cb277b30c9f9e9a699ad483c7ec5ec4f47548b48e85a8d8a35a4c2791255d"}
Sep 16 14:10:44.741300 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:44.741265    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-77f6d9877-g8lwc" event={"ID":"52f3de3b-2520-41d1-9d34-0ad44c966111","Type":"ContainerStarted","Data":"513e41bae64d9541262c55b4fc46a50a4c8ab5e18c55691d489209de84179f80"}
Sep 16 14:10:44.741702 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:44.741375    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-77f6d9877-g8lwc"
Sep 16 14:10:44.765697 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:44.765652    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-77f6d9877-g8lwc" podStartSLOduration=3.004236688 podStartE2EDuration="8.765639837s" totalImagesPullingTime="4.734199208s" totalInitContainerRuntime="1.027203941s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:10:36 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:10:44.764930365 +0000 UTC m=+377.780040226" watchObservedRunningTime="2026-09-16 14:10:44.765639837 +0000 UTC m=+377.780749696"
Sep 16 14:10:55.747018 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:10:55.746988    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-77f6d9877-g8lwc"
Sep 16 14:11:01.381530 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:01.381481    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5b574975b8-rkf8t"]
Sep 16 14:11:01.381903 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:01.381764    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="175c9622-895f-4467-bb33-6462359de0c1" containerName="pull"
Sep 16 14:11:01.381903 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:01.381775    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="175c9622-895f-4467-bb33-6462359de0c1" containerName="pull"
Sep 16 14:11:01.381903 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:01.381797    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="175c9622-895f-4467-bb33-6462359de0c1" containerName="util"
Sep 16 14:11:01.381903 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:01.381803    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="175c9622-895f-4467-bb33-6462359de0c1" containerName="util"
Sep 16 14:11:01.381903 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:01.381808    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="175c9622-895f-4467-bb33-6462359de0c1" containerName="extract"
Sep 16 14:11:01.381903 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:01.381814    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="175c9622-895f-4467-bb33-6462359de0c1" containerName="extract"
Sep 16 14:11:01.381903 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:01.381853    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="175c9622-895f-4467-bb33-6462359de0c1" containerName="extract"
Sep 16 14:11:01.384700 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:01.384685    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5b574975b8-rkf8t"
Sep 16 14:11:01.387863 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:01.387837    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 16 14:11:01.387979 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:01.387900    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 16 14:11:01.388084 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:01.388066    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 16 14:11:01.388860 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:01.388840    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 16 14:11:01.407112 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:01.407089    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5b574975b8-rkf8t"]
Sep 16 14:11:01.449615 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:01.449591    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d8dbea9a-e715-4bd9-88d2-7aea66a83a63-tmp\") pod \"kube-auth-proxy-5b574975b8-rkf8t\" (UID: \"d8dbea9a-e715-4bd9-88d2-7aea66a83a63\") " pod="openshift-ingress/kube-auth-proxy-5b574975b8-rkf8t"
Sep 16 14:11:01.449723 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:01.449646    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8bc4\" (UniqueName: \"kubernetes.io/projected/d8dbea9a-e715-4bd9-88d2-7aea66a83a63-kube-api-access-r8bc4\") pod \"kube-auth-proxy-5b574975b8-rkf8t\" (UID: \"d8dbea9a-e715-4bd9-88d2-7aea66a83a63\") " pod="openshift-ingress/kube-auth-proxy-5b574975b8-rkf8t"
Sep 16 14:11:01.449723 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:01.449691    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d8dbea9a-e715-4bd9-88d2-7aea66a83a63-tls-certs\") pod \"kube-auth-proxy-5b574975b8-rkf8t\" (UID: \"d8dbea9a-e715-4bd9-88d2-7aea66a83a63\") " pod="openshift-ingress/kube-auth-proxy-5b574975b8-rkf8t"
Sep 16 14:11:01.550372 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:01.550336    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8bc4\" (UniqueName: \"kubernetes.io/projected/d8dbea9a-e715-4bd9-88d2-7aea66a83a63-kube-api-access-r8bc4\") pod \"kube-auth-proxy-5b574975b8-rkf8t\" (UID: \"d8dbea9a-e715-4bd9-88d2-7aea66a83a63\") " pod="openshift-ingress/kube-auth-proxy-5b574975b8-rkf8t"
Sep 16 14:11:01.550573 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:01.550381    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d8dbea9a-e715-4bd9-88d2-7aea66a83a63-tls-certs\") pod \"kube-auth-proxy-5b574975b8-rkf8t\" (UID: \"d8dbea9a-e715-4bd9-88d2-7aea66a83a63\") " pod="openshift-ingress/kube-auth-proxy-5b574975b8-rkf8t"
Sep 16 14:11:01.550573 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:01.550415    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d8dbea9a-e715-4bd9-88d2-7aea66a83a63-tmp\") pod \"kube-auth-proxy-5b574975b8-rkf8t\" (UID: \"d8dbea9a-e715-4bd9-88d2-7aea66a83a63\") " pod="openshift-ingress/kube-auth-proxy-5b574975b8-rkf8t"
Sep 16 14:11:01.550573 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:11:01.550543    2580 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Sep 16 14:11:01.550742 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:11:01.550617    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8dbea9a-e715-4bd9-88d2-7aea66a83a63-tls-certs podName:d8dbea9a-e715-4bd9-88d2-7aea66a83a63 nodeName:}" failed. No retries permitted until 2026-09-16 14:11:02.050600186 +0000 UTC m=+395.065710024 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/d8dbea9a-e715-4bd9-88d2-7aea66a83a63-tls-certs") pod "kube-auth-proxy-5b574975b8-rkf8t" (UID: "d8dbea9a-e715-4bd9-88d2-7aea66a83a63") : secret "kube-auth-proxy-tls" not found
Sep 16 14:11:01.552696 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:01.552674    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d8dbea9a-e715-4bd9-88d2-7aea66a83a63-tmp\") pod \"kube-auth-proxy-5b574975b8-rkf8t\" (UID: \"d8dbea9a-e715-4bd9-88d2-7aea66a83a63\") " pod="openshift-ingress/kube-auth-proxy-5b574975b8-rkf8t"
Sep 16 14:11:01.578532 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:01.578512    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8bc4\" (UniqueName: \"kubernetes.io/projected/d8dbea9a-e715-4bd9-88d2-7aea66a83a63-kube-api-access-r8bc4\") pod \"kube-auth-proxy-5b574975b8-rkf8t\" (UID: \"d8dbea9a-e715-4bd9-88d2-7aea66a83a63\") " pod="openshift-ingress/kube-auth-proxy-5b574975b8-rkf8t"
Sep 16 14:11:02.052871 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:02.052833    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d8dbea9a-e715-4bd9-88d2-7aea66a83a63-tls-certs\") pod \"kube-auth-proxy-5b574975b8-rkf8t\" (UID: \"d8dbea9a-e715-4bd9-88d2-7aea66a83a63\") " pod="openshift-ingress/kube-auth-proxy-5b574975b8-rkf8t"
Sep 16 14:11:02.055170 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:02.055141    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d8dbea9a-e715-4bd9-88d2-7aea66a83a63-tls-certs\") pod \"kube-auth-proxy-5b574975b8-rkf8t\" (UID: \"d8dbea9a-e715-4bd9-88d2-7aea66a83a63\") " pod="openshift-ingress/kube-auth-proxy-5b574975b8-rkf8t"
Sep 16 14:11:02.294013 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:02.293980    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5b574975b8-rkf8t"
Sep 16 14:11:02.568533 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:02.568493    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5b574975b8-rkf8t"]
Sep 16 14:11:02.570624 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:11:02.570598    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8dbea9a_e715_4bd9_88d2_7aea66a83a63.slice/crio-d61ffc3a311e0cbb7e92edbb86dd6c54a35313d313dc66ee49810a8a7c2f8a7d WatchSource:0}: Error finding container d61ffc3a311e0cbb7e92edbb86dd6c54a35313d313dc66ee49810a8a7c2f8a7d: Status 404 returned error can't find the container with id d61ffc3a311e0cbb7e92edbb86dd6c54a35313d313dc66ee49810a8a7c2f8a7d
Sep 16 14:11:02.805669 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:02.805635    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5b574975b8-rkf8t" event={"ID":"d8dbea9a-e715-4bd9-88d2-7aea66a83a63","Type":"ContainerStarted","Data":"d61ffc3a311e0cbb7e92edbb86dd6c54a35313d313dc66ee49810a8a7c2f8a7d"}
Sep 16 14:11:03.316610 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:03.316579    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-m74pd"]
Sep 16 14:11:03.319857 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:03.319834    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-m74pd"
Sep 16 14:11:03.323178 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:03.323133    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 16 14:11:03.323178 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:03.323150    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-q22zw\""
Sep 16 14:11:03.323358 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:03.323194    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 16 14:11:03.323358 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:03.323194    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 16 14:11:03.323843 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:03.323827    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 16 14:11:03.340826 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:03.340799    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-m74pd"]
Sep 16 14:11:03.363199 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:03.363169    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c3ff01ed-cf20-4526-96d9-9d540acae326-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-m74pd\" (UID: \"c3ff01ed-cf20-4526-96d9-9d540acae326\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-m74pd"
Sep 16 14:11:03.363351 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:03.363211    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c3ff01ed-cf20-4526-96d9-9d540acae326-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-m74pd\" (UID: \"c3ff01ed-cf20-4526-96d9-9d540acae326\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-m74pd"
Sep 16 14:11:03.363351 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:03.363244    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3ff01ed-cf20-4526-96d9-9d540acae326-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-m74pd\" (UID: \"c3ff01ed-cf20-4526-96d9-9d540acae326\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-m74pd"
Sep 16 14:11:03.363351 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:03.363283    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7r6r\" (UniqueName: \"kubernetes.io/projected/c3ff01ed-cf20-4526-96d9-9d540acae326-kube-api-access-h7r6r\") pod \"istiod-openshift-gateway-7b7995d5cd-m74pd\" (UID: \"c3ff01ed-cf20-4526-96d9-9d540acae326\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-m74pd"
Sep 16 14:11:03.363532 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:03.363364    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c3ff01ed-cf20-4526-96d9-9d540acae326-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-m74pd\" (UID: \"c3ff01ed-cf20-4526-96d9-9d540acae326\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-m74pd"
Sep 16 14:11:03.363532 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:03.363461    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c3ff01ed-cf20-4526-96d9-9d540acae326-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-m74pd\" (UID: \"c3ff01ed-cf20-4526-96d9-9d540acae326\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-m74pd"
Sep 16 14:11:03.363639 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:03.363559    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3ff01ed-cf20-4526-96d9-9d540acae326-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-m74pd\" (UID: \"c3ff01ed-cf20-4526-96d9-9d540acae326\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-m74pd"
Sep 16 14:11:03.464804 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:03.464770    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3ff01ed-cf20-4526-96d9-9d540acae326-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-m74pd\" (UID: \"c3ff01ed-cf20-4526-96d9-9d540acae326\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-m74pd"
Sep 16 14:11:03.464982 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:03.464839    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c3ff01ed-cf20-4526-96d9-9d540acae326-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-m74pd\" (UID: \"c3ff01ed-cf20-4526-96d9-9d540acae326\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-m74pd"
Sep 16 14:11:03.464982 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:03.464865    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c3ff01ed-cf20-4526-96d9-9d540acae326-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-m74pd\" (UID: \"c3ff01ed-cf20-4526-96d9-9d540acae326\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-m74pd"
Sep 16 14:11:03.464982 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:03.464898    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3ff01ed-cf20-4526-96d9-9d540acae326-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-m74pd\" (UID: \"c3ff01ed-cf20-4526-96d9-9d540acae326\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-m74pd"
Sep 16 14:11:03.464982 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:03.464933    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7r6r\" (UniqueName: \"kubernetes.io/projected/c3ff01ed-cf20-4526-96d9-9d540acae326-kube-api-access-h7r6r\") pod \"istiod-openshift-gateway-7b7995d5cd-m74pd\" (UID: \"c3ff01ed-cf20-4526-96d9-9d540acae326\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-m74pd"
Sep 16 14:11:03.464982 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:03.464966    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c3ff01ed-cf20-4526-96d9-9d540acae326-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-m74pd\" (UID: \"c3ff01ed-cf20-4526-96d9-9d540acae326\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-m74pd"
Sep 16 14:11:03.465237 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:03.464993    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c3ff01ed-cf20-4526-96d9-9d540acae326-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-m74pd\" (UID: \"c3ff01ed-cf20-4526-96d9-9d540acae326\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-m74pd"
Sep 16 14:11:03.465982 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:03.465929    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c3ff01ed-cf20-4526-96d9-9d540acae326-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-m74pd\" (UID: \"c3ff01ed-cf20-4526-96d9-9d540acae326\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-m74pd"
Sep 16 14:11:03.467757 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:03.467734    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3ff01ed-cf20-4526-96d9-9d540acae326-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-m74pd\" (UID: \"c3ff01ed-cf20-4526-96d9-9d540acae326\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-m74pd"
Sep 16 14:11:03.467962 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:03.467938    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c3ff01ed-cf20-4526-96d9-9d540acae326-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-m74pd\" (UID: \"c3ff01ed-cf20-4526-96d9-9d540acae326\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-m74pd"
Sep 16 14:11:03.468558 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:03.468536    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c3ff01ed-cf20-4526-96d9-9d540acae326-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-m74pd\" (UID: \"c3ff01ed-cf20-4526-96d9-9d540acae326\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-m74pd"
Sep 16 14:11:03.468909 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:03.468877    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c3ff01ed-cf20-4526-96d9-9d540acae326-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-m74pd\" (UID: \"c3ff01ed-cf20-4526-96d9-9d540acae326\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-m74pd"
Sep 16 14:11:03.473861 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:03.473834    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3ff01ed-cf20-4526-96d9-9d540acae326-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-m74pd\" (UID: \"c3ff01ed-cf20-4526-96d9-9d540acae326\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-m74pd"
Sep 16 14:11:03.475778 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:03.475760    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7r6r\" (UniqueName: \"kubernetes.io/projected/c3ff01ed-cf20-4526-96d9-9d540acae326-kube-api-access-h7r6r\") pod \"istiod-openshift-gateway-7b7995d5cd-m74pd\" (UID: \"c3ff01ed-cf20-4526-96d9-9d540acae326\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-m74pd"
Sep 16 14:11:03.632083 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:03.632001    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-m74pd"
Sep 16 14:11:03.919165 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:03.919089    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-m74pd"]
Sep 16 14:11:03.921777 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:11:03.921747    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3ff01ed_cf20_4526_96d9_9d540acae326.slice/crio-398c67f059843865d0ef8ea54907b840c821df59d166ee551b5251ff26f24cac WatchSource:0}: Error finding container 398c67f059843865d0ef8ea54907b840c821df59d166ee551b5251ff26f24cac: Status 404 returned error can't find the container with id 398c67f059843865d0ef8ea54907b840c821df59d166ee551b5251ff26f24cac
Sep 16 14:11:04.819878 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:04.819839    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-m74pd" event={"ID":"c3ff01ed-cf20-4526-96d9-9d540acae326","Type":"ContainerStarted","Data":"398c67f059843865d0ef8ea54907b840c821df59d166ee551b5251ff26f24cac"}
Sep 16 14:11:05.827924 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:05.827884    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5b574975b8-rkf8t" event={"ID":"d8dbea9a-e715-4bd9-88d2-7aea66a83a63","Type":"ContainerStarted","Data":"17adbdd28a0dafb810d7fa9053858e3b0d1d826a7119431139465581e63e7f33"}
Sep 16 14:11:07.379268 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:07.379232    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 14:11:07.379604 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:07.379295    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 14:11:07.837733 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:07.837702    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-m74pd" event={"ID":"c3ff01ed-cf20-4526-96d9-9d540acae326","Type":"ContainerStarted","Data":"40f02d5c1b7eda8e58aa4b28126465604868c013f6d5af488596aeaa3f3f6a4c"}
Sep 16 14:11:07.837916 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:07.837816    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-m74pd"
Sep 16 14:11:07.857917 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:07.857876    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5b574975b8-rkf8t" podStartSLOduration=3.8186231250000002 podStartE2EDuration="6.857860401s" totalImagesPullingTime="3.039237276s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:11:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:11:05.872876347 +0000 UTC m=+398.887986208" watchObservedRunningTime="2026-09-16 14:11:07.857860401 +0000 UTC m=+400.872970261"
Sep 16 14:11:07.858060 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:07.857947    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-m74pd" podStartSLOduration=1.402603722 podStartE2EDuration="4.857942303s" totalImagesPullingTime="3.455338581s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:11:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:11:07.856939592 +0000 UTC m=+400.872049453" watchObservedRunningTime="2026-09-16 14:11:07.857942303 +0000 UTC m=+400.873052162"
Sep 16 14:11:08.810149 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:08.810111    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-snvvv"]
Sep 16 14:11:08.813414 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:08.813393    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-snvvv"
Sep 16 14:11:08.817196 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:08.817175    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Sep 16 14:11:08.817479 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:08.817462    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-zj4m7\""
Sep 16 14:11:08.849316 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:08.845732    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-snvvv"]
Sep 16 14:11:08.852877 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:08.852855    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-m74pd"
Sep 16 14:11:08.909019 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:08.908988    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2bb4393a-0f0a-4213-a212-19ae51c42bb3-cert\") pod \"odh-model-controller-7998c67dc4-snvvv\" (UID: \"2bb4393a-0f0a-4213-a212-19ae51c42bb3\") " pod="opendatahub/odh-model-controller-7998c67dc4-snvvv"
Sep 16 14:11:08.909136 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:08.909100    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrqbb\" (UniqueName: \"kubernetes.io/projected/2bb4393a-0f0a-4213-a212-19ae51c42bb3-kube-api-access-nrqbb\") pod \"odh-model-controller-7998c67dc4-snvvv\" (UID: \"2bb4393a-0f0a-4213-a212-19ae51c42bb3\") " pod="opendatahub/odh-model-controller-7998c67dc4-snvvv"
Sep 16 14:11:08.960430 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:08.960403    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4t6lk"]
Sep 16 14:11:08.963716 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:08.963695    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4t6lk"
Sep 16 14:11:08.972473 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:08.972454    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-pj2vn\""
Sep 16 14:11:08.988156 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:08.988125    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4t6lk"]
Sep 16 14:11:09.010106 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:09.010080    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6a422ff-e208-4686-9ea5-d7e75901d878-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4t6lk\" (UID: \"c6a422ff-e208-4686-9ea5-d7e75901d878\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4t6lk"
Sep 16 14:11:09.010222 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:09.010122    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6a422ff-e208-4686-9ea5-d7e75901d878-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4t6lk\" (UID: \"c6a422ff-e208-4686-9ea5-d7e75901d878\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4t6lk"
Sep 16 14:11:09.010222 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:09.010153    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6a422ff-e208-4686-9ea5-d7e75901d878-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4t6lk\" (UID: \"c6a422ff-e208-4686-9ea5-d7e75901d878\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4t6lk"
Sep 16 14:11:09.010222 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:09.010216    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmmzj\" (UniqueName: \"kubernetes.io/projected/c6a422ff-e208-4686-9ea5-d7e75901d878-kube-api-access-wmmzj\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4t6lk\" (UID: \"c6a422ff-e208-4686-9ea5-d7e75901d878\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4t6lk"
Sep 16 14:11:09.010376 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:09.010253    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2bb4393a-0f0a-4213-a212-19ae51c42bb3-cert\") pod \"odh-model-controller-7998c67dc4-snvvv\" (UID: \"2bb4393a-0f0a-4213-a212-19ae51c42bb3\") " pod="opendatahub/odh-model-controller-7998c67dc4-snvvv"
Sep 16 14:11:09.010376 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:09.010278    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6a422ff-e208-4686-9ea5-d7e75901d878-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4t6lk\" (UID: \"c6a422ff-e208-4686-9ea5-d7e75901d878\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4t6lk"
Sep 16 14:11:09.010376 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:09.010306    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6a422ff-e208-4686-9ea5-d7e75901d878-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4t6lk\" (UID: \"c6a422ff-e208-4686-9ea5-d7e75901d878\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4t6lk"
Sep 16 14:11:09.010376 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:09.010332    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6a422ff-e208-4686-9ea5-d7e75901d878-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4t6lk\" (UID: \"c6a422ff-e208-4686-9ea5-d7e75901d878\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4t6lk"
Sep 16 14:11:09.010598 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:11:09.010397    2580 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Sep 16 14:11:09.010598 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:09.010421    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrqbb\" (UniqueName: \"kubernetes.io/projected/2bb4393a-0f0a-4213-a212-19ae51c42bb3-kube-api-access-nrqbb\") pod \"odh-model-controller-7998c67dc4-snvvv\" (UID: \"2bb4393a-0f0a-4213-a212-19ae51c42bb3\") " pod="opendatahub/odh-model-controller-7998c67dc4-snvvv"
Sep 16 14:11:09.010598 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:11:09.010448    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2bb4393a-0f0a-4213-a212-19ae51c42bb3-cert podName:2bb4393a-0f0a-4213-a212-19ae51c42bb3 nodeName:}" failed. No retries permitted until 2026-09-16 14:11:09.510434671 +0000 UTC m=+402.525544539 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/2bb4393a-0f0a-4213-a212-19ae51c42bb3-cert") pod "odh-model-controller-7998c67dc4-snvvv" (UID: "2bb4393a-0f0a-4213-a212-19ae51c42bb3") : secret "odh-model-controller-webhook-cert" not found
Sep 16 14:11:09.010598 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:09.010478    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6a422ff-e208-4686-9ea5-d7e75901d878-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4t6lk\" (UID: \"c6a422ff-e208-4686-9ea5-d7e75901d878\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4t6lk"
Sep 16 14:11:09.010598 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:09.010526    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6a422ff-e208-4686-9ea5-d7e75901d878-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4t6lk\" (UID: \"c6a422ff-e208-4686-9ea5-d7e75901d878\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4t6lk"
Sep 16 14:11:09.028082 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:09.028061    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrqbb\" (UniqueName: \"kubernetes.io/projected/2bb4393a-0f0a-4213-a212-19ae51c42bb3-kube-api-access-nrqbb\") pod \"odh-model-controller-7998c67dc4-snvvv\" (UID: \"2bb4393a-0f0a-4213-a212-19ae51c42bb3\") " pod="opendatahub/odh-model-controller-7998c67dc4-snvvv"
Sep 16 14:11:09.111147 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:09.111084    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6a422ff-e208-4686-9ea5-d7e75901d878-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4t6lk\" (UID: \"c6a422ff-e208-4686-9ea5-d7e75901d878\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4t6lk"
Sep 16 14:11:09.111147 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:09.111121    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6a422ff-e208-4686-9ea5-d7e75901d878-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4t6lk\" (UID: \"c6a422ff-e208-4686-9ea5-d7e75901d878\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4t6lk"
Sep 16 14:11:09.111323 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:09.111163    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6a422ff-e208-4686-9ea5-d7e75901d878-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4t6lk\" (UID: \"c6a422ff-e208-4686-9ea5-d7e75901d878\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4t6lk"
Sep 16 14:11:09.111323 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:09.111190    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6a422ff-e208-4686-9ea5-d7e75901d878-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4t6lk\" (UID: \"c6a422ff-e208-4686-9ea5-d7e75901d878\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4t6lk"
Sep 16 14:11:09.111323 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:09.111214    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6a422ff-e208-4686-9ea5-d7e75901d878-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4t6lk\" (UID: \"c6a422ff-e208-4686-9ea5-d7e75901d878\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4t6lk"
Sep 16 14:11:09.111323 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:09.111254    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmmzj\" (UniqueName: \"kubernetes.io/projected/c6a422ff-e208-4686-9ea5-d7e75901d878-kube-api-access-wmmzj\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4t6lk\" (UID: \"c6a422ff-e208-4686-9ea5-d7e75901d878\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4t6lk"
Sep 16 14:11:09.111323 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:09.111295    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6a422ff-e208-4686-9ea5-d7e75901d878-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4t6lk\" (UID: \"c6a422ff-e208-4686-9ea5-d7e75901d878\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4t6lk"
Sep 16 14:11:09.111635 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:09.111442    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6a422ff-e208-4686-9ea5-d7e75901d878-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4t6lk\" (UID: \"c6a422ff-e208-4686-9ea5-d7e75901d878\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4t6lk"
Sep 16 14:11:09.111635 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:09.111476    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6a422ff-e208-4686-9ea5-d7e75901d878-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4t6lk\" (UID: \"c6a422ff-e208-4686-9ea5-d7e75901d878\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4t6lk"
Sep 16 14:11:09.111742 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:09.111727    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6a422ff-e208-4686-9ea5-d7e75901d878-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4t6lk\" (UID: \"c6a422ff-e208-4686-9ea5-d7e75901d878\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4t6lk"
Sep 16 14:11:09.112069 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:09.112044    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6a422ff-e208-4686-9ea5-d7e75901d878-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4t6lk\" (UID: \"c6a422ff-e208-4686-9ea5-d7e75901d878\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4t6lk"
Sep 16 14:11:09.112189 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:09.112170    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6a422ff-e208-4686-9ea5-d7e75901d878-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4t6lk\" (UID: \"c6a422ff-e208-4686-9ea5-d7e75901d878\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4t6lk"
Sep 16 14:11:09.112355 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:09.112330    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6a422ff-e208-4686-9ea5-d7e75901d878-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4t6lk\" (UID: \"c6a422ff-e208-4686-9ea5-d7e75901d878\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4t6lk"
Sep 16 14:11:09.112452 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:09.112404    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6a422ff-e208-4686-9ea5-d7e75901d878-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4t6lk\" (UID: \"c6a422ff-e208-4686-9ea5-d7e75901d878\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4t6lk"
Sep 16 14:11:09.113872 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:09.113834    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6a422ff-e208-4686-9ea5-d7e75901d878-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4t6lk\" (UID: \"c6a422ff-e208-4686-9ea5-d7e75901d878\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4t6lk"
Sep 16 14:11:09.114060 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:09.114038    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6a422ff-e208-4686-9ea5-d7e75901d878-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4t6lk\" (UID: \"c6a422ff-e208-4686-9ea5-d7e75901d878\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4t6lk"
Sep 16 14:11:09.133063 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:09.133041    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6a422ff-e208-4686-9ea5-d7e75901d878-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4t6lk\" (UID: \"c6a422ff-e208-4686-9ea5-d7e75901d878\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4t6lk"
Sep 16 14:11:09.133228 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:09.133207    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmmzj\" (UniqueName: \"kubernetes.io/projected/c6a422ff-e208-4686-9ea5-d7e75901d878-kube-api-access-wmmzj\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4t6lk\" (UID: \"c6a422ff-e208-4686-9ea5-d7e75901d878\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4t6lk"
Sep 16 14:11:09.275228 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:09.275197    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4t6lk"
Sep 16 14:11:09.515213 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:09.515176    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2bb4393a-0f0a-4213-a212-19ae51c42bb3-cert\") pod \"odh-model-controller-7998c67dc4-snvvv\" (UID: \"2bb4393a-0f0a-4213-a212-19ae51c42bb3\") " pod="opendatahub/odh-model-controller-7998c67dc4-snvvv"
Sep 16 14:11:09.517572 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:09.517548    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2bb4393a-0f0a-4213-a212-19ae51c42bb3-cert\") pod \"odh-model-controller-7998c67dc4-snvvv\" (UID: \"2bb4393a-0f0a-4213-a212-19ae51c42bb3\") " pod="opendatahub/odh-model-controller-7998c67dc4-snvvv"
Sep 16 14:11:09.563117 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:09.563093    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4t6lk"]
Sep 16 14:11:09.565352 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:11:09.565327    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6a422ff_e208_4686_9ea5_d7e75901d878.slice/crio-b4a2e4071a1f7ce24ae92fad49e73d06d2a38411c6374bfd8f05480b5f2f543f WatchSource:0}: Error finding container b4a2e4071a1f7ce24ae92fad49e73d06d2a38411c6374bfd8f05480b5f2f543f: Status 404 returned error can't find the container with id b4a2e4071a1f7ce24ae92fad49e73d06d2a38411c6374bfd8f05480b5f2f543f
Sep 16 14:11:09.722697 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:09.722656    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-snvvv"
Sep 16 14:11:09.856987 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:09.856955    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4t6lk" event={"ID":"c6a422ff-e208-4686-9ea5-d7e75901d878","Type":"ContainerStarted","Data":"b4a2e4071a1f7ce24ae92fad49e73d06d2a38411c6374bfd8f05480b5f2f543f"}
Sep 16 14:11:09.995417 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:09.995327    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-snvvv"]
Sep 16 14:11:09.997830 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:11:09.997795    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bb4393a_0f0a_4213_a212_19ae51c42bb3.slice/crio-4d6ab022636f593bf6f8fa8e10e166f27a9bdee8b7c6a87c9ee032bb599da609 WatchSource:0}: Error finding container 4d6ab022636f593bf6f8fa8e10e166f27a9bdee8b7c6a87c9ee032bb599da609: Status 404 returned error can't find the container with id 4d6ab022636f593bf6f8fa8e10e166f27a9bdee8b7c6a87c9ee032bb599da609
Sep 16 14:11:10.862222 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:10.862189    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-snvvv" event={"ID":"2bb4393a-0f0a-4213-a212-19ae51c42bb3","Type":"ContainerStarted","Data":"4d6ab022636f593bf6f8fa8e10e166f27a9bdee8b7c6a87c9ee032bb599da609"}
Sep 16 14:11:14.402573 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:14.402518    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 14:11:14.402874 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:14.402613    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 14:11:14.402874 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:14.402650    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 14:11:14.881127 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:14.881094    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4t6lk" event={"ID":"c6a422ff-e208-4686-9ea5-d7e75901d878","Type":"ContainerStarted","Data":"671c795561124aef126d53931af068a72de175389cb22d8ad8443f2b9c8a9f9e"}
Sep 16 14:11:14.882709 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:14.882684    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-snvvv" event={"ID":"2bb4393a-0f0a-4213-a212-19ae51c42bb3","Type":"ContainerStarted","Data":"a59b0575435ab035fafcfbeaf97a2f6352ffe93f3359079a854546680319a1fc"}
Sep 16 14:11:14.882804 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:14.882748    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-snvvv"
Sep 16 14:11:14.903599 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:14.903556    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4t6lk" podStartSLOduration=2.068737651 podStartE2EDuration="6.903541775s" totalImagesPullingTime="4.834804124s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:11:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:11:14.901784908 +0000 UTC m=+407.916894769" watchObservedRunningTime="2026-09-16 14:11:14.903541775 +0000 UTC m=+407.918651693"
Sep 16 14:11:14.921295 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:14.921236    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-snvvv" podStartSLOduration=2.517906453 podStartE2EDuration="6.921216937s" totalImagesPullingTime="4.403310484s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:11:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:11:14.920666383 +0000 UTC m=+407.935776245" watchObservedRunningTime="2026-09-16 14:11:14.921216937 +0000 UTC m=+407.936326781"
Sep 16 14:11:15.275474 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:15.275439    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4t6lk"
Sep 16 14:11:15.279705 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:15.279681    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4t6lk"
Sep 16 14:11:15.886984 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:15.886951    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4t6lk"
Sep 16 14:11:15.887870 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:15.887852    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4t6lk"
Sep 16 14:11:19.417460 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:19.417426    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-x6xb8"]
Sep 16 14:11:19.420752 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:19.420734    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-x6xb8"
Sep 16 14:11:19.427401 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:19.427145    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Sep 16 14:11:19.427401 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:19.427145    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-7hld5\""
Sep 16 14:11:19.430315 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:19.430295    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-ktsml"]
Sep 16 14:11:19.433298 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:19.433283    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ktsml"
Sep 16 14:11:19.435782 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:19.435760    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-cfhxg\""
Sep 16 14:11:19.435965 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:19.435951    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 16 14:11:19.441782 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:19.441729    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-x6xb8"]
Sep 16 14:11:19.444736 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:19.444713    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-ktsml"]
Sep 16 14:11:19.501885 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:19.501856    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0b1a49bc-75d4-4e7a-baae-7283af813749-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-ktsml\" (UID: \"0b1a49bc-75d4-4e7a-baae-7283af813749\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ktsml"
Sep 16 14:11:19.502042 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:19.501888    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbnqv\" (UniqueName: \"kubernetes.io/projected/c91ff34d-ce75-4dbf-9fb3-15c057742185-kube-api-access-tbnqv\") pod \"kserve-controller-manager-d6f4b65b8-x6xb8\" (UID: \"c91ff34d-ce75-4dbf-9fb3-15c057742185\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-x6xb8"
Sep 16 14:11:19.502042 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:19.501922    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56jr5\" (UniqueName: \"kubernetes.io/projected/0b1a49bc-75d4-4e7a-baae-7283af813749-kube-api-access-56jr5\") pod \"llmisvc-controller-manager-777c6bd7d5-ktsml\" (UID: \"0b1a49bc-75d4-4e7a-baae-7283af813749\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ktsml"
Sep 16 14:11:19.502042 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:19.502002    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c91ff34d-ce75-4dbf-9fb3-15c057742185-cert\") pod \"kserve-controller-manager-d6f4b65b8-x6xb8\" (UID: \"c91ff34d-ce75-4dbf-9fb3-15c057742185\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-x6xb8"
Sep 16 14:11:19.602950 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:19.602915    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0b1a49bc-75d4-4e7a-baae-7283af813749-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-ktsml\" (UID: \"0b1a49bc-75d4-4e7a-baae-7283af813749\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ktsml"
Sep 16 14:11:19.602950 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:19.602953    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbnqv\" (UniqueName: \"kubernetes.io/projected/c91ff34d-ce75-4dbf-9fb3-15c057742185-kube-api-access-tbnqv\") pod \"kserve-controller-manager-d6f4b65b8-x6xb8\" (UID: \"c91ff34d-ce75-4dbf-9fb3-15c057742185\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-x6xb8"
Sep 16 14:11:19.603190 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:19.602981    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56jr5\" (UniqueName: \"kubernetes.io/projected/0b1a49bc-75d4-4e7a-baae-7283af813749-kube-api-access-56jr5\") pod \"llmisvc-controller-manager-777c6bd7d5-ktsml\" (UID: \"0b1a49bc-75d4-4e7a-baae-7283af813749\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ktsml"
Sep 16 14:11:19.603190 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:19.602999    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c91ff34d-ce75-4dbf-9fb3-15c057742185-cert\") pod \"kserve-controller-manager-d6f4b65b8-x6xb8\" (UID: \"c91ff34d-ce75-4dbf-9fb3-15c057742185\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-x6xb8"
Sep 16 14:11:19.603190 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:11:19.603090    2580 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Sep 16 14:11:19.603190 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:11:19.603163    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b1a49bc-75d4-4e7a-baae-7283af813749-cert podName:0b1a49bc-75d4-4e7a-baae-7283af813749 nodeName:}" failed. No retries permitted until 2026-09-16 14:11:20.103142922 +0000 UTC m=+413.118252772 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/0b1a49bc-75d4-4e7a-baae-7283af813749-cert") pod "llmisvc-controller-manager-777c6bd7d5-ktsml" (UID: "0b1a49bc-75d4-4e7a-baae-7283af813749") : secret "llmisvc-webhook-server-cert" not found
Sep 16 14:11:19.605381 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:19.605363    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c91ff34d-ce75-4dbf-9fb3-15c057742185-cert\") pod \"kserve-controller-manager-d6f4b65b8-x6xb8\" (UID: \"c91ff34d-ce75-4dbf-9fb3-15c057742185\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-x6xb8"
Sep 16 14:11:19.617473 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:19.617445    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbnqv\" (UniqueName: \"kubernetes.io/projected/c91ff34d-ce75-4dbf-9fb3-15c057742185-kube-api-access-tbnqv\") pod \"kserve-controller-manager-d6f4b65b8-x6xb8\" (UID: \"c91ff34d-ce75-4dbf-9fb3-15c057742185\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-x6xb8"
Sep 16 14:11:19.617601 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:19.617477    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-56jr5\" (UniqueName: \"kubernetes.io/projected/0b1a49bc-75d4-4e7a-baae-7283af813749-kube-api-access-56jr5\") pod \"llmisvc-controller-manager-777c6bd7d5-ktsml\" (UID: \"0b1a49bc-75d4-4e7a-baae-7283af813749\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ktsml"
Sep 16 14:11:19.731331 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:19.731294    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-x6xb8"
Sep 16 14:11:20.027486 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:20.027407    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-x6xb8"]
Sep 16 14:11:20.030583 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:11:20.030554    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc91ff34d_ce75_4dbf_9fb3_15c057742185.slice/crio-8b581bce136a5f7cbadcb2b9ed37d625ac62f69c97c8c66fc17f9c67197d89f0 WatchSource:0}: Error finding container 8b581bce136a5f7cbadcb2b9ed37d625ac62f69c97c8c66fc17f9c67197d89f0: Status 404 returned error can't find the container with id 8b581bce136a5f7cbadcb2b9ed37d625ac62f69c97c8c66fc17f9c67197d89f0
Sep 16 14:11:20.082054 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:20.082027    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-m74pd"]
Sep 16 14:11:20.082242 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:20.082207    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-m74pd" podUID="c3ff01ed-cf20-4526-96d9-9d540acae326" containerName="discovery" containerID="cri-o://40f02d5c1b7eda8e58aa4b28126465604868c013f6d5af488596aeaa3f3f6a4c" gracePeriod=30
Sep 16 14:11:20.109336 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:20.109312    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0b1a49bc-75d4-4e7a-baae-7283af813749-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-ktsml\" (UID: \"0b1a49bc-75d4-4e7a-baae-7283af813749\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ktsml"
Sep 16 14:11:20.111657 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:20.111639    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0b1a49bc-75d4-4e7a-baae-7283af813749-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-ktsml\" (UID: \"0b1a49bc-75d4-4e7a-baae-7283af813749\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ktsml"
Sep 16 14:11:20.311404 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:20.311381    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-m74pd"
Sep 16 14:11:20.342723 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:20.342698    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ktsml"
Sep 16 14:11:20.411901 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:20.411871    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3ff01ed-cf20-4526-96d9-9d540acae326-istio-token\") pod \"c3ff01ed-cf20-4526-96d9-9d540acae326\" (UID: \"c3ff01ed-cf20-4526-96d9-9d540acae326\") "
Sep 16 14:11:20.412036 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:20.411962    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h7r6r\" (UniqueName: \"kubernetes.io/projected/c3ff01ed-cf20-4526-96d9-9d540acae326-kube-api-access-h7r6r\") pod \"c3ff01ed-cf20-4526-96d9-9d540acae326\" (UID: \"c3ff01ed-cf20-4526-96d9-9d540acae326\") "
Sep 16 14:11:20.412036 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:20.411992    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c3ff01ed-cf20-4526-96d9-9d540acae326-istio-kubeconfig\") pod \"c3ff01ed-cf20-4526-96d9-9d540acae326\" (UID: \"c3ff01ed-cf20-4526-96d9-9d540acae326\") "
Sep 16 14:11:20.412036 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:20.412029    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c3ff01ed-cf20-4526-96d9-9d540acae326-istio-csr-ca-configmap\") pod \"c3ff01ed-cf20-4526-96d9-9d540acae326\" (UID: \"c3ff01ed-cf20-4526-96d9-9d540acae326\") "
Sep 16 14:11:20.412195 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:20.412053    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c3ff01ed-cf20-4526-96d9-9d540acae326-istio-csr-dns-cert\") pod \"c3ff01ed-cf20-4526-96d9-9d540acae326\" (UID: \"c3ff01ed-cf20-4526-96d9-9d540acae326\") "
Sep 16 14:11:20.412195 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:20.412075    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3ff01ed-cf20-4526-96d9-9d540acae326-local-certs\") pod \"c3ff01ed-cf20-4526-96d9-9d540acae326\" (UID: \"c3ff01ed-cf20-4526-96d9-9d540acae326\") "
Sep 16 14:11:20.412195 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:20.412123    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c3ff01ed-cf20-4526-96d9-9d540acae326-cacerts\") pod \"c3ff01ed-cf20-4526-96d9-9d540acae326\" (UID: \"c3ff01ed-cf20-4526-96d9-9d540acae326\") "
Sep 16 14:11:20.412654 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:20.412540    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3ff01ed-cf20-4526-96d9-9d540acae326-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "c3ff01ed-cf20-4526-96d9-9d540acae326" (UID: "c3ff01ed-cf20-4526-96d9-9d540acae326"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:11:20.414806 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:20.414777    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3ff01ed-cf20-4526-96d9-9d540acae326-kube-api-access-h7r6r" (OuterVolumeSpecName: "kube-api-access-h7r6r") pod "c3ff01ed-cf20-4526-96d9-9d540acae326" (UID: "c3ff01ed-cf20-4526-96d9-9d540acae326"). InnerVolumeSpecName "kube-api-access-h7r6r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:11:20.414884 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:20.414777    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c3ff01ed-cf20-4526-96d9-9d540acae326-cacerts" (OuterVolumeSpecName: "cacerts") pod "c3ff01ed-cf20-4526-96d9-9d540acae326" (UID: "c3ff01ed-cf20-4526-96d9-9d540acae326"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:11:20.414884 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:20.414770    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3ff01ed-cf20-4526-96d9-9d540acae326-istio-token" (OuterVolumeSpecName: "istio-token") pod "c3ff01ed-cf20-4526-96d9-9d540acae326" (UID: "c3ff01ed-cf20-4526-96d9-9d540acae326"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:11:20.414884 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:20.414833    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c3ff01ed-cf20-4526-96d9-9d540acae326-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "c3ff01ed-cf20-4526-96d9-9d540acae326" (UID: "c3ff01ed-cf20-4526-96d9-9d540acae326"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:11:20.415001 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:20.414936    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c3ff01ed-cf20-4526-96d9-9d540acae326-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "c3ff01ed-cf20-4526-96d9-9d540acae326" (UID: "c3ff01ed-cf20-4526-96d9-9d540acae326"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:11:20.415628 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:20.415607    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3ff01ed-cf20-4526-96d9-9d540acae326-local-certs" (OuterVolumeSpecName: "local-certs") pod "c3ff01ed-cf20-4526-96d9-9d540acae326" (UID: "c3ff01ed-cf20-4526-96d9-9d540acae326"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:11:20.513456 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:20.513428    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3ff01ed-cf20-4526-96d9-9d540acae326-istio-token\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:11:20.513456 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:20.513451    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h7r6r\" (UniqueName: \"kubernetes.io/projected/c3ff01ed-cf20-4526-96d9-9d540acae326-kube-api-access-h7r6r\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:11:20.513456 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:20.513461    2580 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c3ff01ed-cf20-4526-96d9-9d540acae326-istio-kubeconfig\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:11:20.513869 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:20.513469    2580 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c3ff01ed-cf20-4526-96d9-9d540acae326-istio-csr-ca-configmap\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:11:20.513869 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:20.513477    2580 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c3ff01ed-cf20-4526-96d9-9d540acae326-istio-csr-dns-cert\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:11:20.513869 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:20.513485    2580 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3ff01ed-cf20-4526-96d9-9d540acae326-local-certs\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:11:20.513869 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:20.513493    2580 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c3ff01ed-cf20-4526-96d9-9d540acae326-cacerts\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:11:20.631099 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:20.631076    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-ktsml"]
Sep 16 14:11:20.632985 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:11:20.632959    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod0b1a49bc_75d4_4e7a_baae_7283af813749.slice/crio-68b15b183d5ee983bc5857891004e434feb465a685524dba07138f41b687ed58 WatchSource:0}: Error finding container 68b15b183d5ee983bc5857891004e434feb465a685524dba07138f41b687ed58: Status 404 returned error can't find the container with id 68b15b183d5ee983bc5857891004e434feb465a685524dba07138f41b687ed58
Sep 16 14:11:20.907133 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:20.907044    2580 generic.go:356] "Generic (PLEG): container finished" podID="c3ff01ed-cf20-4526-96d9-9d540acae326" containerID="40f02d5c1b7eda8e58aa4b28126465604868c013f6d5af488596aeaa3f3f6a4c" exitCode=0
Sep 16 14:11:20.907133 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:20.907092    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-m74pd"
Sep 16 14:11:20.907350 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:20.907134    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-m74pd" event={"ID":"c3ff01ed-cf20-4526-96d9-9d540acae326","Type":"ContainerDied","Data":"40f02d5c1b7eda8e58aa4b28126465604868c013f6d5af488596aeaa3f3f6a4c"}
Sep 16 14:11:20.907350 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:20.907183    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-m74pd" event={"ID":"c3ff01ed-cf20-4526-96d9-9d540acae326","Type":"ContainerDied","Data":"398c67f059843865d0ef8ea54907b840c821df59d166ee551b5251ff26f24cac"}
Sep 16 14:11:20.907350 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:20.907213    2580 scope.go:117] "RemoveContainer" containerID="40f02d5c1b7eda8e58aa4b28126465604868c013f6d5af488596aeaa3f3f6a4c"
Sep 16 14:11:20.909064 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:20.908624    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ktsml" event={"ID":"0b1a49bc-75d4-4e7a-baae-7283af813749","Type":"ContainerStarted","Data":"68b15b183d5ee983bc5857891004e434feb465a685524dba07138f41b687ed58"}
Sep 16 14:11:20.910159 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:20.910130    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-x6xb8" event={"ID":"c91ff34d-ce75-4dbf-9fb3-15c057742185","Type":"ContainerStarted","Data":"8b581bce136a5f7cbadcb2b9ed37d625ac62f69c97c8c66fc17f9c67197d89f0"}
Sep 16 14:11:20.919249 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:20.919230    2580 scope.go:117] "RemoveContainer" containerID="40f02d5c1b7eda8e58aa4b28126465604868c013f6d5af488596aeaa3f3f6a4c"
Sep 16 14:11:20.919556 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:11:20.919531    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40f02d5c1b7eda8e58aa4b28126465604868c013f6d5af488596aeaa3f3f6a4c\": container with ID starting with 40f02d5c1b7eda8e58aa4b28126465604868c013f6d5af488596aeaa3f3f6a4c not found: ID does not exist" containerID="40f02d5c1b7eda8e58aa4b28126465604868c013f6d5af488596aeaa3f3f6a4c"
Sep 16 14:11:20.919650 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:20.919564    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40f02d5c1b7eda8e58aa4b28126465604868c013f6d5af488596aeaa3f3f6a4c"} err="failed to get container status \"40f02d5c1b7eda8e58aa4b28126465604868c013f6d5af488596aeaa3f3f6a4c\": rpc error: code = NotFound desc = could not find container \"40f02d5c1b7eda8e58aa4b28126465604868c013f6d5af488596aeaa3f3f6a4c\": container with ID starting with 40f02d5c1b7eda8e58aa4b28126465604868c013f6d5af488596aeaa3f3f6a4c not found: ID does not exist"
Sep 16 14:11:20.936769 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:20.936740    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-m74pd"]
Sep 16 14:11:20.939595 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:20.939573    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-m74pd"]
Sep 16 14:11:21.430070 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:21.430038    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c3ff01ed-cf20-4526-96d9-9d540acae326" path="/var/lib/kubelet/pods/c3ff01ed-cf20-4526-96d9-9d540acae326/volumes"
Sep 16 14:11:22.924355 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:22.924265    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-x6xb8" event={"ID":"c91ff34d-ce75-4dbf-9fb3-15c057742185","Type":"ContainerStarted","Data":"bf538fb602e659b6795ce76db15653aa8ea491a5fbed0eb9dd709384ea279d19"}
Sep 16 14:11:22.924770 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:22.924387    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-x6xb8"
Sep 16 14:11:22.944038 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:22.943986    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-x6xb8" podStartSLOduration=1.318589621 podStartE2EDuration="3.943971206s" totalImagesPullingTime="2.625381585s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:11:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:11:22.941059527 +0000 UTC m=+415.956169424" watchObservedRunningTime="2026-09-16 14:11:22.943971206 +0000 UTC m=+415.959081065"
Sep 16 14:11:24.934480 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:24.934447    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ktsml" event={"ID":"0b1a49bc-75d4-4e7a-baae-7283af813749","Type":"ContainerStarted","Data":"5233b7c844bd96ab1f1d8ce823177282e9815c9c1c90bfb3b0d2e29ea7024573"}
Sep 16 14:11:24.934844 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:24.934592    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ktsml"
Sep 16 14:11:24.961404 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:24.961328    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ktsml" podStartSLOduration=2.12822381 podStartE2EDuration="5.961315553s" totalImagesPullingTime="3.833091743s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:11:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:11:24.960617563 +0000 UTC m=+417.975727444" watchObservedRunningTime="2026-09-16 14:11:24.961315553 +0000 UTC m=+417.976425413"
Sep 16 14:11:25.905907 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:25.905881    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-snvvv"
Sep 16 14:11:53.935046 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:53.934961    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-x6xb8"
Sep 16 14:11:55.941263 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:11:55.941234    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ktsml"
Sep 16 14:12:14.492073 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:14.492040    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7xgfwm"]
Sep 16 14:12:14.492446 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:14.492306    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3ff01ed-cf20-4526-96d9-9d540acae326" containerName="discovery"
Sep 16 14:12:14.492446 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:14.492317    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3ff01ed-cf20-4526-96d9-9d540acae326" containerName="discovery"
Sep 16 14:12:14.492446 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:14.492375    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3ff01ed-cf20-4526-96d9-9d540acae326" containerName="discovery"
Sep 16 14:12:14.499226 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:14.499209    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7xgfwm"
Sep 16 14:12:14.501382 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:14.501358    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 16 14:12:14.501760 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:14.501733    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-gdgch\""
Sep 16 14:12:14.501883 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:14.501735    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 16 14:12:14.503939 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:14.503891    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7xgfwm"]
Sep 16 14:12:14.590957 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:14.590925    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4j9f5"]
Sep 16 14:12:14.594282 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:14.594267    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4j9f5"
Sep 16 14:12:14.601805 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:14.601781    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4j9f5"]
Sep 16 14:12:14.603796 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:14.603775    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9mw4\" (UniqueName: \"kubernetes.io/projected/1db899bb-6934-4409-821a-8dcb90cbd301-kube-api-access-g9mw4\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7xgfwm\" (UID: \"1db899bb-6934-4409-821a-8dcb90cbd301\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7xgfwm"
Sep 16 14:12:14.603896 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:14.603812    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1db899bb-6934-4409-821a-8dcb90cbd301-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7xgfwm\" (UID: \"1db899bb-6934-4409-821a-8dcb90cbd301\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7xgfwm"
Sep 16 14:12:14.603957 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:14.603913    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1db899bb-6934-4409-821a-8dcb90cbd301-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7xgfwm\" (UID: \"1db899bb-6934-4409-821a-8dcb90cbd301\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7xgfwm"
Sep 16 14:12:14.689930 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:14.689897    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cck4x9"]
Sep 16 14:12:14.693248 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:14.693230    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cck4x9"
Sep 16 14:12:14.701968 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:14.701933    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cck4x9"]
Sep 16 14:12:14.704383 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:14.704364    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1db899bb-6934-4409-821a-8dcb90cbd301-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7xgfwm\" (UID: \"1db899bb-6934-4409-821a-8dcb90cbd301\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7xgfwm"
Sep 16 14:12:14.704470 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:14.704403    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d95a3bf2-c2ea-4f89-b356-9d6f511db4ca-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4j9f5\" (UID: \"d95a3bf2-c2ea-4f89-b356-9d6f511db4ca\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4j9f5"
Sep 16 14:12:14.704470 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:14.704427    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vm2mr\" (UniqueName: \"kubernetes.io/projected/d95a3bf2-c2ea-4f89-b356-9d6f511db4ca-kube-api-access-vm2mr\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4j9f5\" (UID: \"d95a3bf2-c2ea-4f89-b356-9d6f511db4ca\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4j9f5"
Sep 16 14:12:14.704625 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:14.704535    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9mw4\" (UniqueName: \"kubernetes.io/projected/1db899bb-6934-4409-821a-8dcb90cbd301-kube-api-access-g9mw4\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7xgfwm\" (UID: \"1db899bb-6934-4409-821a-8dcb90cbd301\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7xgfwm"
Sep 16 14:12:14.704625 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:14.704580    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1db899bb-6934-4409-821a-8dcb90cbd301-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7xgfwm\" (UID: \"1db899bb-6934-4409-821a-8dcb90cbd301\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7xgfwm"
Sep 16 14:12:14.704625 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:14.704618    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d95a3bf2-c2ea-4f89-b356-9d6f511db4ca-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4j9f5\" (UID: \"d95a3bf2-c2ea-4f89-b356-9d6f511db4ca\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4j9f5"
Sep 16 14:12:14.704802 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:14.704783    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1db899bb-6934-4409-821a-8dcb90cbd301-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7xgfwm\" (UID: \"1db899bb-6934-4409-821a-8dcb90cbd301\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7xgfwm"
Sep 16 14:12:14.704953 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:14.704935    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1db899bb-6934-4409-821a-8dcb90cbd301-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7xgfwm\" (UID: \"1db899bb-6934-4409-821a-8dcb90cbd301\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7xgfwm"
Sep 16 14:12:14.712254 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:14.712234    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9mw4\" (UniqueName: \"kubernetes.io/projected/1db899bb-6934-4409-821a-8dcb90cbd301-kube-api-access-g9mw4\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7xgfwm\" (UID: \"1db899bb-6934-4409-821a-8dcb90cbd301\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7xgfwm"
Sep 16 14:12:14.798393 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:14.795615    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wjpzl"]
Sep 16 14:12:14.800834 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:14.800804    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wjpzl"
Sep 16 14:12:14.805100 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:14.805075    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xd89v\" (UniqueName: \"kubernetes.io/projected/d841ae78-746b-4ad2-a0d1-f02b53a7cec0-kube-api-access-xd89v\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cck4x9\" (UID: \"d841ae78-746b-4ad2-a0d1-f02b53a7cec0\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cck4x9"
Sep 16 14:12:14.805226 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:14.805124    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d95a3bf2-c2ea-4f89-b356-9d6f511db4ca-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4j9f5\" (UID: \"d95a3bf2-c2ea-4f89-b356-9d6f511db4ca\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4j9f5"
Sep 16 14:12:14.805226 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:14.805142    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d841ae78-746b-4ad2-a0d1-f02b53a7cec0-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cck4x9\" (UID: \"d841ae78-746b-4ad2-a0d1-f02b53a7cec0\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cck4x9"
Sep 16 14:12:14.805226 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:14.805168    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vm2mr\" (UniqueName: \"kubernetes.io/projected/d95a3bf2-c2ea-4f89-b356-9d6f511db4ca-kube-api-access-vm2mr\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4j9f5\" (UID: \"d95a3bf2-c2ea-4f89-b356-9d6f511db4ca\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4j9f5"
Sep 16 14:12:14.805226 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:14.805195    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d95a3bf2-c2ea-4f89-b356-9d6f511db4ca-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4j9f5\" (UID: \"d95a3bf2-c2ea-4f89-b356-9d6f511db4ca\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4j9f5"
Sep 16 14:12:14.805226 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:14.805221    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d841ae78-746b-4ad2-a0d1-f02b53a7cec0-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cck4x9\" (UID: \"d841ae78-746b-4ad2-a0d1-f02b53a7cec0\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cck4x9"
Sep 16 14:12:14.805535 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:14.805493    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d95a3bf2-c2ea-4f89-b356-9d6f511db4ca-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4j9f5\" (UID: \"d95a3bf2-c2ea-4f89-b356-9d6f511db4ca\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4j9f5"
Sep 16 14:12:14.805607 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:14.805553    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d95a3bf2-c2ea-4f89-b356-9d6f511db4ca-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4j9f5\" (UID: \"d95a3bf2-c2ea-4f89-b356-9d6f511db4ca\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4j9f5"
Sep 16 14:12:14.809014 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:14.808989    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wjpzl"]
Sep 16 14:12:14.809123 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:14.809048    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7xgfwm"
Sep 16 14:12:14.813053 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:14.813033    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vm2mr\" (UniqueName: \"kubernetes.io/projected/d95a3bf2-c2ea-4f89-b356-9d6f511db4ca-kube-api-access-vm2mr\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4j9f5\" (UID: \"d95a3bf2-c2ea-4f89-b356-9d6f511db4ca\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4j9f5"
Sep 16 14:12:14.903047 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:14.903021    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4j9f5"
Sep 16 14:12:14.906336 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:14.906312    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4952aa51-f3c2-4fb7-be41-7d0aa51c6d72-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wjpzl\" (UID: \"4952aa51-f3c2-4fb7-be41-7d0aa51c6d72\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wjpzl"
Sep 16 14:12:14.906410 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:14.906348    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d841ae78-746b-4ad2-a0d1-f02b53a7cec0-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cck4x9\" (UID: \"d841ae78-746b-4ad2-a0d1-f02b53a7cec0\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cck4x9"
Sep 16 14:12:14.906410 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:14.906386    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xd89v\" (UniqueName: \"kubernetes.io/projected/d841ae78-746b-4ad2-a0d1-f02b53a7cec0-kube-api-access-xd89v\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cck4x9\" (UID: \"d841ae78-746b-4ad2-a0d1-f02b53a7cec0\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cck4x9"
Sep 16 14:12:14.906485 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:14.906416    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mm459\" (UniqueName: \"kubernetes.io/projected/4952aa51-f3c2-4fb7-be41-7d0aa51c6d72-kube-api-access-mm459\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wjpzl\" (UID: \"4952aa51-f3c2-4fb7-be41-7d0aa51c6d72\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wjpzl"
Sep 16 14:12:14.906485 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:14.906451    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d841ae78-746b-4ad2-a0d1-f02b53a7cec0-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cck4x9\" (UID: \"d841ae78-746b-4ad2-a0d1-f02b53a7cec0\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cck4x9"
Sep 16 14:12:14.906485 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:14.906476    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4952aa51-f3c2-4fb7-be41-7d0aa51c6d72-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wjpzl\" (UID: \"4952aa51-f3c2-4fb7-be41-7d0aa51c6d72\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wjpzl"
Sep 16 14:12:14.906731 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:14.906711    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d841ae78-746b-4ad2-a0d1-f02b53a7cec0-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cck4x9\" (UID: \"d841ae78-746b-4ad2-a0d1-f02b53a7cec0\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cck4x9"
Sep 16 14:12:14.907024 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:14.906801    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d841ae78-746b-4ad2-a0d1-f02b53a7cec0-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cck4x9\" (UID: \"d841ae78-746b-4ad2-a0d1-f02b53a7cec0\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cck4x9"
Sep 16 14:12:14.914175 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:14.914156    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xd89v\" (UniqueName: \"kubernetes.io/projected/d841ae78-746b-4ad2-a0d1-f02b53a7cec0-kube-api-access-xd89v\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cck4x9\" (UID: \"d841ae78-746b-4ad2-a0d1-f02b53a7cec0\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cck4x9"
Sep 16 14:12:15.003022 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:15.002974    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cck4x9"
Sep 16 14:12:15.006970 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:15.006942    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mm459\" (UniqueName: \"kubernetes.io/projected/4952aa51-f3c2-4fb7-be41-7d0aa51c6d72-kube-api-access-mm459\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wjpzl\" (UID: \"4952aa51-f3c2-4fb7-be41-7d0aa51c6d72\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wjpzl"
Sep 16 14:12:15.007115 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:15.006981    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4952aa51-f3c2-4fb7-be41-7d0aa51c6d72-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wjpzl\" (UID: \"4952aa51-f3c2-4fb7-be41-7d0aa51c6d72\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wjpzl"
Sep 16 14:12:15.007115 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:15.007018    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4952aa51-f3c2-4fb7-be41-7d0aa51c6d72-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wjpzl\" (UID: \"4952aa51-f3c2-4fb7-be41-7d0aa51c6d72\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wjpzl"
Sep 16 14:12:15.007381 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:15.007360    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4952aa51-f3c2-4fb7-be41-7d0aa51c6d72-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wjpzl\" (UID: \"4952aa51-f3c2-4fb7-be41-7d0aa51c6d72\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wjpzl"
Sep 16 14:12:15.007437 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:15.007365    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4952aa51-f3c2-4fb7-be41-7d0aa51c6d72-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wjpzl\" (UID: \"4952aa51-f3c2-4fb7-be41-7d0aa51c6d72\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wjpzl"
Sep 16 14:12:15.015554 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:15.015532    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mm459\" (UniqueName: \"kubernetes.io/projected/4952aa51-f3c2-4fb7-be41-7d0aa51c6d72-kube-api-access-mm459\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wjpzl\" (UID: \"4952aa51-f3c2-4fb7-be41-7d0aa51c6d72\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wjpzl"
Sep 16 14:12:15.078534 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:15.078419    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7xgfwm"]
Sep 16 14:12:15.080568 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:12:15.080536    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1db899bb_6934_4409_821a_8dcb90cbd301.slice/crio-f88e67c09f2e9984e04927db8fb509c4ac9a6d80ef9adb787dc3bdcbe743514d WatchSource:0}: Error finding container f88e67c09f2e9984e04927db8fb509c4ac9a6d80ef9adb787dc3bdcbe743514d: Status 404 returned error can't find the container with id f88e67c09f2e9984e04927db8fb509c4ac9a6d80ef9adb787dc3bdcbe743514d
Sep 16 14:12:15.110860 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:15.110727    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wjpzl"
Sep 16 14:12:15.134556 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:15.134528    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7xgfwm" event={"ID":"1db899bb-6934-4409-821a-8dcb90cbd301","Type":"ContainerStarted","Data":"f88e67c09f2e9984e04927db8fb509c4ac9a6d80ef9adb787dc3bdcbe743514d"}
Sep 16 14:12:15.175417 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:15.175240    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4j9f5"]
Sep 16 14:12:15.178613 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:12:15.178579    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd95a3bf2_c2ea_4f89_b356_9d6f511db4ca.slice/crio-a9683aebe924c535f1f4df645cdd64059903f23f1f5053d56c9e201b68f42fe5 WatchSource:0}: Error finding container a9683aebe924c535f1f4df645cdd64059903f23f1f5053d56c9e201b68f42fe5: Status 404 returned error can't find the container with id a9683aebe924c535f1f4df645cdd64059903f23f1f5053d56c9e201b68f42fe5
Sep 16 14:12:15.272663 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:15.272601    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cck4x9"]
Sep 16 14:12:15.273431 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:12:15.273408    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd841ae78_746b_4ad2_a0d1_f02b53a7cec0.slice/crio-ca280e497882c32aa81267223ad19c5c40c1bd02189118deef1fe5fcfb97cb9e WatchSource:0}: Error finding container ca280e497882c32aa81267223ad19c5c40c1bd02189118deef1fe5fcfb97cb9e: Status 404 returned error can't find the container with id ca280e497882c32aa81267223ad19c5c40c1bd02189118deef1fe5fcfb97cb9e
Sep 16 14:12:15.396346 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:15.396322    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wjpzl"]
Sep 16 14:12:15.429367 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:12:15.429330    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4952aa51_f3c2_4fb7_be41_7d0aa51c6d72.slice/crio-1049bb9036bfd47d9214496f34321c574ab8d026e5681b04cfd52a2d43df6b65 WatchSource:0}: Error finding container 1049bb9036bfd47d9214496f34321c574ab8d026e5681b04cfd52a2d43df6b65: Status 404 returned error can't find the container with id 1049bb9036bfd47d9214496f34321c574ab8d026e5681b04cfd52a2d43df6b65
Sep 16 14:12:16.140570 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:16.140535    2580 generic.go:356] "Generic (PLEG): container finished" podID="d95a3bf2-c2ea-4f89-b356-9d6f511db4ca" containerID="53638a6ed9751c844a08bfd63436309bb59b8f513d1cdef1e93b2f502436395a" exitCode=0
Sep 16 14:12:16.140991 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:16.140624    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4j9f5" event={"ID":"d95a3bf2-c2ea-4f89-b356-9d6f511db4ca","Type":"ContainerDied","Data":"53638a6ed9751c844a08bfd63436309bb59b8f513d1cdef1e93b2f502436395a"}
Sep 16 14:12:16.140991 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:16.140679    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4j9f5" event={"ID":"d95a3bf2-c2ea-4f89-b356-9d6f511db4ca","Type":"ContainerStarted","Data":"a9683aebe924c535f1f4df645cdd64059903f23f1f5053d56c9e201b68f42fe5"}
Sep 16 14:12:16.142182 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:16.142161    2580 generic.go:356] "Generic (PLEG): container finished" podID="4952aa51-f3c2-4fb7-be41-7d0aa51c6d72" containerID="ff5b9c46844c8119054be8f068b18cc58bc64c1f2df9dafb53b5d893e162e999" exitCode=0
Sep 16 14:12:16.142276 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:16.142260    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wjpzl" event={"ID":"4952aa51-f3c2-4fb7-be41-7d0aa51c6d72","Type":"ContainerDied","Data":"ff5b9c46844c8119054be8f068b18cc58bc64c1f2df9dafb53b5d893e162e999"}
Sep 16 14:12:16.142333 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:16.142303    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wjpzl" event={"ID":"4952aa51-f3c2-4fb7-be41-7d0aa51c6d72","Type":"ContainerStarted","Data":"1049bb9036bfd47d9214496f34321c574ab8d026e5681b04cfd52a2d43df6b65"}
Sep 16 14:12:16.143916 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:16.143889    2580 generic.go:356] "Generic (PLEG): container finished" podID="1db899bb-6934-4409-821a-8dcb90cbd301" containerID="e5f50fff8bb4f5fc4b7b387cdcf0fd747fa8ee2be0a2225b40a29d010d06fd69" exitCode=0
Sep 16 14:12:16.144008 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:16.143924    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7xgfwm" event={"ID":"1db899bb-6934-4409-821a-8dcb90cbd301","Type":"ContainerDied","Data":"e5f50fff8bb4f5fc4b7b387cdcf0fd747fa8ee2be0a2225b40a29d010d06fd69"}
Sep 16 14:12:16.145687 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:16.145668    2580 generic.go:356] "Generic (PLEG): container finished" podID="d841ae78-746b-4ad2-a0d1-f02b53a7cec0" containerID="fc743f3bf2e1a33eaa2b438fd5a9959cbf4084b2d6be04e45033d38157d04603" exitCode=0
Sep 16 14:12:16.145779 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:16.145692    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cck4x9" event={"ID":"d841ae78-746b-4ad2-a0d1-f02b53a7cec0","Type":"ContainerDied","Data":"fc743f3bf2e1a33eaa2b438fd5a9959cbf4084b2d6be04e45033d38157d04603"}
Sep 16 14:12:16.145779 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:16.145737    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cck4x9" event={"ID":"d841ae78-746b-4ad2-a0d1-f02b53a7cec0","Type":"ContainerStarted","Data":"ca280e497882c32aa81267223ad19c5c40c1bd02189118deef1fe5fcfb97cb9e"}
Sep 16 14:12:17.152746 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:17.152716    2580 generic.go:356] "Generic (PLEG): container finished" podID="1db899bb-6934-4409-821a-8dcb90cbd301" containerID="89ba0e6cbd510f2da5287bcd973eb349738bedd7ef1b628eb83f92754854877f" exitCode=0
Sep 16 14:12:17.153106 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:17.152801    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7xgfwm" event={"ID":"1db899bb-6934-4409-821a-8dcb90cbd301","Type":"ContainerDied","Data":"89ba0e6cbd510f2da5287bcd973eb349738bedd7ef1b628eb83f92754854877f"}
Sep 16 14:12:18.160111 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:18.160026    2580 generic.go:356] "Generic (PLEG): container finished" podID="4952aa51-f3c2-4fb7-be41-7d0aa51c6d72" containerID="2beb24f30c97e09546e45318c710a1b101264d1be420bb6bc851972c1f3eecd8" exitCode=0
Sep 16 14:12:18.160486 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:18.160110    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wjpzl" event={"ID":"4952aa51-f3c2-4fb7-be41-7d0aa51c6d72","Type":"ContainerDied","Data":"2beb24f30c97e09546e45318c710a1b101264d1be420bb6bc851972c1f3eecd8"}
Sep 16 14:12:18.162335 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:18.162311    2580 generic.go:356] "Generic (PLEG): container finished" podID="1db899bb-6934-4409-821a-8dcb90cbd301" containerID="e526a3dbafc470929df7d23bca0d73819ef2a63c4746254cd8134a9f6abaad92" exitCode=0
Sep 16 14:12:18.162441 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:18.162395    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7xgfwm" event={"ID":"1db899bb-6934-4409-821a-8dcb90cbd301","Type":"ContainerDied","Data":"e526a3dbafc470929df7d23bca0d73819ef2a63c4746254cd8134a9f6abaad92"}
Sep 16 14:12:19.168784 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:19.168756    2580 generic.go:356] "Generic (PLEG): container finished" podID="d95a3bf2-c2ea-4f89-b356-9d6f511db4ca" containerID="ac5ad0e71dad1be645bf2b82b2556ddc7220be3d408858277976770ef82d1b7d" exitCode=0
Sep 16 14:12:19.169113 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:19.168844    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4j9f5" event={"ID":"d95a3bf2-c2ea-4f89-b356-9d6f511db4ca","Type":"ContainerDied","Data":"ac5ad0e71dad1be645bf2b82b2556ddc7220be3d408858277976770ef82d1b7d"}
Sep 16 14:12:19.171221 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:19.171200    2580 generic.go:356] "Generic (PLEG): container finished" podID="4952aa51-f3c2-4fb7-be41-7d0aa51c6d72" containerID="e7fe01148a07537a6fe501fa700baed272ff8e6768dba88ca5add8af938ac55b" exitCode=0
Sep 16 14:12:19.171314 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:19.171277    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wjpzl" event={"ID":"4952aa51-f3c2-4fb7-be41-7d0aa51c6d72","Type":"ContainerDied","Data":"e7fe01148a07537a6fe501fa700baed272ff8e6768dba88ca5add8af938ac55b"}
Sep 16 14:12:19.298688 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:19.298635    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7xgfwm"
Sep 16 14:12:19.341278 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:19.341255    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1db899bb-6934-4409-821a-8dcb90cbd301-util\") pod \"1db899bb-6934-4409-821a-8dcb90cbd301\" (UID: \"1db899bb-6934-4409-821a-8dcb90cbd301\") "
Sep 16 14:12:19.341410 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:19.341308    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g9mw4\" (UniqueName: \"kubernetes.io/projected/1db899bb-6934-4409-821a-8dcb90cbd301-kube-api-access-g9mw4\") pod \"1db899bb-6934-4409-821a-8dcb90cbd301\" (UID: \"1db899bb-6934-4409-821a-8dcb90cbd301\") "
Sep 16 14:12:19.341410 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:19.341328    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1db899bb-6934-4409-821a-8dcb90cbd301-bundle\") pod \"1db899bb-6934-4409-821a-8dcb90cbd301\" (UID: \"1db899bb-6934-4409-821a-8dcb90cbd301\") "
Sep 16 14:12:19.341873 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:19.341845    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1db899bb-6934-4409-821a-8dcb90cbd301-bundle" (OuterVolumeSpecName: "bundle") pod "1db899bb-6934-4409-821a-8dcb90cbd301" (UID: "1db899bb-6934-4409-821a-8dcb90cbd301"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:12:19.343264 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:19.343242    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1db899bb-6934-4409-821a-8dcb90cbd301-kube-api-access-g9mw4" (OuterVolumeSpecName: "kube-api-access-g9mw4") pod "1db899bb-6934-4409-821a-8dcb90cbd301" (UID: "1db899bb-6934-4409-821a-8dcb90cbd301"). InnerVolumeSpecName "kube-api-access-g9mw4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:12:19.346273 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:19.346251    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1db899bb-6934-4409-821a-8dcb90cbd301-util" (OuterVolumeSpecName: "util") pod "1db899bb-6934-4409-821a-8dcb90cbd301" (UID: "1db899bb-6934-4409-821a-8dcb90cbd301"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:12:19.442564 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:19.442539    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1db899bb-6934-4409-821a-8dcb90cbd301-util\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:12:19.442653 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:19.442568    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g9mw4\" (UniqueName: \"kubernetes.io/projected/1db899bb-6934-4409-821a-8dcb90cbd301-kube-api-access-g9mw4\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:12:19.442653 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:19.442582    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1db899bb-6934-4409-821a-8dcb90cbd301-bundle\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:12:20.176995 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:20.176957    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7xgfwm" event={"ID":"1db899bb-6934-4409-821a-8dcb90cbd301","Type":"ContainerDied","Data":"f88e67c09f2e9984e04927db8fb509c4ac9a6d80ef9adb787dc3bdcbe743514d"}
Sep 16 14:12:20.177362 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:20.177010    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f88e67c09f2e9984e04927db8fb509c4ac9a6d80ef9adb787dc3bdcbe743514d"
Sep 16 14:12:20.177362 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:20.176978    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7xgfwm"
Sep 16 14:12:20.178724 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:20.178692    2580 generic.go:356] "Generic (PLEG): container finished" podID="d841ae78-746b-4ad2-a0d1-f02b53a7cec0" containerID="b63c2bc1a48d6fcebb449d85a9e69151413a4024f3735e5f71bcbbf69f3e90a7" exitCode=0
Sep 16 14:12:20.178837 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:20.178771    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cck4x9" event={"ID":"d841ae78-746b-4ad2-a0d1-f02b53a7cec0","Type":"ContainerDied","Data":"b63c2bc1a48d6fcebb449d85a9e69151413a4024f3735e5f71bcbbf69f3e90a7"}
Sep 16 14:12:20.181246 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:20.181226    2580 generic.go:356] "Generic (PLEG): container finished" podID="d95a3bf2-c2ea-4f89-b356-9d6f511db4ca" containerID="411538e114cf3b07cb157fcf2e4ec6bf2fb7672614e659a8a0baf6180691fc0d" exitCode=0
Sep 16 14:12:20.181319 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:20.181291    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4j9f5" event={"ID":"d95a3bf2-c2ea-4f89-b356-9d6f511db4ca","Type":"ContainerDied","Data":"411538e114cf3b07cb157fcf2e4ec6bf2fb7672614e659a8a0baf6180691fc0d"}
Sep 16 14:12:20.311982 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:20.311955    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wjpzl"
Sep 16 14:12:20.350292 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:20.350267    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mm459\" (UniqueName: \"kubernetes.io/projected/4952aa51-f3c2-4fb7-be41-7d0aa51c6d72-kube-api-access-mm459\") pod \"4952aa51-f3c2-4fb7-be41-7d0aa51c6d72\" (UID: \"4952aa51-f3c2-4fb7-be41-7d0aa51c6d72\") "
Sep 16 14:12:20.350416 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:20.350313    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4952aa51-f3c2-4fb7-be41-7d0aa51c6d72-util\") pod \"4952aa51-f3c2-4fb7-be41-7d0aa51c6d72\" (UID: \"4952aa51-f3c2-4fb7-be41-7d0aa51c6d72\") "
Sep 16 14:12:20.350416 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:20.350371    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4952aa51-f3c2-4fb7-be41-7d0aa51c6d72-bundle\") pod \"4952aa51-f3c2-4fb7-be41-7d0aa51c6d72\" (UID: \"4952aa51-f3c2-4fb7-be41-7d0aa51c6d72\") "
Sep 16 14:12:20.350790 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:20.350765    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4952aa51-f3c2-4fb7-be41-7d0aa51c6d72-bundle" (OuterVolumeSpecName: "bundle") pod "4952aa51-f3c2-4fb7-be41-7d0aa51c6d72" (UID: "4952aa51-f3c2-4fb7-be41-7d0aa51c6d72"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:12:20.352199 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:20.352175    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4952aa51-f3c2-4fb7-be41-7d0aa51c6d72-kube-api-access-mm459" (OuterVolumeSpecName: "kube-api-access-mm459") pod "4952aa51-f3c2-4fb7-be41-7d0aa51c6d72" (UID: "4952aa51-f3c2-4fb7-be41-7d0aa51c6d72"). InnerVolumeSpecName "kube-api-access-mm459". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:12:20.355703 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:20.355681    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4952aa51-f3c2-4fb7-be41-7d0aa51c6d72-util" (OuterVolumeSpecName: "util") pod "4952aa51-f3c2-4fb7-be41-7d0aa51c6d72" (UID: "4952aa51-f3c2-4fb7-be41-7d0aa51c6d72"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:12:20.451152 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:20.451125    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4952aa51-f3c2-4fb7-be41-7d0aa51c6d72-util\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:12:20.451152 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:20.451151    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4952aa51-f3c2-4fb7-be41-7d0aa51c6d72-bundle\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:12:20.451324 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:20.451160    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mm459\" (UniqueName: \"kubernetes.io/projected/4952aa51-f3c2-4fb7-be41-7d0aa51c6d72-kube-api-access-mm459\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:12:21.187575 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:21.187543    2580 generic.go:356] "Generic (PLEG): container finished" podID="d841ae78-746b-4ad2-a0d1-f02b53a7cec0" containerID="fb40db993d7a0b8cbb5fb7fb07e0b0bd32be2e3647c836d6d92dc6c6390bc397" exitCode=0
Sep 16 14:12:21.187968 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:21.187624    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cck4x9" event={"ID":"d841ae78-746b-4ad2-a0d1-f02b53a7cec0","Type":"ContainerDied","Data":"fb40db993d7a0b8cbb5fb7fb07e0b0bd32be2e3647c836d6d92dc6c6390bc397"}
Sep 16 14:12:21.189490 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:21.189471    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wjpzl"
Sep 16 14:12:21.189619 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:21.189517    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wjpzl" event={"ID":"4952aa51-f3c2-4fb7-be41-7d0aa51c6d72","Type":"ContainerDied","Data":"1049bb9036bfd47d9214496f34321c574ab8d026e5681b04cfd52a2d43df6b65"}
Sep 16 14:12:21.189619 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:21.189545    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1049bb9036bfd47d9214496f34321c574ab8d026e5681b04cfd52a2d43df6b65"
Sep 16 14:12:21.307843 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:21.307821    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4j9f5"
Sep 16 14:12:21.357955 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:21.357914    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d95a3bf2-c2ea-4f89-b356-9d6f511db4ca-bundle\") pod \"d95a3bf2-c2ea-4f89-b356-9d6f511db4ca\" (UID: \"d95a3bf2-c2ea-4f89-b356-9d6f511db4ca\") "
Sep 16 14:12:21.358094 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:21.357975    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d95a3bf2-c2ea-4f89-b356-9d6f511db4ca-util\") pod \"d95a3bf2-c2ea-4f89-b356-9d6f511db4ca\" (UID: \"d95a3bf2-c2ea-4f89-b356-9d6f511db4ca\") "
Sep 16 14:12:21.358094 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:21.358016    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vm2mr\" (UniqueName: \"kubernetes.io/projected/d95a3bf2-c2ea-4f89-b356-9d6f511db4ca-kube-api-access-vm2mr\") pod \"d95a3bf2-c2ea-4f89-b356-9d6f511db4ca\" (UID: \"d95a3bf2-c2ea-4f89-b356-9d6f511db4ca\") "
Sep 16 14:12:21.358681 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:21.358657    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d95a3bf2-c2ea-4f89-b356-9d6f511db4ca-bundle" (OuterVolumeSpecName: "bundle") pod "d95a3bf2-c2ea-4f89-b356-9d6f511db4ca" (UID: "d95a3bf2-c2ea-4f89-b356-9d6f511db4ca"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:12:21.360176 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:21.360157    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d95a3bf2-c2ea-4f89-b356-9d6f511db4ca-kube-api-access-vm2mr" (OuterVolumeSpecName: "kube-api-access-vm2mr") pod "d95a3bf2-c2ea-4f89-b356-9d6f511db4ca" (UID: "d95a3bf2-c2ea-4f89-b356-9d6f511db4ca"). InnerVolumeSpecName "kube-api-access-vm2mr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:12:21.362652 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:21.362626    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d95a3bf2-c2ea-4f89-b356-9d6f511db4ca-util" (OuterVolumeSpecName: "util") pod "d95a3bf2-c2ea-4f89-b356-9d6f511db4ca" (UID: "d95a3bf2-c2ea-4f89-b356-9d6f511db4ca"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:12:21.458705 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:21.458680    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d95a3bf2-c2ea-4f89-b356-9d6f511db4ca-bundle\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:12:21.458705 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:21.458703    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d95a3bf2-c2ea-4f89-b356-9d6f511db4ca-util\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:12:21.458705 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:21.458713    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vm2mr\" (UniqueName: \"kubernetes.io/projected/d95a3bf2-c2ea-4f89-b356-9d6f511db4ca-kube-api-access-vm2mr\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:12:22.195572 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:22.195538    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4j9f5"
Sep 16 14:12:22.195940 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:22.195537    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4j9f5" event={"ID":"d95a3bf2-c2ea-4f89-b356-9d6f511db4ca","Type":"ContainerDied","Data":"a9683aebe924c535f1f4df645cdd64059903f23f1f5053d56c9e201b68f42fe5"}
Sep 16 14:12:22.195940 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:22.195660    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a9683aebe924c535f1f4df645cdd64059903f23f1f5053d56c9e201b68f42fe5"
Sep 16 14:12:22.324235 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:22.324211    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cck4x9"
Sep 16 14:12:22.365124 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:22.365098    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d841ae78-746b-4ad2-a0d1-f02b53a7cec0-util\") pod \"d841ae78-746b-4ad2-a0d1-f02b53a7cec0\" (UID: \"d841ae78-746b-4ad2-a0d1-f02b53a7cec0\") "
Sep 16 14:12:22.365268 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:22.365137    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d841ae78-746b-4ad2-a0d1-f02b53a7cec0-bundle\") pod \"d841ae78-746b-4ad2-a0d1-f02b53a7cec0\" (UID: \"d841ae78-746b-4ad2-a0d1-f02b53a7cec0\") "
Sep 16 14:12:22.365268 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:22.365227    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xd89v\" (UniqueName: \"kubernetes.io/projected/d841ae78-746b-4ad2-a0d1-f02b53a7cec0-kube-api-access-xd89v\") pod \"d841ae78-746b-4ad2-a0d1-f02b53a7cec0\" (UID: \"d841ae78-746b-4ad2-a0d1-f02b53a7cec0\") "
Sep 16 14:12:22.365697 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:22.365666    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d841ae78-746b-4ad2-a0d1-f02b53a7cec0-bundle" (OuterVolumeSpecName: "bundle") pod "d841ae78-746b-4ad2-a0d1-f02b53a7cec0" (UID: "d841ae78-746b-4ad2-a0d1-f02b53a7cec0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:12:22.367252 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:22.367228    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d841ae78-746b-4ad2-a0d1-f02b53a7cec0-kube-api-access-xd89v" (OuterVolumeSpecName: "kube-api-access-xd89v") pod "d841ae78-746b-4ad2-a0d1-f02b53a7cec0" (UID: "d841ae78-746b-4ad2-a0d1-f02b53a7cec0"). InnerVolumeSpecName "kube-api-access-xd89v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:12:22.369619 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:22.369599    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d841ae78-746b-4ad2-a0d1-f02b53a7cec0-util" (OuterVolumeSpecName: "util") pod "d841ae78-746b-4ad2-a0d1-f02b53a7cec0" (UID: "d841ae78-746b-4ad2-a0d1-f02b53a7cec0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:12:22.466561 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:22.466466    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xd89v\" (UniqueName: \"kubernetes.io/projected/d841ae78-746b-4ad2-a0d1-f02b53a7cec0-kube-api-access-xd89v\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:12:22.466561 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:22.466492    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d841ae78-746b-4ad2-a0d1-f02b53a7cec0-util\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:12:22.466561 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:22.466523    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d841ae78-746b-4ad2-a0d1-f02b53a7cec0-bundle\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:12:23.201836 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:23.201802    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cck4x9" event={"ID":"d841ae78-746b-4ad2-a0d1-f02b53a7cec0","Type":"ContainerDied","Data":"ca280e497882c32aa81267223ad19c5c40c1bd02189118deef1fe5fcfb97cb9e"}
Sep 16 14:12:23.202204 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:23.201844    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cck4x9"
Sep 16 14:12:23.202204 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:23.201847    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ca280e497882c32aa81267223ad19c5c40c1bd02189118deef1fe5fcfb97cb9e"
Sep 16 14:12:30.296242 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:30.296208    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-j4pg9"]
Sep 16 14:12:30.296847 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:30.296620    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d95a3bf2-c2ea-4f89-b356-9d6f511db4ca" containerName="pull"
Sep 16 14:12:30.296847 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:30.296637    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d95a3bf2-c2ea-4f89-b356-9d6f511db4ca" containerName="pull"
Sep 16 14:12:30.296847 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:30.296649    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1db899bb-6934-4409-821a-8dcb90cbd301" containerName="util"
Sep 16 14:12:30.296847 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:30.296656    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1db899bb-6934-4409-821a-8dcb90cbd301" containerName="util"
Sep 16 14:12:30.296847 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:30.296670    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d95a3bf2-c2ea-4f89-b356-9d6f511db4ca" containerName="extract"
Sep 16 14:12:30.296847 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:30.296678    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d95a3bf2-c2ea-4f89-b356-9d6f511db4ca" containerName="extract"
Sep 16 14:12:30.296847 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:30.296688    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4952aa51-f3c2-4fb7-be41-7d0aa51c6d72" containerName="pull"
Sep 16 14:12:30.296847 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:30.296696    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4952aa51-f3c2-4fb7-be41-7d0aa51c6d72" containerName="pull"
Sep 16 14:12:30.296847 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:30.296715    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4952aa51-f3c2-4fb7-be41-7d0aa51c6d72" containerName="util"
Sep 16 14:12:30.296847 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:30.296723    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4952aa51-f3c2-4fb7-be41-7d0aa51c6d72" containerName="util"
Sep 16 14:12:30.296847 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:30.296731    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1db899bb-6934-4409-821a-8dcb90cbd301" containerName="extract"
Sep 16 14:12:30.296847 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:30.296739    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1db899bb-6934-4409-821a-8dcb90cbd301" containerName="extract"
Sep 16 14:12:30.296847 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:30.296748    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4952aa51-f3c2-4fb7-be41-7d0aa51c6d72" containerName="extract"
Sep 16 14:12:30.296847 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:30.296756    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4952aa51-f3c2-4fb7-be41-7d0aa51c6d72" containerName="extract"
Sep 16 14:12:30.296847 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:30.296765    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d841ae78-746b-4ad2-a0d1-f02b53a7cec0" containerName="extract"
Sep 16 14:12:30.296847 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:30.296773    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d841ae78-746b-4ad2-a0d1-f02b53a7cec0" containerName="extract"
Sep 16 14:12:30.296847 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:30.296794    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d95a3bf2-c2ea-4f89-b356-9d6f511db4ca" containerName="util"
Sep 16 14:12:30.296847 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:30.296803    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d95a3bf2-c2ea-4f89-b356-9d6f511db4ca" containerName="util"
Sep 16 14:12:30.296847 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:30.296811    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d841ae78-746b-4ad2-a0d1-f02b53a7cec0" containerName="util"
Sep 16 14:12:30.296847 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:30.296820    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d841ae78-746b-4ad2-a0d1-f02b53a7cec0" containerName="util"
Sep 16 14:12:30.296847 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:30.296829    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1db899bb-6934-4409-821a-8dcb90cbd301" containerName="pull"
Sep 16 14:12:30.296847 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:30.296836    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1db899bb-6934-4409-821a-8dcb90cbd301" containerName="pull"
Sep 16 14:12:30.296847 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:30.296843    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d841ae78-746b-4ad2-a0d1-f02b53a7cec0" containerName="pull"
Sep 16 14:12:30.296847 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:30.296850    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d841ae78-746b-4ad2-a0d1-f02b53a7cec0" containerName="pull"
Sep 16 14:12:30.297981 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:30.296924    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d841ae78-746b-4ad2-a0d1-f02b53a7cec0" containerName="extract"
Sep 16 14:12:30.297981 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:30.296937    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4952aa51-f3c2-4fb7-be41-7d0aa51c6d72" containerName="extract"
Sep 16 14:12:30.297981 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:30.296946    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d95a3bf2-c2ea-4f89-b356-9d6f511db4ca" containerName="extract"
Sep 16 14:12:30.297981 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:30.296957    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1db899bb-6934-4409-821a-8dcb90cbd301" containerName="extract"
Sep 16 14:12:30.301188 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:30.301169    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-j4pg9"
Sep 16 14:12:30.303243 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:30.303221    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 16 14:12:30.303339 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:30.303294    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-b4hpj\""
Sep 16 14:12:30.305293 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:30.305277    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 16 14:12:30.311467 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:30.311447    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-j4pg9"]
Sep 16 14:12:30.426742 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:30.426706    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbkgp\" (UniqueName: \"kubernetes.io/projected/b244a479-4fb3-4bdf-a42a-fad2ff97f288-kube-api-access-zbkgp\") pod \"authorino-operator-76b478bd78-j4pg9\" (UID: \"b244a479-4fb3-4bdf-a42a-fad2ff97f288\") " pod="kuadrant-system/authorino-operator-76b478bd78-j4pg9"
Sep 16 14:12:30.528175 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:30.528133    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbkgp\" (UniqueName: \"kubernetes.io/projected/b244a479-4fb3-4bdf-a42a-fad2ff97f288-kube-api-access-zbkgp\") pod \"authorino-operator-76b478bd78-j4pg9\" (UID: \"b244a479-4fb3-4bdf-a42a-fad2ff97f288\") " pod="kuadrant-system/authorino-operator-76b478bd78-j4pg9"
Sep 16 14:12:30.539917 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:30.539880    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbkgp\" (UniqueName: \"kubernetes.io/projected/b244a479-4fb3-4bdf-a42a-fad2ff97f288-kube-api-access-zbkgp\") pod \"authorino-operator-76b478bd78-j4pg9\" (UID: \"b244a479-4fb3-4bdf-a42a-fad2ff97f288\") " pod="kuadrant-system/authorino-operator-76b478bd78-j4pg9"
Sep 16 14:12:30.611950 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:30.611865    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-j4pg9"
Sep 16 14:12:30.891881 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:30.891861    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-j4pg9"]
Sep 16 14:12:30.893908 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:12:30.893873    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb244a479_4fb3_4bdf_a42a_fad2ff97f288.slice/crio-19e62f3a9a45789d7a8b548c018d09f9da97557532c0b78b29e3de6b9b1c937a WatchSource:0}: Error finding container 19e62f3a9a45789d7a8b548c018d09f9da97557532c0b78b29e3de6b9b1c937a: Status 404 returned error can't find the container with id 19e62f3a9a45789d7a8b548c018d09f9da97557532c0b78b29e3de6b9b1c937a
Sep 16 14:12:31.238493 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:31.238458    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-j4pg9" event={"ID":"b244a479-4fb3-4bdf-a42a-fad2ff97f288","Type":"ContainerStarted","Data":"19e62f3a9a45789d7a8b548c018d09f9da97557532c0b78b29e3de6b9b1c937a"}
Sep 16 14:12:34.253812 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:34.253771    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-j4pg9" event={"ID":"b244a479-4fb3-4bdf-a42a-fad2ff97f288","Type":"ContainerStarted","Data":"19fcaa8e1bb5d4ad9f6c3b3a259c167d58c47cecdb928a85493a1899efdcb99a"}
Sep 16 14:12:34.254182 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:34.253859    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-76b478bd78-j4pg9"
Sep 16 14:12:34.273555 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:34.273492    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-76b478bd78-j4pg9" podStartSLOduration=1.115225322 podStartE2EDuration="4.273478794s" totalImagesPullingTime="3.158253472s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:12:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:12:34.272148382 +0000 UTC m=+487.287258267" watchObservedRunningTime="2026-09-16 14:12:34.273478794 +0000 UTC m=+487.288588654"
Sep 16 14:12:34.696339 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:34.696303    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-trqrn"]
Sep 16 14:12:34.699622 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:34.699607    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-trqrn"
Sep 16 14:12:34.702143 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:34.702124    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-l98d8\""
Sep 16 14:12:34.716279 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:34.716256    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-trqrn"]
Sep 16 14:12:34.765003 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:34.764970    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/99762435-2eca-4cf5-841a-cb8d6e8893cd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-trqrn\" (UID: \"99762435-2eca-4cf5-841a-cb8d6e8893cd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-trqrn"
Sep 16 14:12:34.765153 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:34.765022    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxlf4\" (UniqueName: \"kubernetes.io/projected/99762435-2eca-4cf5-841a-cb8d6e8893cd-kube-api-access-vxlf4\") pod \"kuadrant-operator-controller-manager-76f8f686bf-trqrn\" (UID: \"99762435-2eca-4cf5-841a-cb8d6e8893cd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-trqrn"
Sep 16 14:12:34.865442 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:34.865407    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/99762435-2eca-4cf5-841a-cb8d6e8893cd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-trqrn\" (UID: \"99762435-2eca-4cf5-841a-cb8d6e8893cd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-trqrn"
Sep 16 14:12:34.865628 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:34.865467    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxlf4\" (UniqueName: \"kubernetes.io/projected/99762435-2eca-4cf5-841a-cb8d6e8893cd-kube-api-access-vxlf4\") pod \"kuadrant-operator-controller-manager-76f8f686bf-trqrn\" (UID: \"99762435-2eca-4cf5-841a-cb8d6e8893cd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-trqrn"
Sep 16 14:12:34.865850 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:34.865831    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/99762435-2eca-4cf5-841a-cb8d6e8893cd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-trqrn\" (UID: \"99762435-2eca-4cf5-841a-cb8d6e8893cd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-trqrn"
Sep 16 14:12:34.873783 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:34.873763    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxlf4\" (UniqueName: \"kubernetes.io/projected/99762435-2eca-4cf5-841a-cb8d6e8893cd-kube-api-access-vxlf4\") pod \"kuadrant-operator-controller-manager-76f8f686bf-trqrn\" (UID: \"99762435-2eca-4cf5-841a-cb8d6e8893cd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-trqrn"
Sep 16 14:12:35.008953 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:35.008858    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-trqrn"
Sep 16 14:12:35.302938 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:35.302915    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-trqrn"]
Sep 16 14:12:35.305145 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:12:35.305118    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99762435_2eca_4cf5_841a_cb8d6e8893cd.slice/crio-27b7d22e7288580212094c140ab12297d99e11e9feb3f72a65dfffafad0c0799 WatchSource:0}: Error finding container 27b7d22e7288580212094c140ab12297d99e11e9feb3f72a65dfffafad0c0799: Status 404 returned error can't find the container with id 27b7d22e7288580212094c140ab12297d99e11e9feb3f72a65dfffafad0c0799
Sep 16 14:12:36.265341 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:36.265302    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-trqrn" event={"ID":"99762435-2eca-4cf5-841a-cb8d6e8893cd","Type":"ContainerStarted","Data":"27b7d22e7288580212094c140ab12297d99e11e9feb3f72a65dfffafad0c0799"}
Sep 16 14:12:36.944577 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:36.944540    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-2phrz"]
Sep 16 14:12:36.947998 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:36.947976    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-2phrz"
Sep 16 14:12:36.950227 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:36.950207    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-nvrds\""
Sep 16 14:12:36.950577 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:36.950555    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 16 14:12:36.958485 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:36.958463    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-2phrz"]
Sep 16 14:12:37.083487 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:37.083448    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgtdc\" (UniqueName: \"kubernetes.io/projected/f0ebc0cd-7c60-4324-8820-d5858eb98477-kube-api-access-xgtdc\") pod \"dns-operator-controller-manager-7b84cd7c9c-2phrz\" (UID: \"f0ebc0cd-7c60-4324-8820-d5858eb98477\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-2phrz"
Sep 16 14:12:37.184877 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:37.184834    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgtdc\" (UniqueName: \"kubernetes.io/projected/f0ebc0cd-7c60-4324-8820-d5858eb98477-kube-api-access-xgtdc\") pod \"dns-operator-controller-manager-7b84cd7c9c-2phrz\" (UID: \"f0ebc0cd-7c60-4324-8820-d5858eb98477\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-2phrz"
Sep 16 14:12:37.195827 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:37.195758    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgtdc\" (UniqueName: \"kubernetes.io/projected/f0ebc0cd-7c60-4324-8820-d5858eb98477-kube-api-access-xgtdc\") pod \"dns-operator-controller-manager-7b84cd7c9c-2phrz\" (UID: \"f0ebc0cd-7c60-4324-8820-d5858eb98477\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-2phrz"
Sep 16 14:12:37.261477 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:37.261442    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-2phrz"
Sep 16 14:12:37.549858 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:37.549825    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-2phrz"]
Sep 16 14:12:37.552272 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:12:37.552241    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0ebc0cd_7c60_4324_8820_d5858eb98477.slice/crio-f908ecc7cd992b19380686ec569ffaa17c2c69276c90b4afc231705d93b2a2f8 WatchSource:0}: Error finding container f908ecc7cd992b19380686ec569ffaa17c2c69276c90b4afc231705d93b2a2f8: Status 404 returned error can't find the container with id f908ecc7cd992b19380686ec569ffaa17c2c69276c90b4afc231705d93b2a2f8
Sep 16 14:12:38.277697 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:38.277663    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-2phrz" event={"ID":"f0ebc0cd-7c60-4324-8820-d5858eb98477","Type":"ContainerStarted","Data":"f908ecc7cd992b19380686ec569ffaa17c2c69276c90b4afc231705d93b2a2f8"}
Sep 16 14:12:38.880588 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:38.880554    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-zz4pj"]
Sep 16 14:12:38.884142 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:38.884108    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zz4pj"
Sep 16 14:12:38.886807 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:38.886779    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-r9srf\""
Sep 16 14:12:38.902919 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:38.902891    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-zz4pj"]
Sep 16 14:12:39.000486 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:39.000453    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6l7r\" (UniqueName: \"kubernetes.io/projected/afd526bd-730b-4f26-8a2e-153c122b56ef-kube-api-access-r6l7r\") pod \"limitador-operator-controller-manager-74495c69f8-zz4pj\" (UID: \"afd526bd-730b-4f26-8a2e-153c122b56ef\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zz4pj"
Sep 16 14:12:39.101951 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:39.101916    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6l7r\" (UniqueName: \"kubernetes.io/projected/afd526bd-730b-4f26-8a2e-153c122b56ef-kube-api-access-r6l7r\") pod \"limitador-operator-controller-manager-74495c69f8-zz4pj\" (UID: \"afd526bd-730b-4f26-8a2e-153c122b56ef\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zz4pj"
Sep 16 14:12:39.117676 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:39.117657    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6l7r\" (UniqueName: \"kubernetes.io/projected/afd526bd-730b-4f26-8a2e-153c122b56ef-kube-api-access-r6l7r\") pod \"limitador-operator-controller-manager-74495c69f8-zz4pj\" (UID: \"afd526bd-730b-4f26-8a2e-153c122b56ef\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zz4pj"
Sep 16 14:12:39.195481 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:39.195449    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zz4pj"
Sep 16 14:12:39.286992 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:39.286922    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-trqrn" event={"ID":"99762435-2eca-4cf5-841a-cb8d6e8893cd","Type":"ContainerStarted","Data":"0bde36f0d4d76c77ec1d16b56aa337e51be31c4680eb48ec3272ed8da7dfb0e9"}
Sep 16 14:12:39.287407 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:39.287080    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-trqrn"
Sep 16 14:12:39.308418 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:39.308370    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-trqrn" podStartSLOduration=1.5115435910000001 podStartE2EDuration="5.308350203s" totalImagesPullingTime="3.796806612s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:12:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:12:39.30592892 +0000 UTC m=+492.321038781" watchObservedRunningTime="2026-09-16 14:12:39.308350203 +0000 UTC m=+492.323460072"
Sep 16 14:12:39.489586 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:39.489556    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-zz4pj"]
Sep 16 14:12:39.491535 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:12:39.491488    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafd526bd_730b_4f26_8a2e_153c122b56ef.slice/crio-20b0288fbe87e0a33b73bdd34fd53ca0f94d063ba09c337ae9f3ad94d167558a WatchSource:0}: Error finding container 20b0288fbe87e0a33b73bdd34fd53ca0f94d063ba09c337ae9f3ad94d167558a: Status 404 returned error can't find the container with id 20b0288fbe87e0a33b73bdd34fd53ca0f94d063ba09c337ae9f3ad94d167558a
Sep 16 14:12:40.293039 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:40.292948    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zz4pj" event={"ID":"afd526bd-730b-4f26-8a2e-153c122b56ef","Type":"ContainerStarted","Data":"20b0288fbe87e0a33b73bdd34fd53ca0f94d063ba09c337ae9f3ad94d167558a"}
Sep 16 14:12:40.294664 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:40.294636    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-2phrz" event={"ID":"f0ebc0cd-7c60-4324-8820-d5858eb98477","Type":"ContainerStarted","Data":"4f59f6b4981651d8d92301194124c4a69d98df247182c38807ea6eac067c10e5"}
Sep 16 14:12:40.294800 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:40.294699    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-2phrz"
Sep 16 14:12:40.313559 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:40.313489    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-2phrz" podStartSLOduration=1.8408447209999999 podStartE2EDuration="4.313473163s" totalImagesPullingTime="2.472628442s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:12:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:12:40.311444154 +0000 UTC m=+493.326554029" watchObservedRunningTime="2026-09-16 14:12:40.313473163 +0000 UTC m=+493.328583075"
Sep 16 14:12:41.300620 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:41.300582    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zz4pj" event={"ID":"afd526bd-730b-4f26-8a2e-153c122b56ef","Type":"ContainerStarted","Data":"1eae0e30ac7980d84ae19d9be33f3b0769057bc3484d2d2685a7a99f8aef3f5a"}
Sep 16 14:12:41.301024 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:41.300853    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zz4pj"
Sep 16 14:12:41.326026 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:41.325968    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zz4pj" podStartSLOduration=1.639956983 podStartE2EDuration="3.325952648s" totalImagesPullingTime="1.685995665s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:12:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:12:41.323218856 +0000 UTC m=+494.338328718" watchObservedRunningTime="2026-09-16 14:12:41.325952648 +0000 UTC m=+494.341062508"
Sep 16 14:12:45.260639 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:45.260611    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-76b478bd78-j4pg9"
Sep 16 14:12:50.297815 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:50.297780    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-trqrn"
Sep 16 14:12:51.303000 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:51.302971    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-2phrz"
Sep 16 14:12:52.308427 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:52.308394    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zz4pj"
Sep 16 14:12:52.778284 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:52.778249    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-trqrn"]
Sep 16 14:12:52.778543 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:52.778518    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-trqrn" podUID="99762435-2eca-4cf5-841a-cb8d6e8893cd" containerName="manager" containerID="cri-o://0bde36f0d4d76c77ec1d16b56aa337e51be31c4680eb48ec3272ed8da7dfb0e9" gracePeriod=2
Sep 16 14:12:52.784322 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:52.784286    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-trqrn"]
Sep 16 14:12:52.797862 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:52.797835    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-zz4pj"]
Sep 16 14:12:52.798048 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:52.798025    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zz4pj" podUID="afd526bd-730b-4f26-8a2e-153c122b56ef" containerName="manager" containerID="cri-o://1eae0e30ac7980d84ae19d9be33f3b0769057bc3484d2d2685a7a99f8aef3f5a" gracePeriod=2
Sep 16 14:12:52.803029 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:52.803009    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-zz4pj"]
Sep 16 14:12:52.811869 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:12:52.811798    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-trqrn\" is forbidden: User \"system:node:ip-10-0-129-95.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-95.ec2.internal' and this object" podUID="99762435-2eca-4cf5-841a-cb8d6e8893cd" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-trqrn"
Sep 16 14:12:52.813184 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:52.813158    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cp9lh"]
Sep 16 14:12:52.813443 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:52.813430    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="afd526bd-730b-4f26-8a2e-153c122b56ef" containerName="manager"
Sep 16 14:12:52.813443 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:52.813443    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="afd526bd-730b-4f26-8a2e-153c122b56ef" containerName="manager"
Sep 16 14:12:52.813587 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:52.813461    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99762435-2eca-4cf5-841a-cb8d6e8893cd" containerName="manager"
Sep 16 14:12:52.813587 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:52.813466    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="99762435-2eca-4cf5-841a-cb8d6e8893cd" containerName="manager"
Sep 16 14:12:52.813587 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:52.813533    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="afd526bd-730b-4f26-8a2e-153c122b56ef" containerName="manager"
Sep 16 14:12:52.813587 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:52.813543    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="99762435-2eca-4cf5-841a-cb8d6e8893cd" containerName="manager"
Sep 16 14:12:52.817202 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:52.817185    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cp9lh"
Sep 16 14:12:52.820664 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:12:52.820631    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-zz4pj\" is forbidden: User \"system:node:ip-10-0-129-95.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-95.ec2.internal' and this object" podUID="afd526bd-730b-4f26-8a2e-153c122b56ef" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zz4pj"
Sep 16 14:12:52.834175 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:52.834107    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cp9lh"]
Sep 16 14:12:52.836020 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:52.835999    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-2262r"]
Sep 16 14:12:52.839457 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:52.839438    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2262r"
Sep 16 14:12:52.849946 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:52.849922    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-2262r"]
Sep 16 14:12:52.857255 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:12:52.857223    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-trqrn\" is forbidden: User \"system:node:ip-10-0-129-95.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-95.ec2.internal' and this object" podUID="99762435-2eca-4cf5-841a-cb8d6e8893cd" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-trqrn"
Sep 16 14:12:52.891289 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:12:52.891260    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-trqrn\" is forbidden: User \"system:node:ip-10-0-129-95.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-95.ec2.internal' and this object" podUID="99762435-2eca-4cf5-841a-cb8d6e8893cd" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-trqrn"
Sep 16 14:12:52.892539 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:12:52.892488    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-zz4pj\" is forbidden: User \"system:node:ip-10-0-129-95.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-95.ec2.internal' and this object" podUID="afd526bd-730b-4f26-8a2e-153c122b56ef" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zz4pj"
Sep 16 14:12:52.920452 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:52.920368    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/931901d4-a4a4-427c-acfd-b62eadbdbd24-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-cp9lh\" (UID: \"931901d4-a4a4-427c-acfd-b62eadbdbd24\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cp9lh"
Sep 16 14:12:52.920452 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:52.920412    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f94pn\" (UniqueName: \"kubernetes.io/projected/af8a3a0e-cec2-4fc6-b552-6e5fd93004b0-kube-api-access-f94pn\") pod \"limitador-operator-controller-manager-74495c69f8-2262r\" (UID: \"af8a3a0e-cec2-4fc6-b552-6e5fd93004b0\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2262r"
Sep 16 14:12:52.920663 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:52.920585    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxv6c\" (UniqueName: \"kubernetes.io/projected/931901d4-a4a4-427c-acfd-b62eadbdbd24-kube-api-access-pxv6c\") pod \"kuadrant-operator-controller-manager-76f8f686bf-cp9lh\" (UID: \"931901d4-a4a4-427c-acfd-b62eadbdbd24\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cp9lh"
Sep 16 14:12:53.021649 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:53.021625    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxv6c\" (UniqueName: \"kubernetes.io/projected/931901d4-a4a4-427c-acfd-b62eadbdbd24-kube-api-access-pxv6c\") pod \"kuadrant-operator-controller-manager-76f8f686bf-cp9lh\" (UID: \"931901d4-a4a4-427c-acfd-b62eadbdbd24\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cp9lh"
Sep 16 14:12:53.021753 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:53.021688    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/931901d4-a4a4-427c-acfd-b62eadbdbd24-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-cp9lh\" (UID: \"931901d4-a4a4-427c-acfd-b62eadbdbd24\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cp9lh"
Sep 16 14:12:53.021753 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:53.021717    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f94pn\" (UniqueName: \"kubernetes.io/projected/af8a3a0e-cec2-4fc6-b552-6e5fd93004b0-kube-api-access-f94pn\") pod \"limitador-operator-controller-manager-74495c69f8-2262r\" (UID: \"af8a3a0e-cec2-4fc6-b552-6e5fd93004b0\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2262r"
Sep 16 14:12:53.022072 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:53.022051    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/931901d4-a4a4-427c-acfd-b62eadbdbd24-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-cp9lh\" (UID: \"931901d4-a4a4-427c-acfd-b62eadbdbd24\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cp9lh"
Sep 16 14:12:53.028908 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:53.028858    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-trqrn"
Sep 16 14:12:53.030530 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:12:53.030493    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-zz4pj\" is forbidden: User \"system:node:ip-10-0-129-95.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-95.ec2.internal' and this object" podUID="afd526bd-730b-4f26-8a2e-153c122b56ef" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zz4pj"
Sep 16 14:12:53.031186 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:53.031169    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zz4pj"
Sep 16 14:12:53.031953 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:12:53.031925    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-trqrn\" is forbidden: User \"system:node:ip-10-0-129-95.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-95.ec2.internal' and this object" podUID="99762435-2eca-4cf5-841a-cb8d6e8893cd" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-trqrn"
Sep 16 14:12:53.033039 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:53.033021    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f94pn\" (UniqueName: \"kubernetes.io/projected/af8a3a0e-cec2-4fc6-b552-6e5fd93004b0-kube-api-access-f94pn\") pod \"limitador-operator-controller-manager-74495c69f8-2262r\" (UID: \"af8a3a0e-cec2-4fc6-b552-6e5fd93004b0\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2262r"
Sep 16 14:12:53.033406 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:12:53.033378    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-trqrn\" is forbidden: User \"system:node:ip-10-0-129-95.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-95.ec2.internal' and this object" podUID="99762435-2eca-4cf5-841a-cb8d6e8893cd" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-trqrn"
Sep 16 14:12:53.033645 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:53.033629    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxv6c\" (UniqueName: \"kubernetes.io/projected/931901d4-a4a4-427c-acfd-b62eadbdbd24-kube-api-access-pxv6c\") pod \"kuadrant-operator-controller-manager-76f8f686bf-cp9lh\" (UID: \"931901d4-a4a4-427c-acfd-b62eadbdbd24\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cp9lh"
Sep 16 14:12:53.034935 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:12:53.034917    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-zz4pj\" is forbidden: User \"system:node:ip-10-0-129-95.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-95.ec2.internal' and this object" podUID="afd526bd-730b-4f26-8a2e-153c122b56ef" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zz4pj"
Sep 16 14:12:53.122115 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:53.122081    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vxlf4\" (UniqueName: \"kubernetes.io/projected/99762435-2eca-4cf5-841a-cb8d6e8893cd-kube-api-access-vxlf4\") pod \"99762435-2eca-4cf5-841a-cb8d6e8893cd\" (UID: \"99762435-2eca-4cf5-841a-cb8d6e8893cd\") "
Sep 16 14:12:53.122279 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:53.122144    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r6l7r\" (UniqueName: \"kubernetes.io/projected/afd526bd-730b-4f26-8a2e-153c122b56ef-kube-api-access-r6l7r\") pod \"afd526bd-730b-4f26-8a2e-153c122b56ef\" (UID: \"afd526bd-730b-4f26-8a2e-153c122b56ef\") "
Sep 16 14:12:53.122279 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:53.122180    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/99762435-2eca-4cf5-841a-cb8d6e8893cd-extensions-socket-volume\") pod \"99762435-2eca-4cf5-841a-cb8d6e8893cd\" (UID: \"99762435-2eca-4cf5-841a-cb8d6e8893cd\") "
Sep 16 14:12:53.122703 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:53.122671    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99762435-2eca-4cf5-841a-cb8d6e8893cd-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "99762435-2eca-4cf5-841a-cb8d6e8893cd" (UID: "99762435-2eca-4cf5-841a-cb8d6e8893cd"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:12:53.124310 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:53.124289    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99762435-2eca-4cf5-841a-cb8d6e8893cd-kube-api-access-vxlf4" (OuterVolumeSpecName: "kube-api-access-vxlf4") pod "99762435-2eca-4cf5-841a-cb8d6e8893cd" (UID: "99762435-2eca-4cf5-841a-cb8d6e8893cd"). InnerVolumeSpecName "kube-api-access-vxlf4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:12:53.124310 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:53.124302    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/afd526bd-730b-4f26-8a2e-153c122b56ef-kube-api-access-r6l7r" (OuterVolumeSpecName: "kube-api-access-r6l7r") pod "afd526bd-730b-4f26-8a2e-153c122b56ef" (UID: "afd526bd-730b-4f26-8a2e-153c122b56ef"). InnerVolumeSpecName "kube-api-access-r6l7r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:12:53.217252 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:53.217215    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cp9lh"
Sep 16 14:12:53.223510 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:53.223475    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2262r"
Sep 16 14:12:53.223725 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:53.223705    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vxlf4\" (UniqueName: \"kubernetes.io/projected/99762435-2eca-4cf5-841a-cb8d6e8893cd-kube-api-access-vxlf4\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:12:53.223790 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:53.223741    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r6l7r\" (UniqueName: \"kubernetes.io/projected/afd526bd-730b-4f26-8a2e-153c122b56ef-kube-api-access-r6l7r\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:12:53.223790 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:53.223755    2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/99762435-2eca-4cf5-841a-cb8d6e8893cd-extensions-socket-volume\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:12:53.365193 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:53.365165    2580 generic.go:356] "Generic (PLEG): container finished" podID="99762435-2eca-4cf5-841a-cb8d6e8893cd" containerID="0bde36f0d4d76c77ec1d16b56aa337e51be31c4680eb48ec3272ed8da7dfb0e9" exitCode=0
Sep 16 14:12:53.365674 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:53.365219    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-trqrn"
Sep 16 14:12:53.365674 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:53.365276    2580 scope.go:117] "RemoveContainer" containerID="0bde36f0d4d76c77ec1d16b56aa337e51be31c4680eb48ec3272ed8da7dfb0e9"
Sep 16 14:12:53.366626 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:53.366607    2580 generic.go:356] "Generic (PLEG): container finished" podID="afd526bd-730b-4f26-8a2e-153c122b56ef" containerID="1eae0e30ac7980d84ae19d9be33f3b0769057bc3484d2d2685a7a99f8aef3f5a" exitCode=0
Sep 16 14:12:53.366720 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:53.366642    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zz4pj"
Sep 16 14:12:53.367929 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:12:53.367899    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-trqrn\" is forbidden: User \"system:node:ip-10-0-129-95.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-95.ec2.internal' and this object" podUID="99762435-2eca-4cf5-841a-cb8d6e8893cd" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-trqrn"
Sep 16 14:12:53.369715 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:12:53.369672    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-zz4pj\" is forbidden: User \"system:node:ip-10-0-129-95.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-95.ec2.internal' and this object" podUID="afd526bd-730b-4f26-8a2e-153c122b56ef" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zz4pj"
Sep 16 14:12:53.371209 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:12:53.371185    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-trqrn\" is forbidden: User \"system:node:ip-10-0-129-95.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-95.ec2.internal' and this object" podUID="99762435-2eca-4cf5-841a-cb8d6e8893cd" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-trqrn"
Sep 16 14:12:53.373415 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:12:53.373389    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-zz4pj\" is forbidden: User \"system:node:ip-10-0-129-95.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-95.ec2.internal' and this object" podUID="afd526bd-730b-4f26-8a2e-153c122b56ef" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zz4pj"
Sep 16 14:12:53.375136 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:53.375121    2580 scope.go:117] "RemoveContainer" containerID="0bde36f0d4d76c77ec1d16b56aa337e51be31c4680eb48ec3272ed8da7dfb0e9"
Sep 16 14:12:53.375417 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:12:53.375398    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0bde36f0d4d76c77ec1d16b56aa337e51be31c4680eb48ec3272ed8da7dfb0e9\": container with ID starting with 0bde36f0d4d76c77ec1d16b56aa337e51be31c4680eb48ec3272ed8da7dfb0e9 not found: ID does not exist" containerID="0bde36f0d4d76c77ec1d16b56aa337e51be31c4680eb48ec3272ed8da7dfb0e9"
Sep 16 14:12:53.375489 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:53.375422    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0bde36f0d4d76c77ec1d16b56aa337e51be31c4680eb48ec3272ed8da7dfb0e9"} err="failed to get container status \"0bde36f0d4d76c77ec1d16b56aa337e51be31c4680eb48ec3272ed8da7dfb0e9\": rpc error: code = NotFound desc = could not find container \"0bde36f0d4d76c77ec1d16b56aa337e51be31c4680eb48ec3272ed8da7dfb0e9\": container with ID starting with 0bde36f0d4d76c77ec1d16b56aa337e51be31c4680eb48ec3272ed8da7dfb0e9 not found: ID does not exist"
Sep 16 14:12:53.375489 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:53.375443    2580 scope.go:117] "RemoveContainer" containerID="1eae0e30ac7980d84ae19d9be33f3b0769057bc3484d2d2685a7a99f8aef3f5a"
Sep 16 14:12:53.376292 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:12:53.376271    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-trqrn\" is forbidden: User \"system:node:ip-10-0-129-95.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-95.ec2.internal' and this object" podUID="99762435-2eca-4cf5-841a-cb8d6e8893cd" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-trqrn"
Sep 16 14:12:53.377711 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:12:53.377691    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-zz4pj\" is forbidden: User \"system:node:ip-10-0-129-95.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-95.ec2.internal' and this object" podUID="afd526bd-730b-4f26-8a2e-153c122b56ef" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zz4pj"
Sep 16 14:12:53.379163 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:12:53.379144    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-zz4pj\" is forbidden: User \"system:node:ip-10-0-129-95.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-95.ec2.internal' and this object" podUID="afd526bd-730b-4f26-8a2e-153c122b56ef" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-zz4pj"
Sep 16 14:12:53.380531 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:12:53.380490    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-trqrn\" is forbidden: User \"system:node:ip-10-0-129-95.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-95.ec2.internal' and this object" podUID="99762435-2eca-4cf5-841a-cb8d6e8893cd" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-trqrn"
Sep 16 14:12:53.383021 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:53.383003    2580 scope.go:117] "RemoveContainer" containerID="1eae0e30ac7980d84ae19d9be33f3b0769057bc3484d2d2685a7a99f8aef3f5a"
Sep 16 14:12:53.383273 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:12:53.383256    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1eae0e30ac7980d84ae19d9be33f3b0769057bc3484d2d2685a7a99f8aef3f5a\": container with ID starting with 1eae0e30ac7980d84ae19d9be33f3b0769057bc3484d2d2685a7a99f8aef3f5a not found: ID does not exist" containerID="1eae0e30ac7980d84ae19d9be33f3b0769057bc3484d2d2685a7a99f8aef3f5a"
Sep 16 14:12:53.383314 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:53.383278    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1eae0e30ac7980d84ae19d9be33f3b0769057bc3484d2d2685a7a99f8aef3f5a"} err="failed to get container status \"1eae0e30ac7980d84ae19d9be33f3b0769057bc3484d2d2685a7a99f8aef3f5a\": rpc error: code = NotFound desc = could not find container \"1eae0e30ac7980d84ae19d9be33f3b0769057bc3484d2d2685a7a99f8aef3f5a\": container with ID starting with 1eae0e30ac7980d84ae19d9be33f3b0769057bc3484d2d2685a7a99f8aef3f5a not found: ID does not exist"
Sep 16 14:12:53.429757 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:53.429726    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99762435-2eca-4cf5-841a-cb8d6e8893cd" path="/var/lib/kubelet/pods/99762435-2eca-4cf5-841a-cb8d6e8893cd/volumes"
Sep 16 14:12:53.430039 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:53.430027    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="afd526bd-730b-4f26-8a2e-153c122b56ef" path="/var/lib/kubelet/pods/afd526bd-730b-4f26-8a2e-153c122b56ef/volumes"
Sep 16 14:12:53.528759 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:12:53.528728    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf8a3a0e_cec2_4fc6_b552_6e5fd93004b0.slice/crio-d421985e1d384bb76d68f128ef104d1b0d7ab7e96e78704ef0abcc99928fd3f1 WatchSource:0}: Error finding container d421985e1d384bb76d68f128ef104d1b0d7ab7e96e78704ef0abcc99928fd3f1: Status 404 returned error can't find the container with id d421985e1d384bb76d68f128ef104d1b0d7ab7e96e78704ef0abcc99928fd3f1
Sep 16 14:12:53.531292 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:12:53.531270    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod931901d4_a4a4_427c_acfd_b62eadbdbd24.slice/crio-736f4346f78fbf2b80c6db0900ab4d37f73b761440ba2b419c026dc9cfdf1e52 WatchSource:0}: Error finding container 736f4346f78fbf2b80c6db0900ab4d37f73b761440ba2b419c026dc9cfdf1e52: Status 404 returned error can't find the container with id 736f4346f78fbf2b80c6db0900ab4d37f73b761440ba2b419c026dc9cfdf1e52
Sep 16 14:12:53.534244 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:53.534219    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-2262r"]
Sep 16 14:12:53.536634 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:53.536616    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cp9lh"]
Sep 16 14:12:53.945113 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:53.945078    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zf8st"]
Sep 16 14:12:53.948329 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:53.948313    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zf8st"
Sep 16 14:12:53.967508 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:53.967477    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zf8st"]
Sep 16 14:12:54.029987 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:54.029957    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nz4qg\" (UniqueName: \"kubernetes.io/projected/dd3f16be-5f14-4ca7-98d2-dc26179a97a7-kube-api-access-nz4qg\") pod \"kuadrant-operator-controller-manager-5769f99d9f-zf8st\" (UID: \"dd3f16be-5f14-4ca7-98d2-dc26179a97a7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zf8st"
Sep 16 14:12:54.030174 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:54.030014    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dd3f16be-5f14-4ca7-98d2-dc26179a97a7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-zf8st\" (UID: \"dd3f16be-5f14-4ca7-98d2-dc26179a97a7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zf8st"
Sep 16 14:12:54.130954 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:54.130874    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nz4qg\" (UniqueName: \"kubernetes.io/projected/dd3f16be-5f14-4ca7-98d2-dc26179a97a7-kube-api-access-nz4qg\") pod \"kuadrant-operator-controller-manager-5769f99d9f-zf8st\" (UID: \"dd3f16be-5f14-4ca7-98d2-dc26179a97a7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zf8st"
Sep 16 14:12:54.130954 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:54.130940    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dd3f16be-5f14-4ca7-98d2-dc26179a97a7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-zf8st\" (UID: \"dd3f16be-5f14-4ca7-98d2-dc26179a97a7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zf8st"
Sep 16 14:12:54.131258 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:54.131237    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dd3f16be-5f14-4ca7-98d2-dc26179a97a7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-zf8st\" (UID: \"dd3f16be-5f14-4ca7-98d2-dc26179a97a7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zf8st"
Sep 16 14:12:54.139512 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:54.139474    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nz4qg\" (UniqueName: \"kubernetes.io/projected/dd3f16be-5f14-4ca7-98d2-dc26179a97a7-kube-api-access-nz4qg\") pod \"kuadrant-operator-controller-manager-5769f99d9f-zf8st\" (UID: \"dd3f16be-5f14-4ca7-98d2-dc26179a97a7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zf8st"
Sep 16 14:12:54.257387 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:54.257350    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zf8st"
Sep 16 14:12:54.378265 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:54.378222    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cp9lh" event={"ID":"931901d4-a4a4-427c-acfd-b62eadbdbd24","Type":"ContainerStarted","Data":"e29ebbe8c3fb8e80beeaacbfa04ffec962348c645914069d341b8fcf1764395c"}
Sep 16 14:12:54.378662 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:54.378408    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cp9lh"
Sep 16 14:12:54.378662 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:54.378452    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cp9lh" event={"ID":"931901d4-a4a4-427c-acfd-b62eadbdbd24","Type":"ContainerStarted","Data":"736f4346f78fbf2b80c6db0900ab4d37f73b761440ba2b419c026dc9cfdf1e52"}
Sep 16 14:12:54.381584 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:54.381524    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2262r" event={"ID":"af8a3a0e-cec2-4fc6-b552-6e5fd93004b0","Type":"ContainerStarted","Data":"e11d80a6661887c0916c144b7e25ee65723c2a91b3f52751a649cc804478d1e2"}
Sep 16 14:12:54.381584 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:54.381561    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2262r" event={"ID":"af8a3a0e-cec2-4fc6-b552-6e5fd93004b0","Type":"ContainerStarted","Data":"d421985e1d384bb76d68f128ef104d1b0d7ab7e96e78704ef0abcc99928fd3f1"}
Sep 16 14:12:54.381721 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:54.381593    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2262r"
Sep 16 14:12:54.422888 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:54.422812    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cp9lh" podStartSLOduration=2.422797451 podStartE2EDuration="2.422797451s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:12:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:12:54.399914924 +0000 UTC m=+507.415024784" watchObservedRunningTime="2026-09-16 14:12:54.422797451 +0000 UTC m=+507.437907311"
Sep 16 14:12:54.423424 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:54.423399    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2262r" podStartSLOduration=2.42339068 podStartE2EDuration="2.42339068s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:12:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:12:54.422484072 +0000 UTC m=+507.437593931" watchObservedRunningTime="2026-09-16 14:12:54.42339068 +0000 UTC m=+507.438500540"
Sep 16 14:12:54.554091 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:54.554066    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zf8st"]
Sep 16 14:12:54.557015 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:12:54.556979    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd3f16be_5f14_4ca7_98d2_dc26179a97a7.slice/crio-12f198a2024d6fb64c9626694799980ce1deb464c9af237c585bab9b57c61b11 WatchSource:0}: Error finding container 12f198a2024d6fb64c9626694799980ce1deb464c9af237c585bab9b57c61b11: Status 404 returned error can't find the container with id 12f198a2024d6fb64c9626694799980ce1deb464c9af237c585bab9b57c61b11
Sep 16 14:12:55.387938 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:55.387901    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zf8st" event={"ID":"dd3f16be-5f14-4ca7-98d2-dc26179a97a7","Type":"ContainerStarted","Data":"5d44c1ddeabfba0f37528e40d2009b5351406c3e0921262e09e707a1875d18c0"}
Sep 16 14:12:55.388379 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:55.388188    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zf8st"
Sep 16 14:12:55.388379 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:55.388224    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zf8st" event={"ID":"dd3f16be-5f14-4ca7-98d2-dc26179a97a7","Type":"ContainerStarted","Data":"12f198a2024d6fb64c9626694799980ce1deb464c9af237c585bab9b57c61b11"}
Sep 16 14:12:55.432613 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:12:55.432553    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zf8st" podStartSLOduration=2.432535784 podStartE2EDuration="2.432535784s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:12:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:12:55.430512687 +0000 UTC m=+508.445622538" watchObservedRunningTime="2026-09-16 14:12:55.432535784 +0000 UTC m=+508.447645645"
Sep 16 14:13:05.390762 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:05.390728    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2262r"
Sep 16 14:13:05.391244 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:05.390912    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cp9lh"
Sep 16 14:13:06.395752 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:06.395715    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zf8st"
Sep 16 14:13:06.472913 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:06.472880    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cp9lh"]
Sep 16 14:13:06.473176 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:06.473135    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cp9lh" podUID="931901d4-a4a4-427c-acfd-b62eadbdbd24" containerName="manager" containerID="cri-o://e29ebbe8c3fb8e80beeaacbfa04ffec962348c645914069d341b8fcf1764395c" gracePeriod=10
Sep 16 14:13:06.719813 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:06.719788    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cp9lh"
Sep 16 14:13:06.834907 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:06.834874    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/931901d4-a4a4-427c-acfd-b62eadbdbd24-extensions-socket-volume\") pod \"931901d4-a4a4-427c-acfd-b62eadbdbd24\" (UID: \"931901d4-a4a4-427c-acfd-b62eadbdbd24\") "
Sep 16 14:13:06.835052 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:06.834949    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pxv6c\" (UniqueName: \"kubernetes.io/projected/931901d4-a4a4-427c-acfd-b62eadbdbd24-kube-api-access-pxv6c\") pod \"931901d4-a4a4-427c-acfd-b62eadbdbd24\" (UID: \"931901d4-a4a4-427c-acfd-b62eadbdbd24\") "
Sep 16 14:13:06.835318 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:06.835296    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/931901d4-a4a4-427c-acfd-b62eadbdbd24-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "931901d4-a4a4-427c-acfd-b62eadbdbd24" (UID: "931901d4-a4a4-427c-acfd-b62eadbdbd24"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:13:06.836999 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:06.836977    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/931901d4-a4a4-427c-acfd-b62eadbdbd24-kube-api-access-pxv6c" (OuterVolumeSpecName: "kube-api-access-pxv6c") pod "931901d4-a4a4-427c-acfd-b62eadbdbd24" (UID: "931901d4-a4a4-427c-acfd-b62eadbdbd24"). InnerVolumeSpecName "kube-api-access-pxv6c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:13:06.935613 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:06.935580    2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/931901d4-a4a4-427c-acfd-b62eadbdbd24-extensions-socket-volume\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:13:06.935613 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:06.935608    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pxv6c\" (UniqueName: \"kubernetes.io/projected/931901d4-a4a4-427c-acfd-b62eadbdbd24-kube-api-access-pxv6c\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:13:07.446923 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:07.446895    2580 generic.go:356] "Generic (PLEG): container finished" podID="931901d4-a4a4-427c-acfd-b62eadbdbd24" containerID="e29ebbe8c3fb8e80beeaacbfa04ffec962348c645914069d341b8fcf1764395c" exitCode=0
Sep 16 14:13:07.447253 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:07.446947    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cp9lh"
Sep 16 14:13:07.447253 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:07.446966    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cp9lh" event={"ID":"931901d4-a4a4-427c-acfd-b62eadbdbd24","Type":"ContainerDied","Data":"e29ebbe8c3fb8e80beeaacbfa04ffec962348c645914069d341b8fcf1764395c"}
Sep 16 14:13:07.447253 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:07.447001    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cp9lh" event={"ID":"931901d4-a4a4-427c-acfd-b62eadbdbd24","Type":"ContainerDied","Data":"736f4346f78fbf2b80c6db0900ab4d37f73b761440ba2b419c026dc9cfdf1e52"}
Sep 16 14:13:07.447253 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:07.447022    2580 scope.go:117] "RemoveContainer" containerID="e29ebbe8c3fb8e80beeaacbfa04ffec962348c645914069d341b8fcf1764395c"
Sep 16 14:13:07.456735 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:07.456699    2580 scope.go:117] "RemoveContainer" containerID="e29ebbe8c3fb8e80beeaacbfa04ffec962348c645914069d341b8fcf1764395c"
Sep 16 14:13:07.456984 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:13:07.456963    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e29ebbe8c3fb8e80beeaacbfa04ffec962348c645914069d341b8fcf1764395c\": container with ID starting with e29ebbe8c3fb8e80beeaacbfa04ffec962348c645914069d341b8fcf1764395c not found: ID does not exist" containerID="e29ebbe8c3fb8e80beeaacbfa04ffec962348c645914069d341b8fcf1764395c"
Sep 16 14:13:07.457036 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:07.456992    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e29ebbe8c3fb8e80beeaacbfa04ffec962348c645914069d341b8fcf1764395c"} err="failed to get container status \"e29ebbe8c3fb8e80beeaacbfa04ffec962348c645914069d341b8fcf1764395c\": rpc error: code = NotFound desc = could not find container \"e29ebbe8c3fb8e80beeaacbfa04ffec962348c645914069d341b8fcf1764395c\": container with ID starting with e29ebbe8c3fb8e80beeaacbfa04ffec962348c645914069d341b8fcf1764395c not found: ID does not exist"
Sep 16 14:13:07.468818 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:07.468793    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cp9lh"]
Sep 16 14:13:07.473667 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:07.473647    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cp9lh"]
Sep 16 14:13:09.429671 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:09.429638    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="931901d4-a4a4-427c-acfd-b62eadbdbd24" path="/var/lib/kubelet/pods/931901d4-a4a4-427c-acfd-b62eadbdbd24/volumes"
Sep 16 14:13:22.941731 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:22.941651    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jfn4c"]
Sep 16 14:13:22.942064 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:22.941982    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="931901d4-a4a4-427c-acfd-b62eadbdbd24" containerName="manager"
Sep 16 14:13:22.942064 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:22.941994    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="931901d4-a4a4-427c-acfd-b62eadbdbd24" containerName="manager"
Sep 16 14:13:22.942137 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:22.942065    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="931901d4-a4a4-427c-acfd-b62eadbdbd24" containerName="manager"
Sep 16 14:13:22.944833 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:22.944814    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jfn4c"
Sep 16 14:13:22.948371 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:22.948349    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-6qnhx\""
Sep 16 14:13:22.972445 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:22.972420    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jfn4c"]
Sep 16 14:13:23.059132 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:23.059100    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aedbe6b8-69e8-4213-b355-e4cdcd452d49-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jfn4c\" (UID: \"aedbe6b8-69e8-4213-b355-e4cdcd452d49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jfn4c"
Sep 16 14:13:23.059132 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:23.059136    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aedbe6b8-69e8-4213-b355-e4cdcd452d49-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jfn4c\" (UID: \"aedbe6b8-69e8-4213-b355-e4cdcd452d49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jfn4c"
Sep 16 14:13:23.059347 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:23.059157    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aedbe6b8-69e8-4213-b355-e4cdcd452d49-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jfn4c\" (UID: \"aedbe6b8-69e8-4213-b355-e4cdcd452d49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jfn4c"
Sep 16 14:13:23.059347 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:23.059178    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aedbe6b8-69e8-4213-b355-e4cdcd452d49-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jfn4c\" (UID: \"aedbe6b8-69e8-4213-b355-e4cdcd452d49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jfn4c"
Sep 16 14:13:23.059347 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:23.059265    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6bbc\" (UniqueName: \"kubernetes.io/projected/aedbe6b8-69e8-4213-b355-e4cdcd452d49-kube-api-access-j6bbc\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jfn4c\" (UID: \"aedbe6b8-69e8-4213-b355-e4cdcd452d49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jfn4c"
Sep 16 14:13:23.059347 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:23.059321    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aedbe6b8-69e8-4213-b355-e4cdcd452d49-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jfn4c\" (UID: \"aedbe6b8-69e8-4213-b355-e4cdcd452d49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jfn4c"
Sep 16 14:13:23.059525 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:23.059374    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aedbe6b8-69e8-4213-b355-e4cdcd452d49-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jfn4c\" (UID: \"aedbe6b8-69e8-4213-b355-e4cdcd452d49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jfn4c"
Sep 16 14:13:23.059525 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:23.059409    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aedbe6b8-69e8-4213-b355-e4cdcd452d49-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jfn4c\" (UID: \"aedbe6b8-69e8-4213-b355-e4cdcd452d49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jfn4c"
Sep 16 14:13:23.059525 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:23.059427    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aedbe6b8-69e8-4213-b355-e4cdcd452d49-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jfn4c\" (UID: \"aedbe6b8-69e8-4213-b355-e4cdcd452d49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jfn4c"
Sep 16 14:13:23.160571 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:23.160538    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aedbe6b8-69e8-4213-b355-e4cdcd452d49-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jfn4c\" (UID: \"aedbe6b8-69e8-4213-b355-e4cdcd452d49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jfn4c"
Sep 16 14:13:23.160759 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:23.160584    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aedbe6b8-69e8-4213-b355-e4cdcd452d49-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jfn4c\" (UID: \"aedbe6b8-69e8-4213-b355-e4cdcd452d49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jfn4c"
Sep 16 14:13:23.160759 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:23.160702    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aedbe6b8-69e8-4213-b355-e4cdcd452d49-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jfn4c\" (UID: \"aedbe6b8-69e8-4213-b355-e4cdcd452d49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jfn4c"
Sep 16 14:13:23.160880 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:23.160779    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aedbe6b8-69e8-4213-b355-e4cdcd452d49-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jfn4c\" (UID: \"aedbe6b8-69e8-4213-b355-e4cdcd452d49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jfn4c"
Sep 16 14:13:23.160880 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:23.160817    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aedbe6b8-69e8-4213-b355-e4cdcd452d49-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jfn4c\" (UID: \"aedbe6b8-69e8-4213-b355-e4cdcd452d49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jfn4c"
Sep 16 14:13:23.160880 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:23.160848    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aedbe6b8-69e8-4213-b355-e4cdcd452d49-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jfn4c\" (UID: \"aedbe6b8-69e8-4213-b355-e4cdcd452d49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jfn4c"
Sep 16 14:13:23.160880 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:23.160872    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aedbe6b8-69e8-4213-b355-e4cdcd452d49-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jfn4c\" (UID: \"aedbe6b8-69e8-4213-b355-e4cdcd452d49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jfn4c"
Sep 16 14:13:23.161096 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:23.160908    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6bbc\" (UniqueName: \"kubernetes.io/projected/aedbe6b8-69e8-4213-b355-e4cdcd452d49-kube-api-access-j6bbc\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jfn4c\" (UID: \"aedbe6b8-69e8-4213-b355-e4cdcd452d49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jfn4c"
Sep 16 14:13:23.161096 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:23.160954    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aedbe6b8-69e8-4213-b355-e4cdcd452d49-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jfn4c\" (UID: \"aedbe6b8-69e8-4213-b355-e4cdcd452d49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jfn4c"
Sep 16 14:13:23.161096 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:23.161005    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aedbe6b8-69e8-4213-b355-e4cdcd452d49-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jfn4c\" (UID: \"aedbe6b8-69e8-4213-b355-e4cdcd452d49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jfn4c"
Sep 16 14:13:23.161255 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:23.161196    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aedbe6b8-69e8-4213-b355-e4cdcd452d49-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jfn4c\" (UID: \"aedbe6b8-69e8-4213-b355-e4cdcd452d49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jfn4c"
Sep 16 14:13:23.161255 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:23.161206    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aedbe6b8-69e8-4213-b355-e4cdcd452d49-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jfn4c\" (UID: \"aedbe6b8-69e8-4213-b355-e4cdcd452d49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jfn4c"
Sep 16 14:13:23.161352 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:23.161313    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aedbe6b8-69e8-4213-b355-e4cdcd452d49-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jfn4c\" (UID: \"aedbe6b8-69e8-4213-b355-e4cdcd452d49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jfn4c"
Sep 16 14:13:23.161787 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:23.161762    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aedbe6b8-69e8-4213-b355-e4cdcd452d49-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jfn4c\" (UID: \"aedbe6b8-69e8-4213-b355-e4cdcd452d49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jfn4c"
Sep 16 14:13:23.162908 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:23.162887    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aedbe6b8-69e8-4213-b355-e4cdcd452d49-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jfn4c\" (UID: \"aedbe6b8-69e8-4213-b355-e4cdcd452d49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jfn4c"
Sep 16 14:13:23.163713 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:23.163698    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aedbe6b8-69e8-4213-b355-e4cdcd452d49-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jfn4c\" (UID: \"aedbe6b8-69e8-4213-b355-e4cdcd452d49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jfn4c"
Sep 16 14:13:23.170747 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:23.170722    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aedbe6b8-69e8-4213-b355-e4cdcd452d49-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jfn4c\" (UID: \"aedbe6b8-69e8-4213-b355-e4cdcd452d49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jfn4c"
Sep 16 14:13:23.170747 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:23.170739    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6bbc\" (UniqueName: \"kubernetes.io/projected/aedbe6b8-69e8-4213-b355-e4cdcd452d49-kube-api-access-j6bbc\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jfn4c\" (UID: \"aedbe6b8-69e8-4213-b355-e4cdcd452d49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jfn4c"
Sep 16 14:13:23.255586 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:23.255487    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jfn4c"
Sep 16 14:13:23.530913 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:23.530874    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jfn4c"]
Sep 16 14:13:23.534561 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:13:23.534534    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaedbe6b8_69e8_4213_b355_e4cdcd452d49.slice/crio-f54af3fc20a5476c048d51e233f2c0f2e449c87e2f6e869ae8c4085cc523329e WatchSource:0}: Error finding container f54af3fc20a5476c048d51e233f2c0f2e449c87e2f6e869ae8c4085cc523329e: Status 404 returned error can't find the container with id f54af3fc20a5476c048d51e233f2c0f2e449c87e2f6e869ae8c4085cc523329e
Sep 16 14:13:23.536645 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:23.536609    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 14:13:23.536750 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:23.536676    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 14:13:23.536750 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:23.536704    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 14:13:24.530002 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:24.529965    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jfn4c" event={"ID":"aedbe6b8-69e8-4213-b355-e4cdcd452d49","Type":"ContainerStarted","Data":"83c386a165de6c45771bc31a9f6a53d44a74b09949d60ebe0354dcfd3e70818c"}
Sep 16 14:13:24.530369 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:24.530061    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jfn4c" event={"ID":"aedbe6b8-69e8-4213-b355-e4cdcd452d49","Type":"ContainerStarted","Data":"f54af3fc20a5476c048d51e233f2c0f2e449c87e2f6e869ae8c4085cc523329e"}
Sep 16 14:13:24.560572 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:24.560527    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jfn4c" podStartSLOduration=2.560510835 podStartE2EDuration="2.560510835s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:13:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:13:24.55972191 +0000 UTC m=+537.574831772" watchObservedRunningTime="2026-09-16 14:13:24.560510835 +0000 UTC m=+537.575620749"
Sep 16 14:13:25.255904 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:25.255862    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jfn4c"
Sep 16 14:13:25.260364 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:25.260342    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jfn4c"
Sep 16 14:13:25.535306 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:25.535222    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jfn4c"
Sep 16 14:13:25.536247 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:25.536226    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jfn4c"
Sep 16 14:13:25.800135 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:25.800058    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-csfs9"]
Sep 16 14:13:25.803464 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:25.803447    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-csfs9"
Sep 16 14:13:25.806402 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:25.806379    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-j72bn\""
Sep 16 14:13:25.806491 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:25.806379    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 16 14:13:25.827207 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:25.827180    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-csfs9"]
Sep 16 14:13:25.885666 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:25.885634    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2b2v\" (UniqueName: \"kubernetes.io/projected/5ee29b23-685f-43e4-b6d3-a5d8f9b31e4b-kube-api-access-w2b2v\") pod \"limitador-limitador-8588749977-csfs9\" (UID: \"5ee29b23-685f-43e4-b6d3-a5d8f9b31e4b\") " pod="kuadrant-system/limitador-limitador-8588749977-csfs9"
Sep 16 14:13:25.885836 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:25.885692    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5ee29b23-685f-43e4-b6d3-a5d8f9b31e4b-config-file\") pod \"limitador-limitador-8588749977-csfs9\" (UID: \"5ee29b23-685f-43e4-b6d3-a5d8f9b31e4b\") " pod="kuadrant-system/limitador-limitador-8588749977-csfs9"
Sep 16 14:13:25.899965 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:25.899936    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-csfs9"]
Sep 16 14:13:25.986813 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:25.986782    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2b2v\" (UniqueName: \"kubernetes.io/projected/5ee29b23-685f-43e4-b6d3-a5d8f9b31e4b-kube-api-access-w2b2v\") pod \"limitador-limitador-8588749977-csfs9\" (UID: \"5ee29b23-685f-43e4-b6d3-a5d8f9b31e4b\") " pod="kuadrant-system/limitador-limitador-8588749977-csfs9"
Sep 16 14:13:25.986977 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:25.986829    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5ee29b23-685f-43e4-b6d3-a5d8f9b31e4b-config-file\") pod \"limitador-limitador-8588749977-csfs9\" (UID: \"5ee29b23-685f-43e4-b6d3-a5d8f9b31e4b\") " pod="kuadrant-system/limitador-limitador-8588749977-csfs9"
Sep 16 14:13:25.987479 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:25.987460    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5ee29b23-685f-43e4-b6d3-a5d8f9b31e4b-config-file\") pod \"limitador-limitador-8588749977-csfs9\" (UID: \"5ee29b23-685f-43e4-b6d3-a5d8f9b31e4b\") " pod="kuadrant-system/limitador-limitador-8588749977-csfs9"
Sep 16 14:13:25.998937 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:25.998909    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2b2v\" (UniqueName: \"kubernetes.io/projected/5ee29b23-685f-43e4-b6d3-a5d8f9b31e4b-kube-api-access-w2b2v\") pod \"limitador-limitador-8588749977-csfs9\" (UID: \"5ee29b23-685f-43e4-b6d3-a5d8f9b31e4b\") " pod="kuadrant-system/limitador-limitador-8588749977-csfs9"
Sep 16 14:13:26.114144 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:26.114060    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-csfs9"
Sep 16 14:13:26.425847 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:26.425825    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-csfs9"]
Sep 16 14:13:26.427769 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:13:26.427744    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ee29b23_685f_43e4_b6d3_a5d8f9b31e4b.slice/crio-2006b3fb17c1bbb86ba424f6e9513dbf17b2bf42c761ca041d6e761a504a3b5e WatchSource:0}: Error finding container 2006b3fb17c1bbb86ba424f6e9513dbf17b2bf42c761ca041d6e761a504a3b5e: Status 404 returned error can't find the container with id 2006b3fb17c1bbb86ba424f6e9513dbf17b2bf42c761ca041d6e761a504a3b5e
Sep 16 14:13:26.541393 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:26.541361    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-csfs9" event={"ID":"5ee29b23-685f-43e4-b6d3-a5d8f9b31e4b","Type":"ContainerStarted","Data":"2006b3fb17c1bbb86ba424f6e9513dbf17b2bf42c761ca041d6e761a504a3b5e"}
Sep 16 14:13:26.906068 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:26.906034    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-52ht7"]
Sep 16 14:13:26.910585 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:26.910569    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-52ht7"
Sep 16 14:13:26.913172 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:26.913152    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-8h45d\""
Sep 16 14:13:26.935058 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:26.935029    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-52ht7"]
Sep 16 14:13:26.996522 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:26.996474    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbzt7\" (UniqueName: \"kubernetes.io/projected/9e78ce84-3734-44f2-a159-0724a070e8a3-kube-api-access-xbzt7\") pod \"authorino-7bcfd8f9cd-52ht7\" (UID: \"9e78ce84-3734-44f2-a159-0724a070e8a3\") " pod="kuadrant-system/authorino-7bcfd8f9cd-52ht7"
Sep 16 14:13:27.097001 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:27.096963    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbzt7\" (UniqueName: \"kubernetes.io/projected/9e78ce84-3734-44f2-a159-0724a070e8a3-kube-api-access-xbzt7\") pod \"authorino-7bcfd8f9cd-52ht7\" (UID: \"9e78ce84-3734-44f2-a159-0724a070e8a3\") " pod="kuadrant-system/authorino-7bcfd8f9cd-52ht7"
Sep 16 14:13:27.128049 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:27.128020    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbzt7\" (UniqueName: \"kubernetes.io/projected/9e78ce84-3734-44f2-a159-0724a070e8a3-kube-api-access-xbzt7\") pod \"authorino-7bcfd8f9cd-52ht7\" (UID: \"9e78ce84-3734-44f2-a159-0724a070e8a3\") " pod="kuadrant-system/authorino-7bcfd8f9cd-52ht7"
Sep 16 14:13:27.220202 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:27.220165    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-52ht7"
Sep 16 14:13:27.271147 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:27.271121    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c974bd48-q6j97"]
Sep 16 14:13:27.275659 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:27.275639    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-q6j97"
Sep 16 14:13:27.295331 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:27.295306    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-q6j97"]
Sep 16 14:13:27.399398 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:27.399361    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2p748\" (UniqueName: \"kubernetes.io/projected/d203bde6-cef7-4c8f-a115-e6176de71b37-kube-api-access-2p748\") pod \"authorino-59c974bd48-q6j97\" (UID: \"d203bde6-cef7-4c8f-a115-e6176de71b37\") " pod="kuadrant-system/authorino-59c974bd48-q6j97"
Sep 16 14:13:27.500166 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:27.500080    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2p748\" (UniqueName: \"kubernetes.io/projected/d203bde6-cef7-4c8f-a115-e6176de71b37-kube-api-access-2p748\") pod \"authorino-59c974bd48-q6j97\" (UID: \"d203bde6-cef7-4c8f-a115-e6176de71b37\") " pod="kuadrant-system/authorino-59c974bd48-q6j97"
Sep 16 14:13:27.503833 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:27.503804    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-52ht7"]
Sep 16 14:13:27.507087 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:13:27.507056    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9e78ce84_3734_44f2_a159_0724a070e8a3.slice/crio-9a803a9e2864ef4aaffa465e22de285d0118e4f9da4f1ece36ae78de352c2c0e WatchSource:0}: Error finding container 9a803a9e2864ef4aaffa465e22de285d0118e4f9da4f1ece36ae78de352c2c0e: Status 404 returned error can't find the container with id 9a803a9e2864ef4aaffa465e22de285d0118e4f9da4f1ece36ae78de352c2c0e
Sep 16 14:13:27.510522 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:27.510463    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2p748\" (UniqueName: \"kubernetes.io/projected/d203bde6-cef7-4c8f-a115-e6176de71b37-kube-api-access-2p748\") pod \"authorino-59c974bd48-q6j97\" (UID: \"d203bde6-cef7-4c8f-a115-e6176de71b37\") " pod="kuadrant-system/authorino-59c974bd48-q6j97"
Sep 16 14:13:27.547542 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:27.547480    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-52ht7" event={"ID":"9e78ce84-3734-44f2-a159-0724a070e8a3","Type":"ContainerStarted","Data":"9a803a9e2864ef4aaffa465e22de285d0118e4f9da4f1ece36ae78de352c2c0e"}
Sep 16 14:13:27.586862 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:27.586832    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-q6j97"
Sep 16 14:13:27.883395 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:27.883361    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-q6j97"]
Sep 16 14:13:27.907239 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:13:27.907215    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd203bde6_cef7_4c8f_a115_e6176de71b37.slice/crio-895b9a7c1b1bb0020cef173fda94e2d1b6ca48e4df5554becbdb5c72b356fc9b WatchSource:0}: Error finding container 895b9a7c1b1bb0020cef173fda94e2d1b6ca48e4df5554becbdb5c72b356fc9b: Status 404 returned error can't find the container with id 895b9a7c1b1bb0020cef173fda94e2d1b6ca48e4df5554becbdb5c72b356fc9b
Sep 16 14:13:28.556296 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:28.556226    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-csfs9" event={"ID":"5ee29b23-685f-43e4-b6d3-a5d8f9b31e4b","Type":"ContainerStarted","Data":"c26c43d2e61d58af1ed4ecffd4e1d81eb7f4e3851844f8e546dd458f06c3409f"}
Sep 16 14:13:28.556811 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:28.556414    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-csfs9"
Sep 16 14:13:28.557807 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:28.557777    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-q6j97" event={"ID":"d203bde6-cef7-4c8f-a115-e6176de71b37","Type":"ContainerStarted","Data":"895b9a7c1b1bb0020cef173fda94e2d1b6ca48e4df5554becbdb5c72b356fc9b"}
Sep 16 14:13:28.577142 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:28.577025    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-csfs9" podStartSLOduration=2.043492795 podStartE2EDuration="3.577008474s" totalImagesPullingTime="1.533515679s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:13:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:13:28.576129809 +0000 UTC m=+541.591239670" watchObservedRunningTime="2026-09-16 14:13:28.577008474 +0000 UTC m=+541.592118334"
Sep 16 14:13:30.571883 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:30.571844    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-52ht7" event={"ID":"9e78ce84-3734-44f2-a159-0724a070e8a3","Type":"ContainerStarted","Data":"93e476223b781a032d13c8d95643d1d1bffec71de50f8178f184a91dbfe788a3"}
Sep 16 14:13:30.573429 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:30.573404    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-q6j97" event={"ID":"d203bde6-cef7-4c8f-a115-e6176de71b37","Type":"ContainerStarted","Data":"237d72b0167fc392b13df704d4525652027f10641687d0944641099bad842b86"}
Sep 16 14:13:30.587298 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:30.587257    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bcfd8f9cd-52ht7" podStartSLOduration=2.4247226570000002 podStartE2EDuration="4.587241395s" totalImagesPullingTime="2.162518738s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:13:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:13:30.585812555 +0000 UTC m=+543.600922416" watchObservedRunningTime="2026-09-16 14:13:30.587241395 +0000 UTC m=+543.602351255"
Sep 16 14:13:30.601804 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:30.601750    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c974bd48-q6j97" podStartSLOduration=1.8429292670000001 podStartE2EDuration="3.601734971s" totalImagesPullingTime="1.758805704s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:13:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:13:30.599011677 +0000 UTC m=+543.614121536" watchObservedRunningTime="2026-09-16 14:13:30.601734971 +0000 UTC m=+543.616844832"
Sep 16 14:13:30.628651 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:30.628620    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-52ht7"]
Sep 16 14:13:32.583995 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:32.583934    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7bcfd8f9cd-52ht7" podUID="9e78ce84-3734-44f2-a159-0724a070e8a3" containerName="authorino" containerID="cri-o://93e476223b781a032d13c8d95643d1d1bffec71de50f8178f184a91dbfe788a3" gracePeriod=30
Sep 16 14:13:32.837070 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:32.837016    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-52ht7"
Sep 16 14:13:32.947155 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:32.947121    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xbzt7\" (UniqueName: \"kubernetes.io/projected/9e78ce84-3734-44f2-a159-0724a070e8a3-kube-api-access-xbzt7\") pod \"9e78ce84-3734-44f2-a159-0724a070e8a3\" (UID: \"9e78ce84-3734-44f2-a159-0724a070e8a3\") "
Sep 16 14:13:32.949090 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:32.949059    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e78ce84-3734-44f2-a159-0724a070e8a3-kube-api-access-xbzt7" (OuterVolumeSpecName: "kube-api-access-xbzt7") pod "9e78ce84-3734-44f2-a159-0724a070e8a3" (UID: "9e78ce84-3734-44f2-a159-0724a070e8a3"). InnerVolumeSpecName "kube-api-access-xbzt7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:13:33.048173 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:33.048141    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xbzt7\" (UniqueName: \"kubernetes.io/projected/9e78ce84-3734-44f2-a159-0724a070e8a3-kube-api-access-xbzt7\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:13:33.590083 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:33.590050    2580 generic.go:356] "Generic (PLEG): container finished" podID="9e78ce84-3734-44f2-a159-0724a070e8a3" containerID="93e476223b781a032d13c8d95643d1d1bffec71de50f8178f184a91dbfe788a3" exitCode=0
Sep 16 14:13:33.590473 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:33.590093    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-52ht7"
Sep 16 14:13:33.590473 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:33.590139    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-52ht7" event={"ID":"9e78ce84-3734-44f2-a159-0724a070e8a3","Type":"ContainerDied","Data":"93e476223b781a032d13c8d95643d1d1bffec71de50f8178f184a91dbfe788a3"}
Sep 16 14:13:33.590473 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:33.590177    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-52ht7" event={"ID":"9e78ce84-3734-44f2-a159-0724a070e8a3","Type":"ContainerDied","Data":"9a803a9e2864ef4aaffa465e22de285d0118e4f9da4f1ece36ae78de352c2c0e"}
Sep 16 14:13:33.590473 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:33.590195    2580 scope.go:117] "RemoveContainer" containerID="93e476223b781a032d13c8d95643d1d1bffec71de50f8178f184a91dbfe788a3"
Sep 16 14:13:33.598283 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:33.598267    2580 scope.go:117] "RemoveContainer" containerID="93e476223b781a032d13c8d95643d1d1bffec71de50f8178f184a91dbfe788a3"
Sep 16 14:13:33.598581 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:13:33.598560    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93e476223b781a032d13c8d95643d1d1bffec71de50f8178f184a91dbfe788a3\": container with ID starting with 93e476223b781a032d13c8d95643d1d1bffec71de50f8178f184a91dbfe788a3 not found: ID does not exist" containerID="93e476223b781a032d13c8d95643d1d1bffec71de50f8178f184a91dbfe788a3"
Sep 16 14:13:33.598669 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:33.598589    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93e476223b781a032d13c8d95643d1d1bffec71de50f8178f184a91dbfe788a3"} err="failed to get container status \"93e476223b781a032d13c8d95643d1d1bffec71de50f8178f184a91dbfe788a3\": rpc error: code = NotFound desc = could not find container \"93e476223b781a032d13c8d95643d1d1bffec71de50f8178f184a91dbfe788a3\": container with ID starting with 93e476223b781a032d13c8d95643d1d1bffec71de50f8178f184a91dbfe788a3 not found: ID does not exist"
Sep 16 14:13:33.605393 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:33.605372    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-52ht7"]
Sep 16 14:13:33.610104 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:33.610083    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-52ht7"]
Sep 16 14:13:35.430313 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:35.430280    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e78ce84-3734-44f2-a159-0724a070e8a3" path="/var/lib/kubelet/pods/9e78ce84-3734-44f2-a159-0724a070e8a3/volumes"
Sep 16 14:13:39.567217 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:39.567191    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-csfs9"
Sep 16 14:13:42.545195 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:42.545141    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-csfs9"]
Sep 16 14:13:42.545690 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:42.545466    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-csfs9" podUID="5ee29b23-685f-43e4-b6d3-a5d8f9b31e4b" containerName="limitador" containerID="cri-o://c26c43d2e61d58af1ed4ecffd4e1d81eb7f4e3851844f8e546dd458f06c3409f" gracePeriod=30
Sep 16 14:13:43.081465 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:43.081444    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-csfs9"
Sep 16 14:13:43.232681 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:43.232650    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5ee29b23-685f-43e4-b6d3-a5d8f9b31e4b-config-file\") pod \"5ee29b23-685f-43e4-b6d3-a5d8f9b31e4b\" (UID: \"5ee29b23-685f-43e4-b6d3-a5d8f9b31e4b\") "
Sep 16 14:13:43.232849 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:43.232750    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w2b2v\" (UniqueName: \"kubernetes.io/projected/5ee29b23-685f-43e4-b6d3-a5d8f9b31e4b-kube-api-access-w2b2v\") pod \"5ee29b23-685f-43e4-b6d3-a5d8f9b31e4b\" (UID: \"5ee29b23-685f-43e4-b6d3-a5d8f9b31e4b\") "
Sep 16 14:13:43.233026 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:43.233005    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ee29b23-685f-43e4-b6d3-a5d8f9b31e4b-config-file" (OuterVolumeSpecName: "config-file") pod "5ee29b23-685f-43e4-b6d3-a5d8f9b31e4b" (UID: "5ee29b23-685f-43e4-b6d3-a5d8f9b31e4b"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:13:43.234755 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:43.234735    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ee29b23-685f-43e4-b6d3-a5d8f9b31e4b-kube-api-access-w2b2v" (OuterVolumeSpecName: "kube-api-access-w2b2v") pod "5ee29b23-685f-43e4-b6d3-a5d8f9b31e4b" (UID: "5ee29b23-685f-43e4-b6d3-a5d8f9b31e4b"). InnerVolumeSpecName "kube-api-access-w2b2v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:13:43.334106 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:43.334072    2580 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5ee29b23-685f-43e4-b6d3-a5d8f9b31e4b-config-file\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:13:43.334106 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:43.334100    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w2b2v\" (UniqueName: \"kubernetes.io/projected/5ee29b23-685f-43e4-b6d3-a5d8f9b31e4b-kube-api-access-w2b2v\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:13:43.640239 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:43.640150    2580 generic.go:356] "Generic (PLEG): container finished" podID="5ee29b23-685f-43e4-b6d3-a5d8f9b31e4b" containerID="c26c43d2e61d58af1ed4ecffd4e1d81eb7f4e3851844f8e546dd458f06c3409f" exitCode=0
Sep 16 14:13:43.640239 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:43.640200    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-csfs9"
Sep 16 14:13:43.640715 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:43.640255    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-csfs9" event={"ID":"5ee29b23-685f-43e4-b6d3-a5d8f9b31e4b","Type":"ContainerDied","Data":"c26c43d2e61d58af1ed4ecffd4e1d81eb7f4e3851844f8e546dd458f06c3409f"}
Sep 16 14:13:43.640715 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:43.640302    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-csfs9" event={"ID":"5ee29b23-685f-43e4-b6d3-a5d8f9b31e4b","Type":"ContainerDied","Data":"2006b3fb17c1bbb86ba424f6e9513dbf17b2bf42c761ca041d6e761a504a3b5e"}
Sep 16 14:13:43.640715 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:43.640323    2580 scope.go:117] "RemoveContainer" containerID="c26c43d2e61d58af1ed4ecffd4e1d81eb7f4e3851844f8e546dd458f06c3409f"
Sep 16 14:13:43.648625 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:43.648552    2580 scope.go:117] "RemoveContainer" containerID="c26c43d2e61d58af1ed4ecffd4e1d81eb7f4e3851844f8e546dd458f06c3409f"
Sep 16 14:13:43.648831 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:13:43.648812    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c26c43d2e61d58af1ed4ecffd4e1d81eb7f4e3851844f8e546dd458f06c3409f\": container with ID starting with c26c43d2e61d58af1ed4ecffd4e1d81eb7f4e3851844f8e546dd458f06c3409f not found: ID does not exist" containerID="c26c43d2e61d58af1ed4ecffd4e1d81eb7f4e3851844f8e546dd458f06c3409f"
Sep 16 14:13:43.648879 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:43.648839    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c26c43d2e61d58af1ed4ecffd4e1d81eb7f4e3851844f8e546dd458f06c3409f"} err="failed to get container status \"c26c43d2e61d58af1ed4ecffd4e1d81eb7f4e3851844f8e546dd458f06c3409f\": rpc error: code = NotFound desc = could not find container \"c26c43d2e61d58af1ed4ecffd4e1d81eb7f4e3851844f8e546dd458f06c3409f\": container with ID starting with c26c43d2e61d58af1ed4ecffd4e1d81eb7f4e3851844f8e546dd458f06c3409f not found: ID does not exist"
Sep 16 14:13:43.655479 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:43.655460    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-csfs9"]
Sep 16 14:13:43.660027 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:43.660009    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-csfs9"]
Sep 16 14:13:45.430162 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:13:45.430123    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ee29b23-685f-43e4-b6d3-a5d8f9b31e4b" path="/var/lib/kubelet/pods/5ee29b23-685f-43e4-b6d3-a5d8f9b31e4b/volumes"
Sep 16 14:14:00.274008 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:00.273976    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-557f876f69-knnmv"]
Sep 16 14:14:00.274370 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:00.274293    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ee29b23-685f-43e4-b6d3-a5d8f9b31e4b" containerName="limitador"
Sep 16 14:14:00.274370 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:00.274303    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ee29b23-685f-43e4-b6d3-a5d8f9b31e4b" containerName="limitador"
Sep 16 14:14:00.274370 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:00.274317    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e78ce84-3734-44f2-a159-0724a070e8a3" containerName="authorino"
Sep 16 14:14:00.274370 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:00.274322    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e78ce84-3734-44f2-a159-0724a070e8a3" containerName="authorino"
Sep 16 14:14:00.274517 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:00.274377    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e78ce84-3734-44f2-a159-0724a070e8a3" containerName="authorino"
Sep 16 14:14:00.274517 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:00.274387    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ee29b23-685f-43e4-b6d3-a5d8f9b31e4b" containerName="limitador"
Sep 16 14:14:00.277455 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:00.277433    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-knnmv"
Sep 16 14:14:00.284220 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:00.283940    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557f876f69-knnmv"]
Sep 16 14:14:00.333293 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:00.333266    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-knnmv"]
Sep 16 14:14:00.333547 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:14:00.333526    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-clf9v], unattached volumes=[], failed to process volumes=[kube-api-access-clf9v]: context canceled" pod="kuadrant-system/authorino-557f876f69-knnmv" podUID="b6ba20c9-b004-47b9-936c-2526380ad1e0"
Sep 16 14:14:00.373885 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:00.373856    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-8dzgb"]
Sep 16 14:14:00.376951 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:00.376934    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-8dzgb"
Sep 16 14:14:00.378895 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:00.378867    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 16 14:14:00.385410 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:00.385389    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-8dzgb"]
Sep 16 14:14:00.477101 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:00.477066    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clf9v\" (UniqueName: \"kubernetes.io/projected/b6ba20c9-b004-47b9-936c-2526380ad1e0-kube-api-access-clf9v\") pod \"authorino-557f876f69-knnmv\" (UID: \"b6ba20c9-b004-47b9-936c-2526380ad1e0\") " pod="kuadrant-system/authorino-557f876f69-knnmv"
Sep 16 14:14:00.477249 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:00.477181    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/99a34cfa-e964-4d66-bbdb-3e6c627bd9f2-tls-cert\") pod \"authorino-6d4dfc649f-8dzgb\" (UID: \"99a34cfa-e964-4d66-bbdb-3e6c627bd9f2\") " pod="kuadrant-system/authorino-6d4dfc649f-8dzgb"
Sep 16 14:14:00.477249 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:00.477223    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6jsz\" (UniqueName: \"kubernetes.io/projected/99a34cfa-e964-4d66-bbdb-3e6c627bd9f2-kube-api-access-j6jsz\") pod \"authorino-6d4dfc649f-8dzgb\" (UID: \"99a34cfa-e964-4d66-bbdb-3e6c627bd9f2\") " pod="kuadrant-system/authorino-6d4dfc649f-8dzgb"
Sep 16 14:14:00.512447 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:00.512423    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-8dzgb"]
Sep 16 14:14:00.512656 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:14:00.512638    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-j6jsz tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-6d4dfc649f-8dzgb" podUID="99a34cfa-e964-4d66-bbdb-3e6c627bd9f2"
Sep 16 14:14:00.578205 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:00.578130    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/99a34cfa-e964-4d66-bbdb-3e6c627bd9f2-tls-cert\") pod \"authorino-6d4dfc649f-8dzgb\" (UID: \"99a34cfa-e964-4d66-bbdb-3e6c627bd9f2\") " pod="kuadrant-system/authorino-6d4dfc649f-8dzgb"
Sep 16 14:14:00.578205 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:00.578176    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6jsz\" (UniqueName: \"kubernetes.io/projected/99a34cfa-e964-4d66-bbdb-3e6c627bd9f2-kube-api-access-j6jsz\") pod \"authorino-6d4dfc649f-8dzgb\" (UID: \"99a34cfa-e964-4d66-bbdb-3e6c627bd9f2\") " pod="kuadrant-system/authorino-6d4dfc649f-8dzgb"
Sep 16 14:14:00.578408 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:00.578212    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clf9v\" (UniqueName: \"kubernetes.io/projected/b6ba20c9-b004-47b9-936c-2526380ad1e0-kube-api-access-clf9v\") pod \"authorino-557f876f69-knnmv\" (UID: \"b6ba20c9-b004-47b9-936c-2526380ad1e0\") " pod="kuadrant-system/authorino-557f876f69-knnmv"
Sep 16 14:14:00.580529 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:00.580478    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/99a34cfa-e964-4d66-bbdb-3e6c627bd9f2-tls-cert\") pod \"authorino-6d4dfc649f-8dzgb\" (UID: \"99a34cfa-e964-4d66-bbdb-3e6c627bd9f2\") " pod="kuadrant-system/authorino-6d4dfc649f-8dzgb"
Sep 16 14:14:00.585981 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:00.585942    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-clf9v\" (UniqueName: \"kubernetes.io/projected/b6ba20c9-b004-47b9-936c-2526380ad1e0-kube-api-access-clf9v\") pod \"authorino-557f876f69-knnmv\" (UID: \"b6ba20c9-b004-47b9-936c-2526380ad1e0\") " pod="kuadrant-system/authorino-557f876f69-knnmv"
Sep 16 14:14:00.586096 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:00.585950    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6jsz\" (UniqueName: \"kubernetes.io/projected/99a34cfa-e964-4d66-bbdb-3e6c627bd9f2-kube-api-access-j6jsz\") pod \"authorino-6d4dfc649f-8dzgb\" (UID: \"99a34cfa-e964-4d66-bbdb-3e6c627bd9f2\") " pod="kuadrant-system/authorino-6d4dfc649f-8dzgb"
Sep 16 14:14:00.726374 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:00.726340    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-8dzgb"
Sep 16 14:14:00.726570 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:00.726340    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-knnmv"
Sep 16 14:14:00.734911 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:00.734889    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-8dzgb"
Sep 16 14:14:00.737237 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:00.737221    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-knnmv"
Sep 16 14:14:00.780384 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:00.780355    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-clf9v\" (UniqueName: \"kubernetes.io/projected/b6ba20c9-b004-47b9-936c-2526380ad1e0-kube-api-access-clf9v\") pod \"b6ba20c9-b004-47b9-936c-2526380ad1e0\" (UID: \"b6ba20c9-b004-47b9-936c-2526380ad1e0\") "
Sep 16 14:14:00.780549 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:00.780393    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j6jsz\" (UniqueName: \"kubernetes.io/projected/99a34cfa-e964-4d66-bbdb-3e6c627bd9f2-kube-api-access-j6jsz\") pod \"99a34cfa-e964-4d66-bbdb-3e6c627bd9f2\" (UID: \"99a34cfa-e964-4d66-bbdb-3e6c627bd9f2\") "
Sep 16 14:14:00.780549 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:00.780424    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/99a34cfa-e964-4d66-bbdb-3e6c627bd9f2-tls-cert\") pod \"99a34cfa-e964-4d66-bbdb-3e6c627bd9f2\" (UID: \"99a34cfa-e964-4d66-bbdb-3e6c627bd9f2\") "
Sep 16 14:14:00.782579 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:00.782548    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6ba20c9-b004-47b9-936c-2526380ad1e0-kube-api-access-clf9v" (OuterVolumeSpecName: "kube-api-access-clf9v") pod "b6ba20c9-b004-47b9-936c-2526380ad1e0" (UID: "b6ba20c9-b004-47b9-936c-2526380ad1e0"). InnerVolumeSpecName "kube-api-access-clf9v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:14:00.782669 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:00.782594    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99a34cfa-e964-4d66-bbdb-3e6c627bd9f2-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "99a34cfa-e964-4d66-bbdb-3e6c627bd9f2" (UID: "99a34cfa-e964-4d66-bbdb-3e6c627bd9f2"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:14:00.782669 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:00.782605    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99a34cfa-e964-4d66-bbdb-3e6c627bd9f2-kube-api-access-j6jsz" (OuterVolumeSpecName: "kube-api-access-j6jsz") pod "99a34cfa-e964-4d66-bbdb-3e6c627bd9f2" (UID: "99a34cfa-e964-4d66-bbdb-3e6c627bd9f2"). InnerVolumeSpecName "kube-api-access-j6jsz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:14:00.881311 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:00.881217    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-clf9v\" (UniqueName: \"kubernetes.io/projected/b6ba20c9-b004-47b9-936c-2526380ad1e0-kube-api-access-clf9v\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:14:00.881311 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:00.881259    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j6jsz\" (UniqueName: \"kubernetes.io/projected/99a34cfa-e964-4d66-bbdb-3e6c627bd9f2-kube-api-access-j6jsz\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:14:00.881311 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:00.881270    2580 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/99a34cfa-e964-4d66-bbdb-3e6c627bd9f2-tls-cert\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:14:01.732023 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:01.731940    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-8dzgb"
Sep 16 14:14:01.732448 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:01.731940    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-knnmv"
Sep 16 14:14:01.763013 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:01.762980    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-knnmv"]
Sep 16 14:14:01.766013 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:01.765990    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-557f876f69-knnmv"]
Sep 16 14:14:01.793257 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:01.793228    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-8dzgb"]
Sep 16 14:14:01.799348 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:01.799324    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-8dzgb"]
Sep 16 14:14:03.429591 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:03.429562    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99a34cfa-e964-4d66-bbdb-3e6c627bd9f2" path="/var/lib/kubelet/pods/99a34cfa-e964-4d66-bbdb-3e6c627bd9f2/volumes"
Sep 16 14:14:03.429944 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:03.429794    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6ba20c9-b004-47b9-936c-2526380ad1e0" path="/var/lib/kubelet/pods/b6ba20c9-b004-47b9-936c-2526380ad1e0/volumes"
Sep 16 14:14:03.691279 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:03.691179    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-q6j97"]
Sep 16 14:14:03.691449 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:03.691420    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59c974bd48-q6j97" podUID="d203bde6-cef7-4c8f-a115-e6176de71b37" containerName="authorino" containerID="cri-o://237d72b0167fc392b13df704d4525652027f10641687d0944641099bad842b86" gracePeriod=30
Sep 16 14:14:03.938359 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:03.938338    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-q6j97"
Sep 16 14:14:04.004743 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:04.004677    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2p748\" (UniqueName: \"kubernetes.io/projected/d203bde6-cef7-4c8f-a115-e6176de71b37-kube-api-access-2p748\") pod \"d203bde6-cef7-4c8f-a115-e6176de71b37\" (UID: \"d203bde6-cef7-4c8f-a115-e6176de71b37\") "
Sep 16 14:14:04.006632 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:04.006611    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d203bde6-cef7-4c8f-a115-e6176de71b37-kube-api-access-2p748" (OuterVolumeSpecName: "kube-api-access-2p748") pod "d203bde6-cef7-4c8f-a115-e6176de71b37" (UID: "d203bde6-cef7-4c8f-a115-e6176de71b37"). InnerVolumeSpecName "kube-api-access-2p748". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:14:04.105899 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:04.105859    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2p748\" (UniqueName: \"kubernetes.io/projected/d203bde6-cef7-4c8f-a115-e6176de71b37-kube-api-access-2p748\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:14:04.747161 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:04.747129    2580 generic.go:356] "Generic (PLEG): container finished" podID="d203bde6-cef7-4c8f-a115-e6176de71b37" containerID="237d72b0167fc392b13df704d4525652027f10641687d0944641099bad842b86" exitCode=0
Sep 16 14:14:04.747652 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:04.747170    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-q6j97"
Sep 16 14:14:04.747652 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:04.747211    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-q6j97" event={"ID":"d203bde6-cef7-4c8f-a115-e6176de71b37","Type":"ContainerDied","Data":"237d72b0167fc392b13df704d4525652027f10641687d0944641099bad842b86"}
Sep 16 14:14:04.747652 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:04.747257    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-q6j97" event={"ID":"d203bde6-cef7-4c8f-a115-e6176de71b37","Type":"ContainerDied","Data":"895b9a7c1b1bb0020cef173fda94e2d1b6ca48e4df5554becbdb5c72b356fc9b"}
Sep 16 14:14:04.747652 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:04.747276    2580 scope.go:117] "RemoveContainer" containerID="237d72b0167fc392b13df704d4525652027f10641687d0944641099bad842b86"
Sep 16 14:14:04.755618 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:04.755604    2580 scope.go:117] "RemoveContainer" containerID="237d72b0167fc392b13df704d4525652027f10641687d0944641099bad842b86"
Sep 16 14:14:04.755857 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:14:04.755841    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"237d72b0167fc392b13df704d4525652027f10641687d0944641099bad842b86\": container with ID starting with 237d72b0167fc392b13df704d4525652027f10641687d0944641099bad842b86 not found: ID does not exist" containerID="237d72b0167fc392b13df704d4525652027f10641687d0944641099bad842b86"
Sep 16 14:14:04.755919 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:04.755868    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"237d72b0167fc392b13df704d4525652027f10641687d0944641099bad842b86"} err="failed to get container status \"237d72b0167fc392b13df704d4525652027f10641687d0944641099bad842b86\": rpc error: code = NotFound desc = could not find container \"237d72b0167fc392b13df704d4525652027f10641687d0944641099bad842b86\": container with ID starting with 237d72b0167fc392b13df704d4525652027f10641687d0944641099bad842b86 not found: ID does not exist"
Sep 16 14:14:04.773205 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:04.773183    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-q6j97"]
Sep 16 14:14:04.775990 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:04.775969    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59c974bd48-q6j97"]
Sep 16 14:14:05.433716 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:05.433684    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d203bde6-cef7-4c8f-a115-e6176de71b37" path="/var/lib/kubelet/pods/d203bde6-cef7-4c8f-a115-e6176de71b37/volumes"
Sep 16 14:14:08.615096 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:08.615066    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-75dc6cbdf-4m974"]
Sep 16 14:14:08.615482 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:08.615375    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d203bde6-cef7-4c8f-a115-e6176de71b37" containerName="authorino"
Sep 16 14:14:08.615482 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:08.615386    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d203bde6-cef7-4c8f-a115-e6176de71b37" containerName="authorino"
Sep 16 14:14:08.615482 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:08.615438    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d203bde6-cef7-4c8f-a115-e6176de71b37" containerName="authorino"
Sep 16 14:14:08.620220 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:08.620200    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75dc6cbdf-4m974"
Sep 16 14:14:08.622385 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:08.622362    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 16 14:14:08.622833 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:08.622815    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 16 14:14:08.623008 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:08.622987    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 16 14:14:08.623083 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:08.622893    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-25lqs\""
Sep 16 14:14:08.631742 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:08.631718    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-75dc6cbdf-4m974"]
Sep 16 14:14:08.637766 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:08.637742    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c67fbc12-4d4f-47c1-ad2f-ff528c737943-metrics-certs\") pod \"maas-controller-75dc6cbdf-4m974\" (UID: \"c67fbc12-4d4f-47c1-ad2f-ff528c737943\") " pod="opendatahub/maas-controller-75dc6cbdf-4m974"
Sep 16 14:14:08.637851 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:08.637787    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c67fbc12-4d4f-47c1-ad2f-ff528c737943-webhook-cert\") pod \"maas-controller-75dc6cbdf-4m974\" (UID: \"c67fbc12-4d4f-47c1-ad2f-ff528c737943\") " pod="opendatahub/maas-controller-75dc6cbdf-4m974"
Sep 16 14:14:08.637910 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:08.637859    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfctz\" (UniqueName: \"kubernetes.io/projected/c67fbc12-4d4f-47c1-ad2f-ff528c737943-kube-api-access-cfctz\") pod \"maas-controller-75dc6cbdf-4m974\" (UID: \"c67fbc12-4d4f-47c1-ad2f-ff528c737943\") " pod="opendatahub/maas-controller-75dc6cbdf-4m974"
Sep 16 14:14:08.739125 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:08.739090    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfctz\" (UniqueName: \"kubernetes.io/projected/c67fbc12-4d4f-47c1-ad2f-ff528c737943-kube-api-access-cfctz\") pod \"maas-controller-75dc6cbdf-4m974\" (UID: \"c67fbc12-4d4f-47c1-ad2f-ff528c737943\") " pod="opendatahub/maas-controller-75dc6cbdf-4m974"
Sep 16 14:14:08.739317 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:08.739158    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c67fbc12-4d4f-47c1-ad2f-ff528c737943-metrics-certs\") pod \"maas-controller-75dc6cbdf-4m974\" (UID: \"c67fbc12-4d4f-47c1-ad2f-ff528c737943\") " pod="opendatahub/maas-controller-75dc6cbdf-4m974"
Sep 16 14:14:08.739317 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:08.739183    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c67fbc12-4d4f-47c1-ad2f-ff528c737943-webhook-cert\") pod \"maas-controller-75dc6cbdf-4m974\" (UID: \"c67fbc12-4d4f-47c1-ad2f-ff528c737943\") " pod="opendatahub/maas-controller-75dc6cbdf-4m974"
Sep 16 14:14:08.739317 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:14:08.739269    2580 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Sep 16 14:14:08.739317 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:14:08.739300    2580 secret.go:189] Couldn't get secret opendatahub/maas-controller-metrics-tls: secret "maas-controller-metrics-tls" not found
Sep 16 14:14:08.739317 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:14:08.739320    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c67fbc12-4d4f-47c1-ad2f-ff528c737943-webhook-cert podName:c67fbc12-4d4f-47c1-ad2f-ff528c737943 nodeName:}" failed. No retries permitted until 2026-09-16 14:14:09.239303056 +0000 UTC m=+582.254412911 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/c67fbc12-4d4f-47c1-ad2f-ff528c737943-webhook-cert") pod "maas-controller-75dc6cbdf-4m974" (UID: "c67fbc12-4d4f-47c1-ad2f-ff528c737943") : secret "maas-controller-webhook-cert" not found
Sep 16 14:14:08.739607 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:14:08.739350    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c67fbc12-4d4f-47c1-ad2f-ff528c737943-metrics-certs podName:c67fbc12-4d4f-47c1-ad2f-ff528c737943 nodeName:}" failed. No retries permitted until 2026-09-16 14:14:09.239334199 +0000 UTC m=+582.254444053 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c67fbc12-4d4f-47c1-ad2f-ff528c737943-metrics-certs") pod "maas-controller-75dc6cbdf-4m974" (UID: "c67fbc12-4d4f-47c1-ad2f-ff528c737943") : secret "maas-controller-metrics-tls" not found
Sep 16 14:14:08.748848 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:08.748826    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfctz\" (UniqueName: \"kubernetes.io/projected/c67fbc12-4d4f-47c1-ad2f-ff528c737943-kube-api-access-cfctz\") pod \"maas-controller-75dc6cbdf-4m974\" (UID: \"c67fbc12-4d4f-47c1-ad2f-ff528c737943\") " pod="opendatahub/maas-controller-75dc6cbdf-4m974"
Sep 16 14:14:08.799349 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:08.799321    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-658475d98d-6ncwp"]
Sep 16 14:14:08.802812 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:08.802795    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-658475d98d-6ncwp"
Sep 16 14:14:08.818978 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:08.818954    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-658475d98d-6ncwp"]
Sep 16 14:14:08.840251 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:08.840222    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/793eec9a-8d86-4ebe-8308-836bbdd9e94c-webhook-cert\") pod \"maas-controller-658475d98d-6ncwp\" (UID: \"793eec9a-8d86-4ebe-8308-836bbdd9e94c\") " pod="opendatahub/maas-controller-658475d98d-6ncwp"
Sep 16 14:14:08.840362 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:08.840333    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtd6m\" (UniqueName: \"kubernetes.io/projected/793eec9a-8d86-4ebe-8308-836bbdd9e94c-kube-api-access-xtd6m\") pod \"maas-controller-658475d98d-6ncwp\" (UID: \"793eec9a-8d86-4ebe-8308-836bbdd9e94c\") " pod="opendatahub/maas-controller-658475d98d-6ncwp"
Sep 16 14:14:08.840417 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:08.840360    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/793eec9a-8d86-4ebe-8308-836bbdd9e94c-metrics-certs\") pod \"maas-controller-658475d98d-6ncwp\" (UID: \"793eec9a-8d86-4ebe-8308-836bbdd9e94c\") " pod="opendatahub/maas-controller-658475d98d-6ncwp"
Sep 16 14:14:08.940987 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:08.940958    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/793eec9a-8d86-4ebe-8308-836bbdd9e94c-webhook-cert\") pod \"maas-controller-658475d98d-6ncwp\" (UID: \"793eec9a-8d86-4ebe-8308-836bbdd9e94c\") " pod="opendatahub/maas-controller-658475d98d-6ncwp"
Sep 16 14:14:08.941117 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:08.941020    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtd6m\" (UniqueName: \"kubernetes.io/projected/793eec9a-8d86-4ebe-8308-836bbdd9e94c-kube-api-access-xtd6m\") pod \"maas-controller-658475d98d-6ncwp\" (UID: \"793eec9a-8d86-4ebe-8308-836bbdd9e94c\") " pod="opendatahub/maas-controller-658475d98d-6ncwp"
Sep 16 14:14:08.941117 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:08.941038    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/793eec9a-8d86-4ebe-8308-836bbdd9e94c-metrics-certs\") pod \"maas-controller-658475d98d-6ncwp\" (UID: \"793eec9a-8d86-4ebe-8308-836bbdd9e94c\") " pod="opendatahub/maas-controller-658475d98d-6ncwp"
Sep 16 14:14:08.943541 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:08.943486    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/793eec9a-8d86-4ebe-8308-836bbdd9e94c-metrics-certs\") pod \"maas-controller-658475d98d-6ncwp\" (UID: \"793eec9a-8d86-4ebe-8308-836bbdd9e94c\") " pod="opendatahub/maas-controller-658475d98d-6ncwp"
Sep 16 14:14:08.943666 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:08.943486    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/793eec9a-8d86-4ebe-8308-836bbdd9e94c-webhook-cert\") pod \"maas-controller-658475d98d-6ncwp\" (UID: \"793eec9a-8d86-4ebe-8308-836bbdd9e94c\") " pod="opendatahub/maas-controller-658475d98d-6ncwp"
Sep 16 14:14:08.949837 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:08.949809    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtd6m\" (UniqueName: \"kubernetes.io/projected/793eec9a-8d86-4ebe-8308-836bbdd9e94c-kube-api-access-xtd6m\") pod \"maas-controller-658475d98d-6ncwp\" (UID: \"793eec9a-8d86-4ebe-8308-836bbdd9e94c\") " pod="opendatahub/maas-controller-658475d98d-6ncwp"
Sep 16 14:14:09.112993 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:09.112962    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-658475d98d-6ncwp"
Sep 16 14:14:09.243213 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:09.243144    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c67fbc12-4d4f-47c1-ad2f-ff528c737943-metrics-certs\") pod \"maas-controller-75dc6cbdf-4m974\" (UID: \"c67fbc12-4d4f-47c1-ad2f-ff528c737943\") " pod="opendatahub/maas-controller-75dc6cbdf-4m974"
Sep 16 14:14:09.243213 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:09.243180    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c67fbc12-4d4f-47c1-ad2f-ff528c737943-webhook-cert\") pod \"maas-controller-75dc6cbdf-4m974\" (UID: \"c67fbc12-4d4f-47c1-ad2f-ff528c737943\") " pod="opendatahub/maas-controller-75dc6cbdf-4m974"
Sep 16 14:14:09.245439 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:09.245411    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c67fbc12-4d4f-47c1-ad2f-ff528c737943-webhook-cert\") pod \"maas-controller-75dc6cbdf-4m974\" (UID: \"c67fbc12-4d4f-47c1-ad2f-ff528c737943\") " pod="opendatahub/maas-controller-75dc6cbdf-4m974"
Sep 16 14:14:09.245560 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:09.245445    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c67fbc12-4d4f-47c1-ad2f-ff528c737943-metrics-certs\") pod \"maas-controller-75dc6cbdf-4m974\" (UID: \"c67fbc12-4d4f-47c1-ad2f-ff528c737943\") " pod="opendatahub/maas-controller-75dc6cbdf-4m974"
Sep 16 14:14:09.404615 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:09.404569    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-658475d98d-6ncwp"]
Sep 16 14:14:09.408292 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:14:09.408271    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod793eec9a_8d86_4ebe_8308_836bbdd9e94c.slice/crio-5329971e23942aef5220a6e0860cb584f621c0dc0b269289eac282c15bbfcd9a WatchSource:0}: Error finding container 5329971e23942aef5220a6e0860cb584f621c0dc0b269289eac282c15bbfcd9a: Status 404 returned error can't find the container with id 5329971e23942aef5220a6e0860cb584f621c0dc0b269289eac282c15bbfcd9a
Sep 16 14:14:09.533122 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:09.533048    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75dc6cbdf-4m974"
Sep 16 14:14:09.773535 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:09.773488    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-658475d98d-6ncwp" event={"ID":"793eec9a-8d86-4ebe-8308-836bbdd9e94c","Type":"ContainerStarted","Data":"5329971e23942aef5220a6e0860cb584f621c0dc0b269289eac282c15bbfcd9a"}
Sep 16 14:14:09.807554 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:09.807528    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-75dc6cbdf-4m974"]
Sep 16 14:14:09.809032 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:14:09.809006    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc67fbc12_4d4f_47c1_ad2f_ff528c737943.slice/crio-789c83786932fc66c394b706cc5a908e45578a6744be6fe7ed3be20c6ef6ccae WatchSource:0}: Error finding container 789c83786932fc66c394b706cc5a908e45578a6744be6fe7ed3be20c6ef6ccae: Status 404 returned error can't find the container with id 789c83786932fc66c394b706cc5a908e45578a6744be6fe7ed3be20c6ef6ccae
Sep 16 14:14:10.781104 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:10.781059    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75dc6cbdf-4m974" event={"ID":"c67fbc12-4d4f-47c1-ad2f-ff528c737943","Type":"ContainerStarted","Data":"789c83786932fc66c394b706cc5a908e45578a6744be6fe7ed3be20c6ef6ccae"}
Sep 16 14:14:11.787045 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:11.787011    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-658475d98d-6ncwp" event={"ID":"793eec9a-8d86-4ebe-8308-836bbdd9e94c","Type":"ContainerStarted","Data":"090eeace4aed24bc428874de5239f6d22d0d63925f7a82439e3a35b3d3f9b0f8"}
Sep 16 14:14:11.787464 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:11.787092    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-658475d98d-6ncwp"
Sep 16 14:14:11.788639 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:11.788618    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75dc6cbdf-4m974" event={"ID":"c67fbc12-4d4f-47c1-ad2f-ff528c737943","Type":"ContainerStarted","Data":"0be29f6c084bd6b5e8060ac47c735eee9c4289fda4bc2249e65bada402aedb1e"}
Sep 16 14:14:11.788745 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:11.788673    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-75dc6cbdf-4m974"
Sep 16 14:14:11.805485 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:11.805442    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-658475d98d-6ncwp" podStartSLOduration=1.544396158 podStartE2EDuration="3.805426809s" totalImagesPullingTime="2.261030651s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:14:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:14:11.804944522 +0000 UTC m=+584.820054381" watchObservedRunningTime="2026-09-16 14:14:11.805426809 +0000 UTC m=+584.820536670"
Sep 16 14:14:11.849992 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:11.849949    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-75dc6cbdf-4m974" podStartSLOduration=1.9891924859999999 podStartE2EDuration="3.849937326s" totalImagesPullingTime="1.86074484s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:14:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:14:11.848082705 +0000 UTC m=+584.863192579" watchObservedRunningTime="2026-09-16 14:14:11.849937326 +0000 UTC m=+584.865047186"
Sep 16 14:14:22.796243 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:22.796213    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-75dc6cbdf-4m974"
Sep 16 14:14:22.796744 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:22.796374    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-658475d98d-6ncwp"
Sep 16 14:14:22.882612 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:22.882566    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-75dc6cbdf-4m974"]
Sep 16 14:14:22.882867 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:22.882840    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-75dc6cbdf-4m974" podUID="c67fbc12-4d4f-47c1-ad2f-ff528c737943" containerName="manager" containerID="cri-o://0be29f6c084bd6b5e8060ac47c735eee9c4289fda4bc2249e65bada402aedb1e" gracePeriod=30
Sep 16 14:14:23.116133 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:23.116110    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75dc6cbdf-4m974"
Sep 16 14:14:23.148000 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:23.147968    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c67fbc12-4d4f-47c1-ad2f-ff528c737943-webhook-cert\") pod \"c67fbc12-4d4f-47c1-ad2f-ff528c737943\" (UID: \"c67fbc12-4d4f-47c1-ad2f-ff528c737943\") "
Sep 16 14:14:23.148155 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:23.148008    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cfctz\" (UniqueName: \"kubernetes.io/projected/c67fbc12-4d4f-47c1-ad2f-ff528c737943-kube-api-access-cfctz\") pod \"c67fbc12-4d4f-47c1-ad2f-ff528c737943\" (UID: \"c67fbc12-4d4f-47c1-ad2f-ff528c737943\") "
Sep 16 14:14:23.148155 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:23.148057    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c67fbc12-4d4f-47c1-ad2f-ff528c737943-metrics-certs\") pod \"c67fbc12-4d4f-47c1-ad2f-ff528c737943\" (UID: \"c67fbc12-4d4f-47c1-ad2f-ff528c737943\") "
Sep 16 14:14:23.150202 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:23.150167    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c67fbc12-4d4f-47c1-ad2f-ff528c737943-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "c67fbc12-4d4f-47c1-ad2f-ff528c737943" (UID: "c67fbc12-4d4f-47c1-ad2f-ff528c737943"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:14:23.150202 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:23.150190    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c67fbc12-4d4f-47c1-ad2f-ff528c737943-kube-api-access-cfctz" (OuterVolumeSpecName: "kube-api-access-cfctz") pod "c67fbc12-4d4f-47c1-ad2f-ff528c737943" (UID: "c67fbc12-4d4f-47c1-ad2f-ff528c737943"). InnerVolumeSpecName "kube-api-access-cfctz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:14:23.150378 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:23.150201    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c67fbc12-4d4f-47c1-ad2f-ff528c737943-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "c67fbc12-4d4f-47c1-ad2f-ff528c737943" (UID: "c67fbc12-4d4f-47c1-ad2f-ff528c737943"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:14:23.249466 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:23.249437    2580 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c67fbc12-4d4f-47c1-ad2f-ff528c737943-webhook-cert\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:14:23.249466 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:23.249463    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cfctz\" (UniqueName: \"kubernetes.io/projected/c67fbc12-4d4f-47c1-ad2f-ff528c737943-kube-api-access-cfctz\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:14:23.249641 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:23.249472    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c67fbc12-4d4f-47c1-ad2f-ff528c737943-metrics-certs\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:14:23.847494 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:23.847418    2580 generic.go:356] "Generic (PLEG): container finished" podID="c67fbc12-4d4f-47c1-ad2f-ff528c737943" containerID="0be29f6c084bd6b5e8060ac47c735eee9c4289fda4bc2249e65bada402aedb1e" exitCode=0
Sep 16 14:14:23.847494 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:23.847463    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75dc6cbdf-4m974"
Sep 16 14:14:23.847494 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:23.847472    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75dc6cbdf-4m974" event={"ID":"c67fbc12-4d4f-47c1-ad2f-ff528c737943","Type":"ContainerDied","Data":"0be29f6c084bd6b5e8060ac47c735eee9c4289fda4bc2249e65bada402aedb1e"}
Sep 16 14:14:23.848011 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:23.847537    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75dc6cbdf-4m974" event={"ID":"c67fbc12-4d4f-47c1-ad2f-ff528c737943","Type":"ContainerDied","Data":"789c83786932fc66c394b706cc5a908e45578a6744be6fe7ed3be20c6ef6ccae"}
Sep 16 14:14:23.848011 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:23.847561    2580 scope.go:117] "RemoveContainer" containerID="0be29f6c084bd6b5e8060ac47c735eee9c4289fda4bc2249e65bada402aedb1e"
Sep 16 14:14:23.856003 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:23.855986    2580 scope.go:117] "RemoveContainer" containerID="0be29f6c084bd6b5e8060ac47c735eee9c4289fda4bc2249e65bada402aedb1e"
Sep 16 14:14:23.856265 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:14:23.856246    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0be29f6c084bd6b5e8060ac47c735eee9c4289fda4bc2249e65bada402aedb1e\": container with ID starting with 0be29f6c084bd6b5e8060ac47c735eee9c4289fda4bc2249e65bada402aedb1e not found: ID does not exist" containerID="0be29f6c084bd6b5e8060ac47c735eee9c4289fda4bc2249e65bada402aedb1e"
Sep 16 14:14:23.856311 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:23.856274    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0be29f6c084bd6b5e8060ac47c735eee9c4289fda4bc2249e65bada402aedb1e"} err="failed to get container status \"0be29f6c084bd6b5e8060ac47c735eee9c4289fda4bc2249e65bada402aedb1e\": rpc error: code = NotFound desc = could not find container \"0be29f6c084bd6b5e8060ac47c735eee9c4289fda4bc2249e65bada402aedb1e\": container with ID starting with 0be29f6c084bd6b5e8060ac47c735eee9c4289fda4bc2249e65bada402aedb1e not found: ID does not exist"
Sep 16 14:14:23.870361 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:23.870337    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-75dc6cbdf-4m974"]
Sep 16 14:14:23.879144 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:23.879118    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-75dc6cbdf-4m974"]
Sep 16 14:14:25.430156 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:25.430126    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c67fbc12-4d4f-47c1-ad2f-ff528c737943" path="/var/lib/kubelet/pods/c67fbc12-4d4f-47c1-ad2f-ff528c737943/volumes"
Sep 16 14:14:44.907613 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:44.907579    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-59f6566bbc-npc5m"]
Sep 16 14:14:44.908074 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:44.907927    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c67fbc12-4d4f-47c1-ad2f-ff528c737943" containerName="manager"
Sep 16 14:14:44.908074 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:44.907937    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c67fbc12-4d4f-47c1-ad2f-ff528c737943" containerName="manager"
Sep 16 14:14:44.908074 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:44.907983    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c67fbc12-4d4f-47c1-ad2f-ff528c737943" containerName="manager"
Sep 16 14:14:44.914682 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:44.914663    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-npc5m"
Sep 16 14:14:44.917179 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:44.917159    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 16 14:14:44.918091 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:44.918067    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-b7mcl\""
Sep 16 14:14:44.918203 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:44.918165    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 16 14:14:44.918203 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:44.918183    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 14:14:44.918320 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:44.918077    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 16 14:14:44.925214 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:44.925189    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-59f6566bbc-npc5m"]
Sep 16 14:14:45.028232 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:45.028197    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/40adf329-d01c-4d45-afec-fdc8f1acf8f7-metrics-certs\") pod \"maas-api-59f6566bbc-npc5m\" (UID: \"40adf329-d01c-4d45-afec-fdc8f1acf8f7\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-npc5m"
Sep 16 14:14:45.028232 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:45.028239    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svzzz\" (UniqueName: \"kubernetes.io/projected/40adf329-d01c-4d45-afec-fdc8f1acf8f7-kube-api-access-svzzz\") pod \"maas-api-59f6566bbc-npc5m\" (UID: \"40adf329-d01c-4d45-afec-fdc8f1acf8f7\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-npc5m"
Sep 16 14:14:45.028457 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:45.028259    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40adf329-d01c-4d45-afec-fdc8f1acf8f7-maas-api-tls\") pod \"maas-api-59f6566bbc-npc5m\" (UID: \"40adf329-d01c-4d45-afec-fdc8f1acf8f7\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-npc5m"
Sep 16 14:14:45.028457 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:45.028381    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40adf329-d01c-4d45-afec-fdc8f1acf8f7-openshift-service-ca\") pod \"maas-api-59f6566bbc-npc5m\" (UID: \"40adf329-d01c-4d45-afec-fdc8f1acf8f7\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-npc5m"
Sep 16 14:14:45.129338 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:45.129308    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40adf329-d01c-4d45-afec-fdc8f1acf8f7-openshift-service-ca\") pod \"maas-api-59f6566bbc-npc5m\" (UID: \"40adf329-d01c-4d45-afec-fdc8f1acf8f7\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-npc5m"
Sep 16 14:14:45.129528 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:45.129376    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/40adf329-d01c-4d45-afec-fdc8f1acf8f7-metrics-certs\") pod \"maas-api-59f6566bbc-npc5m\" (UID: \"40adf329-d01c-4d45-afec-fdc8f1acf8f7\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-npc5m"
Sep 16 14:14:45.129528 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:45.129412    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svzzz\" (UniqueName: \"kubernetes.io/projected/40adf329-d01c-4d45-afec-fdc8f1acf8f7-kube-api-access-svzzz\") pod \"maas-api-59f6566bbc-npc5m\" (UID: \"40adf329-d01c-4d45-afec-fdc8f1acf8f7\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-npc5m"
Sep 16 14:14:45.129658 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:14:45.129530    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-metrics-tls: secret "maas-api-metrics-tls" not found
Sep 16 14:14:45.129658 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:14:45.129614    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40adf329-d01c-4d45-afec-fdc8f1acf8f7-metrics-certs podName:40adf329-d01c-4d45-afec-fdc8f1acf8f7 nodeName:}" failed. No retries permitted until 2026-09-16 14:14:45.629591758 +0000 UTC m=+618.644701611 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/40adf329-d01c-4d45-afec-fdc8f1acf8f7-metrics-certs") pod "maas-api-59f6566bbc-npc5m" (UID: "40adf329-d01c-4d45-afec-fdc8f1acf8f7") : secret "maas-api-metrics-tls" not found
Sep 16 14:14:45.129658 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:45.129642    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40adf329-d01c-4d45-afec-fdc8f1acf8f7-maas-api-tls\") pod \"maas-api-59f6566bbc-npc5m\" (UID: \"40adf329-d01c-4d45-afec-fdc8f1acf8f7\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-npc5m"
Sep 16 14:14:45.130068 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:45.130040    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40adf329-d01c-4d45-afec-fdc8f1acf8f7-openshift-service-ca\") pod \"maas-api-59f6566bbc-npc5m\" (UID: \"40adf329-d01c-4d45-afec-fdc8f1acf8f7\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-npc5m"
Sep 16 14:14:45.132250 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:45.132229    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40adf329-d01c-4d45-afec-fdc8f1acf8f7-maas-api-tls\") pod \"maas-api-59f6566bbc-npc5m\" (UID: \"40adf329-d01c-4d45-afec-fdc8f1acf8f7\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-npc5m"
Sep 16 14:14:45.140151 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:45.140125    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-svzzz\" (UniqueName: \"kubernetes.io/projected/40adf329-d01c-4d45-afec-fdc8f1acf8f7-kube-api-access-svzzz\") pod \"maas-api-59f6566bbc-npc5m\" (UID: \"40adf329-d01c-4d45-afec-fdc8f1acf8f7\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-npc5m"
Sep 16 14:14:45.633101 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:45.633067    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/40adf329-d01c-4d45-afec-fdc8f1acf8f7-metrics-certs\") pod \"maas-api-59f6566bbc-npc5m\" (UID: \"40adf329-d01c-4d45-afec-fdc8f1acf8f7\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-npc5m"
Sep 16 14:14:45.635588 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:45.635559    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/40adf329-d01c-4d45-afec-fdc8f1acf8f7-metrics-certs\") pod \"maas-api-59f6566bbc-npc5m\" (UID: \"40adf329-d01c-4d45-afec-fdc8f1acf8f7\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-npc5m"
Sep 16 14:14:45.714171 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:45.714137    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-c48qx"]
Sep 16 14:14:45.718327 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:45.718304    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-c48qx"
Sep 16 14:14:45.724977 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:45.724958    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 16 14:14:45.725051 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:45.724958    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-ccsnl\""
Sep 16 14:14:45.827986 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:45.827949    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-npc5m"
Sep 16 14:14:45.832411 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:45.832386    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-c48qx"]
Sep 16 14:14:45.834899 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:45.834862    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b1ed4abf-7af6-4ef1-9b71-991fbc4ead65-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-c48qx\" (UID: \"b1ed4abf-7af6-4ef1-9b71-991fbc4ead65\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-c48qx"
Sep 16 14:14:45.835069 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:45.835049    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v25v5\" (UniqueName: \"kubernetes.io/projected/b1ed4abf-7af6-4ef1-9b71-991fbc4ead65-kube-api-access-v25v5\") pod \"payload-processing-5b54b7fbb7-c48qx\" (UID: \"b1ed4abf-7af6-4ef1-9b71-991fbc4ead65\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-c48qx"
Sep 16 14:14:45.936131 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:45.936100    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v25v5\" (UniqueName: \"kubernetes.io/projected/b1ed4abf-7af6-4ef1-9b71-991fbc4ead65-kube-api-access-v25v5\") pod \"payload-processing-5b54b7fbb7-c48qx\" (UID: \"b1ed4abf-7af6-4ef1-9b71-991fbc4ead65\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-c48qx"
Sep 16 14:14:45.936508 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:45.936133    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b1ed4abf-7af6-4ef1-9b71-991fbc4ead65-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-c48qx\" (UID: \"b1ed4abf-7af6-4ef1-9b71-991fbc4ead65\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-c48qx"
Sep 16 14:14:45.936876 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:45.936853    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b1ed4abf-7af6-4ef1-9b71-991fbc4ead65-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-c48qx\" (UID: \"b1ed4abf-7af6-4ef1-9b71-991fbc4ead65\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-c48qx"
Sep 16 14:14:45.949953 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:45.949922    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v25v5\" (UniqueName: \"kubernetes.io/projected/b1ed4abf-7af6-4ef1-9b71-991fbc4ead65-kube-api-access-v25v5\") pod \"payload-processing-5b54b7fbb7-c48qx\" (UID: \"b1ed4abf-7af6-4ef1-9b71-991fbc4ead65\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-c48qx"
Sep 16 14:14:46.028140 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:46.028108    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-c48qx"
Sep 16 14:14:46.141782 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:46.141756    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-59f6566bbc-npc5m"]
Sep 16 14:14:46.143297 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:14:46.143274    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40adf329_d01c_4d45_afec_fdc8f1acf8f7.slice/crio-bf94ae94102003f28c2bed3ce1a897dda06f551135c25f102c270610705dc901 WatchSource:0}: Error finding container bf94ae94102003f28c2bed3ce1a897dda06f551135c25f102c270610705dc901: Status 404 returned error can't find the container with id bf94ae94102003f28c2bed3ce1a897dda06f551135c25f102c270610705dc901
Sep 16 14:14:46.306217 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:46.306185    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-c48qx"]
Sep 16 14:14:46.309299 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:14:46.309261    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1ed4abf_7af6_4ef1_9b71_991fbc4ead65.slice/crio-ba622aaeff0d53d384d669e416ede370823127eb06ca19d76bd5ddf15a0b3634 WatchSource:0}: Error finding container ba622aaeff0d53d384d669e416ede370823127eb06ca19d76bd5ddf15a0b3634: Status 404 returned error can't find the container with id ba622aaeff0d53d384d669e416ede370823127eb06ca19d76bd5ddf15a0b3634
Sep 16 14:14:46.960187 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:46.960118    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-npc5m" event={"ID":"40adf329-d01c-4d45-afec-fdc8f1acf8f7","Type":"ContainerStarted","Data":"bf94ae94102003f28c2bed3ce1a897dda06f551135c25f102c270610705dc901"}
Sep 16 14:14:46.961491 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:46.961468    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-c48qx" event={"ID":"b1ed4abf-7af6-4ef1-9b71-991fbc4ead65","Type":"ContainerStarted","Data":"ba622aaeff0d53d384d669e416ede370823127eb06ca19d76bd5ddf15a0b3634"}
Sep 16 14:14:48.974626 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:48.974583    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-npc5m" event={"ID":"40adf329-d01c-4d45-afec-fdc8f1acf8f7","Type":"ContainerStarted","Data":"643206e6f3af898cfe2e02a3e69b4abe34495750465b0bece4fbdea930348940"}
Sep 16 14:14:48.975079 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:48.974724    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-npc5m"
Sep 16 14:14:48.992253 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:48.992192    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-npc5m" podStartSLOduration=3.135555438 podStartE2EDuration="4.992174587s" totalImagesPullingTime="1.856619149s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:14:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:14:48.991444143 +0000 UTC m=+622.006554030" watchObservedRunningTime="2026-09-16 14:14:48.992174587 +0000 UTC m=+622.007284448"
Sep 16 14:14:49.981755 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:49.981718    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-c48qx" event={"ID":"b1ed4abf-7af6-4ef1-9b71-991fbc4ead65","Type":"ContainerStarted","Data":"945c2b0bda0b7b378ae53d83b72d726691e558a05026711d6b3da68a8e553060"}
Sep 16 14:14:49.982128 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:49.982034    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-c48qx"
Sep 16 14:14:54.986936 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:54.986906    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-npc5m"
Sep 16 14:14:55.007766 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:55.007714    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-c48qx" podStartSLOduration=6.538911949 podStartE2EDuration="10.007696217s" totalImagesPullingTime="3.468784268s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:14:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:14:50.019648435 +0000 UTC m=+623.034758296" watchObservedRunningTime="2026-09-16 14:14:55.007696217 +0000 UTC m=+628.022806078"
Sep 16 14:14:55.536642 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:55.536605    2580 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-jfn4c container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body=
Sep 16 14:14:55.536841 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:55.536670    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jfn4c" podUID="aedbe6b8-69e8-4213-b355-e4cdcd452d49" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused"
Sep 16 14:14:56.014069 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:56.014031    2580 generic.go:356] "Generic (PLEG): container finished" podID="aedbe6b8-69e8-4213-b355-e4cdcd452d49" containerID="83c386a165de6c45771bc31a9f6a53d44a74b09949d60ebe0354dcfd3e70818c" exitCode=0
Sep 16 14:14:56.014465 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:56.014098    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jfn4c" event={"ID":"aedbe6b8-69e8-4213-b355-e4cdcd452d49","Type":"ContainerDied","Data":"83c386a165de6c45771bc31a9f6a53d44a74b09949d60ebe0354dcfd3e70818c"}
Sep 16 14:14:56.014535 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:56.014518    2580 scope.go:117] "RemoveContainer" containerID="83c386a165de6c45771bc31a9f6a53d44a74b09949d60ebe0354dcfd3e70818c"
Sep 16 14:14:56.015616 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:56.015584    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 14:14:56.015729 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:56.015675    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 14:14:56.015795 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:56.015753    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 14:14:57.024771 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:57.024688    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jfn4c" event={"ID":"aedbe6b8-69e8-4213-b355-e4cdcd452d49","Type":"ContainerStarted","Data":"48e44f72bf3ccb7c019ace47fcee5275be0890ebbf0334613bd505bf10addee8"}
Sep 16 14:14:57.256089 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:57.256056    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jfn4c"
Sep 16 14:14:57.256262 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:57.256238    2580 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-jfn4c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body=
Sep 16 14:14:57.256317 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:57.256296    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jfn4c" podUID="aedbe6b8-69e8-4213-b355-e4cdcd452d49" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused"
Sep 16 14:14:58.256708 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:58.256673    2580 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-jfn4c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body=
Sep 16 14:14:58.257080 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:58.256741    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jfn4c" podUID="aedbe6b8-69e8-4213-b355-e4cdcd452d49" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused"
Sep 16 14:14:59.256596 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:59.256561    2580 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-jfn4c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body=
Sep 16 14:14:59.256759 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:14:59.256629    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jfn4c" podUID="aedbe6b8-69e8-4213-b355-e4cdcd452d49" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused"
Sep 16 14:15:00.256547 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:15:00.256514    2580 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-jfn4c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body=
Sep 16 14:15:00.256726 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:15:00.256569    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jfn4c" podUID="aedbe6b8-69e8-4213-b355-e4cdcd452d49" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused"
Sep 16 14:15:00.988670 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:15:00.988633    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-c48qx"
Sep 16 14:15:01.256391 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:15:01.256307    2580 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-jfn4c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body=
Sep 16 14:15:01.256391 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:15:01.256366    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jfn4c" podUID="aedbe6b8-69e8-4213-b355-e4cdcd452d49" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused"
Sep 16 14:15:02.259975 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:15:02.259944    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jfn4c"
Sep 16 14:15:02.260399 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:15:02.260202    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jfn4c"
Sep 16 14:15:02.260977 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:15:02.260959    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jfn4c"
Sep 16 14:15:39.883331 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:15:39.883289    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-gkq64"]
Sep 16 14:15:39.886761 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:15:39.886741    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gkq64"
Sep 16 14:15:39.889030 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:15:39.889007    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 16 14:15:39.889122 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:15:39.889019    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 16 14:15:39.889484 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:15:39.889466    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 16 14:15:39.889562 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:15:39.889482    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-k6tqt\""
Sep 16 14:15:39.897737 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:15:39.897712    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-gkq64"]
Sep 16 14:15:39.985713 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:15:39.985679    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfjd6\" (UniqueName: \"kubernetes.io/projected/0f6a2e77-732d-4f09-81d6-0e946934370a-kube-api-access-xfjd6\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gkq64\" (UID: \"0f6a2e77-732d-4f09-81d6-0e946934370a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gkq64"
Sep 16 14:15:39.985881 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:15:39.985744    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0f6a2e77-732d-4f09-81d6-0e946934370a-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gkq64\" (UID: \"0f6a2e77-732d-4f09-81d6-0e946934370a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gkq64"
Sep 16 14:15:39.985881 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:15:39.985788    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f6a2e77-732d-4f09-81d6-0e946934370a-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gkq64\" (UID: \"0f6a2e77-732d-4f09-81d6-0e946934370a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gkq64"
Sep 16 14:15:39.985968 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:15:39.985876    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f6a2e77-732d-4f09-81d6-0e946934370a-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gkq64\" (UID: \"0f6a2e77-732d-4f09-81d6-0e946934370a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gkq64"
Sep 16 14:15:39.985968 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:15:39.985904    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f6a2e77-732d-4f09-81d6-0e946934370a-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gkq64\" (UID: \"0f6a2e77-732d-4f09-81d6-0e946934370a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gkq64"
Sep 16 14:15:39.985968 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:15:39.985939    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f6a2e77-732d-4f09-81d6-0e946934370a-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gkq64\" (UID: \"0f6a2e77-732d-4f09-81d6-0e946934370a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gkq64"
Sep 16 14:15:40.086364 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:15:40.086328    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f6a2e77-732d-4f09-81d6-0e946934370a-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gkq64\" (UID: \"0f6a2e77-732d-4f09-81d6-0e946934370a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gkq64"
Sep 16 14:15:40.086555 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:15:40.086381    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfjd6\" (UniqueName: \"kubernetes.io/projected/0f6a2e77-732d-4f09-81d6-0e946934370a-kube-api-access-xfjd6\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gkq64\" (UID: \"0f6a2e77-732d-4f09-81d6-0e946934370a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gkq64"
Sep 16 14:15:40.086555 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:15:40.086418    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0f6a2e77-732d-4f09-81d6-0e946934370a-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gkq64\" (UID: \"0f6a2e77-732d-4f09-81d6-0e946934370a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gkq64"
Sep 16 14:15:40.086555 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:15:40.086432    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f6a2e77-732d-4f09-81d6-0e946934370a-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gkq64\" (UID: \"0f6a2e77-732d-4f09-81d6-0e946934370a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gkq64"
Sep 16 14:15:40.086555 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:15:40.086476    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f6a2e77-732d-4f09-81d6-0e946934370a-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gkq64\" (UID: \"0f6a2e77-732d-4f09-81d6-0e946934370a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gkq64"
Sep 16 14:15:40.086555 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:15:40.086523    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f6a2e77-732d-4f09-81d6-0e946934370a-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gkq64\" (UID: \"0f6a2e77-732d-4f09-81d6-0e946934370a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gkq64"
Sep 16 14:15:40.086916 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:15:40.086885    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0f6a2e77-732d-4f09-81d6-0e946934370a-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gkq64\" (UID: \"0f6a2e77-732d-4f09-81d6-0e946934370a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gkq64"
Sep 16 14:15:40.087020 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:15:40.086928    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f6a2e77-732d-4f09-81d6-0e946934370a-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gkq64\" (UID: \"0f6a2e77-732d-4f09-81d6-0e946934370a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gkq64"
Sep 16 14:15:40.087020 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:15:40.086989    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f6a2e77-732d-4f09-81d6-0e946934370a-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gkq64\" (UID: \"0f6a2e77-732d-4f09-81d6-0e946934370a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gkq64"
Sep 16 14:15:40.088749 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:15:40.088731    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f6a2e77-732d-4f09-81d6-0e946934370a-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gkq64\" (UID: \"0f6a2e77-732d-4f09-81d6-0e946934370a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gkq64"
Sep 16 14:15:40.088921 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:15:40.088905    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f6a2e77-732d-4f09-81d6-0e946934370a-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gkq64\" (UID: \"0f6a2e77-732d-4f09-81d6-0e946934370a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gkq64"
Sep 16 14:15:40.094598 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:15:40.094574    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfjd6\" (UniqueName: \"kubernetes.io/projected/0f6a2e77-732d-4f09-81d6-0e946934370a-kube-api-access-xfjd6\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gkq64\" (UID: \"0f6a2e77-732d-4f09-81d6-0e946934370a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gkq64"
Sep 16 14:15:40.197556 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:15:40.197523    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gkq64"
Sep 16 14:15:40.478375 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:15:40.478349    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-gkq64"]
Sep 16 14:15:40.480595 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:15:40.480562    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f6a2e77_732d_4f09_81d6_0e946934370a.slice/crio-a741ebda7b56d7831ca36d8eb476642606ce68be0c8f325d6cc07b7e337131f2 WatchSource:0}: Error finding container a741ebda7b56d7831ca36d8eb476642606ce68be0c8f325d6cc07b7e337131f2: Status 404 returned error can't find the container with id a741ebda7b56d7831ca36d8eb476642606ce68be0c8f325d6cc07b7e337131f2
Sep 16 14:15:40.482016 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:15:40.482000    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 14:15:41.261747 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:15:41.261701    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gkq64" event={"ID":"0f6a2e77-732d-4f09-81d6-0e946934370a","Type":"ContainerStarted","Data":"a741ebda7b56d7831ca36d8eb476642606ce68be0c8f325d6cc07b7e337131f2"}
Sep 16 14:15:42.268510 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:15:42.268458    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gkq64" event={"ID":"0f6a2e77-732d-4f09-81d6-0e946934370a","Type":"ContainerStarted","Data":"1872bda5e0fe519d4336cfc60a5fb775db7edb6e18b98a286392d62b425111bb"}
Sep 16 14:15:42.287775 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:15:42.287727    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gkq64" podStartSLOduration=2.278587273 podStartE2EDuration="3.2877087s" totalImagesPullingTime="1.009121427s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:15:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:15:42.284646921 +0000 UTC m=+675.299756779" watchObservedRunningTime="2026-09-16 14:15:42.2877087 +0000 UTC m=+675.302818563"
Sep 16 14:15:50.198618 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:15:50.198581    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gkq64"
Sep 16 14:15:50.199013 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:15:50.198661    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gkq64"
Sep 16 14:15:50.211687 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:15:50.211658    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gkq64"
Sep 16 14:15:50.321150 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:15:50.321122    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gkq64"
Sep 16 14:15:55.165273 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:15:55.165231    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-766hm"]
Sep 16 14:15:55.168552 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:15:55.168534    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-766hm"
Sep 16 14:15:55.171833 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:15:55.171809    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Sep 16 14:15:55.198007 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:15:55.197976    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-766hm"]
Sep 16 14:15:55.221201 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:15:55.221173    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0cd06ff2-ed61-408f-aacf-0dfe837b1c0a-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-766hm\" (UID: \"0cd06ff2-ed61-408f-aacf-0dfe837b1c0a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-766hm"
Sep 16 14:15:55.221354 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:15:55.221229    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0cd06ff2-ed61-408f-aacf-0dfe837b1c0a-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-766hm\" (UID: \"0cd06ff2-ed61-408f-aacf-0dfe837b1c0a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-766hm"
Sep 16 14:15:55.221354 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:15:55.221267    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6x7l9\" (UniqueName: \"kubernetes.io/projected/0cd06ff2-ed61-408f-aacf-0dfe837b1c0a-kube-api-access-6x7l9\") pod \"e2e-trlp-test-simulated-kserve-598466548d-766hm\" (UID: \"0cd06ff2-ed61-408f-aacf-0dfe837b1c0a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-766hm"
Sep 16 14:15:55.221354 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:15:55.221316    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0cd06ff2-ed61-408f-aacf-0dfe837b1c0a-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-766hm\" (UID: \"0cd06ff2-ed61-408f-aacf-0dfe837b1c0a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-766hm"
Sep 16 14:15:55.221354 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:15:55.221344    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0cd06ff2-ed61-408f-aacf-0dfe837b1c0a-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-766hm\" (UID: \"0cd06ff2-ed61-408f-aacf-0dfe837b1c0a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-766hm"
Sep 16 14:15:55.221565 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:15:55.221435    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0cd06ff2-ed61-408f-aacf-0dfe837b1c0a-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-766hm\" (UID: \"0cd06ff2-ed61-408f-aacf-0dfe837b1c0a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-766hm"
Sep 16 14:15:55.322131 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:15:55.322097    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0cd06ff2-ed61-408f-aacf-0dfe837b1c0a-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-766hm\" (UID: \"0cd06ff2-ed61-408f-aacf-0dfe837b1c0a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-766hm"
Sep 16 14:15:55.322298 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:15:55.322163    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0cd06ff2-ed61-408f-aacf-0dfe837b1c0a-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-766hm\" (UID: \"0cd06ff2-ed61-408f-aacf-0dfe837b1c0a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-766hm"
Sep 16 14:15:55.322298 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:15:55.322218    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0cd06ff2-ed61-408f-aacf-0dfe837b1c0a-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-766hm\" (UID: \"0cd06ff2-ed61-408f-aacf-0dfe837b1c0a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-766hm"
Sep 16 14:15:55.322298 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:15:55.322247    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6x7l9\" (UniqueName: \"kubernetes.io/projected/0cd06ff2-ed61-408f-aacf-0dfe837b1c0a-kube-api-access-6x7l9\") pod \"e2e-trlp-test-simulated-kserve-598466548d-766hm\" (UID: \"0cd06ff2-ed61-408f-aacf-0dfe837b1c0a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-766hm"
Sep 16 14:15:55.322298 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:15:55.322277    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0cd06ff2-ed61-408f-aacf-0dfe837b1c0a-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-766hm\" (UID: \"0cd06ff2-ed61-408f-aacf-0dfe837b1c0a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-766hm"
Sep 16 14:15:55.322514 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:15:55.322302    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0cd06ff2-ed61-408f-aacf-0dfe837b1c0a-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-766hm\" (UID: \"0cd06ff2-ed61-408f-aacf-0dfe837b1c0a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-766hm"
Sep 16 14:15:55.322724 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:15:55.322686    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0cd06ff2-ed61-408f-aacf-0dfe837b1c0a-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-766hm\" (UID: \"0cd06ff2-ed61-408f-aacf-0dfe837b1c0a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-766hm"
Sep 16 14:15:55.322724 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:15:55.322705    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0cd06ff2-ed61-408f-aacf-0dfe837b1c0a-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-766hm\" (UID: \"0cd06ff2-ed61-408f-aacf-0dfe837b1c0a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-766hm"
Sep 16 14:15:55.322846 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:15:55.322732    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0cd06ff2-ed61-408f-aacf-0dfe837b1c0a-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-766hm\" (UID: \"0cd06ff2-ed61-408f-aacf-0dfe837b1c0a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-766hm"
Sep 16 14:15:55.324563 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:15:55.324539    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0cd06ff2-ed61-408f-aacf-0dfe837b1c0a-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-766hm\" (UID: \"0cd06ff2-ed61-408f-aacf-0dfe837b1c0a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-766hm"
Sep 16 14:15:55.324720 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:15:55.324702    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0cd06ff2-ed61-408f-aacf-0dfe837b1c0a-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-766hm\" (UID: \"0cd06ff2-ed61-408f-aacf-0dfe837b1c0a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-766hm"
Sep 16 14:15:55.339019 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:15:55.338994    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6x7l9\" (UniqueName: \"kubernetes.io/projected/0cd06ff2-ed61-408f-aacf-0dfe837b1c0a-kube-api-access-6x7l9\") pod \"e2e-trlp-test-simulated-kserve-598466548d-766hm\" (UID: \"0cd06ff2-ed61-408f-aacf-0dfe837b1c0a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-766hm"
Sep 16 14:15:55.478988 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:15:55.478949    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-766hm"
Sep 16 14:15:55.779741 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:15:55.779723    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-766hm"]
Sep 16 14:15:55.781780 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:15:55.781752    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cd06ff2_ed61_408f_aacf_0dfe837b1c0a.slice/crio-f9be4a847284fffe2c7e93f1ec0c8fc1a014926a1177071bc041e3a64f10a6fb WatchSource:0}: Error finding container f9be4a847284fffe2c7e93f1ec0c8fc1a014926a1177071bc041e3a64f10a6fb: Status 404 returned error can't find the container with id f9be4a847284fffe2c7e93f1ec0c8fc1a014926a1177071bc041e3a64f10a6fb
Sep 16 14:15:56.338444 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:15:56.338361    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-766hm" event={"ID":"0cd06ff2-ed61-408f-aacf-0dfe837b1c0a","Type":"ContainerStarted","Data":"1bac6a1f742bb1fe37e071dfe1992f4c3a7691049b228affe6c536e98e41d65d"}
Sep 16 14:15:56.338444 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:15:56.338414    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-766hm" event={"ID":"0cd06ff2-ed61-408f-aacf-0dfe837b1c0a","Type":"ContainerStarted","Data":"f9be4a847284fffe2c7e93f1ec0c8fc1a014926a1177071bc041e3a64f10a6fb"}
Sep 16 14:15:56.371290 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:15:56.371238    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-766hm" podStartSLOduration=1.11100119 podStartE2EDuration="1.371221492s" totalImagesPullingTime="260.220302ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:15:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:15:56.368676724 +0000 UTC m=+689.383786584" watchObservedRunningTime="2026-09-16 14:15:56.371221492 +0000 UTC m=+689.386331352"
Sep 16 14:16:05.479845 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:16:05.479791    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-766hm"
Sep 16 14:16:05.480250 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:16:05.479869    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-766hm"
Sep 16 14:16:05.492377 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:16:05.492352    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-766hm"
Sep 16 14:16:06.399682 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:16:06.399653    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-766hm"
Sep 16 14:16:20.372995 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:16:20.372906    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-j4dnc"]
Sep 16 14:16:20.376743 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:16:20.376719    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j4dnc"
Sep 16 14:16:20.379664 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:16:20.379639    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Sep 16 14:16:20.386551 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:16:20.386529    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-j4dnc"]
Sep 16 14:16:20.449654 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:16:20.449623    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f79d4b44-8a94-42c6-88c5-10cb8a5210f2-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j4dnc\" (UID: \"f79d4b44-8a94-42c6-88c5-10cb8a5210f2\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j4dnc"
Sep 16 14:16:20.449654 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:16:20.449661    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f79d4b44-8a94-42c6-88c5-10cb8a5210f2-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j4dnc\" (UID: \"f79d4b44-8a94-42c6-88c5-10cb8a5210f2\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j4dnc"
Sep 16 14:16:20.449860 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:16:20.449691    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f79d4b44-8a94-42c6-88c5-10cb8a5210f2-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j4dnc\" (UID: \"f79d4b44-8a94-42c6-88c5-10cb8a5210f2\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j4dnc"
Sep 16 14:16:20.449860 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:16:20.449736    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6xpz\" (UniqueName: \"kubernetes.io/projected/f79d4b44-8a94-42c6-88c5-10cb8a5210f2-kube-api-access-m6xpz\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j4dnc\" (UID: \"f79d4b44-8a94-42c6-88c5-10cb8a5210f2\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j4dnc"
Sep 16 14:16:20.449860 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:16:20.449775    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f79d4b44-8a94-42c6-88c5-10cb8a5210f2-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j4dnc\" (UID: \"f79d4b44-8a94-42c6-88c5-10cb8a5210f2\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j4dnc"
Sep 16 14:16:20.449860 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:16:20.449800    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f79d4b44-8a94-42c6-88c5-10cb8a5210f2-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j4dnc\" (UID: \"f79d4b44-8a94-42c6-88c5-10cb8a5210f2\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j4dnc"
Sep 16 14:16:20.550605 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:16:20.550559    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f79d4b44-8a94-42c6-88c5-10cb8a5210f2-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j4dnc\" (UID: \"f79d4b44-8a94-42c6-88c5-10cb8a5210f2\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j4dnc"
Sep 16 14:16:20.550605 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:16:20.550605    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6xpz\" (UniqueName: \"kubernetes.io/projected/f79d4b44-8a94-42c6-88c5-10cb8a5210f2-kube-api-access-m6xpz\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j4dnc\" (UID: \"f79d4b44-8a94-42c6-88c5-10cb8a5210f2\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j4dnc"
Sep 16 14:16:20.550846 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:16:20.550650    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f79d4b44-8a94-42c6-88c5-10cb8a5210f2-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j4dnc\" (UID: \"f79d4b44-8a94-42c6-88c5-10cb8a5210f2\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j4dnc"
Sep 16 14:16:20.550846 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:16:20.550668    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f79d4b44-8a94-42c6-88c5-10cb8a5210f2-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j4dnc\" (UID: \"f79d4b44-8a94-42c6-88c5-10cb8a5210f2\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j4dnc"
Sep 16 14:16:20.550846 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:16:20.550722    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f79d4b44-8a94-42c6-88c5-10cb8a5210f2-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j4dnc\" (UID: \"f79d4b44-8a94-42c6-88c5-10cb8a5210f2\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j4dnc"
Sep 16 14:16:20.550846 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:16:20.550772    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f79d4b44-8a94-42c6-88c5-10cb8a5210f2-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j4dnc\" (UID: \"f79d4b44-8a94-42c6-88c5-10cb8a5210f2\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j4dnc"
Sep 16 14:16:20.551051 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:16:20.551034    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f79d4b44-8a94-42c6-88c5-10cb8a5210f2-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j4dnc\" (UID: \"f79d4b44-8a94-42c6-88c5-10cb8a5210f2\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j4dnc"
Sep 16 14:16:20.551114 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:16:20.551080    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f79d4b44-8a94-42c6-88c5-10cb8a5210f2-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j4dnc\" (UID: \"f79d4b44-8a94-42c6-88c5-10cb8a5210f2\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j4dnc"
Sep 16 14:16:20.551255 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:16:20.551235    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f79d4b44-8a94-42c6-88c5-10cb8a5210f2-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j4dnc\" (UID: \"f79d4b44-8a94-42c6-88c5-10cb8a5210f2\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j4dnc"
Sep 16 14:16:20.553014 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:16:20.552993    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f79d4b44-8a94-42c6-88c5-10cb8a5210f2-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j4dnc\" (UID: \"f79d4b44-8a94-42c6-88c5-10cb8a5210f2\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j4dnc"
Sep 16 14:16:20.553247 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:16:20.553232    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f79d4b44-8a94-42c6-88c5-10cb8a5210f2-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j4dnc\" (UID: \"f79d4b44-8a94-42c6-88c5-10cb8a5210f2\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j4dnc"
Sep 16 14:16:20.559187 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:16:20.559163    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6xpz\" (UniqueName: \"kubernetes.io/projected/f79d4b44-8a94-42c6-88c5-10cb8a5210f2-kube-api-access-m6xpz\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j4dnc\" (UID: \"f79d4b44-8a94-42c6-88c5-10cb8a5210f2\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j4dnc"
Sep 16 14:16:20.687956 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:16:20.687932    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j4dnc"
Sep 16 14:16:20.963074 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:16:20.963048    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-j4dnc"]
Sep 16 14:16:20.964352 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:16:20.964325    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf79d4b44_8a94_42c6_88c5_10cb8a5210f2.slice/crio-f9da72235bfd377f370ad4ef55ea25a5032efc51885b34c551a5fbdefa7d4017 WatchSource:0}: Error finding container f9da72235bfd377f370ad4ef55ea25a5032efc51885b34c551a5fbdefa7d4017: Status 404 returned error can't find the container with id f9da72235bfd377f370ad4ef55ea25a5032efc51885b34c551a5fbdefa7d4017
Sep 16 14:16:21.469404 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:16:21.469329    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j4dnc" event={"ID":"f79d4b44-8a94-42c6-88c5-10cb8a5210f2","Type":"ContainerStarted","Data":"86bc200b6c93c5b91c633f3e1f314fb3c6327807bebfd337aae405e0e05ee84f"}
Sep 16 14:16:21.469404 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:16:21.469364    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j4dnc" event={"ID":"f79d4b44-8a94-42c6-88c5-10cb8a5210f2","Type":"ContainerStarted","Data":"f9da72235bfd377f370ad4ef55ea25a5032efc51885b34c551a5fbdefa7d4017"}
Sep 16 14:16:30.688634 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:16:30.688592    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j4dnc"
Sep 16 14:16:30.689003 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:16:30.688654    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j4dnc"
Sep 16 14:16:30.701192 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:16:30.701171    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j4dnc"
Sep 16 14:16:30.728172 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:16:30.728123    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j4dnc" podStartSLOduration=10.533624761 podStartE2EDuration="10.728107709s" totalImagesPullingTime="194.482948ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:16:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:16:21.492689234 +0000 UTC m=+714.507799105" watchObservedRunningTime="2026-09-16 14:16:30.728107709 +0000 UTC m=+723.743217570"
Sep 16 14:16:31.533055 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:16:31.533019    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j4dnc"
Sep 16 14:17:00.445648 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:00.445614    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7867f86b4c-s2hnz"]
Sep 16 14:17:00.448955 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:00.448938    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7867f86b4c-s2hnz"
Sep 16 14:17:00.451109 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:00.451087    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 16 14:17:00.451714 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:00.451692    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-8h45d\""
Sep 16 14:17:00.461781 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:00.461756    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7867f86b4c-s2hnz"]
Sep 16 14:17:00.602585 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:00.602554    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqt7t\" (UniqueName: \"kubernetes.io/projected/c3ae2ea0-bde3-4337-98cb-96555908f89a-kube-api-access-nqt7t\") pod \"authorino-7867f86b4c-s2hnz\" (UID: \"c3ae2ea0-bde3-4337-98cb-96555908f89a\") " pod="kuadrant-system/authorino-7867f86b4c-s2hnz"
Sep 16 14:17:00.602741 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:00.602624    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c3ae2ea0-bde3-4337-98cb-96555908f89a-tls-cert\") pod \"authorino-7867f86b4c-s2hnz\" (UID: \"c3ae2ea0-bde3-4337-98cb-96555908f89a\") " pod="kuadrant-system/authorino-7867f86b4c-s2hnz"
Sep 16 14:17:00.703361 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:00.703285    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqt7t\" (UniqueName: \"kubernetes.io/projected/c3ae2ea0-bde3-4337-98cb-96555908f89a-kube-api-access-nqt7t\") pod \"authorino-7867f86b4c-s2hnz\" (UID: \"c3ae2ea0-bde3-4337-98cb-96555908f89a\") " pod="kuadrant-system/authorino-7867f86b4c-s2hnz"
Sep 16 14:17:00.703490 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:00.703424    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c3ae2ea0-bde3-4337-98cb-96555908f89a-tls-cert\") pod \"authorino-7867f86b4c-s2hnz\" (UID: \"c3ae2ea0-bde3-4337-98cb-96555908f89a\") " pod="kuadrant-system/authorino-7867f86b4c-s2hnz"
Sep 16 14:17:00.705773 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:00.705746    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c3ae2ea0-bde3-4337-98cb-96555908f89a-tls-cert\") pod \"authorino-7867f86b4c-s2hnz\" (UID: \"c3ae2ea0-bde3-4337-98cb-96555908f89a\") " pod="kuadrant-system/authorino-7867f86b4c-s2hnz"
Sep 16 14:17:00.717603 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:00.717576    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqt7t\" (UniqueName: \"kubernetes.io/projected/c3ae2ea0-bde3-4337-98cb-96555908f89a-kube-api-access-nqt7t\") pod \"authorino-7867f86b4c-s2hnz\" (UID: \"c3ae2ea0-bde3-4337-98cb-96555908f89a\") " pod="kuadrant-system/authorino-7867f86b4c-s2hnz"
Sep 16 14:17:00.761423 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:00.761402    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7867f86b4c-s2hnz"
Sep 16 14:17:01.034359 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:01.034336    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7867f86b4c-s2hnz"]
Sep 16 14:17:01.036406 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:17:01.036381    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc3ae2ea0_bde3_4337_98cb_96555908f89a.slice/crio-240d80f26cc5339a0d382b5a5bf4968c0c3c1b4db7e62ac30e43d9d89c31f837 WatchSource:0}: Error finding container 240d80f26cc5339a0d382b5a5bf4968c0c3c1b4db7e62ac30e43d9d89c31f837: Status 404 returned error can't find the container with id 240d80f26cc5339a0d382b5a5bf4968c0c3c1b4db7e62ac30e43d9d89c31f837
Sep 16 14:17:01.673212 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:01.673177    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7867f86b4c-s2hnz" event={"ID":"c3ae2ea0-bde3-4337-98cb-96555908f89a","Type":"ContainerStarted","Data":"240d80f26cc5339a0d382b5a5bf4968c0c3c1b4db7e62ac30e43d9d89c31f837"}
Sep 16 14:17:02.680235 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:02.680195    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7867f86b4c-s2hnz" event={"ID":"c3ae2ea0-bde3-4337-98cb-96555908f89a","Type":"ContainerStarted","Data":"abef4183032553b6983c397641ed11977c30885d6b1bc3b300523cb39a84bd4b"}
Sep 16 14:17:02.695491 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:02.695433    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7867f86b4c-s2hnz" podStartSLOduration=1.956064925 podStartE2EDuration="2.695408881s" totalImagesPullingTime="739.343956ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:17:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:17:02.692167437 +0000 UTC m=+755.707277298" watchObservedRunningTime="2026-09-16 14:17:02.695408881 +0000 UTC m=+755.710518746"
Sep 16 14:17:03.176919 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:03.176374    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-c48qx"]
Sep 16 14:17:03.176919 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:03.176636    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-5b54b7fbb7-c48qx" podUID="b1ed4abf-7af6-4ef1-9b71-991fbc4ead65" containerName="payload-processing" containerID="cri-o://945c2b0bda0b7b378ae53d83b72d726691e558a05026711d6b3da68a8e553060" gracePeriod=30
Sep 16 14:17:03.288469 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:03.288437    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-658475d98d-6ncwp"]
Sep 16 14:17:03.288696 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:03.288663    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-658475d98d-6ncwp" podUID="793eec9a-8d86-4ebe-8308-836bbdd9e94c" containerName="manager" containerID="cri-o://090eeace4aed24bc428874de5239f6d22d0d63925f7a82439e3a35b3d3f9b0f8" gracePeriod=30
Sep 16 14:17:03.473456 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:03.473176    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-c48qx"
Sep 16 14:17:03.573270 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:03.573240    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-658475d98d-6ncwp"
Sep 16 14:17:03.630264 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:03.630221    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b1ed4abf-7af6-4ef1-9b71-991fbc4ead65-plugins-config-volume\") pod \"b1ed4abf-7af6-4ef1-9b71-991fbc4ead65\" (UID: \"b1ed4abf-7af6-4ef1-9b71-991fbc4ead65\") "
Sep 16 14:17:03.630418 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:03.630333    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v25v5\" (UniqueName: \"kubernetes.io/projected/b1ed4abf-7af6-4ef1-9b71-991fbc4ead65-kube-api-access-v25v5\") pod \"b1ed4abf-7af6-4ef1-9b71-991fbc4ead65\" (UID: \"b1ed4abf-7af6-4ef1-9b71-991fbc4ead65\") "
Sep 16 14:17:03.630598 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:03.630577    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1ed4abf-7af6-4ef1-9b71-991fbc4ead65-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b1ed4abf-7af6-4ef1-9b71-991fbc4ead65" (UID: "b1ed4abf-7af6-4ef1-9b71-991fbc4ead65"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:17:03.630663 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:03.630651    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b1ed4abf-7af6-4ef1-9b71-991fbc4ead65-plugins-config-volume\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:17:03.632429 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:03.632398    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1ed4abf-7af6-4ef1-9b71-991fbc4ead65-kube-api-access-v25v5" (OuterVolumeSpecName: "kube-api-access-v25v5") pod "b1ed4abf-7af6-4ef1-9b71-991fbc4ead65" (UID: "b1ed4abf-7af6-4ef1-9b71-991fbc4ead65"). InnerVolumeSpecName "kube-api-access-v25v5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:17:03.686987 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:03.686953    2580 generic.go:356] "Generic (PLEG): container finished" podID="b1ed4abf-7af6-4ef1-9b71-991fbc4ead65" containerID="945c2b0bda0b7b378ae53d83b72d726691e558a05026711d6b3da68a8e553060" exitCode=0
Sep 16 14:17:03.687376 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:03.686987    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-c48qx" event={"ID":"b1ed4abf-7af6-4ef1-9b71-991fbc4ead65","Type":"ContainerDied","Data":"945c2b0bda0b7b378ae53d83b72d726691e558a05026711d6b3da68a8e553060"}
Sep 16 14:17:03.687376 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:03.687036    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-c48qx"
Sep 16 14:17:03.687376 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:03.687058    2580 scope.go:117] "RemoveContainer" containerID="945c2b0bda0b7b378ae53d83b72d726691e558a05026711d6b3da68a8e553060"
Sep 16 14:17:03.687376 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:03.687047    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-c48qx" event={"ID":"b1ed4abf-7af6-4ef1-9b71-991fbc4ead65","Type":"ContainerDied","Data":"ba622aaeff0d53d384d669e416ede370823127eb06ca19d76bd5ddf15a0b3634"}
Sep 16 14:17:03.688738 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:03.688712    2580 generic.go:356] "Generic (PLEG): container finished" podID="793eec9a-8d86-4ebe-8308-836bbdd9e94c" containerID="090eeace4aed24bc428874de5239f6d22d0d63925f7a82439e3a35b3d3f9b0f8" exitCode=0
Sep 16 14:17:03.688847 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:03.688739    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-658475d98d-6ncwp" event={"ID":"793eec9a-8d86-4ebe-8308-836bbdd9e94c","Type":"ContainerDied","Data":"090eeace4aed24bc428874de5239f6d22d0d63925f7a82439e3a35b3d3f9b0f8"}
Sep 16 14:17:03.688847 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:03.688761    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-658475d98d-6ncwp"
Sep 16 14:17:03.688847 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:03.688789    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-658475d98d-6ncwp" event={"ID":"793eec9a-8d86-4ebe-8308-836bbdd9e94c","Type":"ContainerDied","Data":"5329971e23942aef5220a6e0860cb584f621c0dc0b269289eac282c15bbfcd9a"}
Sep 16 14:17:03.695985 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:03.695968    2580 scope.go:117] "RemoveContainer" containerID="945c2b0bda0b7b378ae53d83b72d726691e558a05026711d6b3da68a8e553060"
Sep 16 14:17:03.696306 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:17:03.696285    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"945c2b0bda0b7b378ae53d83b72d726691e558a05026711d6b3da68a8e553060\": container with ID starting with 945c2b0bda0b7b378ae53d83b72d726691e558a05026711d6b3da68a8e553060 not found: ID does not exist" containerID="945c2b0bda0b7b378ae53d83b72d726691e558a05026711d6b3da68a8e553060"
Sep 16 14:17:03.696393 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:03.696311    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"945c2b0bda0b7b378ae53d83b72d726691e558a05026711d6b3da68a8e553060"} err="failed to get container status \"945c2b0bda0b7b378ae53d83b72d726691e558a05026711d6b3da68a8e553060\": rpc error: code = NotFound desc = could not find container \"945c2b0bda0b7b378ae53d83b72d726691e558a05026711d6b3da68a8e553060\": container with ID starting with 945c2b0bda0b7b378ae53d83b72d726691e558a05026711d6b3da68a8e553060 not found: ID does not exist"
Sep 16 14:17:03.696393 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:03.696327    2580 scope.go:117] "RemoveContainer" containerID="090eeace4aed24bc428874de5239f6d22d0d63925f7a82439e3a35b3d3f9b0f8"
Sep 16 14:17:03.705038 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:03.705020    2580 scope.go:117] "RemoveContainer" containerID="090eeace4aed24bc428874de5239f6d22d0d63925f7a82439e3a35b3d3f9b0f8"
Sep 16 14:17:03.705297 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:17:03.705280    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"090eeace4aed24bc428874de5239f6d22d0d63925f7a82439e3a35b3d3f9b0f8\": container with ID starting with 090eeace4aed24bc428874de5239f6d22d0d63925f7a82439e3a35b3d3f9b0f8 not found: ID does not exist" containerID="090eeace4aed24bc428874de5239f6d22d0d63925f7a82439e3a35b3d3f9b0f8"
Sep 16 14:17:03.705356 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:03.705302    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"090eeace4aed24bc428874de5239f6d22d0d63925f7a82439e3a35b3d3f9b0f8"} err="failed to get container status \"090eeace4aed24bc428874de5239f6d22d0d63925f7a82439e3a35b3d3f9b0f8\": rpc error: code = NotFound desc = could not find container \"090eeace4aed24bc428874de5239f6d22d0d63925f7a82439e3a35b3d3f9b0f8\": container with ID starting with 090eeace4aed24bc428874de5239f6d22d0d63925f7a82439e3a35b3d3f9b0f8 not found: ID does not exist"
Sep 16 14:17:03.712467 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:03.712443    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-c48qx"]
Sep 16 14:17:03.714572 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:03.714549    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-c48qx"]
Sep 16 14:17:03.731192 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:03.731171    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xtd6m\" (UniqueName: \"kubernetes.io/projected/793eec9a-8d86-4ebe-8308-836bbdd9e94c-kube-api-access-xtd6m\") pod \"793eec9a-8d86-4ebe-8308-836bbdd9e94c\" (UID: \"793eec9a-8d86-4ebe-8308-836bbdd9e94c\") "
Sep 16 14:17:03.731290 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:03.731234    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/793eec9a-8d86-4ebe-8308-836bbdd9e94c-webhook-cert\") pod \"793eec9a-8d86-4ebe-8308-836bbdd9e94c\" (UID: \"793eec9a-8d86-4ebe-8308-836bbdd9e94c\") "
Sep 16 14:17:03.731290 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:03.731279    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/793eec9a-8d86-4ebe-8308-836bbdd9e94c-metrics-certs\") pod \"793eec9a-8d86-4ebe-8308-836bbdd9e94c\" (UID: \"793eec9a-8d86-4ebe-8308-836bbdd9e94c\") "
Sep 16 14:17:03.731613 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:03.731576    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v25v5\" (UniqueName: \"kubernetes.io/projected/b1ed4abf-7af6-4ef1-9b71-991fbc4ead65-kube-api-access-v25v5\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:17:03.733290 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:03.733266    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/793eec9a-8d86-4ebe-8308-836bbdd9e94c-kube-api-access-xtd6m" (OuterVolumeSpecName: "kube-api-access-xtd6m") pod "793eec9a-8d86-4ebe-8308-836bbdd9e94c" (UID: "793eec9a-8d86-4ebe-8308-836bbdd9e94c"). InnerVolumeSpecName "kube-api-access-xtd6m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:17:03.733691 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:03.733675    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/793eec9a-8d86-4ebe-8308-836bbdd9e94c-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "793eec9a-8d86-4ebe-8308-836bbdd9e94c" (UID: "793eec9a-8d86-4ebe-8308-836bbdd9e94c"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:17:03.733756 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:03.733698    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/793eec9a-8d86-4ebe-8308-836bbdd9e94c-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "793eec9a-8d86-4ebe-8308-836bbdd9e94c" (UID: "793eec9a-8d86-4ebe-8308-836bbdd9e94c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:17:03.832907 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:03.832876    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xtd6m\" (UniqueName: \"kubernetes.io/projected/793eec9a-8d86-4ebe-8308-836bbdd9e94c-kube-api-access-xtd6m\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:17:03.832907 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:03.832904    2580 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/793eec9a-8d86-4ebe-8308-836bbdd9e94c-webhook-cert\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:17:03.832907 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:03.832913    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/793eec9a-8d86-4ebe-8308-836bbdd9e94c-metrics-certs\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:17:04.023534 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:04.022540    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-658475d98d-6ncwp"]
Sep 16 14:17:04.027036 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:04.027006    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-658475d98d-6ncwp"]
Sep 16 14:17:04.406143 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:04.406071    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-658475d98d-6stb8"]
Sep 16 14:17:04.406435 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:04.406424    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="793eec9a-8d86-4ebe-8308-836bbdd9e94c" containerName="manager"
Sep 16 14:17:04.406478 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:04.406437    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="793eec9a-8d86-4ebe-8308-836bbdd9e94c" containerName="manager"
Sep 16 14:17:04.406478 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:04.406447    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1ed4abf-7af6-4ef1-9b71-991fbc4ead65" containerName="payload-processing"
Sep 16 14:17:04.406478 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:04.406453    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1ed4abf-7af6-4ef1-9b71-991fbc4ead65" containerName="payload-processing"
Sep 16 14:17:04.406589 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:04.406527    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1ed4abf-7af6-4ef1-9b71-991fbc4ead65" containerName="payload-processing"
Sep 16 14:17:04.406589 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:04.406538    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="793eec9a-8d86-4ebe-8308-836bbdd9e94c" containerName="manager"
Sep 16 14:17:04.410892 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:04.410875    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-658475d98d-6stb8"
Sep 16 14:17:04.413148 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:04.413121    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 16 14:17:04.413244 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:04.413121    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 16 14:17:04.413244 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:04.413124    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-25lqs\""
Sep 16 14:17:04.413483 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:04.413465    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 16 14:17:04.420287 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:04.420265    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-658475d98d-6stb8"]
Sep 16 14:17:04.539756 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:04.539728    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wp2kw\" (UniqueName: \"kubernetes.io/projected/1f34b361-5fad-40f6-8f95-14b71f83a416-kube-api-access-wp2kw\") pod \"maas-controller-658475d98d-6stb8\" (UID: \"1f34b361-5fad-40f6-8f95-14b71f83a416\") " pod="opendatahub/maas-controller-658475d98d-6stb8"
Sep 16 14:17:04.539944 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:04.539796    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1f34b361-5fad-40f6-8f95-14b71f83a416-webhook-cert\") pod \"maas-controller-658475d98d-6stb8\" (UID: \"1f34b361-5fad-40f6-8f95-14b71f83a416\") " pod="opendatahub/maas-controller-658475d98d-6stb8"
Sep 16 14:17:04.539944 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:04.539894    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1f34b361-5fad-40f6-8f95-14b71f83a416-metrics-certs\") pod \"maas-controller-658475d98d-6stb8\" (UID: \"1f34b361-5fad-40f6-8f95-14b71f83a416\") " pod="opendatahub/maas-controller-658475d98d-6stb8"
Sep 16 14:17:04.640655 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:04.640624    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1f34b361-5fad-40f6-8f95-14b71f83a416-webhook-cert\") pod \"maas-controller-658475d98d-6stb8\" (UID: \"1f34b361-5fad-40f6-8f95-14b71f83a416\") " pod="opendatahub/maas-controller-658475d98d-6stb8"
Sep 16 14:17:04.640792 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:04.640673    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1f34b361-5fad-40f6-8f95-14b71f83a416-metrics-certs\") pod \"maas-controller-658475d98d-6stb8\" (UID: \"1f34b361-5fad-40f6-8f95-14b71f83a416\") " pod="opendatahub/maas-controller-658475d98d-6stb8"
Sep 16 14:17:04.640792 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:04.640712    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wp2kw\" (UniqueName: \"kubernetes.io/projected/1f34b361-5fad-40f6-8f95-14b71f83a416-kube-api-access-wp2kw\") pod \"maas-controller-658475d98d-6stb8\" (UID: \"1f34b361-5fad-40f6-8f95-14b71f83a416\") " pod="opendatahub/maas-controller-658475d98d-6stb8"
Sep 16 14:17:04.643027 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:04.642999    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1f34b361-5fad-40f6-8f95-14b71f83a416-metrics-certs\") pod \"maas-controller-658475d98d-6stb8\" (UID: \"1f34b361-5fad-40f6-8f95-14b71f83a416\") " pod="opendatahub/maas-controller-658475d98d-6stb8"
Sep 16 14:17:04.643114 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:04.643090    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1f34b361-5fad-40f6-8f95-14b71f83a416-webhook-cert\") pod \"maas-controller-658475d98d-6stb8\" (UID: \"1f34b361-5fad-40f6-8f95-14b71f83a416\") " pod="opendatahub/maas-controller-658475d98d-6stb8"
Sep 16 14:17:04.648660 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:04.648640    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wp2kw\" (UniqueName: \"kubernetes.io/projected/1f34b361-5fad-40f6-8f95-14b71f83a416-kube-api-access-wp2kw\") pod \"maas-controller-658475d98d-6stb8\" (UID: \"1f34b361-5fad-40f6-8f95-14b71f83a416\") " pod="opendatahub/maas-controller-658475d98d-6stb8"
Sep 16 14:17:04.721709 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:04.721685    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-658475d98d-6stb8"
Sep 16 14:17:04.994152 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:04.994130    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-658475d98d-6stb8"]
Sep 16 14:17:04.995934 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:17:04.995907    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f34b361_5fad_40f6_8f95_14b71f83a416.slice/crio-9d1f2a6a2a729580354b9a8989fdbcc36916edb93b45c6c88c6689cdb9b68d41 WatchSource:0}: Error finding container 9d1f2a6a2a729580354b9a8989fdbcc36916edb93b45c6c88c6689cdb9b68d41: Status 404 returned error can't find the container with id 9d1f2a6a2a729580354b9a8989fdbcc36916edb93b45c6c88c6689cdb9b68d41
Sep 16 14:17:05.430125 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:05.430086    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="793eec9a-8d86-4ebe-8308-836bbdd9e94c" path="/var/lib/kubelet/pods/793eec9a-8d86-4ebe-8308-836bbdd9e94c/volumes"
Sep 16 14:17:05.430537 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:05.430519    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1ed4abf-7af6-4ef1-9b71-991fbc4ead65" path="/var/lib/kubelet/pods/b1ed4abf-7af6-4ef1-9b71-991fbc4ead65/volumes"
Sep 16 14:17:05.701976 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:05.701878    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-658475d98d-6stb8" event={"ID":"1f34b361-5fad-40f6-8f95-14b71f83a416","Type":"ContainerStarted","Data":"1b870f4f8496e50685648b98d709898e0694635a67a69bd4f551dbbf683b5603"}
Sep 16 14:17:05.701976 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:05.701931    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-658475d98d-6stb8" event={"ID":"1f34b361-5fad-40f6-8f95-14b71f83a416","Type":"ContainerStarted","Data":"9d1f2a6a2a729580354b9a8989fdbcc36916edb93b45c6c88c6689cdb9b68d41"}
Sep 16 14:17:05.702184 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:05.701976    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-658475d98d-6stb8"
Sep 16 14:17:05.735809 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:05.735763    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-658475d98d-6stb8" podStartSLOduration=1.289671121 podStartE2EDuration="1.735739291s" totalImagesPullingTime="446.06817ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:17:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:17:05.734961233 +0000 UTC m=+758.750071093" watchObservedRunningTime="2026-09-16 14:17:05.735739291 +0000 UTC m=+758.750849150"
Sep 16 14:17:16.709419 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:17:16.709388    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-658475d98d-6stb8"
Sep 16 14:18:05.283633 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:05.283561    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z"]
Sep 16 14:18:05.287155 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:05.287138    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z"
Sep 16 14:18:05.299588 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:05.299569    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-08475e-openshift-default-dockercfg-xg2kk\""
Sep 16 14:18:05.308975 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:05.308952    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z"]
Sep 16 14:18:05.403837 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:05.403807    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk"]
Sep 16 14:18:05.407259 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:05.407237    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk"
Sep 16 14:18:05.410858 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:05.410832    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-2ad565-openshift-default-dockercfg-2ffpn\""
Sep 16 14:18:05.433203 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:05.433178    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk"]
Sep 16 14:18:05.445321 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:05.445292    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9cb6f359-dd14-43e9-a795-d56065b12448-istio-data\") pod \"e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z\" (UID: \"9cb6f359-dd14-43e9-a795-d56065b12448\") " pod="openshift-ingress/e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z"
Sep 16 14:18:05.445431 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:05.445328    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cb6f359-dd14-43e9-a795-d56065b12448-credential-socket\") pod \"e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z\" (UID: \"9cb6f359-dd14-43e9-a795-d56065b12448\") " pod="openshift-ingress/e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z"
Sep 16 14:18:05.445431 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:05.445365    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2k96z\" (UniqueName: \"kubernetes.io/projected/9cb6f359-dd14-43e9-a795-d56065b12448-kube-api-access-2k96z\") pod \"e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z\" (UID: \"9cb6f359-dd14-43e9-a795-d56065b12448\") " pod="openshift-ingress/e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z"
Sep 16 14:18:05.445431 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:05.445415    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9cb6f359-dd14-43e9-a795-d56065b12448-istiod-ca-cert\") pod \"e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z\" (UID: \"9cb6f359-dd14-43e9-a795-d56065b12448\") " pod="openshift-ingress/e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z"
Sep 16 14:18:05.445644 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:05.445437    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9cb6f359-dd14-43e9-a795-d56065b12448-istio-token\") pod \"e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z\" (UID: \"9cb6f359-dd14-43e9-a795-d56065b12448\") " pod="openshift-ingress/e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z"
Sep 16 14:18:05.445644 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:05.445467    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cb6f359-dd14-43e9-a795-d56065b12448-workload-socket\") pod \"e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z\" (UID: \"9cb6f359-dd14-43e9-a795-d56065b12448\") " pod="openshift-ingress/e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z"
Sep 16 14:18:05.445644 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:05.445492    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9cb6f359-dd14-43e9-a795-d56065b12448-workload-certs\") pod \"e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z\" (UID: \"9cb6f359-dd14-43e9-a795-d56065b12448\") " pod="openshift-ingress/e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z"
Sep 16 14:18:05.445644 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:05.445579    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9cb6f359-dd14-43e9-a795-d56065b12448-istio-envoy\") pod \"e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z\" (UID: \"9cb6f359-dd14-43e9-a795-d56065b12448\") " pod="openshift-ingress/e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z"
Sep 16 14:18:05.445644 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:05.445610    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9cb6f359-dd14-43e9-a795-d56065b12448-istio-podinfo\") pod \"e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z\" (UID: \"9cb6f359-dd14-43e9-a795-d56065b12448\") " pod="openshift-ingress/e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z"
Sep 16 14:18:05.546358 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:05.546267    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2k96z\" (UniqueName: \"kubernetes.io/projected/9cb6f359-dd14-43e9-a795-d56065b12448-kube-api-access-2k96z\") pod \"e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z\" (UID: \"9cb6f359-dd14-43e9-a795-d56065b12448\") " pod="openshift-ingress/e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z"
Sep 16 14:18:05.546358 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:05.546307    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9db21092-910f-479a-ae58-4463feec4dfd-istio-envoy\") pod \"e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk\" (UID: \"9db21092-910f-479a-ae58-4463feec4dfd\") " pod="openshift-ingress/e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk"
Sep 16 14:18:05.546358 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:05.546352    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9cb6f359-dd14-43e9-a795-d56065b12448-istiod-ca-cert\") pod \"e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z\" (UID: \"9cb6f359-dd14-43e9-a795-d56065b12448\") " pod="openshift-ingress/e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z"
Sep 16 14:18:05.546643 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:05.546373    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9cb6f359-dd14-43e9-a795-d56065b12448-istio-token\") pod \"e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z\" (UID: \"9cb6f359-dd14-43e9-a795-d56065b12448\") " pod="openshift-ingress/e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z"
Sep 16 14:18:05.546643 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:05.546390    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9db21092-910f-479a-ae58-4463feec4dfd-istiod-ca-cert\") pod \"e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk\" (UID: \"9db21092-910f-479a-ae58-4463feec4dfd\") " pod="openshift-ingress/e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk"
Sep 16 14:18:05.546643 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:05.546492    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9db21092-910f-479a-ae58-4463feec4dfd-istio-token\") pod \"e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk\" (UID: \"9db21092-910f-479a-ae58-4463feec4dfd\") " pod="openshift-ingress/e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk"
Sep 16 14:18:05.546643 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:05.546611    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cb6f359-dd14-43e9-a795-d56065b12448-workload-socket\") pod \"e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z\" (UID: \"9cb6f359-dd14-43e9-a795-d56065b12448\") " pod="openshift-ingress/e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z"
Sep 16 14:18:05.546643 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:05.546637    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9db21092-910f-479a-ae58-4463feec4dfd-credential-socket\") pod \"e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk\" (UID: \"9db21092-910f-479a-ae58-4463feec4dfd\") " pod="openshift-ingress/e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk"
Sep 16 14:18:05.546903 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:05.546669    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9cb6f359-dd14-43e9-a795-d56065b12448-workload-certs\") pod \"e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z\" (UID: \"9cb6f359-dd14-43e9-a795-d56065b12448\") " pod="openshift-ingress/e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z"
Sep 16 14:18:05.546903 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:05.546705    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9db21092-910f-479a-ae58-4463feec4dfd-workload-socket\") pod \"e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk\" (UID: \"9db21092-910f-479a-ae58-4463feec4dfd\") " pod="openshift-ingress/e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk"
Sep 16 14:18:05.546903 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:05.546733    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9db21092-910f-479a-ae58-4463feec4dfd-istio-data\") pod \"e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk\" (UID: \"9db21092-910f-479a-ae58-4463feec4dfd\") " pod="openshift-ingress/e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk"
Sep 16 14:18:05.546903 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:05.546749    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9db21092-910f-479a-ae58-4463feec4dfd-istio-podinfo\") pod \"e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk\" (UID: \"9db21092-910f-479a-ae58-4463feec4dfd\") " pod="openshift-ingress/e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk"
Sep 16 14:18:05.546903 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:05.546772    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9w4fj\" (UniqueName: \"kubernetes.io/projected/9db21092-910f-479a-ae58-4463feec4dfd-kube-api-access-9w4fj\") pod \"e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk\" (UID: \"9db21092-910f-479a-ae58-4463feec4dfd\") " pod="openshift-ingress/e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk"
Sep 16 14:18:05.546903 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:05.546827    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9cb6f359-dd14-43e9-a795-d56065b12448-istio-envoy\") pod \"e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z\" (UID: \"9cb6f359-dd14-43e9-a795-d56065b12448\") " pod="openshift-ingress/e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z"
Sep 16 14:18:05.546903 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:05.546849    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9cb6f359-dd14-43e9-a795-d56065b12448-istio-podinfo\") pod \"e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z\" (UID: \"9cb6f359-dd14-43e9-a795-d56065b12448\") " pod="openshift-ingress/e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z"
Sep 16 14:18:05.546903 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:05.546887    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9cb6f359-dd14-43e9-a795-d56065b12448-istio-data\") pod \"e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z\" (UID: \"9cb6f359-dd14-43e9-a795-d56065b12448\") " pod="openshift-ingress/e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z"
Sep 16 14:18:05.547334 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:05.546927    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cb6f359-dd14-43e9-a795-d56065b12448-credential-socket\") pod \"e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z\" (UID: \"9cb6f359-dd14-43e9-a795-d56065b12448\") " pod="openshift-ingress/e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z"
Sep 16 14:18:05.547334 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:05.546961    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9db21092-910f-479a-ae58-4463feec4dfd-workload-certs\") pod \"e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk\" (UID: \"9db21092-910f-479a-ae58-4463feec4dfd\") " pod="openshift-ingress/e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk"
Sep 16 14:18:05.547334 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:05.547023    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cb6f359-dd14-43e9-a795-d56065b12448-workload-socket\") pod \"e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z\" (UID: \"9cb6f359-dd14-43e9-a795-d56065b12448\") " pod="openshift-ingress/e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z"
Sep 16 14:18:05.547334 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:05.547042    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9cb6f359-dd14-43e9-a795-d56065b12448-workload-certs\") pod \"e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z\" (UID: \"9cb6f359-dd14-43e9-a795-d56065b12448\") " pod="openshift-ingress/e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z"
Sep 16 14:18:05.547334 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:05.547218    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9cb6f359-dd14-43e9-a795-d56065b12448-istiod-ca-cert\") pod \"e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z\" (UID: \"9cb6f359-dd14-43e9-a795-d56065b12448\") " pod="openshift-ingress/e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z"
Sep 16 14:18:05.547334 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:05.547295    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9cb6f359-dd14-43e9-a795-d56065b12448-istio-data\") pod \"e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z\" (UID: \"9cb6f359-dd14-43e9-a795-d56065b12448\") " pod="openshift-ingress/e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z"
Sep 16 14:18:05.547334 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:05.547307    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cb6f359-dd14-43e9-a795-d56065b12448-credential-socket\") pod \"e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z\" (UID: \"9cb6f359-dd14-43e9-a795-d56065b12448\") " pod="openshift-ingress/e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z"
Sep 16 14:18:05.549040 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:05.549017    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9cb6f359-dd14-43e9-a795-d56065b12448-istio-envoy\") pod \"e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z\" (UID: \"9cb6f359-dd14-43e9-a795-d56065b12448\") " pod="openshift-ingress/e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z"
Sep 16 14:18:05.549245 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:05.549229    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9cb6f359-dd14-43e9-a795-d56065b12448-istio-podinfo\") pod \"e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z\" (UID: \"9cb6f359-dd14-43e9-a795-d56065b12448\") " pod="openshift-ingress/e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z"
Sep 16 14:18:05.570980 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:05.570954    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2k96z\" (UniqueName: \"kubernetes.io/projected/9cb6f359-dd14-43e9-a795-d56065b12448-kube-api-access-2k96z\") pod \"e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z\" (UID: \"9cb6f359-dd14-43e9-a795-d56065b12448\") " pod="openshift-ingress/e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z"
Sep 16 14:18:05.573018 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:05.572992    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9cb6f359-dd14-43e9-a795-d56065b12448-istio-token\") pod \"e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z\" (UID: \"9cb6f359-dd14-43e9-a795-d56065b12448\") " pod="openshift-ingress/e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z"
Sep 16 14:18:05.597933 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:05.597909    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z"
Sep 16 14:18:05.648312 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:05.648276    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9db21092-910f-479a-ae58-4463feec4dfd-istio-token\") pod \"e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk\" (UID: \"9db21092-910f-479a-ae58-4463feec4dfd\") " pod="openshift-ingress/e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk"
Sep 16 14:18:05.648312 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:05.648320    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9db21092-910f-479a-ae58-4463feec4dfd-credential-socket\") pod \"e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk\" (UID: \"9db21092-910f-479a-ae58-4463feec4dfd\") " pod="openshift-ingress/e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk"
Sep 16 14:18:05.648564 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:05.648351    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9db21092-910f-479a-ae58-4463feec4dfd-workload-socket\") pod \"e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk\" (UID: \"9db21092-910f-479a-ae58-4463feec4dfd\") " pod="openshift-ingress/e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk"
Sep 16 14:18:05.648564 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:05.648369    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9db21092-910f-479a-ae58-4463feec4dfd-istio-data\") pod \"e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk\" (UID: \"9db21092-910f-479a-ae58-4463feec4dfd\") " pod="openshift-ingress/e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk"
Sep 16 14:18:05.648564 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:05.648387    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9db21092-910f-479a-ae58-4463feec4dfd-istio-podinfo\") pod \"e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk\" (UID: \"9db21092-910f-479a-ae58-4463feec4dfd\") " pod="openshift-ingress/e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk"
Sep 16 14:18:05.648564 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:05.648403    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9w4fj\" (UniqueName: \"kubernetes.io/projected/9db21092-910f-479a-ae58-4463feec4dfd-kube-api-access-9w4fj\") pod \"e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk\" (UID: \"9db21092-910f-479a-ae58-4463feec4dfd\") " pod="openshift-ingress/e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk"
Sep 16 14:18:05.648564 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:05.648445    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9db21092-910f-479a-ae58-4463feec4dfd-workload-certs\") pod \"e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk\" (UID: \"9db21092-910f-479a-ae58-4463feec4dfd\") " pod="openshift-ingress/e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk"
Sep 16 14:18:05.648564 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:05.648472    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9db21092-910f-479a-ae58-4463feec4dfd-istio-envoy\") pod \"e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk\" (UID: \"9db21092-910f-479a-ae58-4463feec4dfd\") " pod="openshift-ingress/e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk"
Sep 16 14:18:05.648564 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:05.648543    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9db21092-910f-479a-ae58-4463feec4dfd-istiod-ca-cert\") pod \"e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk\" (UID: \"9db21092-910f-479a-ae58-4463feec4dfd\") " pod="openshift-ingress/e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk"
Sep 16 14:18:05.648920 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:05.648775    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9db21092-910f-479a-ae58-4463feec4dfd-credential-socket\") pod \"e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk\" (UID: \"9db21092-910f-479a-ae58-4463feec4dfd\") " pod="openshift-ingress/e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk"
Sep 16 14:18:05.649055 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:05.649033    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9db21092-910f-479a-ae58-4463feec4dfd-workload-socket\") pod \"e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk\" (UID: \"9db21092-910f-479a-ae58-4463feec4dfd\") " pod="openshift-ingress/e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk"
Sep 16 14:18:05.649163 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:05.649073    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9db21092-910f-479a-ae58-4463feec4dfd-istiod-ca-cert\") pod \"e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk\" (UID: \"9db21092-910f-479a-ae58-4463feec4dfd\") " pod="openshift-ingress/e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk"
Sep 16 14:18:05.649268 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:05.649245    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9db21092-910f-479a-ae58-4463feec4dfd-workload-certs\") pod \"e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk\" (UID: \"9db21092-910f-479a-ae58-4463feec4dfd\") " pod="openshift-ingress/e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk"
Sep 16 14:18:05.649593 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:05.649553    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9db21092-910f-479a-ae58-4463feec4dfd-istio-data\") pod \"e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk\" (UID: \"9db21092-910f-479a-ae58-4463feec4dfd\") " pod="openshift-ingress/e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk"
Sep 16 14:18:05.651261 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:05.651239    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9db21092-910f-479a-ae58-4463feec4dfd-istio-envoy\") pod \"e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk\" (UID: \"9db21092-910f-479a-ae58-4463feec4dfd\") " pod="openshift-ingress/e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk"
Sep 16 14:18:05.651385 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:05.651368    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9db21092-910f-479a-ae58-4463feec4dfd-istio-podinfo\") pod \"e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk\" (UID: \"9db21092-910f-479a-ae58-4463feec4dfd\") " pod="openshift-ingress/e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk"
Sep 16 14:18:05.677486 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:05.677427    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9db21092-910f-479a-ae58-4463feec4dfd-istio-token\") pod \"e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk\" (UID: \"9db21092-910f-479a-ae58-4463feec4dfd\") " pod="openshift-ingress/e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk"
Sep 16 14:18:05.677939 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:05.677918    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9w4fj\" (UniqueName: \"kubernetes.io/projected/9db21092-910f-479a-ae58-4463feec4dfd-kube-api-access-9w4fj\") pod \"e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk\" (UID: \"9db21092-910f-479a-ae58-4463feec4dfd\") " pod="openshift-ingress/e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk"
Sep 16 14:18:05.720078 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:05.720047    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk"
Sep 16 14:18:05.967025 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:05.966896    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z"]
Sep 16 14:18:05.970030 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:18:05.970001    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cb6f359_dd14_43e9_a795_d56065b12448.slice/crio-9746baea3df4d6044e6885d26800a5628d8f68b8c9a686e5b85380370bd91081 WatchSource:0}: Error finding container 9746baea3df4d6044e6885d26800a5628d8f68b8c9a686e5b85380370bd91081: Status 404 returned error can't find the container with id 9746baea3df4d6044e6885d26800a5628d8f68b8c9a686e5b85380370bd91081
Sep 16 14:18:05.972141 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:05.972111    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 14:18:05.972218 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:05.972177    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 14:18:05.972218 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:05.972205    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 14:18:06.007568 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:06.007532    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z" event={"ID":"9cb6f359-dd14-43e9-a795-d56065b12448","Type":"ContainerStarted","Data":"9746baea3df4d6044e6885d26800a5628d8f68b8c9a686e5b85380370bd91081"}
Sep 16 14:18:06.018339 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:06.018319    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk"]
Sep 16 14:18:06.025539 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:18:06.025489    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9db21092_910f_479a_ae58_4463feec4dfd.slice/crio-b98cace62f61b38e3a4cb6c2fcb920116b6d1314573fc2b53e11891d27d5ce22 WatchSource:0}: Error finding container b98cace62f61b38e3a4cb6c2fcb920116b6d1314573fc2b53e11891d27d5ce22: Status 404 returned error can't find the container with id b98cace62f61b38e3a4cb6c2fcb920116b6d1314573fc2b53e11891d27d5ce22
Sep 16 14:18:06.027738 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:06.027708    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 14:18:06.027813 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:06.027771    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 14:18:06.027813 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:06.027804    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 14:18:07.015882 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:07.015842    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z" event={"ID":"9cb6f359-dd14-43e9-a795-d56065b12448","Type":"ContainerStarted","Data":"823cf112f89ed5507c820a46b4f8fcc167b7eb64c3f4bdb132495d242db621e8"}
Sep 16 14:18:07.017390 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:07.017369    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk" event={"ID":"9db21092-910f-479a-ae58-4463feec4dfd","Type":"ContainerStarted","Data":"6f0235a87d104e93ebafb57294019ee0757189a0965fc15bf09486adf67cab45"}
Sep 16 14:18:07.017534 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:07.017405    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk" event={"ID":"9db21092-910f-479a-ae58-4463feec4dfd","Type":"ContainerStarted","Data":"b98cace62f61b38e3a4cb6c2fcb920116b6d1314573fc2b53e11891d27d5ce22"}
Sep 16 14:18:07.049451 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:07.049403    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z" podStartSLOduration=2.049388517 podStartE2EDuration="2.049388517s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:18:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:18:07.04812992 +0000 UTC m=+820.063239781" watchObservedRunningTime="2026-09-16 14:18:07.049388517 +0000 UTC m=+820.064498377"
Sep 16 14:18:07.068249 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:07.068198    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk" podStartSLOduration=2.06817987 podStartE2EDuration="2.06817987s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:18:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:18:07.0673743 +0000 UTC m=+820.082484176" watchObservedRunningTime="2026-09-16 14:18:07.06817987 +0000 UTC m=+820.083289736"
Sep 16 14:18:07.598665 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:07.598632    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z"
Sep 16 14:18:07.604085 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:07.604061    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z"
Sep 16 14:18:07.721242 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:07.721216    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk"
Sep 16 14:18:07.726444 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:07.726421    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk"
Sep 16 14:18:08.024528 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:08.024484    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z"
Sep 16 14:18:08.024967 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:08.024885    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk"
Sep 16 14:18:08.025366 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:08.025342    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk"
Sep 16 14:18:08.025526 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:08.025506    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z"
Sep 16 14:18:10.424144 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:10.424066    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-2ad565-f46f54678-xc2tg"]
Sep 16 14:18:10.427639 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:10.427612    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-2ad565-f46f54678-xc2tg"
Sep 16 14:18:10.439752 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:10.439732    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w3-2ad565-serving-cert\""
Sep 16 14:18:10.466896 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:10.466869    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-2ad565-f46f54678-xc2tg"]
Sep 16 14:18:10.491899 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:10.491861    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/344de68b-6478-44c6-a0c4-c146b912f4a3-maas-api-tls\") pod \"maas-api-e2e-worker-w3-2ad565-f46f54678-xc2tg\" (UID: \"344de68b-6478-44c6-a0c4-c146b912f4a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-2ad565-f46f54678-xc2tg"
Sep 16 14:18:10.492021 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:10.491913    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/344de68b-6478-44c6-a0c4-c146b912f4a3-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-2ad565-f46f54678-xc2tg\" (UID: \"344de68b-6478-44c6-a0c4-c146b912f4a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-2ad565-f46f54678-xc2tg"
Sep 16 14:18:10.492021 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:10.491954    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtcx8\" (UniqueName: \"kubernetes.io/projected/344de68b-6478-44c6-a0c4-c146b912f4a3-kube-api-access-jtcx8\") pod \"maas-api-e2e-worker-w3-2ad565-f46f54678-xc2tg\" (UID: \"344de68b-6478-44c6-a0c4-c146b912f4a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-2ad565-f46f54678-xc2tg"
Sep 16 14:18:10.492110 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:10.492082    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/344de68b-6478-44c6-a0c4-c146b912f4a3-metrics-certs\") pod \"maas-api-e2e-worker-w3-2ad565-f46f54678-xc2tg\" (UID: \"344de68b-6478-44c6-a0c4-c146b912f4a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-2ad565-f46f54678-xc2tg"
Sep 16 14:18:10.593198 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:10.593154    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/344de68b-6478-44c6-a0c4-c146b912f4a3-maas-api-tls\") pod \"maas-api-e2e-worker-w3-2ad565-f46f54678-xc2tg\" (UID: \"344de68b-6478-44c6-a0c4-c146b912f4a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-2ad565-f46f54678-xc2tg"
Sep 16 14:18:10.593396 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:10.593213    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/344de68b-6478-44c6-a0c4-c146b912f4a3-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-2ad565-f46f54678-xc2tg\" (UID: \"344de68b-6478-44c6-a0c4-c146b912f4a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-2ad565-f46f54678-xc2tg"
Sep 16 14:18:10.593396 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:10.593240    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtcx8\" (UniqueName: \"kubernetes.io/projected/344de68b-6478-44c6-a0c4-c146b912f4a3-kube-api-access-jtcx8\") pod \"maas-api-e2e-worker-w3-2ad565-f46f54678-xc2tg\" (UID: \"344de68b-6478-44c6-a0c4-c146b912f4a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-2ad565-f46f54678-xc2tg"
Sep 16 14:18:10.593396 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:10.593258    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/344de68b-6478-44c6-a0c4-c146b912f4a3-metrics-certs\") pod \"maas-api-e2e-worker-w3-2ad565-f46f54678-xc2tg\" (UID: \"344de68b-6478-44c6-a0c4-c146b912f4a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-2ad565-f46f54678-xc2tg"
Sep 16 14:18:10.593396 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:18:10.593311    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w3-2ad565-serving-cert: secret "maas-api-e2e-worker-w3-2ad565-serving-cert" not found
Sep 16 14:18:10.593681 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:18:10.593400    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/344de68b-6478-44c6-a0c4-c146b912f4a3-maas-api-tls podName:344de68b-6478-44c6-a0c4-c146b912f4a3 nodeName:}" failed. No retries permitted until 2026-09-16 14:18:11.093380151 +0000 UTC m=+824.108489989 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/344de68b-6478-44c6-a0c4-c146b912f4a3-maas-api-tls") pod "maas-api-e2e-worker-w3-2ad565-f46f54678-xc2tg" (UID: "344de68b-6478-44c6-a0c4-c146b912f4a3") : secret "maas-api-e2e-worker-w3-2ad565-serving-cert" not found
Sep 16 14:18:10.593995 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:10.593972    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/344de68b-6478-44c6-a0c4-c146b912f4a3-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-2ad565-f46f54678-xc2tg\" (UID: \"344de68b-6478-44c6-a0c4-c146b912f4a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-2ad565-f46f54678-xc2tg"
Sep 16 14:18:10.595880 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:10.595860    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/344de68b-6478-44c6-a0c4-c146b912f4a3-metrics-certs\") pod \"maas-api-e2e-worker-w3-2ad565-f46f54678-xc2tg\" (UID: \"344de68b-6478-44c6-a0c4-c146b912f4a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-2ad565-f46f54678-xc2tg"
Sep 16 14:18:10.610884 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:10.610854    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtcx8\" (UniqueName: \"kubernetes.io/projected/344de68b-6478-44c6-a0c4-c146b912f4a3-kube-api-access-jtcx8\") pod \"maas-api-e2e-worker-w3-2ad565-f46f54678-xc2tg\" (UID: \"344de68b-6478-44c6-a0c4-c146b912f4a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-2ad565-f46f54678-xc2tg"
Sep 16 14:18:11.097907 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:11.097872    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/344de68b-6478-44c6-a0c4-c146b912f4a3-maas-api-tls\") pod \"maas-api-e2e-worker-w3-2ad565-f46f54678-xc2tg\" (UID: \"344de68b-6478-44c6-a0c4-c146b912f4a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-2ad565-f46f54678-xc2tg"
Sep 16 14:18:11.100229 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:11.100201    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/344de68b-6478-44c6-a0c4-c146b912f4a3-maas-api-tls\") pod \"maas-api-e2e-worker-w3-2ad565-f46f54678-xc2tg\" (UID: \"344de68b-6478-44c6-a0c4-c146b912f4a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-2ad565-f46f54678-xc2tg"
Sep 16 14:18:11.341158 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:11.341128    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-2ad565-f46f54678-xc2tg"
Sep 16 14:18:11.613306 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:11.613281    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-2ad565-f46f54678-xc2tg"]
Sep 16 14:18:11.615427 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:18:11.615391    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod344de68b_6478_44c6_a0c4_c146b912f4a3.slice/crio-25d69e783f883cfa836cdaf14fe410a4ea1995ff1ff83cb79c3c8c8de996b66f WatchSource:0}: Error finding container 25d69e783f883cfa836cdaf14fe410a4ea1995ff1ff83cb79c3c8c8de996b66f: Status 404 returned error can't find the container with id 25d69e783f883cfa836cdaf14fe410a4ea1995ff1ff83cb79c3c8c8de996b66f
Sep 16 14:18:11.886994 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:11.886885    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-08475e-565c574495-zngn7"]
Sep 16 14:18:11.891880 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:11.891857    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-08475e-565c574495-zngn7"
Sep 16 14:18:11.893773 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:11.893749    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w0-08475e-serving-cert\""
Sep 16 14:18:11.904814 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:11.904793    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-08475e-565c574495-zngn7"]
Sep 16 14:18:11.906645 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:11.906624    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/82cb9c30-c55b-4a0a-9198-3d9b4529bb58-maas-api-tls\") pod \"maas-api-e2e-worker-w0-08475e-565c574495-zngn7\" (UID: \"82cb9c30-c55b-4a0a-9198-3d9b4529bb58\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-08475e-565c574495-zngn7"
Sep 16 14:18:11.906953 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:11.906838    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7plc5\" (UniqueName: \"kubernetes.io/projected/82cb9c30-c55b-4a0a-9198-3d9b4529bb58-kube-api-access-7plc5\") pod \"maas-api-e2e-worker-w0-08475e-565c574495-zngn7\" (UID: \"82cb9c30-c55b-4a0a-9198-3d9b4529bb58\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-08475e-565c574495-zngn7"
Sep 16 14:18:11.919030 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:11.919000    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/82cb9c30-c55b-4a0a-9198-3d9b4529bb58-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-08475e-565c574495-zngn7\" (UID: \"82cb9c30-c55b-4a0a-9198-3d9b4529bb58\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-08475e-565c574495-zngn7"
Sep 16 14:18:11.919156 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:11.919067    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/82cb9c30-c55b-4a0a-9198-3d9b4529bb58-metrics-certs\") pod \"maas-api-e2e-worker-w0-08475e-565c574495-zngn7\" (UID: \"82cb9c30-c55b-4a0a-9198-3d9b4529bb58\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-08475e-565c574495-zngn7"
Sep 16 14:18:12.020720 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:12.020671    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/82cb9c30-c55b-4a0a-9198-3d9b4529bb58-maas-api-tls\") pod \"maas-api-e2e-worker-w0-08475e-565c574495-zngn7\" (UID: \"82cb9c30-c55b-4a0a-9198-3d9b4529bb58\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-08475e-565c574495-zngn7"
Sep 16 14:18:12.020720 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:12.020716    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7plc5\" (UniqueName: \"kubernetes.io/projected/82cb9c30-c55b-4a0a-9198-3d9b4529bb58-kube-api-access-7plc5\") pod \"maas-api-e2e-worker-w0-08475e-565c574495-zngn7\" (UID: \"82cb9c30-c55b-4a0a-9198-3d9b4529bb58\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-08475e-565c574495-zngn7"
Sep 16 14:18:12.020953 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:12.020757    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/82cb9c30-c55b-4a0a-9198-3d9b4529bb58-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-08475e-565c574495-zngn7\" (UID: \"82cb9c30-c55b-4a0a-9198-3d9b4529bb58\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-08475e-565c574495-zngn7"
Sep 16 14:18:12.020953 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:12.020782    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/82cb9c30-c55b-4a0a-9198-3d9b4529bb58-metrics-certs\") pod \"maas-api-e2e-worker-w0-08475e-565c574495-zngn7\" (UID: \"82cb9c30-c55b-4a0a-9198-3d9b4529bb58\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-08475e-565c574495-zngn7"
Sep 16 14:18:12.020953 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:18:12.020848    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w0-08475e-serving-cert: secret "maas-api-e2e-worker-w0-08475e-serving-cert" not found
Sep 16 14:18:12.021115 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:18:12.020961    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82cb9c30-c55b-4a0a-9198-3d9b4529bb58-maas-api-tls podName:82cb9c30-c55b-4a0a-9198-3d9b4529bb58 nodeName:}" failed. No retries permitted until 2026-09-16 14:18:12.520938704 +0000 UTC m=+825.536048554 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/82cb9c30-c55b-4a0a-9198-3d9b4529bb58-maas-api-tls") pod "maas-api-e2e-worker-w0-08475e-565c574495-zngn7" (UID: "82cb9c30-c55b-4a0a-9198-3d9b4529bb58") : secret "maas-api-e2e-worker-w0-08475e-serving-cert" not found
Sep 16 14:18:12.021526 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:12.021485    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/82cb9c30-c55b-4a0a-9198-3d9b4529bb58-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-08475e-565c574495-zngn7\" (UID: \"82cb9c30-c55b-4a0a-9198-3d9b4529bb58\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-08475e-565c574495-zngn7"
Sep 16 14:18:12.023507 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:12.023471    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/82cb9c30-c55b-4a0a-9198-3d9b4529bb58-metrics-certs\") pod \"maas-api-e2e-worker-w0-08475e-565c574495-zngn7\" (UID: \"82cb9c30-c55b-4a0a-9198-3d9b4529bb58\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-08475e-565c574495-zngn7"
Sep 16 14:18:12.036230 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:12.036207    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7plc5\" (UniqueName: \"kubernetes.io/projected/82cb9c30-c55b-4a0a-9198-3d9b4529bb58-kube-api-access-7plc5\") pod \"maas-api-e2e-worker-w0-08475e-565c574495-zngn7\" (UID: \"82cb9c30-c55b-4a0a-9198-3d9b4529bb58\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-08475e-565c574495-zngn7"
Sep 16 14:18:12.046294 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:12.046265    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-2ad565-f46f54678-xc2tg" event={"ID":"344de68b-6478-44c6-a0c4-c146b912f4a3","Type":"ContainerStarted","Data":"25d69e783f883cfa836cdaf14fe410a4ea1995ff1ff83cb79c3c8c8de996b66f"}
Sep 16 14:18:12.525609 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:12.525576    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/82cb9c30-c55b-4a0a-9198-3d9b4529bb58-maas-api-tls\") pod \"maas-api-e2e-worker-w0-08475e-565c574495-zngn7\" (UID: \"82cb9c30-c55b-4a0a-9198-3d9b4529bb58\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-08475e-565c574495-zngn7"
Sep 16 14:18:12.527891 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:12.527864    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/82cb9c30-c55b-4a0a-9198-3d9b4529bb58-maas-api-tls\") pod \"maas-api-e2e-worker-w0-08475e-565c574495-zngn7\" (UID: \"82cb9c30-c55b-4a0a-9198-3d9b4529bb58\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-08475e-565c574495-zngn7"
Sep 16 14:18:12.805511 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:12.805419    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-08475e-565c574495-zngn7"
Sep 16 14:18:13.053549 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:13.053483    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-2ad565-f46f54678-xc2tg" event={"ID":"344de68b-6478-44c6-a0c4-c146b912f4a3","Type":"ContainerStarted","Data":"3ad93dc37df134395314150259d45948f0e3ef2d1f607da0a547507c5e70bc5f"}
Sep 16 14:18:13.053753 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:13.053575    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-2ad565-f46f54678-xc2tg"
Sep 16 14:18:13.072400 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:13.072315    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-2ad565-f46f54678-xc2tg" podStartSLOduration=2.636656793 podStartE2EDuration="3.072300263s" totalImagesPullingTime="435.64347ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:18:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:18:13.071605266 +0000 UTC m=+826.086715127" watchObservedRunningTime="2026-09-16 14:18:13.072300263 +0000 UTC m=+826.087410122"
Sep 16 14:18:13.084536 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:13.084493    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-08475e-565c574495-zngn7"]
Sep 16 14:18:13.086290 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:18:13.086258    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82cb9c30_c55b_4a0a_9198_3d9b4529bb58.slice/crio-f3a907629a56c4a294a6752a9895ec76554a9d3f4f1e12d2a4a490cb0e35726c WatchSource:0}: Error finding container f3a907629a56c4a294a6752a9895ec76554a9d3f4f1e12d2a4a490cb0e35726c: Status 404 returned error can't find the container with id f3a907629a56c4a294a6752a9895ec76554a9d3f4f1e12d2a4a490cb0e35726c
Sep 16 14:18:14.061721 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:14.061688    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-08475e-565c574495-zngn7" event={"ID":"82cb9c30-c55b-4a0a-9198-3d9b4529bb58","Type":"ContainerStarted","Data":"f3a907629a56c4a294a6752a9895ec76554a9d3f4f1e12d2a4a490cb0e35726c"}
Sep 16 14:18:15.069230 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:15.069198    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-08475e-565c574495-zngn7" event={"ID":"82cb9c30-c55b-4a0a-9198-3d9b4529bb58","Type":"ContainerStarted","Data":"b16d6550de844430f23826fc3511c392df80505a638fca2f5bb2e59bd4c3f2c7"}
Sep 16 14:18:15.069665 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:15.069314    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-08475e-565c574495-zngn7"
Sep 16 14:18:15.093692 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:15.093650    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-08475e-565c574495-zngn7" podStartSLOduration=2.73023624 podStartE2EDuration="4.093634142s" totalImagesPullingTime="1.363397902s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:18:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:18:15.091342347 +0000 UTC m=+828.106452209" watchObservedRunningTime="2026-09-16 14:18:15.093634142 +0000 UTC m=+828.108744128"
Sep 16 14:18:15.948556 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:15.948525    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-e24c87-5cfb444bd7-gwg85"]
Sep 16 14:18:15.952013 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:15.951989    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-e24c87-5cfb444bd7-gwg85"
Sep 16 14:18:15.954326 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:15.954306    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-e24c87\""
Sep 16 14:18:15.954942 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:15.954923    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-e24c87-dockercfg-k42jb\""
Sep 16 14:18:15.971306 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:15.971285    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-e24c87-5cfb444bd7-gwg85"]
Sep 16 14:18:16.060120 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:16.060084    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4dd9af77-6514-477e-b2c7-4d6fc86f856f-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-e24c87-5cfb444bd7-gwg85\" (UID: \"4dd9af77-6514-477e-b2c7-4d6fc86f856f\") " pod="openshift-ingress/payload-processing-e2e-inf-a-e24c87-5cfb444bd7-gwg85"
Sep 16 14:18:16.060422 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:16.060393    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ns8qm\" (UniqueName: \"kubernetes.io/projected/4dd9af77-6514-477e-b2c7-4d6fc86f856f-kube-api-access-ns8qm\") pod \"payload-processing-e2e-inf-a-e24c87-5cfb444bd7-gwg85\" (UID: \"4dd9af77-6514-477e-b2c7-4d6fc86f856f\") " pod="openshift-ingress/payload-processing-e2e-inf-a-e24c87-5cfb444bd7-gwg85"
Sep 16 14:18:16.160919 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:16.160885    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ns8qm\" (UniqueName: \"kubernetes.io/projected/4dd9af77-6514-477e-b2c7-4d6fc86f856f-kube-api-access-ns8qm\") pod \"payload-processing-e2e-inf-a-e24c87-5cfb444bd7-gwg85\" (UID: \"4dd9af77-6514-477e-b2c7-4d6fc86f856f\") " pod="openshift-ingress/payload-processing-e2e-inf-a-e24c87-5cfb444bd7-gwg85"
Sep 16 14:18:16.161415 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:16.161025    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4dd9af77-6514-477e-b2c7-4d6fc86f856f-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-e24c87-5cfb444bd7-gwg85\" (UID: \"4dd9af77-6514-477e-b2c7-4d6fc86f856f\") " pod="openshift-ingress/payload-processing-e2e-inf-a-e24c87-5cfb444bd7-gwg85"
Sep 16 14:18:16.161794 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:16.161773    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4dd9af77-6514-477e-b2c7-4d6fc86f856f-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-e24c87-5cfb444bd7-gwg85\" (UID: \"4dd9af77-6514-477e-b2c7-4d6fc86f856f\") " pod="openshift-ingress/payload-processing-e2e-inf-a-e24c87-5cfb444bd7-gwg85"
Sep 16 14:18:16.169196 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:16.169166    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ns8qm\" (UniqueName: \"kubernetes.io/projected/4dd9af77-6514-477e-b2c7-4d6fc86f856f-kube-api-access-ns8qm\") pod \"payload-processing-e2e-inf-a-e24c87-5cfb444bd7-gwg85\" (UID: \"4dd9af77-6514-477e-b2c7-4d6fc86f856f\") " pod="openshift-ingress/payload-processing-e2e-inf-a-e24c87-5cfb444bd7-gwg85"
Sep 16 14:18:16.261988 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:16.261908    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-e24c87-5cfb444bd7-gwg85"
Sep 16 14:18:16.553092 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:16.553069    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-e24c87-5cfb444bd7-gwg85"]
Sep 16 14:18:16.555135 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:18:16.555110    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4dd9af77_6514_477e_b2c7_4d6fc86f856f.slice/crio-4b5433bc0f338d1d025e44ebf75b0617ee7d18e2a2f1144969b3192940cee79b WatchSource:0}: Error finding container 4b5433bc0f338d1d025e44ebf75b0617ee7d18e2a2f1144969b3192940cee79b: Status 404 returned error can't find the container with id 4b5433bc0f338d1d025e44ebf75b0617ee7d18e2a2f1144969b3192940cee79b
Sep 16 14:18:17.082271 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:17.082239    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-e24c87-5cfb444bd7-gwg85" event={"ID":"4dd9af77-6514-477e-b2c7-4d6fc86f856f","Type":"ContainerStarted","Data":"ad5b4f37e89759c4a24bde1d5277421196defd7c4e8a3f6293a709e2cea0f59a"}
Sep 16 14:18:17.082486 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:17.082325    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-e24c87-5cfb444bd7-gwg85"
Sep 16 14:18:17.082486 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:17.082346    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-e24c87-5cfb444bd7-gwg85" event={"ID":"4dd9af77-6514-477e-b2c7-4d6fc86f856f","Type":"ContainerStarted","Data":"4b5433bc0f338d1d025e44ebf75b0617ee7d18e2a2f1144969b3192940cee79b"}
Sep 16 14:18:17.180157 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:17.180110    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-e24c87-5cfb444bd7-gwg85" podStartSLOduration=2.180093882 podStartE2EDuration="2.180093882s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:18:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:18:17.177139836 +0000 UTC m=+830.192249695" watchObservedRunningTime="2026-09-16 14:18:17.180093882 +0000 UTC m=+830.195203744"
Sep 16 14:18:19.068847 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:19.068822    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-2ad565-f46f54678-xc2tg"
Sep 16 14:18:21.080872 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:21.080844    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-08475e-565c574495-zngn7"
Sep 16 14:18:28.090251 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:28.090220    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-e24c87-5cfb444bd7-gwg85"
Sep 16 14:18:28.159388 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:28.159357    2580 generic.go:356] "Generic (PLEG): container finished" podID="9db21092-910f-479a-ae58-4463feec4dfd" containerID="6f0235a87d104e93ebafb57294019ee0757189a0965fc15bf09486adf67cab45" exitCode=0
Sep 16 14:18:28.159606 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:28.159433    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk" event={"ID":"9db21092-910f-479a-ae58-4463feec4dfd","Type":"ContainerDied","Data":"6f0235a87d104e93ebafb57294019ee0757189a0965fc15bf09486adf67cab45"}
Sep 16 14:18:28.159888 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:28.159870    2580 scope.go:117] "RemoveContainer" containerID="6f0235a87d104e93ebafb57294019ee0757189a0965fc15bf09486adf67cab45"
Sep 16 14:18:28.161092 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:28.161057    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 14:18:28.161197 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:28.161145    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 14:18:28.161278 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:28.161197    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 14:18:28.161445 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:28.161427    2580 generic.go:356] "Generic (PLEG): container finished" podID="9cb6f359-dd14-43e9-a795-d56065b12448" containerID="823cf112f89ed5507c820a46b4f8fcc167b7eb64c3f4bdb132495d242db621e8" exitCode=0
Sep 16 14:18:28.161544 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:28.161455    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z" event={"ID":"9cb6f359-dd14-43e9-a795-d56065b12448","Type":"ContainerDied","Data":"823cf112f89ed5507c820a46b4f8fcc167b7eb64c3f4bdb132495d242db621e8"}
Sep 16 14:18:28.161810 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:28.161797    2580 scope.go:117] "RemoveContainer" containerID="823cf112f89ed5507c820a46b4f8fcc167b7eb64c3f4bdb132495d242db621e8"
Sep 16 14:18:28.162789 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:28.162767    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 14:18:28.162843 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:28.162825    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 14:18:28.162883 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:28.162858    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 14:18:29.169071 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:29.169036    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk" event={"ID":"9db21092-910f-479a-ae58-4463feec4dfd","Type":"ContainerStarted","Data":"403f0717573202f8a186af8d001bd7f8f7723e282a9024624118ac9a20b665f0"}
Sep 16 14:18:29.171143 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:29.171107    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z" event={"ID":"9cb6f359-dd14-43e9-a795-d56065b12448","Type":"ContainerStarted","Data":"0d0cfbc2ee09057e5aa07767c3ff7e6c61c575d35bcceb7bc296d3bafd683ab9"}
Sep 16 14:18:29.599282 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:29.599242    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z"
Sep 16 14:18:29.599478 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:29.599294    2580 patch_prober.go:28] interesting pod/e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Sep 16 14:18:29.599478 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:29.599360    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z" podUID="9cb6f359-dd14-43e9-a795-d56065b12448" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Sep 16 14:18:29.720766 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:29.720719    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk"
Sep 16 14:18:29.721142 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:29.721117    2580 patch_prober.go:28] interesting pod/e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Sep 16 14:18:29.721258 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:29.721183    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk" podUID="9db21092-910f-479a-ae58-4463feec4dfd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Sep 16 14:18:30.599029 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:30.598996    2580 patch_prober.go:28] interesting pod/e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Sep 16 14:18:30.599384 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:30.599050    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z" podUID="9cb6f359-dd14-43e9-a795-d56065b12448" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Sep 16 14:18:30.721079 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:30.721036    2580 patch_prober.go:28] interesting pod/e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Sep 16 14:18:30.721273 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:30.721109    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk" podUID="9db21092-910f-479a-ae58-4463feec4dfd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Sep 16 14:18:31.598747 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:31.598713    2580 patch_prober.go:28] interesting pod/e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Sep 16 14:18:31.598923 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:31.598773    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z" podUID="9cb6f359-dd14-43e9-a795-d56065b12448" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Sep 16 14:18:31.720483 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:31.720443    2580 patch_prober.go:28] interesting pod/e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Sep 16 14:18:31.720991 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:31.720524    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk" podUID="9db21092-910f-479a-ae58-4463feec4dfd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Sep 16 14:18:32.599080 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:32.599044    2580 patch_prober.go:28] interesting pod/e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Sep 16 14:18:32.599262 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:32.599100    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z" podUID="9cb6f359-dd14-43e9-a795-d56065b12448" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Sep 16 14:18:32.720893 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:32.720852    2580 patch_prober.go:28] interesting pod/e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Sep 16 14:18:32.721391 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:32.720936    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk" podUID="9db21092-910f-479a-ae58-4463feec4dfd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Sep 16 14:18:33.599215 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:33.599167    2580 patch_prober.go:28] interesting pod/e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Sep 16 14:18:33.599406 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:33.599230    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z" podUID="9cb6f359-dd14-43e9-a795-d56065b12448" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Sep 16 14:18:33.720798 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:33.720759    2580 patch_prober.go:28] interesting pod/e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Sep 16 14:18:33.721091 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:33.721058    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk" podUID="9db21092-910f-479a-ae58-4463feec4dfd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Sep 16 14:18:34.602841 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:34.602814    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z"
Sep 16 14:18:34.603119 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:34.603098    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z"
Sep 16 14:18:34.603926 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:34.603902    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z"
Sep 16 14:18:34.725660 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:34.725633    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk"
Sep 16 14:18:34.726126 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:34.726054    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk"
Sep 16 14:18:34.727023 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:34.727004    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk"
Sep 16 14:18:46.945401 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:46.945368    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-46e82d-57b5878894-8m2c8"]
Sep 16 14:18:46.950212 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:46.950191    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-46e82d-57b5878894-8m2c8"
Sep 16 14:18:46.952274 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:46.952253    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-46e82d-dockercfg-cb2kv\""
Sep 16 14:18:46.952578 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:46.952548    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-46e82d\""
Sep 16 14:18:46.958769 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:46.958746    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-46e82d-57b5878894-8m2c8"]
Sep 16 14:18:47.005904 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:47.005859    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d71900a3-4f93-4e27-adac-51ea4e4413c0-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-46e82d-57b5878894-8m2c8\" (UID: \"d71900a3-4f93-4e27-adac-51ea4e4413c0\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-46e82d-57b5878894-8m2c8"
Sep 16 14:18:47.006080 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:47.005932    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6kxbz\" (UniqueName: \"kubernetes.io/projected/d71900a3-4f93-4e27-adac-51ea4e4413c0-kube-api-access-6kxbz\") pod \"payload-pre-processing-e2e-inf-b-46e82d-57b5878894-8m2c8\" (UID: \"d71900a3-4f93-4e27-adac-51ea4e4413c0\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-46e82d-57b5878894-8m2c8"
Sep 16 14:18:47.106994 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:47.106950    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d71900a3-4f93-4e27-adac-51ea4e4413c0-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-46e82d-57b5878894-8m2c8\" (UID: \"d71900a3-4f93-4e27-adac-51ea4e4413c0\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-46e82d-57b5878894-8m2c8"
Sep 16 14:18:47.107175 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:47.107117    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6kxbz\" (UniqueName: \"kubernetes.io/projected/d71900a3-4f93-4e27-adac-51ea4e4413c0-kube-api-access-6kxbz\") pod \"payload-pre-processing-e2e-inf-b-46e82d-57b5878894-8m2c8\" (UID: \"d71900a3-4f93-4e27-adac-51ea4e4413c0\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-46e82d-57b5878894-8m2c8"
Sep 16 14:18:47.107849 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:47.107827    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d71900a3-4f93-4e27-adac-51ea4e4413c0-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-46e82d-57b5878894-8m2c8\" (UID: \"d71900a3-4f93-4e27-adac-51ea4e4413c0\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-46e82d-57b5878894-8m2c8"
Sep 16 14:18:47.118585 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:47.118559    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6kxbz\" (UniqueName: \"kubernetes.io/projected/d71900a3-4f93-4e27-adac-51ea4e4413c0-kube-api-access-6kxbz\") pod \"payload-pre-processing-e2e-inf-b-46e82d-57b5878894-8m2c8\" (UID: \"d71900a3-4f93-4e27-adac-51ea4e4413c0\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-46e82d-57b5878894-8m2c8"
Sep 16 14:18:47.261235 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:47.261152    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-46e82d-57b5878894-8m2c8"
Sep 16 14:18:47.534425 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:47.534403    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-46e82d-57b5878894-8m2c8"]
Sep 16 14:18:47.535880 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:18:47.535854    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd71900a3_4f93_4e27_adac_51ea4e4413c0.slice/crio-27ea9d76f492d8428490c5c143b3f344f7084fe1fc7476f05d315d8bff301355 WatchSource:0}: Error finding container 27ea9d76f492d8428490c5c143b3f344f7084fe1fc7476f05d315d8bff301355: Status 404 returned error can't find the container with id 27ea9d76f492d8428490c5c143b3f344f7084fe1fc7476f05d315d8bff301355
Sep 16 14:18:48.296424 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:48.296383    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-46e82d-57b5878894-8m2c8" event={"ID":"d71900a3-4f93-4e27-adac-51ea4e4413c0","Type":"ContainerStarted","Data":"6d65214d2ab61ed4a0bdddf129b27a6be3773b5221535441dd1b006aa3ff5665"}
Sep 16 14:18:48.296911 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:48.296474    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-46e82d-57b5878894-8m2c8" event={"ID":"d71900a3-4f93-4e27-adac-51ea4e4413c0","Type":"ContainerStarted","Data":"27ea9d76f492d8428490c5c143b3f344f7084fe1fc7476f05d315d8bff301355"}
Sep 16 14:18:48.296911 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:48.296546    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-46e82d-57b5878894-8m2c8"
Sep 16 14:18:48.312699 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:48.312658    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-46e82d-57b5878894-8m2c8" podStartSLOduration=2.312645047 podStartE2EDuration="2.312645047s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:18:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:18:48.311530969 +0000 UTC m=+861.326640834" watchObservedRunningTime="2026-09-16 14:18:48.312645047 +0000 UTC m=+861.327754906"
Sep 16 14:18:55.671853 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:55.671816    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-08475e/simulator-w0-08475e-kserve-8cc86df7b-x7z4t"]
Sep 16 14:18:55.675433 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:55.675412    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-08475e/simulator-w0-08475e-kserve-8cc86df7b-x7z4t"
Sep 16 14:18:55.678254 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:55.678232    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-08475e\"/\"openshift-service-ca.crt\""
Sep 16 14:18:55.678359 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:55.678257    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-08475e\"/\"default-dockercfg-bmbjz\""
Sep 16 14:18:55.678784 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:55.678768    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-08475e\"/\"simulator-w0-08475e-kserve-self-signed-certs\""
Sep 16 14:18:55.678784 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:55.678774    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-08475e\"/\"kube-root-ca.crt\""
Sep 16 14:18:55.683481 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:55.683462    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-08475e/simulator-w0-08475e-kserve-8cc86df7b-x7z4t"]
Sep 16 14:18:55.788066 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:55.788038    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5nrm\" (UniqueName: \"kubernetes.io/projected/9b899509-2d6c-4c9f-82ba-58bbc8fd07d8-kube-api-access-z5nrm\") pod \"simulator-w0-08475e-kserve-8cc86df7b-x7z4t\" (UID: \"9b899509-2d6c-4c9f-82ba-58bbc8fd07d8\") " pod="e2e-models-e2e-worker-w0-08475e/simulator-w0-08475e-kserve-8cc86df7b-x7z4t"
Sep 16 14:18:55.788210 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:55.788098    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b899509-2d6c-4c9f-82ba-58bbc8fd07d8-tmp-dir\") pod \"simulator-w0-08475e-kserve-8cc86df7b-x7z4t\" (UID: \"9b899509-2d6c-4c9f-82ba-58bbc8fd07d8\") " pod="e2e-models-e2e-worker-w0-08475e/simulator-w0-08475e-kserve-8cc86df7b-x7z4t"
Sep 16 14:18:55.788210 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:55.788132    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b899509-2d6c-4c9f-82ba-58bbc8fd07d8-tls-certs\") pod \"simulator-w0-08475e-kserve-8cc86df7b-x7z4t\" (UID: \"9b899509-2d6c-4c9f-82ba-58bbc8fd07d8\") " pod="e2e-models-e2e-worker-w0-08475e/simulator-w0-08475e-kserve-8cc86df7b-x7z4t"
Sep 16 14:18:55.788210 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:55.788154    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b899509-2d6c-4c9f-82ba-58bbc8fd07d8-home\") pod \"simulator-w0-08475e-kserve-8cc86df7b-x7z4t\" (UID: \"9b899509-2d6c-4c9f-82ba-58bbc8fd07d8\") " pod="e2e-models-e2e-worker-w0-08475e/simulator-w0-08475e-kserve-8cc86df7b-x7z4t"
Sep 16 14:18:55.788339 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:55.788237    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b899509-2d6c-4c9f-82ba-58bbc8fd07d8-model-cache\") pod \"simulator-w0-08475e-kserve-8cc86df7b-x7z4t\" (UID: \"9b899509-2d6c-4c9f-82ba-58bbc8fd07d8\") " pod="e2e-models-e2e-worker-w0-08475e/simulator-w0-08475e-kserve-8cc86df7b-x7z4t"
Sep 16 14:18:55.788339 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:55.788317    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b899509-2d6c-4c9f-82ba-58bbc8fd07d8-dshm\") pod \"simulator-w0-08475e-kserve-8cc86df7b-x7z4t\" (UID: \"9b899509-2d6c-4c9f-82ba-58bbc8fd07d8\") " pod="e2e-models-e2e-worker-w0-08475e/simulator-w0-08475e-kserve-8cc86df7b-x7z4t"
Sep 16 14:18:55.889547 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:55.889519    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5nrm\" (UniqueName: \"kubernetes.io/projected/9b899509-2d6c-4c9f-82ba-58bbc8fd07d8-kube-api-access-z5nrm\") pod \"simulator-w0-08475e-kserve-8cc86df7b-x7z4t\" (UID: \"9b899509-2d6c-4c9f-82ba-58bbc8fd07d8\") " pod="e2e-models-e2e-worker-w0-08475e/simulator-w0-08475e-kserve-8cc86df7b-x7z4t"
Sep 16 14:18:55.889720 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:55.889593    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b899509-2d6c-4c9f-82ba-58bbc8fd07d8-tmp-dir\") pod \"simulator-w0-08475e-kserve-8cc86df7b-x7z4t\" (UID: \"9b899509-2d6c-4c9f-82ba-58bbc8fd07d8\") " pod="e2e-models-e2e-worker-w0-08475e/simulator-w0-08475e-kserve-8cc86df7b-x7z4t"
Sep 16 14:18:55.889720 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:55.889621    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b899509-2d6c-4c9f-82ba-58bbc8fd07d8-tls-certs\") pod \"simulator-w0-08475e-kserve-8cc86df7b-x7z4t\" (UID: \"9b899509-2d6c-4c9f-82ba-58bbc8fd07d8\") " pod="e2e-models-e2e-worker-w0-08475e/simulator-w0-08475e-kserve-8cc86df7b-x7z4t"
Sep 16 14:18:55.889720 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:55.889643    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b899509-2d6c-4c9f-82ba-58bbc8fd07d8-home\") pod \"simulator-w0-08475e-kserve-8cc86df7b-x7z4t\" (UID: \"9b899509-2d6c-4c9f-82ba-58bbc8fd07d8\") " pod="e2e-models-e2e-worker-w0-08475e/simulator-w0-08475e-kserve-8cc86df7b-x7z4t"
Sep 16 14:18:55.889720 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:55.889674    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b899509-2d6c-4c9f-82ba-58bbc8fd07d8-model-cache\") pod \"simulator-w0-08475e-kserve-8cc86df7b-x7z4t\" (UID: \"9b899509-2d6c-4c9f-82ba-58bbc8fd07d8\") " pod="e2e-models-e2e-worker-w0-08475e/simulator-w0-08475e-kserve-8cc86df7b-x7z4t"
Sep 16 14:18:55.889720 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:55.889695    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b899509-2d6c-4c9f-82ba-58bbc8fd07d8-dshm\") pod \"simulator-w0-08475e-kserve-8cc86df7b-x7z4t\" (UID: \"9b899509-2d6c-4c9f-82ba-58bbc8fd07d8\") " pod="e2e-models-e2e-worker-w0-08475e/simulator-w0-08475e-kserve-8cc86df7b-x7z4t"
Sep 16 14:18:55.890055 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:55.890028    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b899509-2d6c-4c9f-82ba-58bbc8fd07d8-home\") pod \"simulator-w0-08475e-kserve-8cc86df7b-x7z4t\" (UID: \"9b899509-2d6c-4c9f-82ba-58bbc8fd07d8\") " pod="e2e-models-e2e-worker-w0-08475e/simulator-w0-08475e-kserve-8cc86df7b-x7z4t"
Sep 16 14:18:55.890121 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:55.890058    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b899509-2d6c-4c9f-82ba-58bbc8fd07d8-tmp-dir\") pod \"simulator-w0-08475e-kserve-8cc86df7b-x7z4t\" (UID: \"9b899509-2d6c-4c9f-82ba-58bbc8fd07d8\") " pod="e2e-models-e2e-worker-w0-08475e/simulator-w0-08475e-kserve-8cc86df7b-x7z4t"
Sep 16 14:18:55.890121 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:55.890076    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b899509-2d6c-4c9f-82ba-58bbc8fd07d8-model-cache\") pod \"simulator-w0-08475e-kserve-8cc86df7b-x7z4t\" (UID: \"9b899509-2d6c-4c9f-82ba-58bbc8fd07d8\") " pod="e2e-models-e2e-worker-w0-08475e/simulator-w0-08475e-kserve-8cc86df7b-x7z4t"
Sep 16 14:18:55.891948 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:55.891923    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b899509-2d6c-4c9f-82ba-58bbc8fd07d8-dshm\") pod \"simulator-w0-08475e-kserve-8cc86df7b-x7z4t\" (UID: \"9b899509-2d6c-4c9f-82ba-58bbc8fd07d8\") " pod="e2e-models-e2e-worker-w0-08475e/simulator-w0-08475e-kserve-8cc86df7b-x7z4t"
Sep 16 14:18:55.892191 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:55.892174    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b899509-2d6c-4c9f-82ba-58bbc8fd07d8-tls-certs\") pod \"simulator-w0-08475e-kserve-8cc86df7b-x7z4t\" (UID: \"9b899509-2d6c-4c9f-82ba-58bbc8fd07d8\") " pod="e2e-models-e2e-worker-w0-08475e/simulator-w0-08475e-kserve-8cc86df7b-x7z4t"
Sep 16 14:18:55.898409 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:55.898386    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5nrm\" (UniqueName: \"kubernetes.io/projected/9b899509-2d6c-4c9f-82ba-58bbc8fd07d8-kube-api-access-z5nrm\") pod \"simulator-w0-08475e-kserve-8cc86df7b-x7z4t\" (UID: \"9b899509-2d6c-4c9f-82ba-58bbc8fd07d8\") " pod="e2e-models-e2e-worker-w0-08475e/simulator-w0-08475e-kserve-8cc86df7b-x7z4t"
Sep 16 14:18:55.985699 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:55.985671    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-08475e/simulator-w0-08475e-kserve-8cc86df7b-x7z4t"
Sep 16 14:18:56.260557 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:56.260534    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-08475e/simulator-w0-08475e-kserve-8cc86df7b-x7z4t"]
Sep 16 14:18:56.261733 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:18:56.261706    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9b899509_2d6c_4c9f_82ba_58bbc8fd07d8.slice/crio-64fb20c7e24802dd112407207a6b8c7bdad0b18f2222b5b1a216c8d29ff80bef WatchSource:0}: Error finding container 64fb20c7e24802dd112407207a6b8c7bdad0b18f2222b5b1a216c8d29ff80bef: Status 404 returned error can't find the container with id 64fb20c7e24802dd112407207a6b8c7bdad0b18f2222b5b1a216c8d29ff80bef
Sep 16 14:18:56.342273 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:56.342202    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-08475e/simulator-w0-08475e-kserve-8cc86df7b-x7z4t" event={"ID":"9b899509-2d6c-4c9f-82ba-58bbc8fd07d8","Type":"ContainerStarted","Data":"3c70eaf52334a52522a4c6bfa49a0a9e92c394d0dfd71d3f10b72746a6fc9ed4"}
Sep 16 14:18:56.342273 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:56.342248    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-08475e/simulator-w0-08475e-kserve-8cc86df7b-x7z4t" event={"ID":"9b899509-2d6c-4c9f-82ba-58bbc8fd07d8","Type":"ContainerStarted","Data":"64fb20c7e24802dd112407207a6b8c7bdad0b18f2222b5b1a216c8d29ff80bef"}
Sep 16 14:18:56.361135 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:56.361086    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-08475e/simulator-w0-08475e-kserve-8cc86df7b-x7z4t" podStartSLOduration=1.361070873 podStartE2EDuration="1.361070873s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:18:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:18:56.360158946 +0000 UTC m=+869.375268807" watchObservedRunningTime="2026-09-16 14:18:56.361070873 +0000 UTC m=+869.376180732"
Sep 16 14:18:59.304356 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:18:59.304327    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-46e82d-57b5878894-8m2c8"
Sep 16 14:19:05.985959 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:19:05.985926    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-08475e/simulator-w0-08475e-kserve-8cc86df7b-x7z4t"
Sep 16 14:19:05.986428 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:19:05.985971    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-08475e/simulator-w0-08475e-kserve-8cc86df7b-x7z4t"
Sep 16 14:19:05.998351 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:19:05.998327    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-08475e/simulator-w0-08475e-kserve-8cc86df7b-x7z4t"
Sep 16 14:19:06.410065 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:19:06.409980    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-08475e/simulator-w0-08475e-kserve-8cc86df7b-x7z4t"
Sep 16 14:20:12.287020 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:20:12.286986    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-08475e/premium-w0-08475e-kserve-647898d665-rmzkp"]
Sep 16 14:20:12.296138 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:20:12.296104    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-08475e/premium-w0-08475e-kserve-647898d665-rmzkp"
Sep 16 14:20:12.298346 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:20:12.298286    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-08475e\"/\"premium-w0-08475e-kserve-self-signed-certs\""
Sep 16 14:20:12.303056 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:20:12.303034    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-08475e/premium-w0-08475e-kserve-647898d665-rmzkp"]
Sep 16 14:20:12.396788 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:20:12.396751    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2f76715c-d553-44c4-995e-9d2c1fe9ce5b-model-cache\") pod \"premium-w0-08475e-kserve-647898d665-rmzkp\" (UID: \"2f76715c-d553-44c4-995e-9d2c1fe9ce5b\") " pod="e2e-models-e2e-worker-w0-08475e/premium-w0-08475e-kserve-647898d665-rmzkp"
Sep 16 14:20:12.396979 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:20:12.396812    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2f76715c-d553-44c4-995e-9d2c1fe9ce5b-home\") pod \"premium-w0-08475e-kserve-647898d665-rmzkp\" (UID: \"2f76715c-d553-44c4-995e-9d2c1fe9ce5b\") " pod="e2e-models-e2e-worker-w0-08475e/premium-w0-08475e-kserve-647898d665-rmzkp"
Sep 16 14:20:12.396979 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:20:12.396852    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f76715c-d553-44c4-995e-9d2c1fe9ce5b-tmp-dir\") pod \"premium-w0-08475e-kserve-647898d665-rmzkp\" (UID: \"2f76715c-d553-44c4-995e-9d2c1fe9ce5b\") " pod="e2e-models-e2e-worker-w0-08475e/premium-w0-08475e-kserve-647898d665-rmzkp"
Sep 16 14:20:12.396979 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:20:12.396899    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcwdw\" (UniqueName: \"kubernetes.io/projected/2f76715c-d553-44c4-995e-9d2c1fe9ce5b-kube-api-access-qcwdw\") pod \"premium-w0-08475e-kserve-647898d665-rmzkp\" (UID: \"2f76715c-d553-44c4-995e-9d2c1fe9ce5b\") " pod="e2e-models-e2e-worker-w0-08475e/premium-w0-08475e-kserve-647898d665-rmzkp"
Sep 16 14:20:12.396979 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:20:12.396925    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2f76715c-d553-44c4-995e-9d2c1fe9ce5b-dshm\") pod \"premium-w0-08475e-kserve-647898d665-rmzkp\" (UID: \"2f76715c-d553-44c4-995e-9d2c1fe9ce5b\") " pod="e2e-models-e2e-worker-w0-08475e/premium-w0-08475e-kserve-647898d665-rmzkp"
Sep 16 14:20:12.397155 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:20:12.397001    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f76715c-d553-44c4-995e-9d2c1fe9ce5b-tls-certs\") pod \"premium-w0-08475e-kserve-647898d665-rmzkp\" (UID: \"2f76715c-d553-44c4-995e-9d2c1fe9ce5b\") " pod="e2e-models-e2e-worker-w0-08475e/premium-w0-08475e-kserve-647898d665-rmzkp"
Sep 16 14:20:12.498205 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:20:12.498165    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f76715c-d553-44c4-995e-9d2c1fe9ce5b-tls-certs\") pod \"premium-w0-08475e-kserve-647898d665-rmzkp\" (UID: \"2f76715c-d553-44c4-995e-9d2c1fe9ce5b\") " pod="e2e-models-e2e-worker-w0-08475e/premium-w0-08475e-kserve-647898d665-rmzkp"
Sep 16 14:20:12.498382 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:20:12.498257    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2f76715c-d553-44c4-995e-9d2c1fe9ce5b-model-cache\") pod \"premium-w0-08475e-kserve-647898d665-rmzkp\" (UID: \"2f76715c-d553-44c4-995e-9d2c1fe9ce5b\") " pod="e2e-models-e2e-worker-w0-08475e/premium-w0-08475e-kserve-647898d665-rmzkp"
Sep 16 14:20:12.498382 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:20:12.498292    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2f76715c-d553-44c4-995e-9d2c1fe9ce5b-home\") pod \"premium-w0-08475e-kserve-647898d665-rmzkp\" (UID: \"2f76715c-d553-44c4-995e-9d2c1fe9ce5b\") " pod="e2e-models-e2e-worker-w0-08475e/premium-w0-08475e-kserve-647898d665-rmzkp"
Sep 16 14:20:12.498382 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:20:12.498308    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f76715c-d553-44c4-995e-9d2c1fe9ce5b-tmp-dir\") pod \"premium-w0-08475e-kserve-647898d665-rmzkp\" (UID: \"2f76715c-d553-44c4-995e-9d2c1fe9ce5b\") " pod="e2e-models-e2e-worker-w0-08475e/premium-w0-08475e-kserve-647898d665-rmzkp"
Sep 16 14:20:12.498382 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:20:12.498360    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcwdw\" (UniqueName: \"kubernetes.io/projected/2f76715c-d553-44c4-995e-9d2c1fe9ce5b-kube-api-access-qcwdw\") pod \"premium-w0-08475e-kserve-647898d665-rmzkp\" (UID: \"2f76715c-d553-44c4-995e-9d2c1fe9ce5b\") " pod="e2e-models-e2e-worker-w0-08475e/premium-w0-08475e-kserve-647898d665-rmzkp"
Sep 16 14:20:12.498627 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:20:12.498409    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2f76715c-d553-44c4-995e-9d2c1fe9ce5b-dshm\") pod \"premium-w0-08475e-kserve-647898d665-rmzkp\" (UID: \"2f76715c-d553-44c4-995e-9d2c1fe9ce5b\") " pod="e2e-models-e2e-worker-w0-08475e/premium-w0-08475e-kserve-647898d665-rmzkp"
Sep 16 14:20:12.498776 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:20:12.498753    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f76715c-d553-44c4-995e-9d2c1fe9ce5b-tmp-dir\") pod \"premium-w0-08475e-kserve-647898d665-rmzkp\" (UID: \"2f76715c-d553-44c4-995e-9d2c1fe9ce5b\") " pod="e2e-models-e2e-worker-w0-08475e/premium-w0-08475e-kserve-647898d665-rmzkp"
Sep 16 14:20:12.498856 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:20:12.498806    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2f76715c-d553-44c4-995e-9d2c1fe9ce5b-home\") pod \"premium-w0-08475e-kserve-647898d665-rmzkp\" (UID: \"2f76715c-d553-44c4-995e-9d2c1fe9ce5b\") " pod="e2e-models-e2e-worker-w0-08475e/premium-w0-08475e-kserve-647898d665-rmzkp"
Sep 16 14:20:12.498856 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:20:12.498820    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2f76715c-d553-44c4-995e-9d2c1fe9ce5b-model-cache\") pod \"premium-w0-08475e-kserve-647898d665-rmzkp\" (UID: \"2f76715c-d553-44c4-995e-9d2c1fe9ce5b\") " pod="e2e-models-e2e-worker-w0-08475e/premium-w0-08475e-kserve-647898d665-rmzkp"
Sep 16 14:20:12.500638 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:20:12.500610    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2f76715c-d553-44c4-995e-9d2c1fe9ce5b-dshm\") pod \"premium-w0-08475e-kserve-647898d665-rmzkp\" (UID: \"2f76715c-d553-44c4-995e-9d2c1fe9ce5b\") " pod="e2e-models-e2e-worker-w0-08475e/premium-w0-08475e-kserve-647898d665-rmzkp"
Sep 16 14:20:12.500761 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:20:12.500743    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f76715c-d553-44c4-995e-9d2c1fe9ce5b-tls-certs\") pod \"premium-w0-08475e-kserve-647898d665-rmzkp\" (UID: \"2f76715c-d553-44c4-995e-9d2c1fe9ce5b\") " pod="e2e-models-e2e-worker-w0-08475e/premium-w0-08475e-kserve-647898d665-rmzkp"
Sep 16 14:20:12.506638 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:20:12.506617    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcwdw\" (UniqueName: \"kubernetes.io/projected/2f76715c-d553-44c4-995e-9d2c1fe9ce5b-kube-api-access-qcwdw\") pod \"premium-w0-08475e-kserve-647898d665-rmzkp\" (UID: \"2f76715c-d553-44c4-995e-9d2c1fe9ce5b\") " pod="e2e-models-e2e-worker-w0-08475e/premium-w0-08475e-kserve-647898d665-rmzkp"
Sep 16 14:20:12.606397 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:20:12.606300    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-08475e/premium-w0-08475e-kserve-647898d665-rmzkp"
Sep 16 14:20:12.881185 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:20:12.881157    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-08475e/premium-w0-08475e-kserve-647898d665-rmzkp"]
Sep 16 14:20:12.882672 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:20:12.882646    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2f76715c_d553_44c4_995e_9d2c1fe9ce5b.slice/crio-d98450d2e1a7692b5ddae326080066f91c235e8787041e8e90c56e6c3dcac4c6 WatchSource:0}: Error finding container d98450d2e1a7692b5ddae326080066f91c235e8787041e8e90c56e6c3dcac4c6: Status 404 returned error can't find the container with id d98450d2e1a7692b5ddae326080066f91c235e8787041e8e90c56e6c3dcac4c6
Sep 16 14:20:13.772098 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:20:13.772067    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-08475e/premium-w0-08475e-kserve-647898d665-rmzkp" event={"ID":"2f76715c-d553-44c4-995e-9d2c1fe9ce5b","Type":"ContainerStarted","Data":"4e2e075bca10b0cd17c0e96114130eb1da18bf3ba999a0b09a0f522c0bd4b0c6"}
Sep 16 14:20:13.772485 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:20:13.772107    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-08475e/premium-w0-08475e-kserve-647898d665-rmzkp" event={"ID":"2f76715c-d553-44c4-995e-9d2c1fe9ce5b","Type":"ContainerStarted","Data":"d98450d2e1a7692b5ddae326080066f91c235e8787041e8e90c56e6c3dcac4c6"}
Sep 16 14:20:13.791861 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:20:13.791811    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-08475e/premium-w0-08475e-kserve-647898d665-rmzkp" podStartSLOduration=1.791795491 podStartE2EDuration="1.791795491s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:20:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:20:13.789303881 +0000 UTC m=+946.804413743" watchObservedRunningTime="2026-09-16 14:20:13.791795491 +0000 UTC m=+946.806905352"
Sep 16 14:20:22.606923 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:20:22.606878    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-08475e/premium-w0-08475e-kserve-647898d665-rmzkp"
Sep 16 14:20:22.607366 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:20:22.607183    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-08475e/premium-w0-08475e-kserve-647898d665-rmzkp"
Sep 16 14:20:22.620783 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:20:22.620760    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-08475e/premium-w0-08475e-kserve-647898d665-rmzkp"
Sep 16 14:20:22.834810 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:20:22.834783    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-08475e/premium-w0-08475e-kserve-647898d665-rmzkp"
Sep 16 14:21:04.316329 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:21:04.316243    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-2ad565/distinct-w3-2ad565-kserve-7c9d64d6cf-drp8b"]
Sep 16 14:21:04.319842 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:21:04.319819    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-2ad565/distinct-w3-2ad565-kserve-7c9d64d6cf-drp8b"
Sep 16 14:21:04.323715 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:21:04.323170    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-2ad565\"/\"openshift-service-ca.crt\""
Sep 16 14:21:04.323715 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:21:04.323222    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-2ad565\"/\"distinct-w3-2ad565-kserve-self-signed-certs\""
Sep 16 14:21:04.323715 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:21:04.323314    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-2ad565\"/\"default-dockercfg-f8lkf\""
Sep 16 14:21:04.323715 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:21:04.323450    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-2ad565\"/\"kube-root-ca.crt\""
Sep 16 14:21:04.337021 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:21:04.336993    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-2ad565/distinct-w3-2ad565-kserve-7c9d64d6cf-drp8b"]
Sep 16 14:21:04.413865 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:21:04.413834    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/786b6c75-8be7-4ae5-8719-82847428ecf7-tls-certs\") pod \"distinct-w3-2ad565-kserve-7c9d64d6cf-drp8b\" (UID: \"786b6c75-8be7-4ae5-8719-82847428ecf7\") " pod="e2e-models-e2e-worker-w3-2ad565/distinct-w3-2ad565-kserve-7c9d64d6cf-drp8b"
Sep 16 14:21:04.413865 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:21:04.413869    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/786b6c75-8be7-4ae5-8719-82847428ecf7-home\") pod \"distinct-w3-2ad565-kserve-7c9d64d6cf-drp8b\" (UID: \"786b6c75-8be7-4ae5-8719-82847428ecf7\") " pod="e2e-models-e2e-worker-w3-2ad565/distinct-w3-2ad565-kserve-7c9d64d6cf-drp8b"
Sep 16 14:21:04.414097 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:21:04.413890    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/786b6c75-8be7-4ae5-8719-82847428ecf7-tmp-dir\") pod \"distinct-w3-2ad565-kserve-7c9d64d6cf-drp8b\" (UID: \"786b6c75-8be7-4ae5-8719-82847428ecf7\") " pod="e2e-models-e2e-worker-w3-2ad565/distinct-w3-2ad565-kserve-7c9d64d6cf-drp8b"
Sep 16 14:21:04.414097 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:21:04.413922    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/786b6c75-8be7-4ae5-8719-82847428ecf7-dshm\") pod \"distinct-w3-2ad565-kserve-7c9d64d6cf-drp8b\" (UID: \"786b6c75-8be7-4ae5-8719-82847428ecf7\") " pod="e2e-models-e2e-worker-w3-2ad565/distinct-w3-2ad565-kserve-7c9d64d6cf-drp8b"
Sep 16 14:21:04.414097 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:21:04.413979    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxrdv\" (UniqueName: \"kubernetes.io/projected/786b6c75-8be7-4ae5-8719-82847428ecf7-kube-api-access-qxrdv\") pod \"distinct-w3-2ad565-kserve-7c9d64d6cf-drp8b\" (UID: \"786b6c75-8be7-4ae5-8719-82847428ecf7\") " pod="e2e-models-e2e-worker-w3-2ad565/distinct-w3-2ad565-kserve-7c9d64d6cf-drp8b"
Sep 16 14:21:04.414097 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:21:04.414009    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/786b6c75-8be7-4ae5-8719-82847428ecf7-model-cache\") pod \"distinct-w3-2ad565-kserve-7c9d64d6cf-drp8b\" (UID: \"786b6c75-8be7-4ae5-8719-82847428ecf7\") " pod="e2e-models-e2e-worker-w3-2ad565/distinct-w3-2ad565-kserve-7c9d64d6cf-drp8b"
Sep 16 14:21:04.515057 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:21:04.515018    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxrdv\" (UniqueName: \"kubernetes.io/projected/786b6c75-8be7-4ae5-8719-82847428ecf7-kube-api-access-qxrdv\") pod \"distinct-w3-2ad565-kserve-7c9d64d6cf-drp8b\" (UID: \"786b6c75-8be7-4ae5-8719-82847428ecf7\") " pod="e2e-models-e2e-worker-w3-2ad565/distinct-w3-2ad565-kserve-7c9d64d6cf-drp8b"
Sep 16 14:21:04.515265 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:21:04.515073    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/786b6c75-8be7-4ae5-8719-82847428ecf7-model-cache\") pod \"distinct-w3-2ad565-kserve-7c9d64d6cf-drp8b\" (UID: \"786b6c75-8be7-4ae5-8719-82847428ecf7\") " pod="e2e-models-e2e-worker-w3-2ad565/distinct-w3-2ad565-kserve-7c9d64d6cf-drp8b"
Sep 16 14:21:04.515265 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:21:04.515166    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/786b6c75-8be7-4ae5-8719-82847428ecf7-tls-certs\") pod \"distinct-w3-2ad565-kserve-7c9d64d6cf-drp8b\" (UID: \"786b6c75-8be7-4ae5-8719-82847428ecf7\") " pod="e2e-models-e2e-worker-w3-2ad565/distinct-w3-2ad565-kserve-7c9d64d6cf-drp8b"
Sep 16 14:21:04.515265 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:21:04.515194    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/786b6c75-8be7-4ae5-8719-82847428ecf7-home\") pod \"distinct-w3-2ad565-kserve-7c9d64d6cf-drp8b\" (UID: \"786b6c75-8be7-4ae5-8719-82847428ecf7\") " pod="e2e-models-e2e-worker-w3-2ad565/distinct-w3-2ad565-kserve-7c9d64d6cf-drp8b"
Sep 16 14:21:04.515265 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:21:04.515220    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/786b6c75-8be7-4ae5-8719-82847428ecf7-tmp-dir\") pod \"distinct-w3-2ad565-kserve-7c9d64d6cf-drp8b\" (UID: \"786b6c75-8be7-4ae5-8719-82847428ecf7\") " pod="e2e-models-e2e-worker-w3-2ad565/distinct-w3-2ad565-kserve-7c9d64d6cf-drp8b"
Sep 16 14:21:04.515265 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:21:04.515244    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/786b6c75-8be7-4ae5-8719-82847428ecf7-dshm\") pod \"distinct-w3-2ad565-kserve-7c9d64d6cf-drp8b\" (UID: \"786b6c75-8be7-4ae5-8719-82847428ecf7\") " pod="e2e-models-e2e-worker-w3-2ad565/distinct-w3-2ad565-kserve-7c9d64d6cf-drp8b"
Sep 16 14:21:04.515655 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:21:04.515629    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/786b6c75-8be7-4ae5-8719-82847428ecf7-model-cache\") pod \"distinct-w3-2ad565-kserve-7c9d64d6cf-drp8b\" (UID: \"786b6c75-8be7-4ae5-8719-82847428ecf7\") " pod="e2e-models-e2e-worker-w3-2ad565/distinct-w3-2ad565-kserve-7c9d64d6cf-drp8b"
Sep 16 14:21:04.515792 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:21:04.515728    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/786b6c75-8be7-4ae5-8719-82847428ecf7-tmp-dir\") pod \"distinct-w3-2ad565-kserve-7c9d64d6cf-drp8b\" (UID: \"786b6c75-8be7-4ae5-8719-82847428ecf7\") " pod="e2e-models-e2e-worker-w3-2ad565/distinct-w3-2ad565-kserve-7c9d64d6cf-drp8b"
Sep 16 14:21:04.515792 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:21:04.515685    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/786b6c75-8be7-4ae5-8719-82847428ecf7-home\") pod \"distinct-w3-2ad565-kserve-7c9d64d6cf-drp8b\" (UID: \"786b6c75-8be7-4ae5-8719-82847428ecf7\") " pod="e2e-models-e2e-worker-w3-2ad565/distinct-w3-2ad565-kserve-7c9d64d6cf-drp8b"
Sep 16 14:21:04.517664 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:21:04.517642    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/786b6c75-8be7-4ae5-8719-82847428ecf7-dshm\") pod \"distinct-w3-2ad565-kserve-7c9d64d6cf-drp8b\" (UID: \"786b6c75-8be7-4ae5-8719-82847428ecf7\") " pod="e2e-models-e2e-worker-w3-2ad565/distinct-w3-2ad565-kserve-7c9d64d6cf-drp8b"
Sep 16 14:21:04.517872 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:21:04.517855    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/786b6c75-8be7-4ae5-8719-82847428ecf7-tls-certs\") pod \"distinct-w3-2ad565-kserve-7c9d64d6cf-drp8b\" (UID: \"786b6c75-8be7-4ae5-8719-82847428ecf7\") " pod="e2e-models-e2e-worker-w3-2ad565/distinct-w3-2ad565-kserve-7c9d64d6cf-drp8b"
Sep 16 14:21:04.523733 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:21:04.523707    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxrdv\" (UniqueName: \"kubernetes.io/projected/786b6c75-8be7-4ae5-8719-82847428ecf7-kube-api-access-qxrdv\") pod \"distinct-w3-2ad565-kserve-7c9d64d6cf-drp8b\" (UID: \"786b6c75-8be7-4ae5-8719-82847428ecf7\") " pod="e2e-models-e2e-worker-w3-2ad565/distinct-w3-2ad565-kserve-7c9d64d6cf-drp8b"
Sep 16 14:21:04.630615 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:21:04.630527    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-2ad565/distinct-w3-2ad565-kserve-7c9d64d6cf-drp8b"
Sep 16 14:21:04.913804 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:21:04.913777    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-2ad565/distinct-w3-2ad565-kserve-7c9d64d6cf-drp8b"]
Sep 16 14:21:04.916534 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:21:04.916487    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod786b6c75_8be7_4ae5_8719_82847428ecf7.slice/crio-49fafacb2179f96491f5755fdbd0c863fc02c5a6e38fa8256ab6deb4dd6b8f2b WatchSource:0}: Error finding container 49fafacb2179f96491f5755fdbd0c863fc02c5a6e38fa8256ab6deb4dd6b8f2b: Status 404 returned error can't find the container with id 49fafacb2179f96491f5755fdbd0c863fc02c5a6e38fa8256ab6deb4dd6b8f2b
Sep 16 14:21:05.049998 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:21:05.049964    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-2ad565/distinct-w3-2ad565-kserve-7c9d64d6cf-drp8b" event={"ID":"786b6c75-8be7-4ae5-8719-82847428ecf7","Type":"ContainerStarted","Data":"c326d8b75ceb8521d288ea2f4626afcaf8c433397153dd170300033ed1358b24"}
Sep 16 14:21:05.049998 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:21:05.050005    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-2ad565/distinct-w3-2ad565-kserve-7c9d64d6cf-drp8b" event={"ID":"786b6c75-8be7-4ae5-8719-82847428ecf7","Type":"ContainerStarted","Data":"49fafacb2179f96491f5755fdbd0c863fc02c5a6e38fa8256ab6deb4dd6b8f2b"}
Sep 16 14:21:05.072376 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:21:05.072324    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-2ad565/distinct-w3-2ad565-kserve-7c9d64d6cf-drp8b" podStartSLOduration=1.072306041 podStartE2EDuration="1.072306041s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:21:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:21:05.068996034 +0000 UTC m=+998.084105929" watchObservedRunningTime="2026-09-16 14:21:05.072306041 +0000 UTC m=+998.087415900"
Sep 16 14:21:14.631354 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:21:14.631313    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-2ad565/distinct-w3-2ad565-kserve-7c9d64d6cf-drp8b"
Sep 16 14:21:14.631354 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:21:14.631365    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-2ad565/distinct-w3-2ad565-kserve-7c9d64d6cf-drp8b"
Sep 16 14:21:14.643959 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:21:14.643930    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-2ad565/distinct-w3-2ad565-kserve-7c9d64d6cf-drp8b"
Sep 16 14:21:15.124486 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:21:15.124460    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-2ad565/distinct-w3-2ad565-kserve-7c9d64d6cf-drp8b"
Sep 16 14:22:28.071463 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:22:28.071390    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-08475e/distinct-2-w0-08475e-kserve-66c89bc459-6fc57"]
Sep 16 14:22:28.074641 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:22:28.074622    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-08475e/distinct-2-w0-08475e-kserve-66c89bc459-6fc57"
Sep 16 14:22:28.077302 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:22:28.077280    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-08475e\"/\"distinct-2-w0-08475e-kserve-self-signed-certs\""
Sep 16 14:22:28.094432 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:22:28.094411    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-08475e/distinct-2-w0-08475e-kserve-66c89bc459-6fc57"]
Sep 16 14:22:28.223855 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:22:28.223826    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/da00cc8d-910a-4c6e-86c7-ca474c5e819c-dshm\") pod \"distinct-2-w0-08475e-kserve-66c89bc459-6fc57\" (UID: \"da00cc8d-910a-4c6e-86c7-ca474c5e819c\") " pod="e2e-models-e2e-worker-w0-08475e/distinct-2-w0-08475e-kserve-66c89bc459-6fc57"
Sep 16 14:22:28.223855 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:22:28.223855    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/da00cc8d-910a-4c6e-86c7-ca474c5e819c-model-cache\") pod \"distinct-2-w0-08475e-kserve-66c89bc459-6fc57\" (UID: \"da00cc8d-910a-4c6e-86c7-ca474c5e819c\") " pod="e2e-models-e2e-worker-w0-08475e/distinct-2-w0-08475e-kserve-66c89bc459-6fc57"
Sep 16 14:22:28.224055 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:22:28.223927    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/da00cc8d-910a-4c6e-86c7-ca474c5e819c-tmp-dir\") pod \"distinct-2-w0-08475e-kserve-66c89bc459-6fc57\" (UID: \"da00cc8d-910a-4c6e-86c7-ca474c5e819c\") " pod="e2e-models-e2e-worker-w0-08475e/distinct-2-w0-08475e-kserve-66c89bc459-6fc57"
Sep 16 14:22:28.224055 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:22:28.223946    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6g4x\" (UniqueName: \"kubernetes.io/projected/da00cc8d-910a-4c6e-86c7-ca474c5e819c-kube-api-access-p6g4x\") pod \"distinct-2-w0-08475e-kserve-66c89bc459-6fc57\" (UID: \"da00cc8d-910a-4c6e-86c7-ca474c5e819c\") " pod="e2e-models-e2e-worker-w0-08475e/distinct-2-w0-08475e-kserve-66c89bc459-6fc57"
Sep 16 14:22:28.224055 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:22:28.224011    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/da00cc8d-910a-4c6e-86c7-ca474c5e819c-home\") pod \"distinct-2-w0-08475e-kserve-66c89bc459-6fc57\" (UID: \"da00cc8d-910a-4c6e-86c7-ca474c5e819c\") " pod="e2e-models-e2e-worker-w0-08475e/distinct-2-w0-08475e-kserve-66c89bc459-6fc57"
Sep 16 14:22:28.224055 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:22:28.224050    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/da00cc8d-910a-4c6e-86c7-ca474c5e819c-tls-certs\") pod \"distinct-2-w0-08475e-kserve-66c89bc459-6fc57\" (UID: \"da00cc8d-910a-4c6e-86c7-ca474c5e819c\") " pod="e2e-models-e2e-worker-w0-08475e/distinct-2-w0-08475e-kserve-66c89bc459-6fc57"
Sep 16 14:22:28.325170 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:22:28.325102    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/da00cc8d-910a-4c6e-86c7-ca474c5e819c-tmp-dir\") pod \"distinct-2-w0-08475e-kserve-66c89bc459-6fc57\" (UID: \"da00cc8d-910a-4c6e-86c7-ca474c5e819c\") " pod="e2e-models-e2e-worker-w0-08475e/distinct-2-w0-08475e-kserve-66c89bc459-6fc57"
Sep 16 14:22:28.325170 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:22:28.325139    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6g4x\" (UniqueName: \"kubernetes.io/projected/da00cc8d-910a-4c6e-86c7-ca474c5e819c-kube-api-access-p6g4x\") pod \"distinct-2-w0-08475e-kserve-66c89bc459-6fc57\" (UID: \"da00cc8d-910a-4c6e-86c7-ca474c5e819c\") " pod="e2e-models-e2e-worker-w0-08475e/distinct-2-w0-08475e-kserve-66c89bc459-6fc57"
Sep 16 14:22:28.325170 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:22:28.325168    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/da00cc8d-910a-4c6e-86c7-ca474c5e819c-home\") pod \"distinct-2-w0-08475e-kserve-66c89bc459-6fc57\" (UID: \"da00cc8d-910a-4c6e-86c7-ca474c5e819c\") " pod="e2e-models-e2e-worker-w0-08475e/distinct-2-w0-08475e-kserve-66c89bc459-6fc57"
Sep 16 14:22:28.325385 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:22:28.325190    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/da00cc8d-910a-4c6e-86c7-ca474c5e819c-tls-certs\") pod \"distinct-2-w0-08475e-kserve-66c89bc459-6fc57\" (UID: \"da00cc8d-910a-4c6e-86c7-ca474c5e819c\") " pod="e2e-models-e2e-worker-w0-08475e/distinct-2-w0-08475e-kserve-66c89bc459-6fc57"
Sep 16 14:22:28.325385 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:22:28.325234    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/da00cc8d-910a-4c6e-86c7-ca474c5e819c-dshm\") pod \"distinct-2-w0-08475e-kserve-66c89bc459-6fc57\" (UID: \"da00cc8d-910a-4c6e-86c7-ca474c5e819c\") " pod="e2e-models-e2e-worker-w0-08475e/distinct-2-w0-08475e-kserve-66c89bc459-6fc57"
Sep 16 14:22:28.325385 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:22:28.325250    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/da00cc8d-910a-4c6e-86c7-ca474c5e819c-model-cache\") pod \"distinct-2-w0-08475e-kserve-66c89bc459-6fc57\" (UID: \"da00cc8d-910a-4c6e-86c7-ca474c5e819c\") " pod="e2e-models-e2e-worker-w0-08475e/distinct-2-w0-08475e-kserve-66c89bc459-6fc57"
Sep 16 14:22:28.325592 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:22:28.325572    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/da00cc8d-910a-4c6e-86c7-ca474c5e819c-tmp-dir\") pod \"distinct-2-w0-08475e-kserve-66c89bc459-6fc57\" (UID: \"da00cc8d-910a-4c6e-86c7-ca474c5e819c\") " pod="e2e-models-e2e-worker-w0-08475e/distinct-2-w0-08475e-kserve-66c89bc459-6fc57"
Sep 16 14:22:28.325657 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:22:28.325577    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/da00cc8d-910a-4c6e-86c7-ca474c5e819c-home\") pod \"distinct-2-w0-08475e-kserve-66c89bc459-6fc57\" (UID: \"da00cc8d-910a-4c6e-86c7-ca474c5e819c\") " pod="e2e-models-e2e-worker-w0-08475e/distinct-2-w0-08475e-kserve-66c89bc459-6fc57"
Sep 16 14:22:28.325657 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:22:28.325632    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/da00cc8d-910a-4c6e-86c7-ca474c5e819c-model-cache\") pod \"distinct-2-w0-08475e-kserve-66c89bc459-6fc57\" (UID: \"da00cc8d-910a-4c6e-86c7-ca474c5e819c\") " pod="e2e-models-e2e-worker-w0-08475e/distinct-2-w0-08475e-kserve-66c89bc459-6fc57"
Sep 16 14:22:28.327616 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:22:28.327593    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/da00cc8d-910a-4c6e-86c7-ca474c5e819c-dshm\") pod \"distinct-2-w0-08475e-kserve-66c89bc459-6fc57\" (UID: \"da00cc8d-910a-4c6e-86c7-ca474c5e819c\") " pod="e2e-models-e2e-worker-w0-08475e/distinct-2-w0-08475e-kserve-66c89bc459-6fc57"
Sep 16 14:22:28.327864 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:22:28.327843    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/da00cc8d-910a-4c6e-86c7-ca474c5e819c-tls-certs\") pod \"distinct-2-w0-08475e-kserve-66c89bc459-6fc57\" (UID: \"da00cc8d-910a-4c6e-86c7-ca474c5e819c\") " pod="e2e-models-e2e-worker-w0-08475e/distinct-2-w0-08475e-kserve-66c89bc459-6fc57"
Sep 16 14:22:28.334489 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:22:28.334466    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6g4x\" (UniqueName: \"kubernetes.io/projected/da00cc8d-910a-4c6e-86c7-ca474c5e819c-kube-api-access-p6g4x\") pod \"distinct-2-w0-08475e-kserve-66c89bc459-6fc57\" (UID: \"da00cc8d-910a-4c6e-86c7-ca474c5e819c\") " pod="e2e-models-e2e-worker-w0-08475e/distinct-2-w0-08475e-kserve-66c89bc459-6fc57"
Sep 16 14:22:28.383713 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:22:28.383688    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-08475e/distinct-2-w0-08475e-kserve-66c89bc459-6fc57"
Sep 16 14:22:28.677860 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:22:28.677832    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-08475e/distinct-2-w0-08475e-kserve-66c89bc459-6fc57"]
Sep 16 14:22:28.678919 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:22:28.678878    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podda00cc8d_910a_4c6e_86c7_ca474c5e819c.slice/crio-5401a000fe12e5b120e240eff1d6e921f0c9ebf0cb254aa5162c944f38041753 WatchSource:0}: Error finding container 5401a000fe12e5b120e240eff1d6e921f0c9ebf0cb254aa5162c944f38041753: Status 404 returned error can't find the container with id 5401a000fe12e5b120e240eff1d6e921f0c9ebf0cb254aa5162c944f38041753
Sep 16 14:22:29.536581 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:22:29.536547    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-08475e/distinct-2-w0-08475e-kserve-66c89bc459-6fc57" event={"ID":"da00cc8d-910a-4c6e-86c7-ca474c5e819c","Type":"ContainerStarted","Data":"955ffbd85ac1d5d3915802e700a8a2784a64c486e94549a39ab78ad50bdefbf1"}
Sep 16 14:22:29.536581 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:22:29.536587    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-08475e/distinct-2-w0-08475e-kserve-66c89bc459-6fc57" event={"ID":"da00cc8d-910a-4c6e-86c7-ca474c5e819c","Type":"ContainerStarted","Data":"5401a000fe12e5b120e240eff1d6e921f0c9ebf0cb254aa5162c944f38041753"}
Sep 16 14:22:29.566777 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:22:29.566732    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-08475e/distinct-2-w0-08475e-kserve-66c89bc459-6fc57" podStartSLOduration=1.5667170910000001 podStartE2EDuration="1.566717091s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:22:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:22:29.566329961 +0000 UTC m=+1082.581439821" watchObservedRunningTime="2026-09-16 14:22:29.566717091 +0000 UTC m=+1082.581826976"
Sep 16 14:22:38.384673 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:22:38.384638    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-08475e/distinct-2-w0-08475e-kserve-66c89bc459-6fc57"
Sep 16 14:22:38.385056 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:22:38.384723    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-08475e/distinct-2-w0-08475e-kserve-66c89bc459-6fc57"
Sep 16 14:22:38.397982 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:22:38.397956    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-08475e/distinct-2-w0-08475e-kserve-66c89bc459-6fc57"
Sep 16 14:22:38.596660 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:22:38.596633    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-08475e/distinct-2-w0-08475e-kserve-66c89bc459-6fc57"
Sep 16 14:23:16.919756 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:23:16.919721    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-e24c87-5cfb444bd7-gwg85"]
Sep 16 14:23:16.920251 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:23:16.919970    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-e24c87-5cfb444bd7-gwg85" podUID="4dd9af77-6514-477e-b2c7-4d6fc86f856f" containerName="payload-processing" containerID="cri-o://ad5b4f37e89759c4a24bde1d5277421196defd7c4e8a3f6293a709e2cea0f59a" gracePeriod=30
Sep 16 14:23:17.182675 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:23:17.182651    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-e24c87-5cfb444bd7-gwg85"
Sep 16 14:23:17.201430 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:23:17.201408    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4dd9af77-6514-477e-b2c7-4d6fc86f856f-plugins-config-volume\") pod \"4dd9af77-6514-477e-b2c7-4d6fc86f856f\" (UID: \"4dd9af77-6514-477e-b2c7-4d6fc86f856f\") "
Sep 16 14:23:17.201562 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:23:17.201443    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ns8qm\" (UniqueName: \"kubernetes.io/projected/4dd9af77-6514-477e-b2c7-4d6fc86f856f-kube-api-access-ns8qm\") pod \"4dd9af77-6514-477e-b2c7-4d6fc86f856f\" (UID: \"4dd9af77-6514-477e-b2c7-4d6fc86f856f\") "
Sep 16 14:23:17.201866 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:23:17.201837    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4dd9af77-6514-477e-b2c7-4d6fc86f856f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4dd9af77-6514-477e-b2c7-4d6fc86f856f" (UID: "4dd9af77-6514-477e-b2c7-4d6fc86f856f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:23:17.203408 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:23:17.203389    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4dd9af77-6514-477e-b2c7-4d6fc86f856f-kube-api-access-ns8qm" (OuterVolumeSpecName: "kube-api-access-ns8qm") pod "4dd9af77-6514-477e-b2c7-4d6fc86f856f" (UID: "4dd9af77-6514-477e-b2c7-4d6fc86f856f"). InnerVolumeSpecName "kube-api-access-ns8qm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:23:17.302651 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:23:17.302622    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4dd9af77-6514-477e-b2c7-4d6fc86f856f-plugins-config-volume\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:23:17.302651 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:23:17.302647    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ns8qm\" (UniqueName: \"kubernetes.io/projected/4dd9af77-6514-477e-b2c7-4d6fc86f856f-kube-api-access-ns8qm\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:23:17.804300 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:23:17.804223    2580 generic.go:356] "Generic (PLEG): container finished" podID="4dd9af77-6514-477e-b2c7-4d6fc86f856f" containerID="ad5b4f37e89759c4a24bde1d5277421196defd7c4e8a3f6293a709e2cea0f59a" exitCode=0
Sep 16 14:23:17.804300 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:23:17.804276    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-e24c87-5cfb444bd7-gwg85"
Sep 16 14:23:17.804477 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:23:17.804316    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-e24c87-5cfb444bd7-gwg85" event={"ID":"4dd9af77-6514-477e-b2c7-4d6fc86f856f","Type":"ContainerDied","Data":"ad5b4f37e89759c4a24bde1d5277421196defd7c4e8a3f6293a709e2cea0f59a"}
Sep 16 14:23:17.804477 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:23:17.804377    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-e24c87-5cfb444bd7-gwg85" event={"ID":"4dd9af77-6514-477e-b2c7-4d6fc86f856f","Type":"ContainerDied","Data":"4b5433bc0f338d1d025e44ebf75b0617ee7d18e2a2f1144969b3192940cee79b"}
Sep 16 14:23:17.804477 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:23:17.804403    2580 scope.go:117] "RemoveContainer" containerID="ad5b4f37e89759c4a24bde1d5277421196defd7c4e8a3f6293a709e2cea0f59a"
Sep 16 14:23:17.820238 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:23:17.820213    2580 scope.go:117] "RemoveContainer" containerID="ad5b4f37e89759c4a24bde1d5277421196defd7c4e8a3f6293a709e2cea0f59a"
Sep 16 14:23:17.820626 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:23:17.820604    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad5b4f37e89759c4a24bde1d5277421196defd7c4e8a3f6293a709e2cea0f59a\": container with ID starting with ad5b4f37e89759c4a24bde1d5277421196defd7c4e8a3f6293a709e2cea0f59a not found: ID does not exist" containerID="ad5b4f37e89759c4a24bde1d5277421196defd7c4e8a3f6293a709e2cea0f59a"
Sep 16 14:23:17.820742 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:23:17.820638    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad5b4f37e89759c4a24bde1d5277421196defd7c4e8a3f6293a709e2cea0f59a"} err="failed to get container status \"ad5b4f37e89759c4a24bde1d5277421196defd7c4e8a3f6293a709e2cea0f59a\": rpc error: code = NotFound desc = could not find container \"ad5b4f37e89759c4a24bde1d5277421196defd7c4e8a3f6293a709e2cea0f59a\": container with ID starting with ad5b4f37e89759c4a24bde1d5277421196defd7c4e8a3f6293a709e2cea0f59a not found: ID does not exist"
Sep 16 14:23:17.839946 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:23:17.839922    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-e24c87-5cfb444bd7-gwg85"]
Sep 16 14:23:17.846938 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:23:17.846912    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-e24c87-5cfb444bd7-gwg85"]
Sep 16 14:23:19.430583 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:23:19.430552    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4dd9af77-6514-477e-b2c7-4d6fc86f856f" path="/var/lib/kubelet/pods/4dd9af77-6514-477e-b2c7-4d6fc86f856f/volumes"
Sep 16 14:23:38.065366 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:23:38.065327    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-46e82d-57b5878894-8m2c8"]
Sep 16 14:23:38.068108 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:23:38.065616    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-46e82d-57b5878894-8m2c8" podUID="d71900a3-4f93-4e27-adac-51ea4e4413c0" containerName="payload-pre-processing" containerID="cri-o://6d65214d2ab61ed4a0bdddf129b27a6be3773b5221535441dd1b006aa3ff5665" gracePeriod=30
Sep 16 14:23:38.350071 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:23:38.350046    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-46e82d-57b5878894-8m2c8"
Sep 16 14:23:38.517924 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:23:38.517885    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6kxbz\" (UniqueName: \"kubernetes.io/projected/d71900a3-4f93-4e27-adac-51ea4e4413c0-kube-api-access-6kxbz\") pod \"d71900a3-4f93-4e27-adac-51ea4e4413c0\" (UID: \"d71900a3-4f93-4e27-adac-51ea4e4413c0\") "
Sep 16 14:23:38.518095 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:23:38.517955    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d71900a3-4f93-4e27-adac-51ea4e4413c0-plugins-config-volume\") pod \"d71900a3-4f93-4e27-adac-51ea4e4413c0\" (UID: \"d71900a3-4f93-4e27-adac-51ea4e4413c0\") "
Sep 16 14:23:38.518352 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:23:38.518304    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d71900a3-4f93-4e27-adac-51ea4e4413c0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d71900a3-4f93-4e27-adac-51ea4e4413c0" (UID: "d71900a3-4f93-4e27-adac-51ea4e4413c0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:23:38.518522 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:23:38.518415    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d71900a3-4f93-4e27-adac-51ea4e4413c0-plugins-config-volume\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:23:38.520136 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:23:38.520113    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d71900a3-4f93-4e27-adac-51ea4e4413c0-kube-api-access-6kxbz" (OuterVolumeSpecName: "kube-api-access-6kxbz") pod "d71900a3-4f93-4e27-adac-51ea4e4413c0" (UID: "d71900a3-4f93-4e27-adac-51ea4e4413c0"). InnerVolumeSpecName "kube-api-access-6kxbz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:23:38.619653 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:23:38.619578    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6kxbz\" (UniqueName: \"kubernetes.io/projected/d71900a3-4f93-4e27-adac-51ea4e4413c0-kube-api-access-6kxbz\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:23:38.929286 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:23:38.929247    2580 generic.go:356] "Generic (PLEG): container finished" podID="d71900a3-4f93-4e27-adac-51ea4e4413c0" containerID="6d65214d2ab61ed4a0bdddf129b27a6be3773b5221535441dd1b006aa3ff5665" exitCode=0
Sep 16 14:23:38.929471 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:23:38.929298    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-46e82d-57b5878894-8m2c8"
Sep 16 14:23:38.929471 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:23:38.929337    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-46e82d-57b5878894-8m2c8" event={"ID":"d71900a3-4f93-4e27-adac-51ea4e4413c0","Type":"ContainerDied","Data":"6d65214d2ab61ed4a0bdddf129b27a6be3773b5221535441dd1b006aa3ff5665"}
Sep 16 14:23:38.929471 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:23:38.929404    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-46e82d-57b5878894-8m2c8" event={"ID":"d71900a3-4f93-4e27-adac-51ea4e4413c0","Type":"ContainerDied","Data":"27ea9d76f492d8428490c5c143b3f344f7084fe1fc7476f05d315d8bff301355"}
Sep 16 14:23:38.929471 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:23:38.929436    2580 scope.go:117] "RemoveContainer" containerID="6d65214d2ab61ed4a0bdddf129b27a6be3773b5221535441dd1b006aa3ff5665"
Sep 16 14:23:38.939410 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:23:38.939389    2580 scope.go:117] "RemoveContainer" containerID="6d65214d2ab61ed4a0bdddf129b27a6be3773b5221535441dd1b006aa3ff5665"
Sep 16 14:23:38.939749 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:23:38.939729    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d65214d2ab61ed4a0bdddf129b27a6be3773b5221535441dd1b006aa3ff5665\": container with ID starting with 6d65214d2ab61ed4a0bdddf129b27a6be3773b5221535441dd1b006aa3ff5665 not found: ID does not exist" containerID="6d65214d2ab61ed4a0bdddf129b27a6be3773b5221535441dd1b006aa3ff5665"
Sep 16 14:23:38.939834 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:23:38.939759    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d65214d2ab61ed4a0bdddf129b27a6be3773b5221535441dd1b006aa3ff5665"} err="failed to get container status \"6d65214d2ab61ed4a0bdddf129b27a6be3773b5221535441dd1b006aa3ff5665\": rpc error: code = NotFound desc = could not find container \"6d65214d2ab61ed4a0bdddf129b27a6be3773b5221535441dd1b006aa3ff5665\": container with ID starting with 6d65214d2ab61ed4a0bdddf129b27a6be3773b5221535441dd1b006aa3ff5665 not found: ID does not exist"
Sep 16 14:23:38.963345 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:23:38.963322    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-46e82d-57b5878894-8m2c8"]
Sep 16 14:23:38.969423 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:23:38.969401    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-46e82d-57b5878894-8m2c8"]
Sep 16 14:23:39.430898 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:23:39.430863    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d71900a3-4f93-4e27-adac-51ea4e4413c0" path="/var/lib/kubelet/pods/d71900a3-4f93-4e27-adac-51ea4e4413c0/volumes"
Sep 16 14:23:58.291610 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:23:58.291526    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-50b57c-57b44f6f78-5d29x"]
Sep 16 14:23:58.292055 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:23:58.291956    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4dd9af77-6514-477e-b2c7-4d6fc86f856f" containerName="payload-processing"
Sep 16 14:23:58.292055 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:23:58.291968    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4dd9af77-6514-477e-b2c7-4d6fc86f856f" containerName="payload-processing"
Sep 16 14:23:58.292055 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:23:58.291983    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d71900a3-4f93-4e27-adac-51ea4e4413c0" containerName="payload-pre-processing"
Sep 16 14:23:58.292055 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:23:58.291989    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d71900a3-4f93-4e27-adac-51ea4e4413c0" containerName="payload-pre-processing"
Sep 16 14:23:58.292055 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:23:58.292048    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4dd9af77-6514-477e-b2c7-4d6fc86f856f" containerName="payload-processing"
Sep 16 14:23:58.292055 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:23:58.292057    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d71900a3-4f93-4e27-adac-51ea4e4413c0" containerName="payload-pre-processing"
Sep 16 14:23:58.295104 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:23:58.295088    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-50b57c-57b44f6f78-5d29x"
Sep 16 14:23:58.300567 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:23:58.300545    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-50b57c-dockercfg-pgd9v\""
Sep 16 14:23:58.300698 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:23:58.300675    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-50b57c\""
Sep 16 14:23:58.317277 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:23:58.317256    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-50b57c-57b44f6f78-5d29x"]
Sep 16 14:23:58.404656 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:23:58.404620    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ab77b2d-c006-43b0-b94f-ec5fbfda3f62-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-50b57c-57b44f6f78-5d29x\" (UID: \"7ab77b2d-c006-43b0-b94f-ec5fbfda3f62\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-50b57c-57b44f6f78-5d29x"
Sep 16 14:23:58.404810 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:23:58.404754    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fr775\" (UniqueName: \"kubernetes.io/projected/7ab77b2d-c006-43b0-b94f-ec5fbfda3f62-kube-api-access-fr775\") pod \"payload-pre-processing-e2e-shared-a-w2-50b57c-57b44f6f78-5d29x\" (UID: \"7ab77b2d-c006-43b0-b94f-ec5fbfda3f62\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-50b57c-57b44f6f78-5d29x"
Sep 16 14:23:58.506146 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:23:58.506111    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fr775\" (UniqueName: \"kubernetes.io/projected/7ab77b2d-c006-43b0-b94f-ec5fbfda3f62-kube-api-access-fr775\") pod \"payload-pre-processing-e2e-shared-a-w2-50b57c-57b44f6f78-5d29x\" (UID: \"7ab77b2d-c006-43b0-b94f-ec5fbfda3f62\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-50b57c-57b44f6f78-5d29x"
Sep 16 14:23:58.506347 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:23:58.506184    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ab77b2d-c006-43b0-b94f-ec5fbfda3f62-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-50b57c-57b44f6f78-5d29x\" (UID: \"7ab77b2d-c006-43b0-b94f-ec5fbfda3f62\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-50b57c-57b44f6f78-5d29x"
Sep 16 14:23:58.507008 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:23:58.506987    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ab77b2d-c006-43b0-b94f-ec5fbfda3f62-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-50b57c-57b44f6f78-5d29x\" (UID: \"7ab77b2d-c006-43b0-b94f-ec5fbfda3f62\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-50b57c-57b44f6f78-5d29x"
Sep 16 14:23:58.521452 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:23:58.521423    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fr775\" (UniqueName: \"kubernetes.io/projected/7ab77b2d-c006-43b0-b94f-ec5fbfda3f62-kube-api-access-fr775\") pod \"payload-pre-processing-e2e-shared-a-w2-50b57c-57b44f6f78-5d29x\" (UID: \"7ab77b2d-c006-43b0-b94f-ec5fbfda3f62\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-50b57c-57b44f6f78-5d29x"
Sep 16 14:23:58.605912 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:23:58.605841    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-50b57c-57b44f6f78-5d29x"
Sep 16 14:23:58.898494 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:23:58.898470    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-50b57c-57b44f6f78-5d29x"]
Sep 16 14:23:58.900332 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:23:58.900298    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ab77b2d_c006_43b0_b94f_ec5fbfda3f62.slice/crio-be404a21e1384be38084d1816e4618706b5b65b60bf52b3186d09f18a66f0a76 WatchSource:0}: Error finding container be404a21e1384be38084d1816e4618706b5b65b60bf52b3186d09f18a66f0a76: Status 404 returned error can't find the container with id be404a21e1384be38084d1816e4618706b5b65b60bf52b3186d09f18a66f0a76
Sep 16 14:23:59.055205 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:23:59.055170    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-50b57c-57b44f6f78-5d29x" event={"ID":"7ab77b2d-c006-43b0-b94f-ec5fbfda3f62","Type":"ContainerStarted","Data":"4f3699b916166cb2ab2dbb579183c3542a2c8e788049de1ed48c1cf21177b244"}
Sep 16 14:23:59.055411 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:23:59.055227    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-50b57c-57b44f6f78-5d29x" event={"ID":"7ab77b2d-c006-43b0-b94f-ec5fbfda3f62","Type":"ContainerStarted","Data":"be404a21e1384be38084d1816e4618706b5b65b60bf52b3186d09f18a66f0a76"}
Sep 16 14:23:59.055411 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:23:59.055294    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-50b57c-57b44f6f78-5d29x"
Sep 16 14:23:59.098289 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:23:59.098233    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-50b57c-57b44f6f78-5d29x" podStartSLOduration=1.098215873 podStartE2EDuration="1.098215873s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:23:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:23:59.09660225 +0000 UTC m=+1172.111712109" watchObservedRunningTime="2026-09-16 14:23:59.098215873 +0000 UTC m=+1172.113325711"
Sep 16 14:24:10.062984 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:24:10.062951    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-50b57c-57b44f6f78-5d29x"
Sep 16 14:24:27.475381 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:24:27.475347    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-f72402-5f595b4b77-j97vl"]
Sep 16 14:24:27.480010 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:24:27.479987    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f72402-5f595b4b77-j97vl"
Sep 16 14:24:27.483598 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:24:27.483555    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-f72402-dockercfg-ph7t8\""
Sep 16 14:24:27.483698 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:24:27.483580    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-f72402\""
Sep 16 14:24:27.499134 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:24:27.499105    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-f72402-5f595b4b77-j97vl"]
Sep 16 14:24:27.603135 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:24:27.603103    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/118c3a58-d101-4668-9da2-e9a824500df4-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-f72402-5f595b4b77-j97vl\" (UID: \"118c3a58-d101-4668-9da2-e9a824500df4\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f72402-5f595b4b77-j97vl"
Sep 16 14:24:27.603298 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:24:27.603157    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fxcf\" (UniqueName: \"kubernetes.io/projected/118c3a58-d101-4668-9da2-e9a824500df4-kube-api-access-6fxcf\") pod \"payload-processing-e2e-shared-b-w2-f72402-5f595b4b77-j97vl\" (UID: \"118c3a58-d101-4668-9da2-e9a824500df4\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f72402-5f595b4b77-j97vl"
Sep 16 14:24:27.703904 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:24:27.703874    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/118c3a58-d101-4668-9da2-e9a824500df4-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-f72402-5f595b4b77-j97vl\" (UID: \"118c3a58-d101-4668-9da2-e9a824500df4\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f72402-5f595b4b77-j97vl"
Sep 16 14:24:27.704058 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:24:27.703930    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fxcf\" (UniqueName: \"kubernetes.io/projected/118c3a58-d101-4668-9da2-e9a824500df4-kube-api-access-6fxcf\") pod \"payload-processing-e2e-shared-b-w2-f72402-5f595b4b77-j97vl\" (UID: \"118c3a58-d101-4668-9da2-e9a824500df4\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f72402-5f595b4b77-j97vl"
Sep 16 14:24:27.704570 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:24:27.704552    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/118c3a58-d101-4668-9da2-e9a824500df4-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-f72402-5f595b4b77-j97vl\" (UID: \"118c3a58-d101-4668-9da2-e9a824500df4\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f72402-5f595b4b77-j97vl"
Sep 16 14:24:27.722591 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:24:27.722527    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fxcf\" (UniqueName: \"kubernetes.io/projected/118c3a58-d101-4668-9da2-e9a824500df4-kube-api-access-6fxcf\") pod \"payload-processing-e2e-shared-b-w2-f72402-5f595b4b77-j97vl\" (UID: \"118c3a58-d101-4668-9da2-e9a824500df4\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f72402-5f595b4b77-j97vl"
Sep 16 14:24:27.790742 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:24:27.790713    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f72402-5f595b4b77-j97vl"
Sep 16 14:24:28.103075 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:24:28.103052    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-f72402-5f595b4b77-j97vl"]
Sep 16 14:24:28.104898 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:24:28.104865    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod118c3a58_d101_4668_9da2_e9a824500df4.slice/crio-0b0d8f69f691fd283906962059e3847e61407e49b39325682a235bb2dd7d407b WatchSource:0}: Error finding container 0b0d8f69f691fd283906962059e3847e61407e49b39325682a235bb2dd7d407b: Status 404 returned error can't find the container with id 0b0d8f69f691fd283906962059e3847e61407e49b39325682a235bb2dd7d407b
Sep 16 14:24:28.230087 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:24:28.230049    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f72402-5f595b4b77-j97vl" event={"ID":"118c3a58-d101-4668-9da2-e9a824500df4","Type":"ContainerStarted","Data":"29ddb00408e01bb5b82992408c531b3ece71abd93e693f5575c641cef28dff3b"}
Sep 16 14:24:28.230263 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:24:28.230116    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f72402-5f595b4b77-j97vl"
Sep 16 14:24:28.230263 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:24:28.230136    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f72402-5f595b4b77-j97vl" event={"ID":"118c3a58-d101-4668-9da2-e9a824500df4","Type":"ContainerStarted","Data":"0b0d8f69f691fd283906962059e3847e61407e49b39325682a235bb2dd7d407b"}
Sep 16 14:24:28.260399 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:24:28.260347    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f72402-5f595b4b77-j97vl" podStartSLOduration=1.260330642 podStartE2EDuration="1.260330642s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:24:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:24:28.258222167 +0000 UTC m=+1201.273332040" watchObservedRunningTime="2026-09-16 14:24:28.260330642 +0000 UTC m=+1201.275440543"
Sep 16 14:24:39.238292 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:24:39.238262    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f72402-5f595b4b77-j97vl"
Sep 16 14:24:50.284031 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:24:50.283997    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-2ad565/distinct-w3-2ad565-kserve-7c9d64d6cf-drp8b"]
Sep 16 14:25:01.839398 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:01.839367    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-2ad565-f46f54678-xc2tg"]
Sep 16 14:25:01.839837 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:01.839611    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-2ad565-f46f54678-xc2tg" podUID="344de68b-6478-44c6-a0c4-c146b912f4a3" containerName="maas-api" containerID="cri-o://3ad93dc37df134395314150259d45948f0e3ef2d1f607da0a547507c5e70bc5f" gracePeriod=30
Sep 16 14:25:02.109880 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:02.109849    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-2ad565-f46f54678-xc2tg"
Sep 16 14:25:02.165459 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:02.165430    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jtcx8\" (UniqueName: \"kubernetes.io/projected/344de68b-6478-44c6-a0c4-c146b912f4a3-kube-api-access-jtcx8\") pod \"344de68b-6478-44c6-a0c4-c146b912f4a3\" (UID: \"344de68b-6478-44c6-a0c4-c146b912f4a3\") "
Sep 16 14:25:02.165656 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:02.165492    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/344de68b-6478-44c6-a0c4-c146b912f4a3-openshift-service-ca\") pod \"344de68b-6478-44c6-a0c4-c146b912f4a3\" (UID: \"344de68b-6478-44c6-a0c4-c146b912f4a3\") "
Sep 16 14:25:02.165656 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:02.165578    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/344de68b-6478-44c6-a0c4-c146b912f4a3-metrics-certs\") pod \"344de68b-6478-44c6-a0c4-c146b912f4a3\" (UID: \"344de68b-6478-44c6-a0c4-c146b912f4a3\") "
Sep 16 14:25:02.165791 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:02.165717    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/344de68b-6478-44c6-a0c4-c146b912f4a3-maas-api-tls\") pod \"344de68b-6478-44c6-a0c4-c146b912f4a3\" (UID: \"344de68b-6478-44c6-a0c4-c146b912f4a3\") "
Sep 16 14:25:02.165916 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:02.165887    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/344de68b-6478-44c6-a0c4-c146b912f4a3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "344de68b-6478-44c6-a0c4-c146b912f4a3" (UID: "344de68b-6478-44c6-a0c4-c146b912f4a3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:25:02.166113 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:02.166086    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/344de68b-6478-44c6-a0c4-c146b912f4a3-openshift-service-ca\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:25:02.168666 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:02.168627    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/344de68b-6478-44c6-a0c4-c146b912f4a3-kube-api-access-jtcx8" (OuterVolumeSpecName: "kube-api-access-jtcx8") pod "344de68b-6478-44c6-a0c4-c146b912f4a3" (UID: "344de68b-6478-44c6-a0c4-c146b912f4a3"). InnerVolumeSpecName "kube-api-access-jtcx8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:25:02.168780 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:02.168665    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/344de68b-6478-44c6-a0c4-c146b912f4a3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "344de68b-6478-44c6-a0c4-c146b912f4a3" (UID: "344de68b-6478-44c6-a0c4-c146b912f4a3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:25:02.168845 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:02.168810    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/344de68b-6478-44c6-a0c4-c146b912f4a3-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "344de68b-6478-44c6-a0c4-c146b912f4a3" (UID: "344de68b-6478-44c6-a0c4-c146b912f4a3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:25:02.267035 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:02.267000    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/344de68b-6478-44c6-a0c4-c146b912f4a3-metrics-certs\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:25:02.267035 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:02.267027    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/344de68b-6478-44c6-a0c4-c146b912f4a3-maas-api-tls\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:25:02.267035 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:02.267036    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jtcx8\" (UniqueName: \"kubernetes.io/projected/344de68b-6478-44c6-a0c4-c146b912f4a3-kube-api-access-jtcx8\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:25:02.441872 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:02.441839    2580 generic.go:356] "Generic (PLEG): container finished" podID="344de68b-6478-44c6-a0c4-c146b912f4a3" containerID="3ad93dc37df134395314150259d45948f0e3ef2d1f607da0a547507c5e70bc5f" exitCode=0
Sep 16 14:25:02.442038 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:02.441883    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-2ad565-f46f54678-xc2tg"
Sep 16 14:25:02.442038 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:02.441905    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-2ad565-f46f54678-xc2tg" event={"ID":"344de68b-6478-44c6-a0c4-c146b912f4a3","Type":"ContainerDied","Data":"3ad93dc37df134395314150259d45948f0e3ef2d1f607da0a547507c5e70bc5f"}
Sep 16 14:25:02.442038 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:02.441944    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-2ad565-f46f54678-xc2tg" event={"ID":"344de68b-6478-44c6-a0c4-c146b912f4a3","Type":"ContainerDied","Data":"25d69e783f883cfa836cdaf14fe410a4ea1995ff1ff83cb79c3c8c8de996b66f"}
Sep 16 14:25:02.442038 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:02.441963    2580 scope.go:117] "RemoveContainer" containerID="3ad93dc37df134395314150259d45948f0e3ef2d1f607da0a547507c5e70bc5f"
Sep 16 14:25:02.452624 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:02.452606    2580 scope.go:117] "RemoveContainer" containerID="3ad93dc37df134395314150259d45948f0e3ef2d1f607da0a547507c5e70bc5f"
Sep 16 14:25:02.452912 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:25:02.452892    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ad93dc37df134395314150259d45948f0e3ef2d1f607da0a547507c5e70bc5f\": container with ID starting with 3ad93dc37df134395314150259d45948f0e3ef2d1f607da0a547507c5e70bc5f not found: ID does not exist" containerID="3ad93dc37df134395314150259d45948f0e3ef2d1f607da0a547507c5e70bc5f"
Sep 16 14:25:02.452962 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:02.452921    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ad93dc37df134395314150259d45948f0e3ef2d1f607da0a547507c5e70bc5f"} err="failed to get container status \"3ad93dc37df134395314150259d45948f0e3ef2d1f607da0a547507c5e70bc5f\": rpc error: code = NotFound desc = could not find container \"3ad93dc37df134395314150259d45948f0e3ef2d1f607da0a547507c5e70bc5f\": container with ID starting with 3ad93dc37df134395314150259d45948f0e3ef2d1f607da0a547507c5e70bc5f not found: ID does not exist"
Sep 16 14:25:02.484271 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:02.484244    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-2ad565-f46f54678-xc2tg"]
Sep 16 14:25:02.493909 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:02.493888    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-2ad565-f46f54678-xc2tg"]
Sep 16 14:25:03.431010 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:03.430980    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="344de68b-6478-44c6-a0c4-c146b912f4a3" path="/var/lib/kubelet/pods/344de68b-6478-44c6-a0c4-c146b912f4a3/volumes"
Sep 16 14:25:05.303297 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:05.303249    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-2ad565/distinct-w3-2ad565-kserve-7c9d64d6cf-drp8b" podUID="786b6c75-8be7-4ae5-8719-82847428ecf7" containerName="main" containerID="cri-o://c326d8b75ceb8521d288ea2f4626afcaf8c433397153dd170300033ed1358b24" gracePeriod=45
Sep 16 14:25:05.462748 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:05.462714    2580 generic.go:356] "Generic (PLEG): container finished" podID="786b6c75-8be7-4ae5-8719-82847428ecf7" containerID="c326d8b75ceb8521d288ea2f4626afcaf8c433397153dd170300033ed1358b24" exitCode=0
Sep 16 14:25:05.462912 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:05.462779    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-2ad565/distinct-w3-2ad565-kserve-7c9d64d6cf-drp8b" event={"ID":"786b6c75-8be7-4ae5-8719-82847428ecf7","Type":"ContainerDied","Data":"c326d8b75ceb8521d288ea2f4626afcaf8c433397153dd170300033ed1358b24"}
Sep 16 14:25:05.570266 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:05.570242    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-2ad565/distinct-w3-2ad565-kserve-7c9d64d6cf-drp8b"
Sep 16 14:25:05.601835 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:05.601813    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/786b6c75-8be7-4ae5-8719-82847428ecf7-dshm\") pod \"786b6c75-8be7-4ae5-8719-82847428ecf7\" (UID: \"786b6c75-8be7-4ae5-8719-82847428ecf7\") "
Sep 16 14:25:05.601960 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:05.601887    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/786b6c75-8be7-4ae5-8719-82847428ecf7-tmp-dir\") pod \"786b6c75-8be7-4ae5-8719-82847428ecf7\" (UID: \"786b6c75-8be7-4ae5-8719-82847428ecf7\") "
Sep 16 14:25:05.601960 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:05.601924    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qxrdv\" (UniqueName: \"kubernetes.io/projected/786b6c75-8be7-4ae5-8719-82847428ecf7-kube-api-access-qxrdv\") pod \"786b6c75-8be7-4ae5-8719-82847428ecf7\" (UID: \"786b6c75-8be7-4ae5-8719-82847428ecf7\") "
Sep 16 14:25:05.601960 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:05.601953    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/786b6c75-8be7-4ae5-8719-82847428ecf7-home\") pod \"786b6c75-8be7-4ae5-8719-82847428ecf7\" (UID: \"786b6c75-8be7-4ae5-8719-82847428ecf7\") "
Sep 16 14:25:05.602080 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:05.601981    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/786b6c75-8be7-4ae5-8719-82847428ecf7-tls-certs\") pod \"786b6c75-8be7-4ae5-8719-82847428ecf7\" (UID: \"786b6c75-8be7-4ae5-8719-82847428ecf7\") "
Sep 16 14:25:05.602080 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:05.602020    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/786b6c75-8be7-4ae5-8719-82847428ecf7-model-cache\") pod \"786b6c75-8be7-4ae5-8719-82847428ecf7\" (UID: \"786b6c75-8be7-4ae5-8719-82847428ecf7\") "
Sep 16 14:25:05.602193 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:05.602073    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/786b6c75-8be7-4ae5-8719-82847428ecf7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "786b6c75-8be7-4ae5-8719-82847428ecf7" (UID: "786b6c75-8be7-4ae5-8719-82847428ecf7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:25:05.602282 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:05.602260    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/786b6c75-8be7-4ae5-8719-82847428ecf7-home" (OuterVolumeSpecName: "home") pod "786b6c75-8be7-4ae5-8719-82847428ecf7" (UID: "786b6c75-8be7-4ae5-8719-82847428ecf7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:25:05.602348 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:05.602280    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/786b6c75-8be7-4ae5-8719-82847428ecf7-model-cache" (OuterVolumeSpecName: "model-cache") pod "786b6c75-8be7-4ae5-8719-82847428ecf7" (UID: "786b6c75-8be7-4ae5-8719-82847428ecf7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:25:05.602348 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:05.602329    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/786b6c75-8be7-4ae5-8719-82847428ecf7-tmp-dir\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:25:05.604316 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:05.604281    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/786b6c75-8be7-4ae5-8719-82847428ecf7-dshm" (OuterVolumeSpecName: "dshm") pod "786b6c75-8be7-4ae5-8719-82847428ecf7" (UID: "786b6c75-8be7-4ae5-8719-82847428ecf7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:25:05.604601 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:05.604575    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/786b6c75-8be7-4ae5-8719-82847428ecf7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "786b6c75-8be7-4ae5-8719-82847428ecf7" (UID: "786b6c75-8be7-4ae5-8719-82847428ecf7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:25:05.604826 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:05.604807    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/786b6c75-8be7-4ae5-8719-82847428ecf7-kube-api-access-qxrdv" (OuterVolumeSpecName: "kube-api-access-qxrdv") pod "786b6c75-8be7-4ae5-8719-82847428ecf7" (UID: "786b6c75-8be7-4ae5-8719-82847428ecf7"). InnerVolumeSpecName "kube-api-access-qxrdv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:25:05.702983 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:05.702953    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qxrdv\" (UniqueName: \"kubernetes.io/projected/786b6c75-8be7-4ae5-8719-82847428ecf7-kube-api-access-qxrdv\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:25:05.702983 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:05.702980    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/786b6c75-8be7-4ae5-8719-82847428ecf7-home\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:25:05.702983 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:05.702991    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/786b6c75-8be7-4ae5-8719-82847428ecf7-tls-certs\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:25:05.703213 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:05.702999    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/786b6c75-8be7-4ae5-8719-82847428ecf7-model-cache\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:25:05.703213 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:05.703007    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/786b6c75-8be7-4ae5-8719-82847428ecf7-dshm\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:25:06.469715 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:06.469681    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-2ad565/distinct-w3-2ad565-kserve-7c9d64d6cf-drp8b"
Sep 16 14:25:06.469715 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:06.469687    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-2ad565/distinct-w3-2ad565-kserve-7c9d64d6cf-drp8b" event={"ID":"786b6c75-8be7-4ae5-8719-82847428ecf7","Type":"ContainerDied","Data":"49fafacb2179f96491f5755fdbd0c863fc02c5a6e38fa8256ab6deb4dd6b8f2b"}
Sep 16 14:25:06.470466 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:06.469734    2580 scope.go:117] "RemoveContainer" containerID="c326d8b75ceb8521d288ea2f4626afcaf8c433397153dd170300033ed1358b24"
Sep 16 14:25:06.519796 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:06.513044    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-2ad565/distinct-w3-2ad565-kserve-7c9d64d6cf-drp8b"]
Sep 16 14:25:06.523001 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:06.522975    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-2ad565/distinct-w3-2ad565-kserve-7c9d64d6cf-drp8b"]
Sep 16 14:25:07.430790 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:07.430755    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="786b6c75-8be7-4ae5-8719-82847428ecf7" path="/var/lib/kubelet/pods/786b6c75-8be7-4ae5-8719-82847428ecf7/volumes"
Sep 16 14:25:09.585530 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:09.585485    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk"]
Sep 16 14:25:09.586001 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:09.585743    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk" podUID="9db21092-910f-479a-ae58-4463feec4dfd" containerName="istio-proxy" containerID="cri-o://403f0717573202f8a186af8d001bd7f8f7723e282a9024624118ac9a20b665f0" gracePeriod=30
Sep 16 14:25:14.850783 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:14.850713    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk"
Sep 16 14:25:14.997169 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:14.997135    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9w4fj\" (UniqueName: \"kubernetes.io/projected/9db21092-910f-479a-ae58-4463feec4dfd-kube-api-access-9w4fj\") pod \"9db21092-910f-479a-ae58-4463feec4dfd\" (UID: \"9db21092-910f-479a-ae58-4463feec4dfd\") "
Sep 16 14:25:14.997364 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:14.997187    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9db21092-910f-479a-ae58-4463feec4dfd-credential-socket\") pod \"9db21092-910f-479a-ae58-4463feec4dfd\" (UID: \"9db21092-910f-479a-ae58-4463feec4dfd\") "
Sep 16 14:25:14.997364 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:14.997218    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9db21092-910f-479a-ae58-4463feec4dfd-istiod-ca-cert\") pod \"9db21092-910f-479a-ae58-4463feec4dfd\" (UID: \"9db21092-910f-479a-ae58-4463feec4dfd\") "
Sep 16 14:25:14.997364 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:14.997247    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9db21092-910f-479a-ae58-4463feec4dfd-workload-certs\") pod \"9db21092-910f-479a-ae58-4463feec4dfd\" (UID: \"9db21092-910f-479a-ae58-4463feec4dfd\") "
Sep 16 14:25:14.997364 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:14.997334    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9db21092-910f-479a-ae58-4463feec4dfd-istio-data\") pod \"9db21092-910f-479a-ae58-4463feec4dfd\" (UID: \"9db21092-910f-479a-ae58-4463feec4dfd\") "
Sep 16 14:25:14.997364 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:14.997357    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9db21092-910f-479a-ae58-4463feec4dfd-istio-token\") pod \"9db21092-910f-479a-ae58-4463feec4dfd\" (UID: \"9db21092-910f-479a-ae58-4463feec4dfd\") "
Sep 16 14:25:14.997662 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:14.997389    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9db21092-910f-479a-ae58-4463feec4dfd-istio-envoy\") pod \"9db21092-910f-479a-ae58-4463feec4dfd\" (UID: \"9db21092-910f-479a-ae58-4463feec4dfd\") "
Sep 16 14:25:14.997662 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:14.997459    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9db21092-910f-479a-ae58-4463feec4dfd-workload-socket\") pod \"9db21092-910f-479a-ae58-4463feec4dfd\" (UID: \"9db21092-910f-479a-ae58-4463feec4dfd\") "
Sep 16 14:25:14.997662 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:14.997464    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9db21092-910f-479a-ae58-4463feec4dfd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9db21092-910f-479a-ae58-4463feec4dfd" (UID: "9db21092-910f-479a-ae58-4463feec4dfd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:25:14.997662 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:14.997490    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9db21092-910f-479a-ae58-4463feec4dfd-istio-podinfo\") pod \"9db21092-910f-479a-ae58-4463feec4dfd\" (UID: \"9db21092-910f-479a-ae58-4463feec4dfd\") "
Sep 16 14:25:14.997662 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:14.997586    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9db21092-910f-479a-ae58-4463feec4dfd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9db21092-910f-479a-ae58-4463feec4dfd" (UID: "9db21092-910f-479a-ae58-4463feec4dfd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:25:14.997662 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:14.997599    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9db21092-910f-479a-ae58-4463feec4dfd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9db21092-910f-479a-ae58-4463feec4dfd" (UID: "9db21092-910f-479a-ae58-4463feec4dfd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:25:14.997917 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:14.997677    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9db21092-910f-479a-ae58-4463feec4dfd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9db21092-910f-479a-ae58-4463feec4dfd" (UID: "9db21092-910f-479a-ae58-4463feec4dfd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:25:14.997917 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:14.997893    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9db21092-910f-479a-ae58-4463feec4dfd-workload-socket\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:25:14.997917 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:14.997895    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9db21092-910f-479a-ae58-4463feec4dfd-istio-data" (OuterVolumeSpecName: "istio-data") pod "9db21092-910f-479a-ae58-4463feec4dfd" (UID: "9db21092-910f-479a-ae58-4463feec4dfd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:25:14.998051 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:14.997914    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9db21092-910f-479a-ae58-4463feec4dfd-credential-socket\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:25:14.998051 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:14.997949    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9db21092-910f-479a-ae58-4463feec4dfd-istiod-ca-cert\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:25:14.998051 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:14.997966    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9db21092-910f-479a-ae58-4463feec4dfd-workload-certs\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:25:14.999691 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:14.999659    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9db21092-910f-479a-ae58-4463feec4dfd-istio-token" (OuterVolumeSpecName: "istio-token") pod "9db21092-910f-479a-ae58-4463feec4dfd" (UID: "9db21092-910f-479a-ae58-4463feec4dfd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:25:14.999691 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:14.999680    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9db21092-910f-479a-ae58-4463feec4dfd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9db21092-910f-479a-ae58-4463feec4dfd" (UID: "9db21092-910f-479a-ae58-4463feec4dfd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:25:14.999936 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:14.999919    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9db21092-910f-479a-ae58-4463feec4dfd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9db21092-910f-479a-ae58-4463feec4dfd" (UID: "9db21092-910f-479a-ae58-4463feec4dfd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 14:25:14.999991 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:14.999931    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9db21092-910f-479a-ae58-4463feec4dfd-kube-api-access-9w4fj" (OuterVolumeSpecName: "kube-api-access-9w4fj") pod "9db21092-910f-479a-ae58-4463feec4dfd" (UID: "9db21092-910f-479a-ae58-4463feec4dfd"). InnerVolumeSpecName "kube-api-access-9w4fj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:25:15.099192 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:15.099155    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9w4fj\" (UniqueName: \"kubernetes.io/projected/9db21092-910f-479a-ae58-4463feec4dfd-kube-api-access-9w4fj\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:25:15.099192 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:15.099183    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9db21092-910f-479a-ae58-4463feec4dfd-istio-data\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:25:15.099192 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:15.099192    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9db21092-910f-479a-ae58-4463feec4dfd-istio-token\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:25:15.099192 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:15.099200    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9db21092-910f-479a-ae58-4463feec4dfd-istio-envoy\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:25:15.099460 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:15.099207    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9db21092-910f-479a-ae58-4463feec4dfd-istio-podinfo\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:25:15.523378 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:15.523339    2580 generic.go:356] "Generic (PLEG): container finished" podID="9db21092-910f-479a-ae58-4463feec4dfd" containerID="403f0717573202f8a186af8d001bd7f8f7723e282a9024624118ac9a20b665f0" exitCode=0
Sep 16 14:25:15.523567 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:15.523383    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk" event={"ID":"9db21092-910f-479a-ae58-4463feec4dfd","Type":"ContainerDied","Data":"403f0717573202f8a186af8d001bd7f8f7723e282a9024624118ac9a20b665f0"}
Sep 16 14:25:15.523567 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:15.523397    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk"
Sep 16 14:25:15.523567 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:15.523436    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk" event={"ID":"9db21092-910f-479a-ae58-4463feec4dfd","Type":"ContainerDied","Data":"b98cace62f61b38e3a4cb6c2fcb920116b6d1314573fc2b53e11891d27d5ce22"}
Sep 16 14:25:15.523567 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:15.523466    2580 scope.go:117] "RemoveContainer" containerID="403f0717573202f8a186af8d001bd7f8f7723e282a9024624118ac9a20b665f0"
Sep 16 14:25:15.532460 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:15.532443    2580 scope.go:117] "RemoveContainer" containerID="6f0235a87d104e93ebafb57294019ee0757189a0965fc15bf09486adf67cab45"
Sep 16 14:25:15.540946 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:15.540921    2580 scope.go:117] "RemoveContainer" containerID="403f0717573202f8a186af8d001bd7f8f7723e282a9024624118ac9a20b665f0"
Sep 16 14:25:15.541193 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:25:15.541171    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"403f0717573202f8a186af8d001bd7f8f7723e282a9024624118ac9a20b665f0\": container with ID starting with 403f0717573202f8a186af8d001bd7f8f7723e282a9024624118ac9a20b665f0 not found: ID does not exist" containerID="403f0717573202f8a186af8d001bd7f8f7723e282a9024624118ac9a20b665f0"
Sep 16 14:25:15.541283 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:15.541200    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"403f0717573202f8a186af8d001bd7f8f7723e282a9024624118ac9a20b665f0"} err="failed to get container status \"403f0717573202f8a186af8d001bd7f8f7723e282a9024624118ac9a20b665f0\": rpc error: code = NotFound desc = could not find container \"403f0717573202f8a186af8d001bd7f8f7723e282a9024624118ac9a20b665f0\": container with ID starting with 403f0717573202f8a186af8d001bd7f8f7723e282a9024624118ac9a20b665f0 not found: ID does not exist"
Sep 16 14:25:15.541283 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:15.541214    2580 scope.go:117] "RemoveContainer" containerID="6f0235a87d104e93ebafb57294019ee0757189a0965fc15bf09486adf67cab45"
Sep 16 14:25:15.541487 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:25:15.541469    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f0235a87d104e93ebafb57294019ee0757189a0965fc15bf09486adf67cab45\": container with ID starting with 6f0235a87d104e93ebafb57294019ee0757189a0965fc15bf09486adf67cab45 not found: ID does not exist" containerID="6f0235a87d104e93ebafb57294019ee0757189a0965fc15bf09486adf67cab45"
Sep 16 14:25:15.541570 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:15.541493    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f0235a87d104e93ebafb57294019ee0757189a0965fc15bf09486adf67cab45"} err="failed to get container status \"6f0235a87d104e93ebafb57294019ee0757189a0965fc15bf09486adf67cab45\": rpc error: code = NotFound desc = could not find container \"6f0235a87d104e93ebafb57294019ee0757189a0965fc15bf09486adf67cab45\": container with ID starting with 6f0235a87d104e93ebafb57294019ee0757189a0965fc15bf09486adf67cab45 not found: ID does not exist"
Sep 16 14:25:15.542443 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:15.542422    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk"]
Sep 16 14:25:15.548973 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:15.548954    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-2ad565-openshift-default-5c59847c5b-8s6xk"]
Sep 16 14:25:17.430995 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:25:17.430962    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9db21092-910f-479a-ae58-4463feec4dfd" path="/var/lib/kubelet/pods/9db21092-910f-479a-ae58-4463feec4dfd/volumes"
Sep 16 14:26:18.688470 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:18.688430    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-08475e/distinct-2-w0-08475e-kserve-66c89bc459-6fc57"]
Sep 16 14:26:18.713073 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:18.713041    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-08475e/premium-w0-08475e-kserve-647898d665-rmzkp"]
Sep 16 14:26:18.721532 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:18.721492    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-08475e/simulator-w0-08475e-kserve-8cc86df7b-x7z4t"]
Sep 16 14:26:31.701632 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:31.701601    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-08475e-565c574495-zngn7"]
Sep 16 14:26:31.702050 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:31.701790    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-08475e-565c574495-zngn7" podUID="82cb9c30-c55b-4a0a-9198-3d9b4529bb58" containerName="maas-api" containerID="cri-o://b16d6550de844430f23826fc3511c392df80505a638fca2f5bb2e59bd4c3f2c7" gracePeriod=30
Sep 16 14:26:31.948595 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:31.948562    2580 generic.go:356] "Generic (PLEG): container finished" podID="82cb9c30-c55b-4a0a-9198-3d9b4529bb58" containerID="b16d6550de844430f23826fc3511c392df80505a638fca2f5bb2e59bd4c3f2c7" exitCode=0
Sep 16 14:26:31.948706 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:31.948608    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-08475e-565c574495-zngn7" event={"ID":"82cb9c30-c55b-4a0a-9198-3d9b4529bb58","Type":"ContainerDied","Data":"b16d6550de844430f23826fc3511c392df80505a638fca2f5bb2e59bd4c3f2c7"}
Sep 16 14:26:31.964147 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:31.964097    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-08475e-565c574495-zngn7"
Sep 16 14:26:32.021670 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:32.021647    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/82cb9c30-c55b-4a0a-9198-3d9b4529bb58-maas-api-tls\") pod \"82cb9c30-c55b-4a0a-9198-3d9b4529bb58\" (UID: \"82cb9c30-c55b-4a0a-9198-3d9b4529bb58\") "
Sep 16 14:26:32.021806 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:32.021733    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7plc5\" (UniqueName: \"kubernetes.io/projected/82cb9c30-c55b-4a0a-9198-3d9b4529bb58-kube-api-access-7plc5\") pod \"82cb9c30-c55b-4a0a-9198-3d9b4529bb58\" (UID: \"82cb9c30-c55b-4a0a-9198-3d9b4529bb58\") "
Sep 16 14:26:32.021806 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:32.021765    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/82cb9c30-c55b-4a0a-9198-3d9b4529bb58-metrics-certs\") pod \"82cb9c30-c55b-4a0a-9198-3d9b4529bb58\" (UID: \"82cb9c30-c55b-4a0a-9198-3d9b4529bb58\") "
Sep 16 14:26:32.021806 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:32.021795    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/82cb9c30-c55b-4a0a-9198-3d9b4529bb58-openshift-service-ca\") pod \"82cb9c30-c55b-4a0a-9198-3d9b4529bb58\" (UID: \"82cb9c30-c55b-4a0a-9198-3d9b4529bb58\") "
Sep 16 14:26:32.022160 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:32.022130    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82cb9c30-c55b-4a0a-9198-3d9b4529bb58-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "82cb9c30-c55b-4a0a-9198-3d9b4529bb58" (UID: "82cb9c30-c55b-4a0a-9198-3d9b4529bb58"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:26:32.023846 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:32.023815    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82cb9c30-c55b-4a0a-9198-3d9b4529bb58-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "82cb9c30-c55b-4a0a-9198-3d9b4529bb58" (UID: "82cb9c30-c55b-4a0a-9198-3d9b4529bb58"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:26:32.023935 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:32.023825    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82cb9c30-c55b-4a0a-9198-3d9b4529bb58-kube-api-access-7plc5" (OuterVolumeSpecName: "kube-api-access-7plc5") pod "82cb9c30-c55b-4a0a-9198-3d9b4529bb58" (UID: "82cb9c30-c55b-4a0a-9198-3d9b4529bb58"). InnerVolumeSpecName "kube-api-access-7plc5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:26:32.023935 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:32.023848    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82cb9c30-c55b-4a0a-9198-3d9b4529bb58-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "82cb9c30-c55b-4a0a-9198-3d9b4529bb58" (UID: "82cb9c30-c55b-4a0a-9198-3d9b4529bb58"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:26:32.122609 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:32.122579    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7plc5\" (UniqueName: \"kubernetes.io/projected/82cb9c30-c55b-4a0a-9198-3d9b4529bb58-kube-api-access-7plc5\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:26:32.122609 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:32.122605    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/82cb9c30-c55b-4a0a-9198-3d9b4529bb58-metrics-certs\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:26:32.122781 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:32.122618    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/82cb9c30-c55b-4a0a-9198-3d9b4529bb58-openshift-service-ca\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:26:32.122781 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:32.122630    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/82cb9c30-c55b-4a0a-9198-3d9b4529bb58-maas-api-tls\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:26:32.588929 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:32.588891    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-dbb84810-549db4cd8-djk7f"]
Sep 16 14:26:32.589283 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:32.589272    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="786b6c75-8be7-4ae5-8719-82847428ecf7" containerName="main"
Sep 16 14:26:32.589329 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:32.589286    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="786b6c75-8be7-4ae5-8719-82847428ecf7" containerName="main"
Sep 16 14:26:32.589329 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:32.589300    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="344de68b-6478-44c6-a0c4-c146b912f4a3" containerName="maas-api"
Sep 16 14:26:32.589329 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:32.589306    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="344de68b-6478-44c6-a0c4-c146b912f4a3" containerName="maas-api"
Sep 16 14:26:32.589329 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:32.589315    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9db21092-910f-479a-ae58-4463feec4dfd" containerName="istio-proxy"
Sep 16 14:26:32.589329 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:32.589321    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9db21092-910f-479a-ae58-4463feec4dfd" containerName="istio-proxy"
Sep 16 14:26:32.589510 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:32.589331    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82cb9c30-c55b-4a0a-9198-3d9b4529bb58" containerName="maas-api"
Sep 16 14:26:32.589510 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:32.589337    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="82cb9c30-c55b-4a0a-9198-3d9b4529bb58" containerName="maas-api"
Sep 16 14:26:32.589510 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:32.589397    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="786b6c75-8be7-4ae5-8719-82847428ecf7" containerName="main"
Sep 16 14:26:32.589510 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:32.589405    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9db21092-910f-479a-ae58-4463feec4dfd" containerName="istio-proxy"
Sep 16 14:26:32.589510 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:32.589412    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="82cb9c30-c55b-4a0a-9198-3d9b4529bb58" containerName="maas-api"
Sep 16 14:26:32.589510 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:32.589417    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="344de68b-6478-44c6-a0c4-c146b912f4a3" containerName="maas-api"
Sep 16 14:26:32.589510 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:32.589423    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9db21092-910f-479a-ae58-4463feec4dfd" containerName="istio-proxy"
Sep 16 14:26:32.592422 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:32.592403    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dbb84810-549db4cd8-djk7f"
Sep 16 14:26:32.601424 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:32.601400    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-dbb84810-serving-cert\""
Sep 16 14:26:32.627427 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:32.627401    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-dbb84810-549db4cd8-djk7f"]
Sep 16 14:26:32.730022 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:32.729990    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e12a76d-2722-46ce-9149-ce3487392be4-metrics-certs\") pod \"maas-api-e2e-mt-dbb84810-549db4cd8-djk7f\" (UID: \"1e12a76d-2722-46ce-9149-ce3487392be4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dbb84810-549db4cd8-djk7f"
Sep 16 14:26:32.730509 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:32.730056    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1e12a76d-2722-46ce-9149-ce3487392be4-maas-api-tls\") pod \"maas-api-e2e-mt-dbb84810-549db4cd8-djk7f\" (UID: \"1e12a76d-2722-46ce-9149-ce3487392be4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dbb84810-549db4cd8-djk7f"
Sep 16 14:26:32.730509 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:32.730164    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e12a76d-2722-46ce-9149-ce3487392be4-openshift-service-ca\") pod \"maas-api-e2e-mt-dbb84810-549db4cd8-djk7f\" (UID: \"1e12a76d-2722-46ce-9149-ce3487392be4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dbb84810-549db4cd8-djk7f"
Sep 16 14:26:32.730509 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:32.730218    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxjt9\" (UniqueName: \"kubernetes.io/projected/1e12a76d-2722-46ce-9149-ce3487392be4-kube-api-access-kxjt9\") pod \"maas-api-e2e-mt-dbb84810-549db4cd8-djk7f\" (UID: \"1e12a76d-2722-46ce-9149-ce3487392be4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dbb84810-549db4cd8-djk7f"
Sep 16 14:26:32.831575 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:32.831542    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxjt9\" (UniqueName: \"kubernetes.io/projected/1e12a76d-2722-46ce-9149-ce3487392be4-kube-api-access-kxjt9\") pod \"maas-api-e2e-mt-dbb84810-549db4cd8-djk7f\" (UID: \"1e12a76d-2722-46ce-9149-ce3487392be4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dbb84810-549db4cd8-djk7f"
Sep 16 14:26:32.831753 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:32.831586    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e12a76d-2722-46ce-9149-ce3487392be4-metrics-certs\") pod \"maas-api-e2e-mt-dbb84810-549db4cd8-djk7f\" (UID: \"1e12a76d-2722-46ce-9149-ce3487392be4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dbb84810-549db4cd8-djk7f"
Sep 16 14:26:32.831753 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:32.831652    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1e12a76d-2722-46ce-9149-ce3487392be4-maas-api-tls\") pod \"maas-api-e2e-mt-dbb84810-549db4cd8-djk7f\" (UID: \"1e12a76d-2722-46ce-9149-ce3487392be4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dbb84810-549db4cd8-djk7f"
Sep 16 14:26:32.831753 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:32.831721    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e12a76d-2722-46ce-9149-ce3487392be4-openshift-service-ca\") pod \"maas-api-e2e-mt-dbb84810-549db4cd8-djk7f\" (UID: \"1e12a76d-2722-46ce-9149-ce3487392be4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dbb84810-549db4cd8-djk7f"
Sep 16 14:26:32.832510 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:32.832469    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e12a76d-2722-46ce-9149-ce3487392be4-openshift-service-ca\") pod \"maas-api-e2e-mt-dbb84810-549db4cd8-djk7f\" (UID: \"1e12a76d-2722-46ce-9149-ce3487392be4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dbb84810-549db4cd8-djk7f"
Sep 16 14:26:32.834235 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:32.834213    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e12a76d-2722-46ce-9149-ce3487392be4-metrics-certs\") pod \"maas-api-e2e-mt-dbb84810-549db4cd8-djk7f\" (UID: \"1e12a76d-2722-46ce-9149-ce3487392be4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dbb84810-549db4cd8-djk7f"
Sep 16 14:26:32.834316 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:32.834283    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1e12a76d-2722-46ce-9149-ce3487392be4-maas-api-tls\") pod \"maas-api-e2e-mt-dbb84810-549db4cd8-djk7f\" (UID: \"1e12a76d-2722-46ce-9149-ce3487392be4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dbb84810-549db4cd8-djk7f"
Sep 16 14:26:32.852269 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:32.852210    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxjt9\" (UniqueName: \"kubernetes.io/projected/1e12a76d-2722-46ce-9149-ce3487392be4-kube-api-access-kxjt9\") pod \"maas-api-e2e-mt-dbb84810-549db4cd8-djk7f\" (UID: \"1e12a76d-2722-46ce-9149-ce3487392be4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dbb84810-549db4cd8-djk7f"
Sep 16 14:26:32.903046 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:32.903025    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dbb84810-549db4cd8-djk7f"
Sep 16 14:26:32.958315 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:32.957798    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-08475e-565c574495-zngn7"
Sep 16 14:26:32.958991 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:32.957816    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-08475e-565c574495-zngn7" event={"ID":"82cb9c30-c55b-4a0a-9198-3d9b4529bb58","Type":"ContainerDied","Data":"f3a907629a56c4a294a6752a9895ec76554a9d3f4f1e12d2a4a490cb0e35726c"}
Sep 16 14:26:32.958991 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:32.958574    2580 scope.go:117] "RemoveContainer" containerID="b16d6550de844430f23826fc3511c392df80505a638fca2f5bb2e59bd4c3f2c7"
Sep 16 14:26:33.019660 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:33.019634    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-08475e-565c574495-zngn7"]
Sep 16 14:26:33.022159 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:33.022138    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-08475e-565c574495-zngn7"]
Sep 16 14:26:33.188424 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:33.188397    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-dbb84810-549db4cd8-djk7f"]
Sep 16 14:26:33.190076 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:26:33.190056    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e12a76d_2722_46ce_9149_ce3487392be4.slice/crio-e4fd13db39eb622fab622eb72cd1296f383f56c5deaf40fe6dbdc49ceaae7c41 WatchSource:0}: Error finding container e4fd13db39eb622fab622eb72cd1296f383f56c5deaf40fe6dbdc49ceaae7c41: Status 404 returned error can't find the container with id e4fd13db39eb622fab622eb72cd1296f383f56c5deaf40fe6dbdc49ceaae7c41
Sep 16 14:26:33.191428 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:33.191412    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 14:26:33.436723 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:33.436686    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82cb9c30-c55b-4a0a-9198-3d9b4529bb58" path="/var/lib/kubelet/pods/82cb9c30-c55b-4a0a-9198-3d9b4529bb58/volumes"
Sep 16 14:26:33.708699 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:33.708620    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-08475e/distinct-2-w0-08475e-kserve-66c89bc459-6fc57" podUID="da00cc8d-910a-4c6e-86c7-ca474c5e819c" containerName="main" containerID="cri-o://955ffbd85ac1d5d3915802e700a8a2784a64c486e94549a39ab78ad50bdefbf1" gracePeriod=45
Sep 16 14:26:33.732682 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:33.732633    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-08475e/premium-w0-08475e-kserve-647898d665-rmzkp" podUID="2f76715c-d553-44c4-995e-9d2c1fe9ce5b" containerName="main" containerID="cri-o://4e2e075bca10b0cd17c0e96114130eb1da18bf3ba999a0b09a0f522c0bd4b0c6" gracePeriod=45
Sep 16 14:26:33.740006 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:33.739962    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-08475e/simulator-w0-08475e-kserve-8cc86df7b-x7z4t" podUID="9b899509-2d6c-4c9f-82ba-58bbc8fd07d8" containerName="main" containerID="cri-o://3c70eaf52334a52522a4c6bfa49a0a9e92c394d0dfd71d3f10b72746a6fc9ed4" gracePeriod=45
Sep 16 14:26:33.966775 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:33.966743    2580 generic.go:356] "Generic (PLEG): container finished" podID="9b899509-2d6c-4c9f-82ba-58bbc8fd07d8" containerID="3c70eaf52334a52522a4c6bfa49a0a9e92c394d0dfd71d3f10b72746a6fc9ed4" exitCode=0
Sep 16 14:26:33.966920 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:33.966816    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-08475e/simulator-w0-08475e-kserve-8cc86df7b-x7z4t" event={"ID":"9b899509-2d6c-4c9f-82ba-58bbc8fd07d8","Type":"ContainerDied","Data":"3c70eaf52334a52522a4c6bfa49a0a9e92c394d0dfd71d3f10b72746a6fc9ed4"}
Sep 16 14:26:33.968715 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:33.968694    2580 generic.go:356] "Generic (PLEG): container finished" podID="2f76715c-d553-44c4-995e-9d2c1fe9ce5b" containerID="4e2e075bca10b0cd17c0e96114130eb1da18bf3ba999a0b09a0f522c0bd4b0c6" exitCode=0
Sep 16 14:26:33.968827 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:33.968752    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-08475e/premium-w0-08475e-kserve-647898d665-rmzkp" event={"ID":"2f76715c-d553-44c4-995e-9d2c1fe9ce5b","Type":"ContainerDied","Data":"4e2e075bca10b0cd17c0e96114130eb1da18bf3ba999a0b09a0f522c0bd4b0c6"}
Sep 16 14:26:33.975429 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:33.975407    2580 generic.go:356] "Generic (PLEG): container finished" podID="da00cc8d-910a-4c6e-86c7-ca474c5e819c" containerID="955ffbd85ac1d5d3915802e700a8a2784a64c486e94549a39ab78ad50bdefbf1" exitCode=0
Sep 16 14:26:33.975568 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:33.975433    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-08475e/distinct-2-w0-08475e-kserve-66c89bc459-6fc57" event={"ID":"da00cc8d-910a-4c6e-86c7-ca474c5e819c","Type":"ContainerDied","Data":"955ffbd85ac1d5d3915802e700a8a2784a64c486e94549a39ab78ad50bdefbf1"}
Sep 16 14:26:33.976948 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:33.976922    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dbb84810-549db4cd8-djk7f" event={"ID":"1e12a76d-2722-46ce-9149-ce3487392be4","Type":"ContainerStarted","Data":"e4fd13db39eb622fab622eb72cd1296f383f56c5deaf40fe6dbdc49ceaae7c41"}
Sep 16 14:26:34.015043 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.015012    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-08475e/distinct-2-w0-08475e-kserve-66c89bc459-6fc57"
Sep 16 14:26:34.056446 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.056422    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-08475e/simulator-w0-08475e-kserve-8cc86df7b-x7z4t"
Sep 16 14:26:34.061364 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.061310    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-08475e/premium-w0-08475e-kserve-647898d665-rmzkp"
Sep 16 14:26:34.144236 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.144191    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2f76715c-d553-44c4-995e-9d2c1fe9ce5b-home\") pod \"2f76715c-d553-44c4-995e-9d2c1fe9ce5b\" (UID: \"2f76715c-d553-44c4-995e-9d2c1fe9ce5b\") "
Sep 16 14:26:34.144384 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.144262    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/da00cc8d-910a-4c6e-86c7-ca474c5e819c-dshm\") pod \"da00cc8d-910a-4c6e-86c7-ca474c5e819c\" (UID: \"da00cc8d-910a-4c6e-86c7-ca474c5e819c\") "
Sep 16 14:26:34.144384 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.144294    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p6g4x\" (UniqueName: \"kubernetes.io/projected/da00cc8d-910a-4c6e-86c7-ca474c5e819c-kube-api-access-p6g4x\") pod \"da00cc8d-910a-4c6e-86c7-ca474c5e819c\" (UID: \"da00cc8d-910a-4c6e-86c7-ca474c5e819c\") "
Sep 16 14:26:34.144384 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.144353    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/da00cc8d-910a-4c6e-86c7-ca474c5e819c-tmp-dir\") pod \"da00cc8d-910a-4c6e-86c7-ca474c5e819c\" (UID: \"da00cc8d-910a-4c6e-86c7-ca474c5e819c\") "
Sep 16 14:26:34.144384 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.144380    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b899509-2d6c-4c9f-82ba-58bbc8fd07d8-home\") pod \"9b899509-2d6c-4c9f-82ba-58bbc8fd07d8\" (UID: \"9b899509-2d6c-4c9f-82ba-58bbc8fd07d8\") "
Sep 16 14:26:34.144688 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.144404    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b899509-2d6c-4c9f-82ba-58bbc8fd07d8-dshm\") pod \"9b899509-2d6c-4c9f-82ba-58bbc8fd07d8\" (UID: \"9b899509-2d6c-4c9f-82ba-58bbc8fd07d8\") "
Sep 16 14:26:34.144688 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.144429    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b899509-2d6c-4c9f-82ba-58bbc8fd07d8-tmp-dir\") pod \"9b899509-2d6c-4c9f-82ba-58bbc8fd07d8\" (UID: \"9b899509-2d6c-4c9f-82ba-58bbc8fd07d8\") "
Sep 16 14:26:34.144688 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.144459    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b899509-2d6c-4c9f-82ba-58bbc8fd07d8-tls-certs\") pod \"9b899509-2d6c-4c9f-82ba-58bbc8fd07d8\" (UID: \"9b899509-2d6c-4c9f-82ba-58bbc8fd07d8\") "
Sep 16 14:26:34.144688 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.144528    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2f76715c-d553-44c4-995e-9d2c1fe9ce5b-dshm\") pod \"2f76715c-d553-44c4-995e-9d2c1fe9ce5b\" (UID: \"2f76715c-d553-44c4-995e-9d2c1fe9ce5b\") "
Sep 16 14:26:34.144688 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.144559    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2f76715c-d553-44c4-995e-9d2c1fe9ce5b-model-cache\") pod \"2f76715c-d553-44c4-995e-9d2c1fe9ce5b\" (UID: \"2f76715c-d553-44c4-995e-9d2c1fe9ce5b\") "
Sep 16 14:26:34.144688 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.144589    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/da00cc8d-910a-4c6e-86c7-ca474c5e819c-model-cache\") pod \"da00cc8d-910a-4c6e-86c7-ca474c5e819c\" (UID: \"da00cc8d-910a-4c6e-86c7-ca474c5e819c\") "
Sep 16 14:26:34.144688 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.144632    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f76715c-d553-44c4-995e-9d2c1fe9ce5b-tls-certs\") pod \"2f76715c-d553-44c4-995e-9d2c1fe9ce5b\" (UID: \"2f76715c-d553-44c4-995e-9d2c1fe9ce5b\") "
Sep 16 14:26:34.144688 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.144669    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b899509-2d6c-4c9f-82ba-58bbc8fd07d8-model-cache\") pod \"9b899509-2d6c-4c9f-82ba-58bbc8fd07d8\" (UID: \"9b899509-2d6c-4c9f-82ba-58bbc8fd07d8\") "
Sep 16 14:26:34.145101 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.144719    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/da00cc8d-910a-4c6e-86c7-ca474c5e819c-home\") pod \"da00cc8d-910a-4c6e-86c7-ca474c5e819c\" (UID: \"da00cc8d-910a-4c6e-86c7-ca474c5e819c\") "
Sep 16 14:26:34.145101 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.144745    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z5nrm\" (UniqueName: \"kubernetes.io/projected/9b899509-2d6c-4c9f-82ba-58bbc8fd07d8-kube-api-access-z5nrm\") pod \"9b899509-2d6c-4c9f-82ba-58bbc8fd07d8\" (UID: \"9b899509-2d6c-4c9f-82ba-58bbc8fd07d8\") "
Sep 16 14:26:34.145101 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.144779    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qcwdw\" (UniqueName: \"kubernetes.io/projected/2f76715c-d553-44c4-995e-9d2c1fe9ce5b-kube-api-access-qcwdw\") pod \"2f76715c-d553-44c4-995e-9d2c1fe9ce5b\" (UID: \"2f76715c-d553-44c4-995e-9d2c1fe9ce5b\") "
Sep 16 14:26:34.145101 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.144805    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f76715c-d553-44c4-995e-9d2c1fe9ce5b-tmp-dir\") pod \"2f76715c-d553-44c4-995e-9d2c1fe9ce5b\" (UID: \"2f76715c-d553-44c4-995e-9d2c1fe9ce5b\") "
Sep 16 14:26:34.145101 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.144829    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/da00cc8d-910a-4c6e-86c7-ca474c5e819c-tls-certs\") pod \"da00cc8d-910a-4c6e-86c7-ca474c5e819c\" (UID: \"da00cc8d-910a-4c6e-86c7-ca474c5e819c\") "
Sep 16 14:26:34.146241 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.145568    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da00cc8d-910a-4c6e-86c7-ca474c5e819c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "da00cc8d-910a-4c6e-86c7-ca474c5e819c" (UID: "da00cc8d-910a-4c6e-86c7-ca474c5e819c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:26:34.146241 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.145674    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b899509-2d6c-4c9f-82ba-58bbc8fd07d8-model-cache" (OuterVolumeSpecName: "model-cache") pod "9b899509-2d6c-4c9f-82ba-58bbc8fd07d8" (UID: "9b899509-2d6c-4c9f-82ba-58bbc8fd07d8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:26:34.146241 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.145899    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f76715c-d553-44c4-995e-9d2c1fe9ce5b-model-cache" (OuterVolumeSpecName: "model-cache") pod "2f76715c-d553-44c4-995e-9d2c1fe9ce5b" (UID: "2f76715c-d553-44c4-995e-9d2c1fe9ce5b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:26:34.146241 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.146144    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da00cc8d-910a-4c6e-86c7-ca474c5e819c-model-cache" (OuterVolumeSpecName: "model-cache") pod "da00cc8d-910a-4c6e-86c7-ca474c5e819c" (UID: "da00cc8d-910a-4c6e-86c7-ca474c5e819c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:26:34.146610 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.146581    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-dbb84810-f646d745-hkfg4"]
Sep 16 14:26:34.146948 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.146916    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f76715c-d553-44c4-995e-9d2c1fe9ce5b-home" (OuterVolumeSpecName: "home") pod "2f76715c-d553-44c4-995e-9d2c1fe9ce5b" (UID: "2f76715c-d553-44c4-995e-9d2c1fe9ce5b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:26:34.147190 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.147168    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9db21092-910f-479a-ae58-4463feec4dfd" containerName="istio-proxy"
Sep 16 14:26:34.147190 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.147194    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9db21092-910f-479a-ae58-4463feec4dfd" containerName="istio-proxy"
Sep 16 14:26:34.147190 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.147209    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f76715c-d553-44c4-995e-9d2c1fe9ce5b" containerName="main"
Sep 16 14:26:34.147575 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.147218    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f76715c-d553-44c4-995e-9d2c1fe9ce5b" containerName="main"
Sep 16 14:26:34.147575 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.147233    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b899509-2d6c-4c9f-82ba-58bbc8fd07d8" containerName="main"
Sep 16 14:26:34.147575 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.147241    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b899509-2d6c-4c9f-82ba-58bbc8fd07d8" containerName="main"
Sep 16 14:26:34.147575 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.147260    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da00cc8d-910a-4c6e-86c7-ca474c5e819c" containerName="main"
Sep 16 14:26:34.147575 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.147270    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="da00cc8d-910a-4c6e-86c7-ca474c5e819c" containerName="main"
Sep 16 14:26:34.147575 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.147388    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f76715c-d553-44c4-995e-9d2c1fe9ce5b" containerName="main"
Sep 16 14:26:34.147575 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.147403    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b899509-2d6c-4c9f-82ba-58bbc8fd07d8" containerName="main"
Sep 16 14:26:34.147575 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.147418    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="da00cc8d-910a-4c6e-86c7-ca474c5e819c" containerName="main"
Sep 16 14:26:34.147575 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.147473    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f76715c-d553-44c4-995e-9d2c1fe9ce5b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2f76715c-d553-44c4-995e-9d2c1fe9ce5b" (UID: "2f76715c-d553-44c4-995e-9d2c1fe9ce5b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:26:34.148361 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.147673    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b899509-2d6c-4c9f-82ba-58bbc8fd07d8-home" (OuterVolumeSpecName: "home") pod "9b899509-2d6c-4c9f-82ba-58bbc8fd07d8" (UID: "9b899509-2d6c-4c9f-82ba-58bbc8fd07d8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:26:34.148361 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.147850    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b899509-2d6c-4c9f-82ba-58bbc8fd07d8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9b899509-2d6c-4c9f-82ba-58bbc8fd07d8" (UID: "9b899509-2d6c-4c9f-82ba-58bbc8fd07d8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:26:34.148361 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.148025    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da00cc8d-910a-4c6e-86c7-ca474c5e819c-home" (OuterVolumeSpecName: "home") pod "da00cc8d-910a-4c6e-86c7-ca474c5e819c" (UID: "da00cc8d-910a-4c6e-86c7-ca474c5e819c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:26:34.149961 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.149925    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f76715c-d553-44c4-995e-9d2c1fe9ce5b-dshm" (OuterVolumeSpecName: "dshm") pod "2f76715c-d553-44c4-995e-9d2c1fe9ce5b" (UID: "2f76715c-d553-44c4-995e-9d2c1fe9ce5b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:26:34.150388 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.150339    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da00cc8d-910a-4c6e-86c7-ca474c5e819c-dshm" (OuterVolumeSpecName: "dshm") pod "da00cc8d-910a-4c6e-86c7-ca474c5e819c" (UID: "da00cc8d-910a-4c6e-86c7-ca474c5e819c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:26:34.150388 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.150351    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da00cc8d-910a-4c6e-86c7-ca474c5e819c-kube-api-access-p6g4x" (OuterVolumeSpecName: "kube-api-access-p6g4x") pod "da00cc8d-910a-4c6e-86c7-ca474c5e819c" (UID: "da00cc8d-910a-4c6e-86c7-ca474c5e819c"). InnerVolumeSpecName "kube-api-access-p6g4x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:26:34.150388 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.150353    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da00cc8d-910a-4c6e-86c7-ca474c5e819c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "da00cc8d-910a-4c6e-86c7-ca474c5e819c" (UID: "da00cc8d-910a-4c6e-86c7-ca474c5e819c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:26:34.150723 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.150693    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f76715c-d553-44c4-995e-9d2c1fe9ce5b-kube-api-access-qcwdw" (OuterVolumeSpecName: "kube-api-access-qcwdw") pod "2f76715c-d553-44c4-995e-9d2c1fe9ce5b" (UID: "2f76715c-d553-44c4-995e-9d2c1fe9ce5b"). InnerVolumeSpecName "kube-api-access-qcwdw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:26:34.150809 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.150719    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f76715c-d553-44c4-995e-9d2c1fe9ce5b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2f76715c-d553-44c4-995e-9d2c1fe9ce5b" (UID: "2f76715c-d553-44c4-995e-9d2c1fe9ce5b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:26:34.150930 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.150895    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b899509-2d6c-4c9f-82ba-58bbc8fd07d8-dshm" (OuterVolumeSpecName: "dshm") pod "9b899509-2d6c-4c9f-82ba-58bbc8fd07d8" (UID: "9b899509-2d6c-4c9f-82ba-58bbc8fd07d8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:26:34.151698 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.151672    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b899509-2d6c-4c9f-82ba-58bbc8fd07d8-kube-api-access-z5nrm" (OuterVolumeSpecName: "kube-api-access-z5nrm") pod "9b899509-2d6c-4c9f-82ba-58bbc8fd07d8" (UID: "9b899509-2d6c-4c9f-82ba-58bbc8fd07d8"). InnerVolumeSpecName "kube-api-access-z5nrm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:26:34.152077 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.152059    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-dbb84810-f646d745-hkfg4"
Sep 16 14:26:34.152269 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.152244    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b899509-2d6c-4c9f-82ba-58bbc8fd07d8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9b899509-2d6c-4c9f-82ba-58bbc8fd07d8" (UID: "9b899509-2d6c-4c9f-82ba-58bbc8fd07d8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:26:34.155117 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.155096    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-dbb84810\""
Sep 16 14:26:34.155393 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.155373    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-dbb84810-dockercfg-jcnpc\""
Sep 16 14:26:34.183582 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.183551    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-dbb84810-f646d745-hkfg4"]
Sep 16 14:26:34.246627 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.246537    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3264f8ac-0b3c-4f2c-be20-d7976302daed-plugins-config-volume\") pod \"payload-processing-e2e-mt-dbb84810-f646d745-hkfg4\" (UID: \"3264f8ac-0b3c-4f2c-be20-d7976302daed\") " pod="openshift-ingress/payload-processing-e2e-mt-dbb84810-f646d745-hkfg4"
Sep 16 14:26:34.246627 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.246611    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msmx8\" (UniqueName: \"kubernetes.io/projected/3264f8ac-0b3c-4f2c-be20-d7976302daed-kube-api-access-msmx8\") pod \"payload-processing-e2e-mt-dbb84810-f646d745-hkfg4\" (UID: \"3264f8ac-0b3c-4f2c-be20-d7976302daed\") " pod="openshift-ingress/payload-processing-e2e-mt-dbb84810-f646d745-hkfg4"
Sep 16 14:26:34.246900 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.246746    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/da00cc8d-910a-4c6e-86c7-ca474c5e819c-home\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:26:34.246900 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.246766    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z5nrm\" (UniqueName: \"kubernetes.io/projected/9b899509-2d6c-4c9f-82ba-58bbc8fd07d8-kube-api-access-z5nrm\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:26:34.246900 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.246776    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qcwdw\" (UniqueName: \"kubernetes.io/projected/2f76715c-d553-44c4-995e-9d2c1fe9ce5b-kube-api-access-qcwdw\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:26:34.246900 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.246786    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f76715c-d553-44c4-995e-9d2c1fe9ce5b-tmp-dir\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:26:34.246900 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.246795    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/da00cc8d-910a-4c6e-86c7-ca474c5e819c-tls-certs\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:26:34.246900 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.246802    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2f76715c-d553-44c4-995e-9d2c1fe9ce5b-home\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:26:34.246900 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.246809    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/da00cc8d-910a-4c6e-86c7-ca474c5e819c-dshm\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:26:34.246900 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.246817    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p6g4x\" (UniqueName: \"kubernetes.io/projected/da00cc8d-910a-4c6e-86c7-ca474c5e819c-kube-api-access-p6g4x\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:26:34.246900 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.246825    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/da00cc8d-910a-4c6e-86c7-ca474c5e819c-tmp-dir\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:26:34.246900 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.246834    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b899509-2d6c-4c9f-82ba-58bbc8fd07d8-home\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:26:34.246900 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.246840    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b899509-2d6c-4c9f-82ba-58bbc8fd07d8-dshm\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:26:34.246900 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.246848    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b899509-2d6c-4c9f-82ba-58bbc8fd07d8-tmp-dir\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:26:34.246900 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.246855    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b899509-2d6c-4c9f-82ba-58bbc8fd07d8-tls-certs\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:26:34.246900 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.246861    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2f76715c-d553-44c4-995e-9d2c1fe9ce5b-dshm\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:26:34.246900 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.246868    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2f76715c-d553-44c4-995e-9d2c1fe9ce5b-model-cache\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:26:34.246900 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.246876    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/da00cc8d-910a-4c6e-86c7-ca474c5e819c-model-cache\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:26:34.246900 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.246885    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f76715c-d553-44c4-995e-9d2c1fe9ce5b-tls-certs\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:26:34.246900 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.246892    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b899509-2d6c-4c9f-82ba-58bbc8fd07d8-model-cache\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:26:34.347364 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.347332    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3264f8ac-0b3c-4f2c-be20-d7976302daed-plugins-config-volume\") pod \"payload-processing-e2e-mt-dbb84810-f646d745-hkfg4\" (UID: \"3264f8ac-0b3c-4f2c-be20-d7976302daed\") " pod="openshift-ingress/payload-processing-e2e-mt-dbb84810-f646d745-hkfg4"
Sep 16 14:26:34.347568 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.347395    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msmx8\" (UniqueName: \"kubernetes.io/projected/3264f8ac-0b3c-4f2c-be20-d7976302daed-kube-api-access-msmx8\") pod \"payload-processing-e2e-mt-dbb84810-f646d745-hkfg4\" (UID: \"3264f8ac-0b3c-4f2c-be20-d7976302daed\") " pod="openshift-ingress/payload-processing-e2e-mt-dbb84810-f646d745-hkfg4"
Sep 16 14:26:34.348066 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.348045    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3264f8ac-0b3c-4f2c-be20-d7976302daed-plugins-config-volume\") pod \"payload-processing-e2e-mt-dbb84810-f646d745-hkfg4\" (UID: \"3264f8ac-0b3c-4f2c-be20-d7976302daed\") " pod="openshift-ingress/payload-processing-e2e-mt-dbb84810-f646d745-hkfg4"
Sep 16 14:26:34.359770 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.359752    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-msmx8\" (UniqueName: \"kubernetes.io/projected/3264f8ac-0b3c-4f2c-be20-d7976302daed-kube-api-access-msmx8\") pod \"payload-processing-e2e-mt-dbb84810-f646d745-hkfg4\" (UID: \"3264f8ac-0b3c-4f2c-be20-d7976302daed\") " pod="openshift-ingress/payload-processing-e2e-mt-dbb84810-f646d745-hkfg4"
Sep 16 14:26:34.468390 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.468356    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-dbb84810-f646d745-hkfg4"
Sep 16 14:26:34.767304 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.767275    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-dbb84810-f646d745-hkfg4"]
Sep 16 14:26:34.768401 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:26:34.768371    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3264f8ac_0b3c_4f2c_be20_d7976302daed.slice/crio-3903f4455eee6efd150358a16a74687d71cc76a224370c0cf15350a4d2ecfb57 WatchSource:0}: Error finding container 3903f4455eee6efd150358a16a74687d71cc76a224370c0cf15350a4d2ecfb57: Status 404 returned error can't find the container with id 3903f4455eee6efd150358a16a74687d71cc76a224370c0cf15350a4d2ecfb57
Sep 16 14:26:34.984061 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.984027    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-08475e/distinct-2-w0-08475e-kserve-66c89bc459-6fc57"
Sep 16 14:26:34.984061 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.984040    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-08475e/distinct-2-w0-08475e-kserve-66c89bc459-6fc57" event={"ID":"da00cc8d-910a-4c6e-86c7-ca474c5e819c","Type":"ContainerDied","Data":"5401a000fe12e5b120e240eff1d6e921f0c9ebf0cb254aa5162c944f38041753"}
Sep 16 14:26:34.984586 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.984095    2580 scope.go:117] "RemoveContainer" containerID="955ffbd85ac1d5d3915802e700a8a2784a64c486e94549a39ab78ad50bdefbf1"
Sep 16 14:26:34.986461 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.986417    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dbb84810-549db4cd8-djk7f" event={"ID":"1e12a76d-2722-46ce-9149-ce3487392be4","Type":"ContainerStarted","Data":"b9b4cd469a2d58c037f98e8bdbdf0eb7d26df7a21695633949756fbee97fb3df"}
Sep 16 14:26:34.986628 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.986476    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dbb84810-549db4cd8-djk7f"
Sep 16 14:26:34.988391 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.988343    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-dbb84810-f646d745-hkfg4" event={"ID":"3264f8ac-0b3c-4f2c-be20-d7976302daed","Type":"ContainerStarted","Data":"0ba8bd13b8d59a8a9afe0df731966330c22c72797948cbcaabf40dc5b10ed7b1"}
Sep 16 14:26:34.988591 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.988566    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-dbb84810-f646d745-hkfg4"
Sep 16 14:26:34.988718 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.988602    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-dbb84810-f646d745-hkfg4" event={"ID":"3264f8ac-0b3c-4f2c-be20-d7976302daed","Type":"ContainerStarted","Data":"3903f4455eee6efd150358a16a74687d71cc76a224370c0cf15350a4d2ecfb57"}
Sep 16 14:26:34.989952 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.989912    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-08475e/simulator-w0-08475e-kserve-8cc86df7b-x7z4t"
Sep 16 14:26:34.989952 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.989921    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-08475e/simulator-w0-08475e-kserve-8cc86df7b-x7z4t" event={"ID":"9b899509-2d6c-4c9f-82ba-58bbc8fd07d8","Type":"ContainerDied","Data":"64fb20c7e24802dd112407207a6b8c7bdad0b18f2222b5b1a216c8d29ff80bef"}
Sep 16 14:26:34.991529 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.991475    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-08475e/premium-w0-08475e-kserve-647898d665-rmzkp"
Sep 16 14:26:34.991529 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.991493    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-08475e/premium-w0-08475e-kserve-647898d665-rmzkp" event={"ID":"2f76715c-d553-44c4-995e-9d2c1fe9ce5b","Type":"ContainerDied","Data":"d98450d2e1a7692b5ddae326080066f91c235e8787041e8e90c56e6c3dcac4c6"}
Sep 16 14:26:34.996263 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:34.996045    2580 scope.go:117] "RemoveContainer" containerID="3c70eaf52334a52522a4c6bfa49a0a9e92c394d0dfd71d3f10b72746a6fc9ed4"
Sep 16 14:26:35.005869 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:35.005850    2580 scope.go:117] "RemoveContainer" containerID="4e2e075bca10b0cd17c0e96114130eb1da18bf3ba999a0b09a0f522c0bd4b0c6"
Sep 16 14:26:35.036204 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:35.036122    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-dbb84810-f646d745-hkfg4" podStartSLOduration=1.036106037 podStartE2EDuration="1.036106037s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:26:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:26:35.033138537 +0000 UTC m=+1328.048248397" watchObservedRunningTime="2026-09-16 14:26:35.036106037 +0000 UTC m=+1328.051215896"
Sep 16 14:26:35.110886 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:35.110836    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dbb84810-549db4cd8-djk7f" podStartSLOduration=2.229341781 podStartE2EDuration="3.110817302s" totalImagesPullingTime="881.475521ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:26:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:26:35.108706931 +0000 UTC m=+1328.123816792" watchObservedRunningTime="2026-09-16 14:26:35.110817302 +0000 UTC m=+1328.125927161"
Sep 16 14:26:35.141631 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:35.141596    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-08475e/premium-w0-08475e-kserve-647898d665-rmzkp"]
Sep 16 14:26:35.146854 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:35.146831    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-08475e/premium-w0-08475e-kserve-647898d665-rmzkp"]
Sep 16 14:26:35.168818 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:35.168793    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-08475e/simulator-w0-08475e-kserve-8cc86df7b-x7z4t"]
Sep 16 14:26:35.178780 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:35.178760    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-08475e/simulator-w0-08475e-kserve-8cc86df7b-x7z4t"]
Sep 16 14:26:35.206546 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:35.206524    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-08475e/distinct-2-w0-08475e-kserve-66c89bc459-6fc57"]
Sep 16 14:26:35.218276 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:35.218252    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-08475e/distinct-2-w0-08475e-kserve-66c89bc459-6fc57"]
Sep 16 14:26:35.433536 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:35.433492    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f76715c-d553-44c4-995e-9d2c1fe9ce5b" path="/var/lib/kubelet/pods/2f76715c-d553-44c4-995e-9d2c1fe9ce5b/volumes"
Sep 16 14:26:35.433937 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:35.433924    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b899509-2d6c-4c9f-82ba-58bbc8fd07d8" path="/var/lib/kubelet/pods/9b899509-2d6c-4c9f-82ba-58bbc8fd07d8/volumes"
Sep 16 14:26:35.434290 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:35.434277    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da00cc8d-910a-4c6e-86c7-ca474c5e819c" path="/var/lib/kubelet/pods/da00cc8d-910a-4c6e-86c7-ca474c5e819c/volumes"
Sep 16 14:26:39.794375 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:39.794336    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z"]
Sep 16 14:26:39.795018 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:39.794978    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z" podUID="9cb6f359-dd14-43e9-a795-d56065b12448" containerName="istio-proxy" containerID="cri-o://0d0cfbc2ee09057e5aa07767c3ff7e6c61c575d35bcceb7bc296d3bafd683ab9" gracePeriod=30
Sep 16 14:26:41.004855 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:41.004827    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dbb84810-549db4cd8-djk7f"
Sep 16 14:26:45.049821 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:45.049792    2580 generic.go:356] "Generic (PLEG): container finished" podID="9cb6f359-dd14-43e9-a795-d56065b12448" containerID="0d0cfbc2ee09057e5aa07767c3ff7e6c61c575d35bcceb7bc296d3bafd683ab9" exitCode=0
Sep 16 14:26:45.050133 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:45.049866    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z" event={"ID":"9cb6f359-dd14-43e9-a795-d56065b12448","Type":"ContainerDied","Data":"0d0cfbc2ee09057e5aa07767c3ff7e6c61c575d35bcceb7bc296d3bafd683ab9"}
Sep 16 14:26:45.050133 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:45.049928    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z" event={"ID":"9cb6f359-dd14-43e9-a795-d56065b12448","Type":"ContainerDied","Data":"9746baea3df4d6044e6885d26800a5628d8f68b8c9a686e5b85380370bd91081"}
Sep 16 14:26:45.050133 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:45.049946    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9746baea3df4d6044e6885d26800a5628d8f68b8c9a686e5b85380370bd91081"
Sep 16 14:26:45.050133 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:45.049959    2580 scope.go:117] "RemoveContainer" containerID="823cf112f89ed5507c820a46b4f8fcc167b7eb64c3f4bdb132495d242db621e8"
Sep 16 14:26:45.050568 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:45.050551    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z"
Sep 16 14:26:45.151008 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:45.150930    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9cb6f359-dd14-43e9-a795-d56065b12448-istio-data\") pod \"9cb6f359-dd14-43e9-a795-d56065b12448\" (UID: \"9cb6f359-dd14-43e9-a795-d56065b12448\") "
Sep 16 14:26:45.151008 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:45.150995    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cb6f359-dd14-43e9-a795-d56065b12448-credential-socket\") pod \"9cb6f359-dd14-43e9-a795-d56065b12448\" (UID: \"9cb6f359-dd14-43e9-a795-d56065b12448\") "
Sep 16 14:26:45.151233 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:45.151012    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cb6f359-dd14-43e9-a795-d56065b12448-workload-socket\") pod \"9cb6f359-dd14-43e9-a795-d56065b12448\" (UID: \"9cb6f359-dd14-43e9-a795-d56065b12448\") "
Sep 16 14:26:45.151233 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:45.151050    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9cb6f359-dd14-43e9-a795-d56065b12448-istio-token\") pod \"9cb6f359-dd14-43e9-a795-d56065b12448\" (UID: \"9cb6f359-dd14-43e9-a795-d56065b12448\") "
Sep 16 14:26:45.151233 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:45.151064    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9cb6f359-dd14-43e9-a795-d56065b12448-istio-envoy\") pod \"9cb6f359-dd14-43e9-a795-d56065b12448\" (UID: \"9cb6f359-dd14-43e9-a795-d56065b12448\") "
Sep 16 14:26:45.151233 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:45.151115    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2k96z\" (UniqueName: \"kubernetes.io/projected/9cb6f359-dd14-43e9-a795-d56065b12448-kube-api-access-2k96z\") pod \"9cb6f359-dd14-43e9-a795-d56065b12448\" (UID: \"9cb6f359-dd14-43e9-a795-d56065b12448\") "
Sep 16 14:26:45.151233 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:45.151150    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9cb6f359-dd14-43e9-a795-d56065b12448-istiod-ca-cert\") pod \"9cb6f359-dd14-43e9-a795-d56065b12448\" (UID: \"9cb6f359-dd14-43e9-a795-d56065b12448\") "
Sep 16 14:26:45.151233 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:45.151197    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9cb6f359-dd14-43e9-a795-d56065b12448-istio-podinfo\") pod \"9cb6f359-dd14-43e9-a795-d56065b12448\" (UID: \"9cb6f359-dd14-43e9-a795-d56065b12448\") "
Sep 16 14:26:45.151605 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:45.151218    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cb6f359-dd14-43e9-a795-d56065b12448-istio-data" (OuterVolumeSpecName: "istio-data") pod "9cb6f359-dd14-43e9-a795-d56065b12448" (UID: "9cb6f359-dd14-43e9-a795-d56065b12448"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:26:45.151605 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:45.151241    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9cb6f359-dd14-43e9-a795-d56065b12448-workload-certs\") pod \"9cb6f359-dd14-43e9-a795-d56065b12448\" (UID: \"9cb6f359-dd14-43e9-a795-d56065b12448\") "
Sep 16 14:26:45.151605 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:45.151332    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cb6f359-dd14-43e9-a795-d56065b12448-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9cb6f359-dd14-43e9-a795-d56065b12448" (UID: "9cb6f359-dd14-43e9-a795-d56065b12448"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:26:45.151605 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:45.151335    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cb6f359-dd14-43e9-a795-d56065b12448-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9cb6f359-dd14-43e9-a795-d56065b12448" (UID: "9cb6f359-dd14-43e9-a795-d56065b12448"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:26:45.151605 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:45.151455    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cb6f359-dd14-43e9-a795-d56065b12448-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9cb6f359-dd14-43e9-a795-d56065b12448" (UID: "9cb6f359-dd14-43e9-a795-d56065b12448"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:26:45.151605 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:45.151569    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9cb6f359-dd14-43e9-a795-d56065b12448-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9cb6f359-dd14-43e9-a795-d56065b12448" (UID: "9cb6f359-dd14-43e9-a795-d56065b12448"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:26:45.151909 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:45.151649    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9cb6f359-dd14-43e9-a795-d56065b12448-istiod-ca-cert\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:26:45.151909 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:45.151668    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9cb6f359-dd14-43e9-a795-d56065b12448-workload-certs\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:26:45.151909 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:45.151682    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9cb6f359-dd14-43e9-a795-d56065b12448-istio-data\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:26:45.151909 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:45.151694    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cb6f359-dd14-43e9-a795-d56065b12448-credential-socket\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:26:45.151909 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:45.151709    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cb6f359-dd14-43e9-a795-d56065b12448-workload-socket\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:26:45.153275 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:45.153244    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9cb6f359-dd14-43e9-a795-d56065b12448-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9cb6f359-dd14-43e9-a795-d56065b12448" (UID: "9cb6f359-dd14-43e9-a795-d56065b12448"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 14:26:45.153406 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:45.153289    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9cb6f359-dd14-43e9-a795-d56065b12448-istio-token" (OuterVolumeSpecName: "istio-token") pod "9cb6f359-dd14-43e9-a795-d56065b12448" (UID: "9cb6f359-dd14-43e9-a795-d56065b12448"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:26:45.153406 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:45.153289    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cb6f359-dd14-43e9-a795-d56065b12448-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9cb6f359-dd14-43e9-a795-d56065b12448" (UID: "9cb6f359-dd14-43e9-a795-d56065b12448"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:26:45.153559 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:45.153447    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9cb6f359-dd14-43e9-a795-d56065b12448-kube-api-access-2k96z" (OuterVolumeSpecName: "kube-api-access-2k96z") pod "9cb6f359-dd14-43e9-a795-d56065b12448" (UID: "9cb6f359-dd14-43e9-a795-d56065b12448"). InnerVolumeSpecName "kube-api-access-2k96z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:26:45.252337 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:45.252307    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2k96z\" (UniqueName: \"kubernetes.io/projected/9cb6f359-dd14-43e9-a795-d56065b12448-kube-api-access-2k96z\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:26:45.252337 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:45.252335    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9cb6f359-dd14-43e9-a795-d56065b12448-istio-podinfo\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:26:45.252337 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:45.252344    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9cb6f359-dd14-43e9-a795-d56065b12448-istio-token\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:26:45.252559 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:45.252351    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9cb6f359-dd14-43e9-a795-d56065b12448-istio-envoy\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:26:46.002029 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:46.002000    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-dbb84810-f646d745-hkfg4"
Sep 16 14:26:46.057814 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:46.057788    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z"
Sep 16 14:26:46.114307 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:46.114277    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z"]
Sep 16 14:26:46.122839 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:46.122816    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-08475e-openshift-default-7b9f9dcc8b-tkl2z"]
Sep 16 14:26:47.430887 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:26:47.430858    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9cb6f359-dd14-43e9-a795-d56065b12448" path="/var/lib/kubelet/pods/9cb6f359-dd14-43e9-a795-d56065b12448/volumes"
Sep 16 14:27:19.838603 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:19.838572    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-dbb84810-549db4cd8-djk7f"]
Sep 16 14:27:19.839115 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:19.838768    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dbb84810-549db4cd8-djk7f" podUID="1e12a76d-2722-46ce-9149-ce3487392be4" containerName="maas-api" containerID="cri-o://b9b4cd469a2d58c037f98e8bdbdf0eb7d26df7a21695633949756fbee97fb3df" gracePeriod=30
Sep 16 14:27:20.008971 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:20.008488    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-dbb84810-f646d745-hkfg4"]
Sep 16 14:27:20.008971 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:20.008771    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-dbb84810-f646d745-hkfg4" podUID="3264f8ac-0b3c-4f2c-be20-d7976302daed" containerName="payload-processing" containerID="cri-o://0ba8bd13b8d59a8a9afe0df731966330c22c72797948cbcaabf40dc5b10ed7b1" gracePeriod=30
Sep 16 14:27:20.140983 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:20.140959    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dbb84810-549db4cd8-djk7f"
Sep 16 14:27:20.253269 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:20.253237    2580 generic.go:356] "Generic (PLEG): container finished" podID="1e12a76d-2722-46ce-9149-ce3487392be4" containerID="b9b4cd469a2d58c037f98e8bdbdf0eb7d26df7a21695633949756fbee97fb3df" exitCode=0
Sep 16 14:27:20.253457 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:20.253290    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dbb84810-549db4cd8-djk7f"
Sep 16 14:27:20.253457 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:20.253277    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dbb84810-549db4cd8-djk7f" event={"ID":"1e12a76d-2722-46ce-9149-ce3487392be4","Type":"ContainerDied","Data":"b9b4cd469a2d58c037f98e8bdbdf0eb7d26df7a21695633949756fbee97fb3df"}
Sep 16 14:27:20.253457 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:20.253352    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dbb84810-549db4cd8-djk7f" event={"ID":"1e12a76d-2722-46ce-9149-ce3487392be4","Type":"ContainerDied","Data":"e4fd13db39eb622fab622eb72cd1296f383f56c5deaf40fe6dbdc49ceaae7c41"}
Sep 16 14:27:20.253457 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:20.253382    2580 scope.go:117] "RemoveContainer" containerID="b9b4cd469a2d58c037f98e8bdbdf0eb7d26df7a21695633949756fbee97fb3df"
Sep 16 14:27:20.255259 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:20.255236    2580 generic.go:356] "Generic (PLEG): container finished" podID="3264f8ac-0b3c-4f2c-be20-d7976302daed" containerID="0ba8bd13b8d59a8a9afe0df731966330c22c72797948cbcaabf40dc5b10ed7b1" exitCode=0
Sep 16 14:27:20.255372 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:20.255304    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-dbb84810-f646d745-hkfg4" event={"ID":"3264f8ac-0b3c-4f2c-be20-d7976302daed","Type":"ContainerDied","Data":"0ba8bd13b8d59a8a9afe0df731966330c22c72797948cbcaabf40dc5b10ed7b1"}
Sep 16 14:27:20.263338 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:20.263319    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e12a76d-2722-46ce-9149-ce3487392be4-openshift-service-ca\") pod \"1e12a76d-2722-46ce-9149-ce3487392be4\" (UID: \"1e12a76d-2722-46ce-9149-ce3487392be4\") "
Sep 16 14:27:20.263469 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:20.263421    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e12a76d-2722-46ce-9149-ce3487392be4-metrics-certs\") pod \"1e12a76d-2722-46ce-9149-ce3487392be4\" (UID: \"1e12a76d-2722-46ce-9149-ce3487392be4\") "
Sep 16 14:27:20.263469 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:20.263444    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1e12a76d-2722-46ce-9149-ce3487392be4-maas-api-tls\") pod \"1e12a76d-2722-46ce-9149-ce3487392be4\" (UID: \"1e12a76d-2722-46ce-9149-ce3487392be4\") "
Sep 16 14:27:20.263619 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:20.263536    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kxjt9\" (UniqueName: \"kubernetes.io/projected/1e12a76d-2722-46ce-9149-ce3487392be4-kube-api-access-kxjt9\") pod \"1e12a76d-2722-46ce-9149-ce3487392be4\" (UID: \"1e12a76d-2722-46ce-9149-ce3487392be4\") "
Sep 16 14:27:20.263981 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:20.263893    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e12a76d-2722-46ce-9149-ce3487392be4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1e12a76d-2722-46ce-9149-ce3487392be4" (UID: "1e12a76d-2722-46ce-9149-ce3487392be4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:27:20.265670 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:20.265644    2580 scope.go:117] "RemoveContainer" containerID="b9b4cd469a2d58c037f98e8bdbdf0eb7d26df7a21695633949756fbee97fb3df"
Sep 16 14:27:20.265836 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:20.265799    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e12a76d-2722-46ce-9149-ce3487392be4-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "1e12a76d-2722-46ce-9149-ce3487392be4" (UID: "1e12a76d-2722-46ce-9149-ce3487392be4"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:27:20.266046 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:20.266017    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e12a76d-2722-46ce-9149-ce3487392be4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1e12a76d-2722-46ce-9149-ce3487392be4" (UID: "1e12a76d-2722-46ce-9149-ce3487392be4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:27:20.266147 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:27:20.266013    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9b4cd469a2d58c037f98e8bdbdf0eb7d26df7a21695633949756fbee97fb3df\": container with ID starting with b9b4cd469a2d58c037f98e8bdbdf0eb7d26df7a21695633949756fbee97fb3df not found: ID does not exist" containerID="b9b4cd469a2d58c037f98e8bdbdf0eb7d26df7a21695633949756fbee97fb3df"
Sep 16 14:27:20.266147 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:20.266097    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9b4cd469a2d58c037f98e8bdbdf0eb7d26df7a21695633949756fbee97fb3df"} err="failed to get container status \"b9b4cd469a2d58c037f98e8bdbdf0eb7d26df7a21695633949756fbee97fb3df\": rpc error: code = NotFound desc = could not find container \"b9b4cd469a2d58c037f98e8bdbdf0eb7d26df7a21695633949756fbee97fb3df\": container with ID starting with b9b4cd469a2d58c037f98e8bdbdf0eb7d26df7a21695633949756fbee97fb3df not found: ID does not exist"
Sep 16 14:27:20.266263 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:20.266192    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e12a76d-2722-46ce-9149-ce3487392be4-kube-api-access-kxjt9" (OuterVolumeSpecName: "kube-api-access-kxjt9") pod "1e12a76d-2722-46ce-9149-ce3487392be4" (UID: "1e12a76d-2722-46ce-9149-ce3487392be4"). InnerVolumeSpecName "kube-api-access-kxjt9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:27:20.277184 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:20.277162    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-dbb84810-f646d745-hkfg4"
Sep 16 14:27:20.365066 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:20.365027    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-msmx8\" (UniqueName: \"kubernetes.io/projected/3264f8ac-0b3c-4f2c-be20-d7976302daed-kube-api-access-msmx8\") pod \"3264f8ac-0b3c-4f2c-be20-d7976302daed\" (UID: \"3264f8ac-0b3c-4f2c-be20-d7976302daed\") "
Sep 16 14:27:20.365247 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:20.365118    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3264f8ac-0b3c-4f2c-be20-d7976302daed-plugins-config-volume\") pod \"3264f8ac-0b3c-4f2c-be20-d7976302daed\" (UID: \"3264f8ac-0b3c-4f2c-be20-d7976302daed\") "
Sep 16 14:27:20.365353 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:20.365340    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e12a76d-2722-46ce-9149-ce3487392be4-openshift-service-ca\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:27:20.365415 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:20.365356    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e12a76d-2722-46ce-9149-ce3487392be4-metrics-certs\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:27:20.365415 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:20.365365    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1e12a76d-2722-46ce-9149-ce3487392be4-maas-api-tls\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:27:20.365415 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:20.365374    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kxjt9\" (UniqueName: \"kubernetes.io/projected/1e12a76d-2722-46ce-9149-ce3487392be4-kube-api-access-kxjt9\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:27:20.365563 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:20.365476    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3264f8ac-0b3c-4f2c-be20-d7976302daed-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3264f8ac-0b3c-4f2c-be20-d7976302daed" (UID: "3264f8ac-0b3c-4f2c-be20-d7976302daed"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:27:20.367094 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:20.367075    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3264f8ac-0b3c-4f2c-be20-d7976302daed-kube-api-access-msmx8" (OuterVolumeSpecName: "kube-api-access-msmx8") pod "3264f8ac-0b3c-4f2c-be20-d7976302daed" (UID: "3264f8ac-0b3c-4f2c-be20-d7976302daed"). InnerVolumeSpecName "kube-api-access-msmx8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:27:20.466452 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:20.466418    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3264f8ac-0b3c-4f2c-be20-d7976302daed-plugins-config-volume\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:27:20.466452 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:20.466449    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-msmx8\" (UniqueName: \"kubernetes.io/projected/3264f8ac-0b3c-4f2c-be20-d7976302daed-kube-api-access-msmx8\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:27:20.584339 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:20.584309    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-dbb84810-549db4cd8-djk7f"]
Sep 16 14:27:20.590540 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:20.590515    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-dbb84810-549db4cd8-djk7f"]
Sep 16 14:27:21.263238 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:21.263213    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-dbb84810-f646d745-hkfg4"
Sep 16 14:27:21.263238 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:21.263228    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-dbb84810-f646d745-hkfg4" event={"ID":"3264f8ac-0b3c-4f2c-be20-d7976302daed","Type":"ContainerDied","Data":"3903f4455eee6efd150358a16a74687d71cc76a224370c0cf15350a4d2ecfb57"}
Sep 16 14:27:21.263752 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:21.263289    2580 scope.go:117] "RemoveContainer" containerID="0ba8bd13b8d59a8a9afe0df731966330c22c72797948cbcaabf40dc5b10ed7b1"
Sep 16 14:27:21.293622 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:21.293599    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-dbb84810-f646d745-hkfg4"]
Sep 16 14:27:21.298987 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:21.298966    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-dbb84810-f646d745-hkfg4"]
Sep 16 14:27:21.430152 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:21.430123    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e12a76d-2722-46ce-9149-ce3487392be4" path="/var/lib/kubelet/pods/1e12a76d-2722-46ce-9149-ce3487392be4/volumes"
Sep 16 14:27:21.430483 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:21.430470    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3264f8ac-0b3c-4f2c-be20-d7976302daed" path="/var/lib/kubelet/pods/3264f8ac-0b3c-4f2c-be20-d7976302daed/volumes"
Sep 16 14:27:27.508271 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:27.508235    2580 scope.go:117] "RemoveContainer" containerID="0d0cfbc2ee09057e5aa07767c3ff7e6c61c575d35bcceb7bc296d3bafd683ab9"
Sep 16 14:27:53.986933 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:53.986899    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-2f9be3-96bb88d79-nvqx7"]
Sep 16 14:27:53.987480 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:53.987458    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e12a76d-2722-46ce-9149-ce3487392be4" containerName="maas-api"
Sep 16 14:27:53.987644 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:53.987621    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e12a76d-2722-46ce-9149-ce3487392be4" containerName="maas-api"
Sep 16 14:27:53.987777 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:53.987649    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3264f8ac-0b3c-4f2c-be20-d7976302daed" containerName="payload-processing"
Sep 16 14:27:53.987777 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:53.987661    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3264f8ac-0b3c-4f2c-be20-d7976302daed" containerName="payload-processing"
Sep 16 14:27:53.987777 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:53.987677    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9cb6f359-dd14-43e9-a795-d56065b12448" containerName="istio-proxy"
Sep 16 14:27:53.987777 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:53.987685    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9cb6f359-dd14-43e9-a795-d56065b12448" containerName="istio-proxy"
Sep 16 14:27:53.987988 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:53.987806    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="3264f8ac-0b3c-4f2c-be20-d7976302daed" containerName="payload-processing"
Sep 16 14:27:53.987988 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:53.987818    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9cb6f359-dd14-43e9-a795-d56065b12448" containerName="istio-proxy"
Sep 16 14:27:53.987988 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:53.987828    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9cb6f359-dd14-43e9-a795-d56065b12448" containerName="istio-proxy"
Sep 16 14:27:53.987988 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:53.987838    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e12a76d-2722-46ce-9149-ce3487392be4" containerName="maas-api"
Sep 16 14:27:53.992709 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:53.992689    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-2f9be3-96bb88d79-nvqx7"
Sep 16 14:27:53.995426 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:53.995384    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w1-2f9be3-serving-cert\""
Sep 16 14:27:54.006696 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:54.006674    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-2f9be3-96bb88d79-nvqx7"]
Sep 16 14:27:54.059680 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:54.059648    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac9b3cd6-0c4e-4aa5-96d8-4d92646548ca-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w1-2f9be3-96bb88d79-nvqx7\" (UID: \"ac9b3cd6-0c4e-4aa5-96d8-4d92646548ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-2f9be3-96bb88d79-nvqx7"
Sep 16 14:27:54.059865 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:54.059702    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djwwg\" (UniqueName: \"kubernetes.io/projected/ac9b3cd6-0c4e-4aa5-96d8-4d92646548ca-kube-api-access-djwwg\") pod \"maas-api-e2e-shared-a-w1-2f9be3-96bb88d79-nvqx7\" (UID: \"ac9b3cd6-0c4e-4aa5-96d8-4d92646548ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-2f9be3-96bb88d79-nvqx7"
Sep 16 14:27:54.059865 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:54.059736    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac9b3cd6-0c4e-4aa5-96d8-4d92646548ca-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-2f9be3-96bb88d79-nvqx7\" (UID: \"ac9b3cd6-0c4e-4aa5-96d8-4d92646548ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-2f9be3-96bb88d79-nvqx7"
Sep 16 14:27:54.059865 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:54.059842    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac9b3cd6-0c4e-4aa5-96d8-4d92646548ca-metrics-certs\") pod \"maas-api-e2e-shared-a-w1-2f9be3-96bb88d79-nvqx7\" (UID: \"ac9b3cd6-0c4e-4aa5-96d8-4d92646548ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-2f9be3-96bb88d79-nvqx7"
Sep 16 14:27:54.160641 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:54.160603    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac9b3cd6-0c4e-4aa5-96d8-4d92646548ca-metrics-certs\") pod \"maas-api-e2e-shared-a-w1-2f9be3-96bb88d79-nvqx7\" (UID: \"ac9b3cd6-0c4e-4aa5-96d8-4d92646548ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-2f9be3-96bb88d79-nvqx7"
Sep 16 14:27:54.160641 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:54.160646    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac9b3cd6-0c4e-4aa5-96d8-4d92646548ca-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w1-2f9be3-96bb88d79-nvqx7\" (UID: \"ac9b3cd6-0c4e-4aa5-96d8-4d92646548ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-2f9be3-96bb88d79-nvqx7"
Sep 16 14:27:54.160895 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:54.160682    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djwwg\" (UniqueName: \"kubernetes.io/projected/ac9b3cd6-0c4e-4aa5-96d8-4d92646548ca-kube-api-access-djwwg\") pod \"maas-api-e2e-shared-a-w1-2f9be3-96bb88d79-nvqx7\" (UID: \"ac9b3cd6-0c4e-4aa5-96d8-4d92646548ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-2f9be3-96bb88d79-nvqx7"
Sep 16 14:27:54.160895 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:54.160705    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac9b3cd6-0c4e-4aa5-96d8-4d92646548ca-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-2f9be3-96bb88d79-nvqx7\" (UID: \"ac9b3cd6-0c4e-4aa5-96d8-4d92646548ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-2f9be3-96bb88d79-nvqx7"
Sep 16 14:27:54.161327 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:54.161305    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac9b3cd6-0c4e-4aa5-96d8-4d92646548ca-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w1-2f9be3-96bb88d79-nvqx7\" (UID: \"ac9b3cd6-0c4e-4aa5-96d8-4d92646548ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-2f9be3-96bb88d79-nvqx7"
Sep 16 14:27:54.163088 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:54.163065    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac9b3cd6-0c4e-4aa5-96d8-4d92646548ca-metrics-certs\") pod \"maas-api-e2e-shared-a-w1-2f9be3-96bb88d79-nvqx7\" (UID: \"ac9b3cd6-0c4e-4aa5-96d8-4d92646548ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-2f9be3-96bb88d79-nvqx7"
Sep 16 14:27:54.163251 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:54.163232    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac9b3cd6-0c4e-4aa5-96d8-4d92646548ca-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-2f9be3-96bb88d79-nvqx7\" (UID: \"ac9b3cd6-0c4e-4aa5-96d8-4d92646548ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-2f9be3-96bb88d79-nvqx7"
Sep 16 14:27:54.171394 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:54.171374    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djwwg\" (UniqueName: \"kubernetes.io/projected/ac9b3cd6-0c4e-4aa5-96d8-4d92646548ca-kube-api-access-djwwg\") pod \"maas-api-e2e-shared-a-w1-2f9be3-96bb88d79-nvqx7\" (UID: \"ac9b3cd6-0c4e-4aa5-96d8-4d92646548ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-2f9be3-96bb88d79-nvqx7"
Sep 16 14:27:54.329353 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:54.329256    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-2f9be3-96bb88d79-nvqx7"
Sep 16 14:27:54.618550 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:54.618524    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-2f9be3-96bb88d79-nvqx7"]
Sep 16 14:27:54.620868 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:27:54.620843    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac9b3cd6_0c4e_4aa5_96d8_4d92646548ca.slice/crio-cfc844f8d94cf0ed6be2eaa94367ba426f8b045715e101493e53f7f4ab389666 WatchSource:0}: Error finding container cfc844f8d94cf0ed6be2eaa94367ba426f8b045715e101493e53f7f4ab389666: Status 404 returned error can't find the container with id cfc844f8d94cf0ed6be2eaa94367ba426f8b045715e101493e53f7f4ab389666
Sep 16 14:27:55.074651 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:55.074626    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-2f9be3-b566b5c74-6m2r7"]
Sep 16 14:27:55.075024 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:55.075010    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9cb6f359-dd14-43e9-a795-d56065b12448" containerName="istio-proxy"
Sep 16 14:27:55.075024 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:55.075025    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9cb6f359-dd14-43e9-a795-d56065b12448" containerName="istio-proxy"
Sep 16 14:27:55.079316 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:55.079297    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-2f9be3-b566b5c74-6m2r7"
Sep 16 14:27:55.083531 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:55.083510    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w1-2f9be3-dockercfg-xhmq8\""
Sep 16 14:27:55.084483 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:55.084462    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w1-2f9be3\""
Sep 16 14:27:55.098995 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:55.098969    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-2f9be3-b566b5c74-6m2r7"]
Sep 16 14:27:55.172284 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:55.172251    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hnjw\" (UniqueName: \"kubernetes.io/projected/fa6c5500-4785-4955-bb2b-ceec12a2ea84-kube-api-access-4hnjw\") pod \"payload-pre-processing-e2e-shared-a-w1-2f9be3-b566b5c74-6m2r7\" (UID: \"fa6c5500-4785-4955-bb2b-ceec12a2ea84\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-2f9be3-b566b5c74-6m2r7"
Sep 16 14:27:55.172460 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:55.172289    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa6c5500-4785-4955-bb2b-ceec12a2ea84-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w1-2f9be3-b566b5c74-6m2r7\" (UID: \"fa6c5500-4785-4955-bb2b-ceec12a2ea84\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-2f9be3-b566b5c74-6m2r7"
Sep 16 14:27:55.273587 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:55.273553    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hnjw\" (UniqueName: \"kubernetes.io/projected/fa6c5500-4785-4955-bb2b-ceec12a2ea84-kube-api-access-4hnjw\") pod \"payload-pre-processing-e2e-shared-a-w1-2f9be3-b566b5c74-6m2r7\" (UID: \"fa6c5500-4785-4955-bb2b-ceec12a2ea84\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-2f9be3-b566b5c74-6m2r7"
Sep 16 14:27:55.273758 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:55.273594    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa6c5500-4785-4955-bb2b-ceec12a2ea84-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w1-2f9be3-b566b5c74-6m2r7\" (UID: \"fa6c5500-4785-4955-bb2b-ceec12a2ea84\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-2f9be3-b566b5c74-6m2r7"
Sep 16 14:27:55.274475 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:55.274450    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa6c5500-4785-4955-bb2b-ceec12a2ea84-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w1-2f9be3-b566b5c74-6m2r7\" (UID: \"fa6c5500-4785-4955-bb2b-ceec12a2ea84\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-2f9be3-b566b5c74-6m2r7"
Sep 16 14:27:55.281997 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:55.281972    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hnjw\" (UniqueName: \"kubernetes.io/projected/fa6c5500-4785-4955-bb2b-ceec12a2ea84-kube-api-access-4hnjw\") pod \"payload-pre-processing-e2e-shared-a-w1-2f9be3-b566b5c74-6m2r7\" (UID: \"fa6c5500-4785-4955-bb2b-ceec12a2ea84\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-2f9be3-b566b5c74-6m2r7"
Sep 16 14:27:55.391107 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:55.391031    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-2f9be3-b566b5c74-6m2r7"
Sep 16 14:27:55.459729 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:55.459698    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-2f9be3-96bb88d79-nvqx7" event={"ID":"ac9b3cd6-0c4e-4aa5-96d8-4d92646548ca","Type":"ContainerStarted","Data":"62569fb35ab1aa1a7784a79e989790492e1a8b02fc0c43a2628c7771f865e1b6"}
Sep 16 14:27:55.459864 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:55.459758    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-2f9be3-96bb88d79-nvqx7" event={"ID":"ac9b3cd6-0c4e-4aa5-96d8-4d92646548ca","Type":"ContainerStarted","Data":"cfc844f8d94cf0ed6be2eaa94367ba426f8b045715e101493e53f7f4ab389666"}
Sep 16 14:27:55.459937 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:55.459877    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-2f9be3-96bb88d79-nvqx7"
Sep 16 14:27:55.488210 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:55.488164    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-2f9be3-96bb88d79-nvqx7" podStartSLOduration=2.075085085 podStartE2EDuration="2.488148536s" totalImagesPullingTime="413.063451ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:27:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:27:55.486556638 +0000 UTC m=+1408.501666498" watchObservedRunningTime="2026-09-16 14:27:55.488148536 +0000 UTC m=+1408.503258397"
Sep 16 14:27:55.672620 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:55.672592    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-2f9be3-b566b5c74-6m2r7"]
Sep 16 14:27:55.674078 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:27:55.674048    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa6c5500_4785_4955_bb2b_ceec12a2ea84.slice/crio-007c361b64c301e3d6644a1943102a39db195f3039837e8b6ab335ce9c3c5c16 WatchSource:0}: Error finding container 007c361b64c301e3d6644a1943102a39db195f3039837e8b6ab335ce9c3c5c16: Status 404 returned error can't find the container with id 007c361b64c301e3d6644a1943102a39db195f3039837e8b6ab335ce9c3c5c16
Sep 16 14:27:56.467813 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:56.467783    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-2f9be3-b566b5c74-6m2r7" event={"ID":"fa6c5500-4785-4955-bb2b-ceec12a2ea84","Type":"ContainerStarted","Data":"391c9023067ea2f0efb23da5ccac9461072bf09d200977d6a02a7303e2b3e491"}
Sep 16 14:27:56.468263 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:56.467830    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-2f9be3-b566b5c74-6m2r7" event={"ID":"fa6c5500-4785-4955-bb2b-ceec12a2ea84","Type":"ContainerStarted","Data":"007c361b64c301e3d6644a1943102a39db195f3039837e8b6ab335ce9c3c5c16"}
Sep 16 14:27:56.468263 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:56.468015    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-2f9be3-b566b5c74-6m2r7"
Sep 16 14:27:56.502534 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:27:56.502474    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-2f9be3-b566b5c74-6m2r7" podStartSLOduration=1.502460275 podStartE2EDuration="1.502460275s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:27:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:27:56.501050036 +0000 UTC m=+1409.516159895" watchObservedRunningTime="2026-09-16 14:27:56.502460275 +0000 UTC m=+1409.517570155"
Sep 16 14:28:01.473543 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:01.473512    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-2f9be3-96bb88d79-nvqx7"
Sep 16 14:28:07.475122 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:07.475092    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-2f9be3-b566b5c74-6m2r7"
Sep 16 14:28:22.246189 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:22.246105    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv"]
Sep 16 14:28:22.249866 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:22.249846    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv"
Sep 16 14:28:22.255809 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:22.255788    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w1-cbeb11-openshift-default-dockercfg-jxvq5\""
Sep 16 14:28:22.280006 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:22.279978    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv"]
Sep 16 14:28:22.427650 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:22.427620    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0f4134f-f3be-492c-921f-a371c8d204a4-credential-socket\") pod \"e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv\" (UID: \"e0f4134f-f3be-492c-921f-a371c8d204a4\") " pod="openshift-ingress/e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv"
Sep 16 14:28:22.427650 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:22.427656    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtv4b\" (UniqueName: \"kubernetes.io/projected/e0f4134f-f3be-492c-921f-a371c8d204a4-kube-api-access-wtv4b\") pod \"e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv\" (UID: \"e0f4134f-f3be-492c-921f-a371c8d204a4\") " pod="openshift-ingress/e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv"
Sep 16 14:28:22.427894 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:22.427676    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0f4134f-f3be-492c-921f-a371c8d204a4-workload-certs\") pod \"e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv\" (UID: \"e0f4134f-f3be-492c-921f-a371c8d204a4\") " pod="openshift-ingress/e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv"
Sep 16 14:28:22.427894 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:22.427729    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0f4134f-f3be-492c-921f-a371c8d204a4-istio-token\") pod \"e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv\" (UID: \"e0f4134f-f3be-492c-921f-a371c8d204a4\") " pod="openshift-ingress/e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv"
Sep 16 14:28:22.427894 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:22.427756    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0f4134f-f3be-492c-921f-a371c8d204a4-istio-envoy\") pod \"e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv\" (UID: \"e0f4134f-f3be-492c-921f-a371c8d204a4\") " pod="openshift-ingress/e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv"
Sep 16 14:28:22.427894 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:22.427783    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0f4134f-f3be-492c-921f-a371c8d204a4-istio-podinfo\") pod \"e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv\" (UID: \"e0f4134f-f3be-492c-921f-a371c8d204a4\") " pod="openshift-ingress/e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv"
Sep 16 14:28:22.427894 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:22.427837    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0f4134f-f3be-492c-921f-a371c8d204a4-istiod-ca-cert\") pod \"e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv\" (UID: \"e0f4134f-f3be-492c-921f-a371c8d204a4\") " pod="openshift-ingress/e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv"
Sep 16 14:28:22.428109 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:22.427897    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0f4134f-f3be-492c-921f-a371c8d204a4-istio-data\") pod \"e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv\" (UID: \"e0f4134f-f3be-492c-921f-a371c8d204a4\") " pod="openshift-ingress/e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv"
Sep 16 14:28:22.428109 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:22.427944    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0f4134f-f3be-492c-921f-a371c8d204a4-workload-socket\") pod \"e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv\" (UID: \"e0f4134f-f3be-492c-921f-a371c8d204a4\") " pod="openshift-ingress/e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv"
Sep 16 14:28:22.529370 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:22.529292    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0f4134f-f3be-492c-921f-a371c8d204a4-istio-podinfo\") pod \"e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv\" (UID: \"e0f4134f-f3be-492c-921f-a371c8d204a4\") " pod="openshift-ingress/e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv"
Sep 16 14:28:22.529370 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:22.529330    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0f4134f-f3be-492c-921f-a371c8d204a4-istiod-ca-cert\") pod \"e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv\" (UID: \"e0f4134f-f3be-492c-921f-a371c8d204a4\") " pod="openshift-ingress/e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv"
Sep 16 14:28:22.529370 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:22.529350    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0f4134f-f3be-492c-921f-a371c8d204a4-istio-data\") pod \"e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv\" (UID: \"e0f4134f-f3be-492c-921f-a371c8d204a4\") " pod="openshift-ingress/e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv"
Sep 16 14:28:22.529681 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:22.529374    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0f4134f-f3be-492c-921f-a371c8d204a4-workload-socket\") pod \"e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv\" (UID: \"e0f4134f-f3be-492c-921f-a371c8d204a4\") " pod="openshift-ingress/e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv"
Sep 16 14:28:22.529681 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:22.529421    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0f4134f-f3be-492c-921f-a371c8d204a4-credential-socket\") pod \"e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv\" (UID: \"e0f4134f-f3be-492c-921f-a371c8d204a4\") " pod="openshift-ingress/e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv"
Sep 16 14:28:22.529681 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:22.529442    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtv4b\" (UniqueName: \"kubernetes.io/projected/e0f4134f-f3be-492c-921f-a371c8d204a4-kube-api-access-wtv4b\") pod \"e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv\" (UID: \"e0f4134f-f3be-492c-921f-a371c8d204a4\") " pod="openshift-ingress/e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv"
Sep 16 14:28:22.529681 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:22.529469    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0f4134f-f3be-492c-921f-a371c8d204a4-workload-certs\") pod \"e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv\" (UID: \"e0f4134f-f3be-492c-921f-a371c8d204a4\") " pod="openshift-ingress/e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv"
Sep 16 14:28:22.529681 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:22.529557    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0f4134f-f3be-492c-921f-a371c8d204a4-istio-token\") pod \"e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv\" (UID: \"e0f4134f-f3be-492c-921f-a371c8d204a4\") " pod="openshift-ingress/e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv"
Sep 16 14:28:22.529681 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:22.529634    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0f4134f-f3be-492c-921f-a371c8d204a4-istio-envoy\") pod \"e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv\" (UID: \"e0f4134f-f3be-492c-921f-a371c8d204a4\") " pod="openshift-ingress/e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv"
Sep 16 14:28:22.529988 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:22.529783    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0f4134f-f3be-492c-921f-a371c8d204a4-istio-data\") pod \"e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv\" (UID: \"e0f4134f-f3be-492c-921f-a371c8d204a4\") " pod="openshift-ingress/e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv"
Sep 16 14:28:22.529988 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:22.529956    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0f4134f-f3be-492c-921f-a371c8d204a4-credential-socket\") pod \"e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv\" (UID: \"e0f4134f-f3be-492c-921f-a371c8d204a4\") " pod="openshift-ingress/e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv"
Sep 16 14:28:22.530174 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:22.530013    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0f4134f-f3be-492c-921f-a371c8d204a4-workload-certs\") pod \"e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv\" (UID: \"e0f4134f-f3be-492c-921f-a371c8d204a4\") " pod="openshift-ingress/e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv"
Sep 16 14:28:22.530174 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:22.530069    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0f4134f-f3be-492c-921f-a371c8d204a4-workload-socket\") pod \"e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv\" (UID: \"e0f4134f-f3be-492c-921f-a371c8d204a4\") " pod="openshift-ingress/e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv"
Sep 16 14:28:22.530174 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:22.530130    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0f4134f-f3be-492c-921f-a371c8d204a4-istiod-ca-cert\") pod \"e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv\" (UID: \"e0f4134f-f3be-492c-921f-a371c8d204a4\") " pod="openshift-ingress/e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv"
Sep 16 14:28:22.531956 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:22.531936    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0f4134f-f3be-492c-921f-a371c8d204a4-istio-envoy\") pod \"e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv\" (UID: \"e0f4134f-f3be-492c-921f-a371c8d204a4\") " pod="openshift-ingress/e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv"
Sep 16 14:28:22.532233 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:22.532215    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0f4134f-f3be-492c-921f-a371c8d204a4-istio-podinfo\") pod \"e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv\" (UID: \"e0f4134f-f3be-492c-921f-a371c8d204a4\") " pod="openshift-ingress/e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv"
Sep 16 14:28:22.547695 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:22.547671    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0f4134f-f3be-492c-921f-a371c8d204a4-istio-token\") pod \"e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv\" (UID: \"e0f4134f-f3be-492c-921f-a371c8d204a4\") " pod="openshift-ingress/e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv"
Sep 16 14:28:22.547898 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:22.547883    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtv4b\" (UniqueName: \"kubernetes.io/projected/e0f4134f-f3be-492c-921f-a371c8d204a4-kube-api-access-wtv4b\") pod \"e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv\" (UID: \"e0f4134f-f3be-492c-921f-a371c8d204a4\") " pod="openshift-ingress/e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv"
Sep 16 14:28:22.560438 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:22.560418    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv"
Sep 16 14:28:22.852256 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:22.852223    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv"]
Sep 16 14:28:22.853589 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:28:22.853561    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0f4134f_f3be_492c_921f_a371c8d204a4.slice/crio-96eb3bc906e1a28d2020a92cc885edf57e145922c6c08596f4c45612c67c3cce WatchSource:0}: Error finding container 96eb3bc906e1a28d2020a92cc885edf57e145922c6c08596f4c45612c67c3cce: Status 404 returned error can't find the container with id 96eb3bc906e1a28d2020a92cc885edf57e145922c6c08596f4c45612c67c3cce
Sep 16 14:28:22.855702 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:22.855670    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 14:28:22.855786 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:22.855732    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 14:28:22.855786 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:22.855760    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 14:28:23.615658 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:23.615609    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv" event={"ID":"e0f4134f-f3be-492c-921f-a371c8d204a4","Type":"ContainerStarted","Data":"866add6938b768b31ac90398c44051bacc61dee65be546d7094ccbc4dd5b0468"}
Sep 16 14:28:23.616090 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:23.615678    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv" event={"ID":"e0f4134f-f3be-492c-921f-a371c8d204a4","Type":"ContainerStarted","Data":"96eb3bc906e1a28d2020a92cc885edf57e145922c6c08596f4c45612c67c3cce"}
Sep 16 14:28:23.643153 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:23.643100    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv" podStartSLOduration=1.643083836 podStartE2EDuration="1.643083836s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:28:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:28:23.639821504 +0000 UTC m=+1436.654931365" watchObservedRunningTime="2026-09-16 14:28:23.643083836 +0000 UTC m=+1436.658193696"
Sep 16 14:28:24.560846 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:24.560816    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv"
Sep 16 14:28:24.566011 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:24.565984    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv"
Sep 16 14:28:24.621330 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:24.621293    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv"
Sep 16 14:28:24.622205 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:24.622185    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv"
Sep 16 14:28:27.967358 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:27.967260    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-cbeb11-69d98645fc-bwhqw"]
Sep 16 14:28:27.971591 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:27.971567    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-cbeb11-69d98645fc-bwhqw"
Sep 16 14:28:27.974349 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:27.974324    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w1-cbeb11-serving-cert\""
Sep 16 14:28:27.975697 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:27.975665    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/26764d22-8352-4c06-98c1-1097e10c7e7d-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-cbeb11-69d98645fc-bwhqw\" (UID: \"26764d22-8352-4c06-98c1-1097e10c7e7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-cbeb11-69d98645fc-bwhqw"
Sep 16 14:28:27.975817 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:27.975802    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26764d22-8352-4c06-98c1-1097e10c7e7d-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-cbeb11-69d98645fc-bwhqw\" (UID: \"26764d22-8352-4c06-98c1-1097e10c7e7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-cbeb11-69d98645fc-bwhqw"
Sep 16 14:28:27.975885 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:27.975836    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/26764d22-8352-4c06-98c1-1097e10c7e7d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-cbeb11-69d98645fc-bwhqw\" (UID: \"26764d22-8352-4c06-98c1-1097e10c7e7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-cbeb11-69d98645fc-bwhqw"
Sep 16 14:28:27.975885 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:27.975869    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-956ws\" (UniqueName: \"kubernetes.io/projected/26764d22-8352-4c06-98c1-1097e10c7e7d-kube-api-access-956ws\") pod \"maas-api-e2e-shared-b-w1-cbeb11-69d98645fc-bwhqw\" (UID: \"26764d22-8352-4c06-98c1-1097e10c7e7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-cbeb11-69d98645fc-bwhqw"
Sep 16 14:28:27.983551 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:27.983526    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-cbeb11-69d98645fc-bwhqw"]
Sep 16 14:28:28.076726 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:28.076697    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-956ws\" (UniqueName: \"kubernetes.io/projected/26764d22-8352-4c06-98c1-1097e10c7e7d-kube-api-access-956ws\") pod \"maas-api-e2e-shared-b-w1-cbeb11-69d98645fc-bwhqw\" (UID: \"26764d22-8352-4c06-98c1-1097e10c7e7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-cbeb11-69d98645fc-bwhqw"
Sep 16 14:28:28.076970 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:28.076948    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/26764d22-8352-4c06-98c1-1097e10c7e7d-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-cbeb11-69d98645fc-bwhqw\" (UID: \"26764d22-8352-4c06-98c1-1097e10c7e7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-cbeb11-69d98645fc-bwhqw"
Sep 16 14:28:28.077137 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:28:28.077113    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-cbeb11-serving-cert: secret "maas-api-e2e-shared-b-w1-cbeb11-serving-cert" not found
Sep 16 14:28:28.077363 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:28:28.077347    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26764d22-8352-4c06-98c1-1097e10c7e7d-maas-api-tls podName:26764d22-8352-4c06-98c1-1097e10c7e7d nodeName:}" failed. No retries permitted until 2026-09-16 14:28:28.577323234 +0000 UTC m=+1441.592433072 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/26764d22-8352-4c06-98c1-1097e10c7e7d-maas-api-tls") pod "maas-api-e2e-shared-b-w1-cbeb11-69d98645fc-bwhqw" (UID: "26764d22-8352-4c06-98c1-1097e10c7e7d") : secret "maas-api-e2e-shared-b-w1-cbeb11-serving-cert" not found
Sep 16 14:28:28.077575 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:28.077141    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26764d22-8352-4c06-98c1-1097e10c7e7d-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-cbeb11-69d98645fc-bwhqw\" (UID: \"26764d22-8352-4c06-98c1-1097e10c7e7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-cbeb11-69d98645fc-bwhqw"
Sep 16 14:28:28.077818 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:28.077785    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/26764d22-8352-4c06-98c1-1097e10c7e7d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-cbeb11-69d98645fc-bwhqw\" (UID: \"26764d22-8352-4c06-98c1-1097e10c7e7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-cbeb11-69d98645fc-bwhqw"
Sep 16 14:28:28.078466 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:28.078442    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/26764d22-8352-4c06-98c1-1097e10c7e7d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-cbeb11-69d98645fc-bwhqw\" (UID: \"26764d22-8352-4c06-98c1-1097e10c7e7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-cbeb11-69d98645fc-bwhqw"
Sep 16 14:28:28.081052 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:28.081029    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26764d22-8352-4c06-98c1-1097e10c7e7d-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-cbeb11-69d98645fc-bwhqw\" (UID: \"26764d22-8352-4c06-98c1-1097e10c7e7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-cbeb11-69d98645fc-bwhqw"
Sep 16 14:28:28.085814 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:28.085790    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-956ws\" (UniqueName: \"kubernetes.io/projected/26764d22-8352-4c06-98c1-1097e10c7e7d-kube-api-access-956ws\") pod \"maas-api-e2e-shared-b-w1-cbeb11-69d98645fc-bwhqw\" (UID: \"26764d22-8352-4c06-98c1-1097e10c7e7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-cbeb11-69d98645fc-bwhqw"
Sep 16 14:28:28.581098 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:28.581067    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/26764d22-8352-4c06-98c1-1097e10c7e7d-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-cbeb11-69d98645fc-bwhqw\" (UID: \"26764d22-8352-4c06-98c1-1097e10c7e7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-cbeb11-69d98645fc-bwhqw"
Sep 16 14:28:28.583358 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:28.583331    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/26764d22-8352-4c06-98c1-1097e10c7e7d-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-cbeb11-69d98645fc-bwhqw\" (UID: \"26764d22-8352-4c06-98c1-1097e10c7e7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-cbeb11-69d98645fc-bwhqw"
Sep 16 14:28:28.585109 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:28.585093    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-cbeb11-69d98645fc-bwhqw"
Sep 16 14:28:28.776793 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:28.776763    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-cbeb11-549d695445-rztrz"]
Sep 16 14:28:28.780440 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:28.780423    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-cbeb11-549d695445-rztrz"
Sep 16 14:28:28.782841 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:28.782820    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d25388e0-508a-4169-b297-82e555257170-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w1-cbeb11-549d695445-rztrz\" (UID: \"d25388e0-508a-4169-b297-82e555257170\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-cbeb11-549d695445-rztrz"
Sep 16 14:28:28.782938 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:28.782857    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9md5v\" (UniqueName: \"kubernetes.io/projected/d25388e0-508a-4169-b297-82e555257170-kube-api-access-9md5v\") pod \"payload-processing-e2e-shared-b-w1-cbeb11-549d695445-rztrz\" (UID: \"d25388e0-508a-4169-b297-82e555257170\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-cbeb11-549d695445-rztrz"
Sep 16 14:28:28.789725 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:28.789677    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w1-cbeb11\""
Sep 16 14:28:28.789838 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:28.789745    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w1-cbeb11-dockercfg-twskv\""
Sep 16 14:28:28.809451 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:28.809423    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-cbeb11-549d695445-rztrz"]
Sep 16 14:28:28.884377 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:28.884297    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9md5v\" (UniqueName: \"kubernetes.io/projected/d25388e0-508a-4169-b297-82e555257170-kube-api-access-9md5v\") pod \"payload-processing-e2e-shared-b-w1-cbeb11-549d695445-rztrz\" (UID: \"d25388e0-508a-4169-b297-82e555257170\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-cbeb11-549d695445-rztrz"
Sep 16 14:28:28.884551 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:28.884387    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d25388e0-508a-4169-b297-82e555257170-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w1-cbeb11-549d695445-rztrz\" (UID: \"d25388e0-508a-4169-b297-82e555257170\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-cbeb11-549d695445-rztrz"
Sep 16 14:28:28.885022 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:28.885005    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d25388e0-508a-4169-b297-82e555257170-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w1-cbeb11-549d695445-rztrz\" (UID: \"d25388e0-508a-4169-b297-82e555257170\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-cbeb11-549d695445-rztrz"
Sep 16 14:28:28.897598 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:28.897572    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-cbeb11-69d98645fc-bwhqw"]
Sep 16 14:28:28.900173 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:28:28.900149    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26764d22_8352_4c06_98c1_1097e10c7e7d.slice/crio-c154cb268489bcceb616c2522a01583ba683c854268ad153aa295ba6b01dbfd2 WatchSource:0}: Error finding container c154cb268489bcceb616c2522a01583ba683c854268ad153aa295ba6b01dbfd2: Status 404 returned error can't find the container with id c154cb268489bcceb616c2522a01583ba683c854268ad153aa295ba6b01dbfd2
Sep 16 14:28:28.901140 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:28.901123    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9md5v\" (UniqueName: \"kubernetes.io/projected/d25388e0-508a-4169-b297-82e555257170-kube-api-access-9md5v\") pod \"payload-processing-e2e-shared-b-w1-cbeb11-549d695445-rztrz\" (UID: \"d25388e0-508a-4169-b297-82e555257170\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-cbeb11-549d695445-rztrz"
Sep 16 14:28:29.092879 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:29.092852    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-cbeb11-549d695445-rztrz"
Sep 16 14:28:29.381256 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:29.381201    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-cbeb11-549d695445-rztrz"]
Sep 16 14:28:29.387682 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:28:29.387658    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd25388e0_508a_4169_b297_82e555257170.slice/crio-86e6eb2eae532f0ac374ee8ffd8b59d9d3dff0a746209de86e0cab116ecb385c WatchSource:0}: Error finding container 86e6eb2eae532f0ac374ee8ffd8b59d9d3dff0a746209de86e0cab116ecb385c: Status 404 returned error can't find the container with id 86e6eb2eae532f0ac374ee8ffd8b59d9d3dff0a746209de86e0cab116ecb385c
Sep 16 14:28:29.654282 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:29.654177    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-cbeb11-69d98645fc-bwhqw" event={"ID":"26764d22-8352-4c06-98c1-1097e10c7e7d","Type":"ContainerStarted","Data":"ef2e7e91490f915656d47f7cca2cafbdc2985b543617ab7b872a3d0a6c30d51e"}
Sep 16 14:28:29.654282 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:29.654241    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-cbeb11-69d98645fc-bwhqw" event={"ID":"26764d22-8352-4c06-98c1-1097e10c7e7d","Type":"ContainerStarted","Data":"c154cb268489bcceb616c2522a01583ba683c854268ad153aa295ba6b01dbfd2"}
Sep 16 14:28:29.654282 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:29.654269    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-cbeb11-69d98645fc-bwhqw"
Sep 16 14:28:29.655762 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:29.655737    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-cbeb11-549d695445-rztrz" event={"ID":"d25388e0-508a-4169-b297-82e555257170","Type":"ContainerStarted","Data":"1263deecacb26c6e62233ca31250cf922072c50f5238b518f11f983f261cc659"}
Sep 16 14:28:29.655900 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:29.655774    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-cbeb11-549d695445-rztrz" event={"ID":"d25388e0-508a-4169-b297-82e555257170","Type":"ContainerStarted","Data":"86e6eb2eae532f0ac374ee8ffd8b59d9d3dff0a746209de86e0cab116ecb385c"}
Sep 16 14:28:29.655900 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:29.655852    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-cbeb11-549d695445-rztrz"
Sep 16 14:28:29.689254 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:29.689197    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-cbeb11-69d98645fc-bwhqw" podStartSLOduration=2.239747161 podStartE2EDuration="2.689181359s" totalImagesPullingTime="449.434198ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:28:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:28:29.687198561 +0000 UTC m=+1442.702308421" watchObservedRunningTime="2026-09-16 14:28:29.689181359 +0000 UTC m=+1442.704291219"
Sep 16 14:28:29.717459 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:29.717398    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-cbeb11-549d695445-rztrz" podStartSLOduration=1.7173762799999999 podStartE2EDuration="1.71737628s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:28:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:28:29.716819363 +0000 UTC m=+1442.731929224" watchObservedRunningTime="2026-09-16 14:28:29.71737628 +0000 UTC m=+1442.732486140"
Sep 16 14:28:35.668000 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:35.667970    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-cbeb11-69d98645fc-bwhqw"
Sep 16 14:28:40.664005 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:40.663971    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-cbeb11-549d695445-rztrz"
Sep 16 14:28:43.744248 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:43.744221    2580 generic.go:356] "Generic (PLEG): container finished" podID="e0f4134f-f3be-492c-921f-a371c8d204a4" containerID="866add6938b768b31ac90398c44051bacc61dee65be546d7094ccbc4dd5b0468" exitCode=0
Sep 16 14:28:43.744601 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:43.744290    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv" event={"ID":"e0f4134f-f3be-492c-921f-a371c8d204a4","Type":"ContainerDied","Data":"866add6938b768b31ac90398c44051bacc61dee65be546d7094ccbc4dd5b0468"}
Sep 16 14:28:43.744683 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:43.744670    2580 scope.go:117] "RemoveContainer" containerID="866add6938b768b31ac90398c44051bacc61dee65be546d7094ccbc4dd5b0468"
Sep 16 14:28:43.745748 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:43.745715    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 14:28:43.745863 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:43.745800    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 14:28:43.745863 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:43.745852    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 14:28:44.752220 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:44.752187    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv" event={"ID":"e0f4134f-f3be-492c-921f-a371c8d204a4","Type":"ContainerStarted","Data":"e21dad2079bdc022ebad1238cd803d1156aa95936899e202ab4458b0974777fd"}
Sep 16 14:28:45.560861 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:45.560828    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv"
Sep 16 14:28:45.561054 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:45.561000    2580 patch_prober.go:28] interesting pod/e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Sep 16 14:28:45.561054 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:45.561042    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv" podUID="e0f4134f-f3be-492c-921f-a371c8d204a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Sep 16 14:28:46.561355 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:46.561321    2580 patch_prober.go:28] interesting pod/e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Sep 16 14:28:46.561756 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:46.561374    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv" podUID="e0f4134f-f3be-492c-921f-a371c8d204a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Sep 16 14:28:47.561361 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:47.561326    2580 patch_prober.go:28] interesting pod/e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Sep 16 14:28:47.561782 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:47.561375    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv" podUID="e0f4134f-f3be-492c-921f-a371c8d204a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Sep 16 14:28:48.561406 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:48.561366    2580 patch_prober.go:28] interesting pod/e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Sep 16 14:28:48.561844 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:48.561422    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv" podUID="e0f4134f-f3be-492c-921f-a371c8d204a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Sep 16 14:28:49.561416 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:49.561378    2580 patch_prober.go:28] interesting pod/e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Sep 16 14:28:49.561839 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:49.561435    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv" podUID="e0f4134f-f3be-492c-921f-a371c8d204a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Sep 16 14:28:50.565376 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:50.565346    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv"
Sep 16 14:28:50.565755 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:50.565719    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv"
Sep 16 14:28:50.566390 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:28:50.566371    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv"
Sep 16 14:29:36.567001 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:29:36.566966    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-dffae957/e2e-lifecycle-model-dffae957-kserve-c7646b987-9rjn8"]
Sep 16 14:29:36.570867 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:29:36.570848    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-dffae957/e2e-lifecycle-model-dffae957-kserve-c7646b987-9rjn8"
Sep 16 14:29:36.573310 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:29:36.573276    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-dffae957\"/\"kube-root-ca.crt\""
Sep 16 14:29:36.573412 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:29:36.573319    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-dffae957\"/\"openshift-service-ca.crt\""
Sep 16 14:29:36.573466 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:29:36.573429    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-dffae957\"/\"default-dockercfg-ghjrx\""
Sep 16 14:29:36.574173 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:29:36.574155    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-dffae957\"/\"e2e-lifecycle-model-dffae957-kserve-self-signed-certs\""
Sep 16 14:29:36.582959 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:29:36.582934    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-dffae957/e2e-lifecycle-model-dffae957-kserve-c7646b987-9rjn8"]
Sep 16 14:29:36.644613 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:29:36.644588    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7-dshm\") pod \"e2e-lifecycle-model-dffae957-kserve-c7646b987-9rjn8\" (UID: \"6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7\") " pod="ai-tenant-e2e-mt-dffae957/e2e-lifecycle-model-dffae957-kserve-c7646b987-9rjn8"
Sep 16 14:29:36.644757 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:29:36.644631    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7tj7\" (UniqueName: \"kubernetes.io/projected/6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7-kube-api-access-n7tj7\") pod \"e2e-lifecycle-model-dffae957-kserve-c7646b987-9rjn8\" (UID: \"6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7\") " pod="ai-tenant-e2e-mt-dffae957/e2e-lifecycle-model-dffae957-kserve-c7646b987-9rjn8"
Sep 16 14:29:36.644757 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:29:36.644659    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7-tls-certs\") pod \"e2e-lifecycle-model-dffae957-kserve-c7646b987-9rjn8\" (UID: \"6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7\") " pod="ai-tenant-e2e-mt-dffae957/e2e-lifecycle-model-dffae957-kserve-c7646b987-9rjn8"
Sep 16 14:29:36.644757 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:29:36.644709    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7-model-cache\") pod \"e2e-lifecycle-model-dffae957-kserve-c7646b987-9rjn8\" (UID: \"6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7\") " pod="ai-tenant-e2e-mt-dffae957/e2e-lifecycle-model-dffae957-kserve-c7646b987-9rjn8"
Sep 16 14:29:36.644757 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:29:36.644750    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7-home\") pod \"e2e-lifecycle-model-dffae957-kserve-c7646b987-9rjn8\" (UID: \"6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7\") " pod="ai-tenant-e2e-mt-dffae957/e2e-lifecycle-model-dffae957-kserve-c7646b987-9rjn8"
Sep 16 14:29:36.644973 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:29:36.644793    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7-tmp-dir\") pod \"e2e-lifecycle-model-dffae957-kserve-c7646b987-9rjn8\" (UID: \"6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7\") " pod="ai-tenant-e2e-mt-dffae957/e2e-lifecycle-model-dffae957-kserve-c7646b987-9rjn8"
Sep 16 14:29:36.745982 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:29:36.745949    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7-dshm\") pod \"e2e-lifecycle-model-dffae957-kserve-c7646b987-9rjn8\" (UID: \"6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7\") " pod="ai-tenant-e2e-mt-dffae957/e2e-lifecycle-model-dffae957-kserve-c7646b987-9rjn8"
Sep 16 14:29:36.746171 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:29:36.746030    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7tj7\" (UniqueName: \"kubernetes.io/projected/6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7-kube-api-access-n7tj7\") pod \"e2e-lifecycle-model-dffae957-kserve-c7646b987-9rjn8\" (UID: \"6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7\") " pod="ai-tenant-e2e-mt-dffae957/e2e-lifecycle-model-dffae957-kserve-c7646b987-9rjn8"
Sep 16 14:29:36.746171 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:29:36.746076    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7-tls-certs\") pod \"e2e-lifecycle-model-dffae957-kserve-c7646b987-9rjn8\" (UID: \"6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7\") " pod="ai-tenant-e2e-mt-dffae957/e2e-lifecycle-model-dffae957-kserve-c7646b987-9rjn8"
Sep 16 14:29:36.746171 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:29:36.746114    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7-model-cache\") pod \"e2e-lifecycle-model-dffae957-kserve-c7646b987-9rjn8\" (UID: \"6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7\") " pod="ai-tenant-e2e-mt-dffae957/e2e-lifecycle-model-dffae957-kserve-c7646b987-9rjn8"
Sep 16 14:29:36.746335 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:29:36.746182    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7-home\") pod \"e2e-lifecycle-model-dffae957-kserve-c7646b987-9rjn8\" (UID: \"6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7\") " pod="ai-tenant-e2e-mt-dffae957/e2e-lifecycle-model-dffae957-kserve-c7646b987-9rjn8"
Sep 16 14:29:36.746335 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:29:36.746243    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7-tmp-dir\") pod \"e2e-lifecycle-model-dffae957-kserve-c7646b987-9rjn8\" (UID: \"6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7\") " pod="ai-tenant-e2e-mt-dffae957/e2e-lifecycle-model-dffae957-kserve-c7646b987-9rjn8"
Sep 16 14:29:36.746825 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:29:36.746796    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7-tmp-dir\") pod \"e2e-lifecycle-model-dffae957-kserve-c7646b987-9rjn8\" (UID: \"6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7\") " pod="ai-tenant-e2e-mt-dffae957/e2e-lifecycle-model-dffae957-kserve-c7646b987-9rjn8"
Sep 16 14:29:36.747118 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:29:36.747091    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7-model-cache\") pod \"e2e-lifecycle-model-dffae957-kserve-c7646b987-9rjn8\" (UID: \"6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7\") " pod="ai-tenant-e2e-mt-dffae957/e2e-lifecycle-model-dffae957-kserve-c7646b987-9rjn8"
Sep 16 14:29:36.747272 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:29:36.747119    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7-home\") pod \"e2e-lifecycle-model-dffae957-kserve-c7646b987-9rjn8\" (UID: \"6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7\") " pod="ai-tenant-e2e-mt-dffae957/e2e-lifecycle-model-dffae957-kserve-c7646b987-9rjn8"
Sep 16 14:29:36.749112 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:29:36.749086    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7-dshm\") pod \"e2e-lifecycle-model-dffae957-kserve-c7646b987-9rjn8\" (UID: \"6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7\") " pod="ai-tenant-e2e-mt-dffae957/e2e-lifecycle-model-dffae957-kserve-c7646b987-9rjn8"
Sep 16 14:29:36.749273 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:29:36.749253    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7-tls-certs\") pod \"e2e-lifecycle-model-dffae957-kserve-c7646b987-9rjn8\" (UID: \"6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7\") " pod="ai-tenant-e2e-mt-dffae957/e2e-lifecycle-model-dffae957-kserve-c7646b987-9rjn8"
Sep 16 14:29:36.755090 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:29:36.755042    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7tj7\" (UniqueName: \"kubernetes.io/projected/6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7-kube-api-access-n7tj7\") pod \"e2e-lifecycle-model-dffae957-kserve-c7646b987-9rjn8\" (UID: \"6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7\") " pod="ai-tenant-e2e-mt-dffae957/e2e-lifecycle-model-dffae957-kserve-c7646b987-9rjn8"
Sep 16 14:29:36.881739 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:29:36.881654    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-dffae957/e2e-lifecycle-model-dffae957-kserve-c7646b987-9rjn8"
Sep 16 14:29:37.159542 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:29:37.159486    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-dffae957/e2e-lifecycle-model-dffae957-kserve-c7646b987-9rjn8"]
Sep 16 14:29:37.161230 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:29:37.161202    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6c2aea0a_9c82_4dd6_9a9e_85350bdd9ae7.slice/crio-5844be9d1337da838b037180def8458ddc1da97f28c5280c92d07ac2ba31ae8a WatchSource:0}: Error finding container 5844be9d1337da838b037180def8458ddc1da97f28c5280c92d07ac2ba31ae8a: Status 404 returned error can't find the container with id 5844be9d1337da838b037180def8458ddc1da97f28c5280c92d07ac2ba31ae8a
Sep 16 14:29:38.064203 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:29:38.064164    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-dffae957/e2e-lifecycle-model-dffae957-kserve-c7646b987-9rjn8" event={"ID":"6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7","Type":"ContainerStarted","Data":"8a60ee3f49f5ba000ad354dbf1987443f5d950fa5626f3675135a78e75db87ba"}
Sep 16 14:29:38.064203 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:29:38.064207    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-dffae957/e2e-lifecycle-model-dffae957-kserve-c7646b987-9rjn8" event={"ID":"6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7","Type":"ContainerStarted","Data":"5844be9d1337da838b037180def8458ddc1da97f28c5280c92d07ac2ba31ae8a"}
Sep 16 14:29:38.083254 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:29:38.083206    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-dffae957/e2e-lifecycle-model-dffae957-kserve-c7646b987-9rjn8" podStartSLOduration=2.083192196 podStartE2EDuration="2.083192196s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:29:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:29:38.081273595 +0000 UTC m=+1511.096383466" watchObservedRunningTime="2026-09-16 14:29:38.083192196 +0000 UTC m=+1511.098302055"
Sep 16 14:29:46.882044 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:29:46.882011    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-dffae957/e2e-lifecycle-model-dffae957-kserve-c7646b987-9rjn8"
Sep 16 14:29:46.882559 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:29:46.882062    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-dffae957/e2e-lifecycle-model-dffae957-kserve-c7646b987-9rjn8"
Sep 16 14:29:46.894734 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:29:46.894710    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-dffae957/e2e-lifecycle-model-dffae957-kserve-c7646b987-9rjn8"
Sep 16 14:29:47.131260 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:29:47.131231    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-dffae957/e2e-lifecycle-model-dffae957-kserve-c7646b987-9rjn8"
Sep 16 14:30:00.222680 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:00.222602    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-cbeb11-29826150-zjnb4"]
Sep 16 14:30:00.225717 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:00.225700    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-cbeb11-29826150-zjnb4"
Sep 16 14:30:00.228342 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:00.228326    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-g952z\""
Sep 16 14:30:00.235895 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:00.235880    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-cbeb11-29826150-zjnb4"
Sep 16 14:30:00.241337 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:00.241311    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-cbeb11-29826150-zjnb4"]
Sep 16 14:30:00.510820 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:00.510791    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-cbeb11-29826150-zjnb4"]
Sep 16 14:30:00.513224 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:30:00.513194    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67048f3c_d226_46d0_8535_04f5db62ea5f.slice/crio-1da37c4a7d64e6e856764c4b2d671b07dd949955d7ad526cd67acd060b92a28b WatchSource:0}: Error finding container 1da37c4a7d64e6e856764c4b2d671b07dd949955d7ad526cd67acd060b92a28b: Status 404 returned error can't find the container with id 1da37c4a7d64e6e856764c4b2d671b07dd949955d7ad526cd67acd060b92a28b
Sep 16 14:30:01.198090 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:01.198055    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-cbeb11-29826150-zjnb4" event={"ID":"67048f3c-d226-46d0-8535-04f5db62ea5f","Type":"ContainerStarted","Data":"1da37c4a7d64e6e856764c4b2d671b07dd949955d7ad526cd67acd060b92a28b"}
Sep 16 14:30:04.224593 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:04.224469    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-cbeb11-29826150-zjnb4" event={"ID":"67048f3c-d226-46d0-8535-04f5db62ea5f","Type":"ContainerStarted","Data":"41536a673d68eee8f227933d9c08a115e15ec1401c1500d368ca63647b499164"}
Sep 16 14:30:04.245707 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:04.245658    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-cbeb11-29826150-zjnb4" podStartSLOduration=1.470970497 podStartE2EDuration="4.245643994s" totalImagesPullingTime="2.774673497s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:30:04.243455592 +0000 UTC m=+1537.258565452" watchObservedRunningTime="2026-09-16 14:30:04.245643994 +0000 UTC m=+1537.260753853"
Sep 16 14:30:05.232236 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:05.232203    2580 generic.go:356] "Generic (PLEG): container finished" podID="67048f3c-d226-46d0-8535-04f5db62ea5f" containerID="41536a673d68eee8f227933d9c08a115e15ec1401c1500d368ca63647b499164" exitCode=0
Sep 16 14:30:05.232663 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:05.232291    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-cbeb11-29826150-zjnb4" event={"ID":"67048f3c-d226-46d0-8535-04f5db62ea5f","Type":"ContainerDied","Data":"41536a673d68eee8f227933d9c08a115e15ec1401c1500d368ca63647b499164"}
Sep 16 14:30:06.374795 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:06.374773    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-cbeb11-29826150-zjnb4"
Sep 16 14:30:07.244214 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:07.244175    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-cbeb11-29826150-zjnb4" event={"ID":"67048f3c-d226-46d0-8535-04f5db62ea5f","Type":"ContainerDied","Data":"1da37c4a7d64e6e856764c4b2d671b07dd949955d7ad526cd67acd060b92a28b"}
Sep 16 14:30:07.244214 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:07.244200    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-cbeb11-29826150-zjnb4"
Sep 16 14:30:07.244424 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:07.244228    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1da37c4a7d64e6e856764c4b2d671b07dd949955d7ad526cd67acd060b92a28b"
Sep 16 14:30:14.211280 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:14.211243    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8959-14c5eb"]
Sep 16 14:30:14.211701 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:14.211687    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67048f3c-d226-46d0-8535-04f5db62ea5f" containerName="cleanup"
Sep 16 14:30:14.211756 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:14.211703    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="67048f3c-d226-46d0-8535-04f5db62ea5f" containerName="cleanup"
Sep 16 14:30:14.211789 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:14.211779    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="67048f3c-d226-46d0-8535-04f5db62ea5f" containerName="cleanup"
Sep 16 14:30:14.215033 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:14.215019    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8959-14c5eb"
Sep 16 14:30:14.219623 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:14.219600    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-8t2sn\""
Sep 16 14:30:14.226239 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:14.226217    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8959-14c5eb"]
Sep 16 14:30:14.295946 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:14.295921    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxrfs\" (UniqueName: \"kubernetes.io/projected/04ae4eb2-71cd-4bac-8c05-cd940a35099c-kube-api-access-mxrfs\") pod \"test-curl-8959-14c5eb\" (UID: \"04ae4eb2-71cd-4bac-8c05-cd940a35099c\") " pod="opendatahub/test-curl-8959-14c5eb"
Sep 16 14:30:14.397018 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:14.396981    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxrfs\" (UniqueName: \"kubernetes.io/projected/04ae4eb2-71cd-4bac-8c05-cd940a35099c-kube-api-access-mxrfs\") pod \"test-curl-8959-14c5eb\" (UID: \"04ae4eb2-71cd-4bac-8c05-cd940a35099c\") " pod="opendatahub/test-curl-8959-14c5eb"
Sep 16 14:30:14.405314 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:14.405286    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxrfs\" (UniqueName: \"kubernetes.io/projected/04ae4eb2-71cd-4bac-8c05-cd940a35099c-kube-api-access-mxrfs\") pod \"test-curl-8959-14c5eb\" (UID: \"04ae4eb2-71cd-4bac-8c05-cd940a35099c\") " pod="opendatahub/test-curl-8959-14c5eb"
Sep 16 14:30:14.524180 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:14.524101    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8959-14c5eb"
Sep 16 14:30:14.820625 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:14.820529    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8959-14c5eb"]
Sep 16 14:30:14.825460 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:30:14.825435    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod04ae4eb2_71cd_4bac_8c05_cd940a35099c.slice/crio-0221d6a2fcd7b48bcb96256d1321d857a854df80c3d5ad50c90f27fea811f179 WatchSource:0}: Error finding container 0221d6a2fcd7b48bcb96256d1321d857a854df80c3d5ad50c90f27fea811f179: Status 404 returned error can't find the container with id 0221d6a2fcd7b48bcb96256d1321d857a854df80c3d5ad50c90f27fea811f179
Sep 16 14:30:15.289354 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:15.289315    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8959-14c5eb" event={"ID":"04ae4eb2-71cd-4bac-8c05-cd940a35099c","Type":"ContainerStarted","Data":"0221d6a2fcd7b48bcb96256d1321d857a854df80c3d5ad50c90f27fea811f179"}
Sep 16 14:30:32.241583 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:32.241546    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-dffae957/e2e-lifecycle-model-dffae957-kserve-c7646b987-9rjn8"]
Sep 16 14:30:37.417402 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:37.417361    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8959-14c5eb" event={"ID":"04ae4eb2-71cd-4bac-8c05-cd940a35099c","Type":"ContainerStarted","Data":"fa5ac184560c9e5dc0614edd9dbd28fe6ec9f2d57eb48d0e274977d237d971d0"}
Sep 16 14:30:37.449617 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:37.449571    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-8959-14c5eb" podStartSLOduration=1.709524829 podStartE2EDuration="23.449556594s" totalImagesPullingTime="21.740031765s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:30:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:30:37.447252418 +0000 UTC m=+1570.462362281" watchObservedRunningTime="2026-09-16 14:30:37.449556594 +0000 UTC m=+1570.464666454"
Sep 16 14:30:38.098048 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:38.098017    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8959-14c5eb"]
Sep 16 14:30:38.105523 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:38.105482    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8959-14c5eb"]
Sep 16 14:30:38.422803 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:30:38.422775    2580 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-8959-14c5eb" container="test-curl-8959-14c5eb"
Sep 16 14:30:39.427923 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:39.427863    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-8959-14c5eb" podUID="04ae4eb2-71cd-4bac-8c05-cd940a35099c" containerName="test-curl-8959-14c5eb" containerID="cri-o://fa5ac184560c9e5dc0614edd9dbd28fe6ec9f2d57eb48d0e274977d237d971d0" gracePeriod=2
Sep 16 14:30:39.429835 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:30:39.429791    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8959-14c5eb\" is forbidden: User \"system:node:ip-10-0-129-95.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-95.ec2.internal' and this object" podUID="04ae4eb2-71cd-4bac-8c05-cd940a35099c" pod="opendatahub/test-curl-8959-14c5eb"
Sep 16 14:30:41.604546 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:41.604525    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8959-14c5eb"
Sep 16 14:30:41.652098 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:41.652078    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mxrfs\" (UniqueName: \"kubernetes.io/projected/04ae4eb2-71cd-4bac-8c05-cd940a35099c-kube-api-access-mxrfs\") pod \"04ae4eb2-71cd-4bac-8c05-cd940a35099c\" (UID: \"04ae4eb2-71cd-4bac-8c05-cd940a35099c\") "
Sep 16 14:30:41.654053 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:41.654032    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04ae4eb2-71cd-4bac-8c05-cd940a35099c-kube-api-access-mxrfs" (OuterVolumeSpecName: "kube-api-access-mxrfs") pod "04ae4eb2-71cd-4bac-8c05-cd940a35099c" (UID: "04ae4eb2-71cd-4bac-8c05-cd940a35099c"). InnerVolumeSpecName "kube-api-access-mxrfs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:30:41.753417 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:41.753356    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mxrfs\" (UniqueName: \"kubernetes.io/projected/04ae4eb2-71cd-4bac-8c05-cd940a35099c-kube-api-access-mxrfs\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:30:42.445310 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:42.445276    2580 generic.go:356] "Generic (PLEG): container finished" podID="04ae4eb2-71cd-4bac-8c05-cd940a35099c" containerID="fa5ac184560c9e5dc0614edd9dbd28fe6ec9f2d57eb48d0e274977d237d971d0" exitCode=137
Sep 16 14:30:42.445543 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:42.445332    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8959-14c5eb"
Sep 16 14:30:42.445543 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:42.445389    2580 scope.go:117] "RemoveContainer" containerID="fa5ac184560c9e5dc0614edd9dbd28fe6ec9f2d57eb48d0e274977d237d971d0"
Sep 16 14:30:42.454668 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:42.454655    2580 scope.go:117] "RemoveContainer" containerID="fa5ac184560c9e5dc0614edd9dbd28fe6ec9f2d57eb48d0e274977d237d971d0"
Sep 16 14:30:42.454912 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:30:42.454894    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa5ac184560c9e5dc0614edd9dbd28fe6ec9f2d57eb48d0e274977d237d971d0\": container with ID starting with fa5ac184560c9e5dc0614edd9dbd28fe6ec9f2d57eb48d0e274977d237d971d0 not found: ID does not exist" containerID="fa5ac184560c9e5dc0614edd9dbd28fe6ec9f2d57eb48d0e274977d237d971d0"
Sep 16 14:30:42.454961 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:42.454920    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa5ac184560c9e5dc0614edd9dbd28fe6ec9f2d57eb48d0e274977d237d971d0"} err="failed to get container status \"fa5ac184560c9e5dc0614edd9dbd28fe6ec9f2d57eb48d0e274977d237d971d0\": rpc error: code = NotFound desc = could not find container \"fa5ac184560c9e5dc0614edd9dbd28fe6ec9f2d57eb48d0e274977d237d971d0\": container with ID starting with fa5ac184560c9e5dc0614edd9dbd28fe6ec9f2d57eb48d0e274977d237d971d0 not found: ID does not exist"
Sep 16 14:30:42.455712 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:30:42.455692    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8959-14c5eb\" is forbidden: User \"system:node:ip-10-0-129-95.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-95.ec2.internal' and this object" podUID="04ae4eb2-71cd-4bac-8c05-cd940a35099c" pod="opendatahub/test-curl-8959-14c5eb"
Sep 16 14:30:43.431733 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:43.431698    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04ae4eb2-71cd-4bac-8c05-cd940a35099c" path="/var/lib/kubelet/pods/04ae4eb2-71cd-4bac-8c05-cd940a35099c/volumes"
Sep 16 14:30:47.260629 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:47.260567    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-dffae957/e2e-lifecycle-model-dffae957-kserve-c7646b987-9rjn8" podUID="6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7" containerName="main" containerID="cri-o://8a60ee3f49f5ba000ad354dbf1987443f5d950fa5626f3675135a78e75db87ba" gracePeriod=45
Sep 16 14:30:47.474730 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:47.474698    2580 generic.go:356] "Generic (PLEG): container finished" podID="6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7" containerID="8a60ee3f49f5ba000ad354dbf1987443f5d950fa5626f3675135a78e75db87ba" exitCode=0
Sep 16 14:30:47.474880 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:47.474767    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-dffae957/e2e-lifecycle-model-dffae957-kserve-c7646b987-9rjn8" event={"ID":"6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7","Type":"ContainerDied","Data":"8a60ee3f49f5ba000ad354dbf1987443f5d950fa5626f3675135a78e75db87ba"}
Sep 16 14:30:47.514454 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:47.514398    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-dffae957/e2e-lifecycle-model-dffae957-kserve-c7646b987-9rjn8"
Sep 16 14:30:47.609465 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:47.609435    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7-home\") pod \"6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7\" (UID: \"6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7\") "
Sep 16 14:30:47.609649 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:47.609481    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7-tls-certs\") pod \"6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7\" (UID: \"6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7\") "
Sep 16 14:30:47.609649 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:47.609597    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7-model-cache\") pod \"6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7\" (UID: \"6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7\") "
Sep 16 14:30:47.609801 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:47.609751    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7-home" (OuterVolumeSpecName: "home") pod "6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7" (UID: "6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:30:47.609865 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:47.609819    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7-model-cache" (OuterVolumeSpecName: "model-cache") pod "6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7" (UID: "6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:30:47.609865 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:47.609834    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7-dshm\") pod \"6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7\" (UID: \"6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7\") "
Sep 16 14:30:47.609974 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:47.609864    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7-tmp-dir\") pod \"6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7\" (UID: \"6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7\") "
Sep 16 14:30:47.609974 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:47.609926    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n7tj7\" (UniqueName: \"kubernetes.io/projected/6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7-kube-api-access-n7tj7\") pod \"6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7\" (UID: \"6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7\") "
Sep 16 14:30:47.610149 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:47.610124    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7" (UID: "6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:30:47.610348 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:47.610291    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7-home\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:30:47.610458 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:47.610357    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7-model-cache\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:30:47.612178 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:47.612150    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7-dshm" (OuterVolumeSpecName: "dshm") pod "6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7" (UID: "6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:30:47.612261 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:47.612176    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7" (UID: "6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:30:47.612349 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:47.612330    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7-kube-api-access-n7tj7" (OuterVolumeSpecName: "kube-api-access-n7tj7") pod "6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7" (UID: "6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7"). InnerVolumeSpecName "kube-api-access-n7tj7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:30:47.711192 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:47.711161    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7-tls-certs\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:30:47.711192 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:47.711193    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7-dshm\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:30:47.711362 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:47.711209    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7-tmp-dir\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:30:47.711362 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:47.711222    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n7tj7\" (UniqueName: \"kubernetes.io/projected/6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7-kube-api-access-n7tj7\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:30:48.481522 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:48.481464    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-dffae957/e2e-lifecycle-model-dffae957-kserve-c7646b987-9rjn8" event={"ID":"6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7","Type":"ContainerDied","Data":"5844be9d1337da838b037180def8458ddc1da97f28c5280c92d07ac2ba31ae8a"}
Sep 16 14:30:48.481522 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:48.481507    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-dffae957/e2e-lifecycle-model-dffae957-kserve-c7646b987-9rjn8"
Sep 16 14:30:48.481522 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:48.481532    2580 scope.go:117] "RemoveContainer" containerID="8a60ee3f49f5ba000ad354dbf1987443f5d950fa5626f3675135a78e75db87ba"
Sep 16 14:30:48.509691 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:48.509655    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-dffae957/e2e-lifecycle-model-dffae957-kserve-c7646b987-9rjn8"]
Sep 16 14:30:48.514875 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:48.514843    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-dffae957/e2e-lifecycle-model-dffae957-kserve-c7646b987-9rjn8"]
Sep 16 14:30:49.431234 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:30:49.431200    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7" path="/var/lib/kubelet/pods/6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7/volumes"
Sep 16 14:32:48.251994 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:32:48.251958    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-b77fde-68ddfb7b5d-4wjc8"]
Sep 16 14:32:48.252655 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:32:48.252636    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04ae4eb2-71cd-4bac-8c05-cd940a35099c" containerName="test-curl-8959-14c5eb"
Sep 16 14:32:48.252748 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:32:48.252657    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="04ae4eb2-71cd-4bac-8c05-cd940a35099c" containerName="test-curl-8959-14c5eb"
Sep 16 14:32:48.252748 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:32:48.252690    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7" containerName="main"
Sep 16 14:32:48.252748 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:32:48.252700    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7" containerName="main"
Sep 16 14:32:48.252905 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:32:48.252814    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c2aea0a-9c82-4dd6-9a9e-85350bdd9ae7" containerName="main"
Sep 16 14:32:48.252905 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:32:48.252832    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="04ae4eb2-71cd-4bac-8c05-cd940a35099c" containerName="test-curl-8959-14c5eb"
Sep 16 14:32:48.256021 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:32:48.256001    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b77fde-68ddfb7b5d-4wjc8"
Sep 16 14:32:48.258391 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:32:48.258366    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-b77fde-dockercfg-55c8t\""
Sep 16 14:32:48.258476 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:32:48.258405    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-b77fde\""
Sep 16 14:32:48.271695 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:32:48.271673    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-b77fde-68ddfb7b5d-4wjc8"]
Sep 16 14:32:48.335317 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:32:48.335286    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2fsh\" (UniqueName: \"kubernetes.io/projected/ef0b01c7-42dc-466d-bb9a-0d6918e83d91-kube-api-access-r2fsh\") pod \"payload-pre-processing-e2e-ipp-a-b77fde-68ddfb7b5d-4wjc8\" (UID: \"ef0b01c7-42dc-466d-bb9a-0d6918e83d91\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b77fde-68ddfb7b5d-4wjc8"
Sep 16 14:32:48.335487 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:32:48.335328    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef0b01c7-42dc-466d-bb9a-0d6918e83d91-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-b77fde-68ddfb7b5d-4wjc8\" (UID: \"ef0b01c7-42dc-466d-bb9a-0d6918e83d91\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b77fde-68ddfb7b5d-4wjc8"
Sep 16 14:32:48.435921 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:32:48.435887    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2fsh\" (UniqueName: \"kubernetes.io/projected/ef0b01c7-42dc-466d-bb9a-0d6918e83d91-kube-api-access-r2fsh\") pod \"payload-pre-processing-e2e-ipp-a-b77fde-68ddfb7b5d-4wjc8\" (UID: \"ef0b01c7-42dc-466d-bb9a-0d6918e83d91\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b77fde-68ddfb7b5d-4wjc8"
Sep 16 14:32:48.436107 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:32:48.435930    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef0b01c7-42dc-466d-bb9a-0d6918e83d91-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-b77fde-68ddfb7b5d-4wjc8\" (UID: \"ef0b01c7-42dc-466d-bb9a-0d6918e83d91\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b77fde-68ddfb7b5d-4wjc8"
Sep 16 14:32:48.436689 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:32:48.436665    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef0b01c7-42dc-466d-bb9a-0d6918e83d91-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-b77fde-68ddfb7b5d-4wjc8\" (UID: \"ef0b01c7-42dc-466d-bb9a-0d6918e83d91\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b77fde-68ddfb7b5d-4wjc8"
Sep 16 14:32:48.445782 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:32:48.445758    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2fsh\" (UniqueName: \"kubernetes.io/projected/ef0b01c7-42dc-466d-bb9a-0d6918e83d91-kube-api-access-r2fsh\") pod \"payload-pre-processing-e2e-ipp-a-b77fde-68ddfb7b5d-4wjc8\" (UID: \"ef0b01c7-42dc-466d-bb9a-0d6918e83d91\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b77fde-68ddfb7b5d-4wjc8"
Sep 16 14:32:48.566751 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:32:48.566665    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b77fde-68ddfb7b5d-4wjc8"
Sep 16 14:32:48.859508 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:32:48.859467    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-b77fde-68ddfb7b5d-4wjc8"]
Sep 16 14:32:48.862201 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:32:48.862173    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef0b01c7_42dc_466d_bb9a_0d6918e83d91.slice/crio-ad404017651ac51b0acdf6b64426b8fe4c3186044e78fd2b104927384db12cfe WatchSource:0}: Error finding container ad404017651ac51b0acdf6b64426b8fe4c3186044e78fd2b104927384db12cfe: Status 404 returned error can't find the container with id ad404017651ac51b0acdf6b64426b8fe4c3186044e78fd2b104927384db12cfe
Sep 16 14:32:49.177835 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:32:49.177804    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b77fde-68ddfb7b5d-4wjc8" event={"ID":"ef0b01c7-42dc-466d-bb9a-0d6918e83d91","Type":"ContainerStarted","Data":"c4e15bed0ccb72903e831891c511f89188bfc89a8a2cd30eec4ca23f6d27637a"}
Sep 16 14:32:49.177985 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:32:49.177865    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b77fde-68ddfb7b5d-4wjc8" event={"ID":"ef0b01c7-42dc-466d-bb9a-0d6918e83d91","Type":"ContainerStarted","Data":"ad404017651ac51b0acdf6b64426b8fe4c3186044e78fd2b104927384db12cfe"}
Sep 16 14:32:49.177985 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:32:49.177910    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b77fde-68ddfb7b5d-4wjc8"
Sep 16 14:32:49.204602 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:32:49.204546    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b77fde-68ddfb7b5d-4wjc8" podStartSLOduration=1.204527106 podStartE2EDuration="1.204527106s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:32:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:32:49.202616543 +0000 UTC m=+1702.217726404" watchObservedRunningTime="2026-09-16 14:32:49.204527106 +0000 UTC m=+1702.219636968"
Sep 16 14:32:50.099970 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:32:50.099887    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-7b98f7-75d74965fc-fqpc6"]
Sep 16 14:32:50.103830 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:32:50.103805    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-7b98f7-75d74965fc-fqpc6"
Sep 16 14:32:50.110902 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:32:50.110420    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-7b98f7-dockercfg-w55kd\""
Sep 16 14:32:50.110902 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:32:50.110423    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-7b98f7\""
Sep 16 14:32:50.121506 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:32:50.121479    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-7b98f7-75d74965fc-fqpc6"]
Sep 16 14:32:50.154914 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:32:50.154886    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5sjm4\" (UniqueName: \"kubernetes.io/projected/188c78a5-068c-44da-a90d-45917ba5de3b-kube-api-access-5sjm4\") pod \"payload-processing-e2e-api-a-7b98f7-75d74965fc-fqpc6\" (UID: \"188c78a5-068c-44da-a90d-45917ba5de3b\") " pod="openshift-ingress/payload-processing-e2e-api-a-7b98f7-75d74965fc-fqpc6"
Sep 16 14:32:50.155074 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:32:50.154979    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/188c78a5-068c-44da-a90d-45917ba5de3b-plugins-config-volume\") pod \"payload-processing-e2e-api-a-7b98f7-75d74965fc-fqpc6\" (UID: \"188c78a5-068c-44da-a90d-45917ba5de3b\") " pod="openshift-ingress/payload-processing-e2e-api-a-7b98f7-75d74965fc-fqpc6"
Sep 16 14:32:50.255781 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:32:50.255739    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5sjm4\" (UniqueName: \"kubernetes.io/projected/188c78a5-068c-44da-a90d-45917ba5de3b-kube-api-access-5sjm4\") pod \"payload-processing-e2e-api-a-7b98f7-75d74965fc-fqpc6\" (UID: \"188c78a5-068c-44da-a90d-45917ba5de3b\") " pod="openshift-ingress/payload-processing-e2e-api-a-7b98f7-75d74965fc-fqpc6"
Sep 16 14:32:50.255978 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:32:50.255840    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/188c78a5-068c-44da-a90d-45917ba5de3b-plugins-config-volume\") pod \"payload-processing-e2e-api-a-7b98f7-75d74965fc-fqpc6\" (UID: \"188c78a5-068c-44da-a90d-45917ba5de3b\") " pod="openshift-ingress/payload-processing-e2e-api-a-7b98f7-75d74965fc-fqpc6"
Sep 16 14:32:50.256614 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:32:50.256594    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/188c78a5-068c-44da-a90d-45917ba5de3b-plugins-config-volume\") pod \"payload-processing-e2e-api-a-7b98f7-75d74965fc-fqpc6\" (UID: \"188c78a5-068c-44da-a90d-45917ba5de3b\") " pod="openshift-ingress/payload-processing-e2e-api-a-7b98f7-75d74965fc-fqpc6"
Sep 16 14:32:50.265037 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:32:50.265012    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5sjm4\" (UniqueName: \"kubernetes.io/projected/188c78a5-068c-44da-a90d-45917ba5de3b-kube-api-access-5sjm4\") pod \"payload-processing-e2e-api-a-7b98f7-75d74965fc-fqpc6\" (UID: \"188c78a5-068c-44da-a90d-45917ba5de3b\") " pod="openshift-ingress/payload-processing-e2e-api-a-7b98f7-75d74965fc-fqpc6"
Sep 16 14:32:50.419281 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:32:50.419203    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-7b98f7-75d74965fc-fqpc6"
Sep 16 14:32:50.713396 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:32:50.713375    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-7b98f7-75d74965fc-fqpc6"]
Sep 16 14:32:50.714443 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:32:50.714405    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod188c78a5_068c_44da_a90d_45917ba5de3b.slice/crio-fbbe764ce807a4f776f379f8664709213f430c1f3a82b8a57137f14c24fea7b4 WatchSource:0}: Error finding container fbbe764ce807a4f776f379f8664709213f430c1f3a82b8a57137f14c24fea7b4: Status 404 returned error can't find the container with id fbbe764ce807a4f776f379f8664709213f430c1f3a82b8a57137f14c24fea7b4
Sep 16 14:32:51.193627 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:32:51.193593    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-7b98f7-75d74965fc-fqpc6" event={"ID":"188c78a5-068c-44da-a90d-45917ba5de3b","Type":"ContainerStarted","Data":"f97d8a8db167adac5b5641cb761af51d2d531ce739e577fad92ece29617480fa"}
Sep 16 14:32:51.194023 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:32:51.193651    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-7b98f7-75d74965fc-fqpc6" event={"ID":"188c78a5-068c-44da-a90d-45917ba5de3b","Type":"ContainerStarted","Data":"fbbe764ce807a4f776f379f8664709213f430c1f3a82b8a57137f14c24fea7b4"}
Sep 16 14:32:51.194023 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:32:51.193809    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-7b98f7-75d74965fc-fqpc6"
Sep 16 14:32:51.213443 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:32:51.213392    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-7b98f7-75d74965fc-fqpc6" podStartSLOduration=1.213373293 podStartE2EDuration="1.213373293s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:32:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:32:51.212947945 +0000 UTC m=+1704.228057832" watchObservedRunningTime="2026-09-16 14:32:51.213373293 +0000 UTC m=+1704.228483155"
Sep 16 14:33:00.187094 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:33:00.187060    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b77fde-68ddfb7b5d-4wjc8"
Sep 16 14:33:02.201226 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:33:02.201195    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-7b98f7-75d74965fc-fqpc6"
Sep 16 14:33:19.192045 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:33:19.192010    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-7eeba8-656566c466-gw78b"]
Sep 16 14:33:19.196569 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:33:19.196547    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7eeba8-656566c466-gw78b"
Sep 16 14:33:19.198626 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:33:19.198602    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-7eeba8-serving-cert\""
Sep 16 14:33:19.211612 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:33:19.211586    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-7eeba8-656566c466-gw78b"]
Sep 16 14:33:19.341223 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:33:19.341180    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/71ad7864-113f-434c-aaa7-998dd7774203-metrics-certs\") pod \"maas-api-e2e-ipp-b-7eeba8-656566c466-gw78b\" (UID: \"71ad7864-113f-434c-aaa7-998dd7774203\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7eeba8-656566c466-gw78b"
Sep 16 14:33:19.341223 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:33:19.341226    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71ad7864-113f-434c-aaa7-998dd7774203-maas-api-tls\") pod \"maas-api-e2e-ipp-b-7eeba8-656566c466-gw78b\" (UID: \"71ad7864-113f-434c-aaa7-998dd7774203\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7eeba8-656566c466-gw78b"
Sep 16 14:33:19.341540 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:33:19.341246    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4mb8\" (UniqueName: \"kubernetes.io/projected/71ad7864-113f-434c-aaa7-998dd7774203-kube-api-access-g4mb8\") pod \"maas-api-e2e-ipp-b-7eeba8-656566c466-gw78b\" (UID: \"71ad7864-113f-434c-aaa7-998dd7774203\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7eeba8-656566c466-gw78b"
Sep 16 14:33:19.341540 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:33:19.341323    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71ad7864-113f-434c-aaa7-998dd7774203-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-7eeba8-656566c466-gw78b\" (UID: \"71ad7864-113f-434c-aaa7-998dd7774203\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7eeba8-656566c466-gw78b"
Sep 16 14:33:19.442032 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:33:19.441993    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/71ad7864-113f-434c-aaa7-998dd7774203-metrics-certs\") pod \"maas-api-e2e-ipp-b-7eeba8-656566c466-gw78b\" (UID: \"71ad7864-113f-434c-aaa7-998dd7774203\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7eeba8-656566c466-gw78b"
Sep 16 14:33:19.442266 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:33:19.442155    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71ad7864-113f-434c-aaa7-998dd7774203-maas-api-tls\") pod \"maas-api-e2e-ipp-b-7eeba8-656566c466-gw78b\" (UID: \"71ad7864-113f-434c-aaa7-998dd7774203\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7eeba8-656566c466-gw78b"
Sep 16 14:33:19.442266 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:33:19.442207    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4mb8\" (UniqueName: \"kubernetes.io/projected/71ad7864-113f-434c-aaa7-998dd7774203-kube-api-access-g4mb8\") pod \"maas-api-e2e-ipp-b-7eeba8-656566c466-gw78b\" (UID: \"71ad7864-113f-434c-aaa7-998dd7774203\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7eeba8-656566c466-gw78b"
Sep 16 14:33:19.442266 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:33:19.442262    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71ad7864-113f-434c-aaa7-998dd7774203-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-7eeba8-656566c466-gw78b\" (UID: \"71ad7864-113f-434c-aaa7-998dd7774203\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7eeba8-656566c466-gw78b"
Sep 16 14:33:19.442858 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:33:19.442831    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71ad7864-113f-434c-aaa7-998dd7774203-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-7eeba8-656566c466-gw78b\" (UID: \"71ad7864-113f-434c-aaa7-998dd7774203\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7eeba8-656566c466-gw78b"
Sep 16 14:33:19.444489 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:33:19.444467    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/71ad7864-113f-434c-aaa7-998dd7774203-metrics-certs\") pod \"maas-api-e2e-ipp-b-7eeba8-656566c466-gw78b\" (UID: \"71ad7864-113f-434c-aaa7-998dd7774203\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7eeba8-656566c466-gw78b"
Sep 16 14:33:19.444749 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:33:19.444727    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71ad7864-113f-434c-aaa7-998dd7774203-maas-api-tls\") pod \"maas-api-e2e-ipp-b-7eeba8-656566c466-gw78b\" (UID: \"71ad7864-113f-434c-aaa7-998dd7774203\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7eeba8-656566c466-gw78b"
Sep 16 14:33:19.451833 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:33:19.451810    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4mb8\" (UniqueName: \"kubernetes.io/projected/71ad7864-113f-434c-aaa7-998dd7774203-kube-api-access-g4mb8\") pod \"maas-api-e2e-ipp-b-7eeba8-656566c466-gw78b\" (UID: \"71ad7864-113f-434c-aaa7-998dd7774203\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7eeba8-656566c466-gw78b"
Sep 16 14:33:19.508443 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:33:19.508408    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7eeba8-656566c466-gw78b"
Sep 16 14:33:19.795149 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:33:19.795114    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-7eeba8-656566c466-gw78b"]
Sep 16 14:33:19.795591 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:33:19.795566    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71ad7864_113f_434c_aaa7_998dd7774203.slice/crio-88c1964c2fe1a91a0a17c73b7923163942117a206613fd14a5cde505c0bd166f WatchSource:0}: Error finding container 88c1964c2fe1a91a0a17c73b7923163942117a206613fd14a5cde505c0bd166f: Status 404 returned error can't find the container with id 88c1964c2fe1a91a0a17c73b7923163942117a206613fd14a5cde505c0bd166f
Sep 16 14:33:19.797132 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:33:19.797110    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 14:33:20.361320 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:33:20.361280    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7eeba8-656566c466-gw78b" event={"ID":"71ad7864-113f-434c-aaa7-998dd7774203","Type":"ContainerStarted","Data":"a9fdc7c99f4ae4f1a9cb0d669ce28011e11fbd6f8ea607ff0db9bb9c2241a299"}
Sep 16 14:33:20.361737 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:33:20.361340    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7eeba8-656566c466-gw78b" event={"ID":"71ad7864-113f-434c-aaa7-998dd7774203","Type":"ContainerStarted","Data":"88c1964c2fe1a91a0a17c73b7923163942117a206613fd14a5cde505c0bd166f"}
Sep 16 14:33:20.361737 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:33:20.361368    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7eeba8-656566c466-gw78b"
Sep 16 14:33:20.393359 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:33:20.393314    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7eeba8-656566c466-gw78b" podStartSLOduration=1.018055884 podStartE2EDuration="1.393298385s" totalImagesPullingTime="375.242501ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:33:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:33:20.391973344 +0000 UTC m=+1733.407083195" watchObservedRunningTime="2026-09-16 14:33:20.393298385 +0000 UTC m=+1733.408408245"
Sep 16 14:33:24.263702 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:33:24.263665    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-927f98-cd4ff9fc4-5xmv5"]
Sep 16 14:33:24.267409 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:33:24.267391    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-927f98-cd4ff9fc4-5xmv5"
Sep 16 14:33:24.269669 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:33:24.269648    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-927f98-dockercfg-k88zr\""
Sep 16 14:33:24.269796 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:33:24.269726    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-927f98\""
Sep 16 14:33:24.284444 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:33:24.284423    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-927f98-cd4ff9fc4-5xmv5"]
Sep 16 14:33:24.288461 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:33:24.288439    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdnm7\" (UniqueName: \"kubernetes.io/projected/c56923cb-d5e0-4c17-8d6f-5f3cfa3878b1-kube-api-access-jdnm7\") pod \"payload-pre-processing-e2e-api-b-927f98-cd4ff9fc4-5xmv5\" (UID: \"c56923cb-d5e0-4c17-8d6f-5f3cfa3878b1\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-927f98-cd4ff9fc4-5xmv5"
Sep 16 14:33:24.288599 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:33:24.288521    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c56923cb-d5e0-4c17-8d6f-5f3cfa3878b1-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-927f98-cd4ff9fc4-5xmv5\" (UID: \"c56923cb-d5e0-4c17-8d6f-5f3cfa3878b1\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-927f98-cd4ff9fc4-5xmv5"
Sep 16 14:33:24.330187 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:33:24.330155    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-927f98-74b8f5ddcf-wstvh"]
Sep 16 14:33:24.334949 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:33:24.334924    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-927f98-74b8f5ddcf-wstvh"
Sep 16 14:33:24.347097 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:33:24.347073    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-927f98-74b8f5ddcf-wstvh"]
Sep 16 14:33:24.390084 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:33:24.390052    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdnm7\" (UniqueName: \"kubernetes.io/projected/c56923cb-d5e0-4c17-8d6f-5f3cfa3878b1-kube-api-access-jdnm7\") pod \"payload-pre-processing-e2e-api-b-927f98-cd4ff9fc4-5xmv5\" (UID: \"c56923cb-d5e0-4c17-8d6f-5f3cfa3878b1\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-927f98-cd4ff9fc4-5xmv5"
Sep 16 14:33:24.390269 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:33:24.390107    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c56923cb-d5e0-4c17-8d6f-5f3cfa3878b1-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-927f98-cd4ff9fc4-5xmv5\" (UID: \"c56923cb-d5e0-4c17-8d6f-5f3cfa3878b1\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-927f98-cd4ff9fc4-5xmv5"
Sep 16 14:33:24.390269 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:33:24.390167    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6c6d408-ef8b-4e6a-83db-22c6834a59c3-plugins-config-volume\") pod \"payload-processing-e2e-api-b-927f98-74b8f5ddcf-wstvh\" (UID: \"b6c6d408-ef8b-4e6a-83db-22c6834a59c3\") " pod="openshift-ingress/payload-processing-e2e-api-b-927f98-74b8f5ddcf-wstvh"
Sep 16 14:33:24.390269 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:33:24.390207    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tct28\" (UniqueName: \"kubernetes.io/projected/b6c6d408-ef8b-4e6a-83db-22c6834a59c3-kube-api-access-tct28\") pod \"payload-processing-e2e-api-b-927f98-74b8f5ddcf-wstvh\" (UID: \"b6c6d408-ef8b-4e6a-83db-22c6834a59c3\") " pod="openshift-ingress/payload-processing-e2e-api-b-927f98-74b8f5ddcf-wstvh"
Sep 16 14:33:24.390835 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:33:24.390816    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c56923cb-d5e0-4c17-8d6f-5f3cfa3878b1-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-927f98-cd4ff9fc4-5xmv5\" (UID: \"c56923cb-d5e0-4c17-8d6f-5f3cfa3878b1\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-927f98-cd4ff9fc4-5xmv5"
Sep 16 14:33:24.409022 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:33:24.408990    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdnm7\" (UniqueName: \"kubernetes.io/projected/c56923cb-d5e0-4c17-8d6f-5f3cfa3878b1-kube-api-access-jdnm7\") pod \"payload-pre-processing-e2e-api-b-927f98-cd4ff9fc4-5xmv5\" (UID: \"c56923cb-d5e0-4c17-8d6f-5f3cfa3878b1\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-927f98-cd4ff9fc4-5xmv5"
Sep 16 14:33:24.490895 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:33:24.490857    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6c6d408-ef8b-4e6a-83db-22c6834a59c3-plugins-config-volume\") pod \"payload-processing-e2e-api-b-927f98-74b8f5ddcf-wstvh\" (UID: \"b6c6d408-ef8b-4e6a-83db-22c6834a59c3\") " pod="openshift-ingress/payload-processing-e2e-api-b-927f98-74b8f5ddcf-wstvh"
Sep 16 14:33:24.491093 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:33:24.490905    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tct28\" (UniqueName: \"kubernetes.io/projected/b6c6d408-ef8b-4e6a-83db-22c6834a59c3-kube-api-access-tct28\") pod \"payload-processing-e2e-api-b-927f98-74b8f5ddcf-wstvh\" (UID: \"b6c6d408-ef8b-4e6a-83db-22c6834a59c3\") " pod="openshift-ingress/payload-processing-e2e-api-b-927f98-74b8f5ddcf-wstvh"
Sep 16 14:33:24.491593 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:33:24.491575    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6c6d408-ef8b-4e6a-83db-22c6834a59c3-plugins-config-volume\") pod \"payload-processing-e2e-api-b-927f98-74b8f5ddcf-wstvh\" (UID: \"b6c6d408-ef8b-4e6a-83db-22c6834a59c3\") " pod="openshift-ingress/payload-processing-e2e-api-b-927f98-74b8f5ddcf-wstvh"
Sep 16 14:33:24.500614 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:33:24.500590    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tct28\" (UniqueName: \"kubernetes.io/projected/b6c6d408-ef8b-4e6a-83db-22c6834a59c3-kube-api-access-tct28\") pod \"payload-processing-e2e-api-b-927f98-74b8f5ddcf-wstvh\" (UID: \"b6c6d408-ef8b-4e6a-83db-22c6834a59c3\") " pod="openshift-ingress/payload-processing-e2e-api-b-927f98-74b8f5ddcf-wstvh"
Sep 16 14:33:24.577986 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:33:24.577896    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-927f98-cd4ff9fc4-5xmv5"
Sep 16 14:33:24.648458 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:33:24.648416    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-927f98-74b8f5ddcf-wstvh"
Sep 16 14:33:24.869079 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:33:24.869054    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-927f98-cd4ff9fc4-5xmv5"]
Sep 16 14:33:24.870086 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:33:24.870062    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc56923cb_d5e0_4c17_8d6f_5f3cfa3878b1.slice/crio-7b1e69cb5430caa0f4534576837e526978c20be43d7ad8fdb42c2b61a6f3ab0d WatchSource:0}: Error finding container 7b1e69cb5430caa0f4534576837e526978c20be43d7ad8fdb42c2b61a6f3ab0d: Status 404 returned error can't find the container with id 7b1e69cb5430caa0f4534576837e526978c20be43d7ad8fdb42c2b61a6f3ab0d
Sep 16 14:33:24.937699 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:33:24.937669    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-927f98-74b8f5ddcf-wstvh"]
Sep 16 14:33:24.938801 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:33:24.938777    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6c6d408_ef8b_4e6a_83db_22c6834a59c3.slice/crio-c74293373fdc501771abdc285d554ed690b1c64d8ddd56d5dc8ab036153a6c2e WatchSource:0}: Error finding container c74293373fdc501771abdc285d554ed690b1c64d8ddd56d5dc8ab036153a6c2e: Status 404 returned error can't find the container with id c74293373fdc501771abdc285d554ed690b1c64d8ddd56d5dc8ab036153a6c2e
Sep 16 14:33:25.394286 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:33:25.394248    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-927f98-cd4ff9fc4-5xmv5" event={"ID":"c56923cb-d5e0-4c17-8d6f-5f3cfa3878b1","Type":"ContainerStarted","Data":"7ca0e2a5b34bc2566a3169bffc8ec1d33f2f3015a9a550e9696f80f0d715a4dd"}
Sep 16 14:33:25.394757 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:33:25.394447    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-927f98-cd4ff9fc4-5xmv5"
Sep 16 14:33:25.394757 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:33:25.394480    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-927f98-cd4ff9fc4-5xmv5" event={"ID":"c56923cb-d5e0-4c17-8d6f-5f3cfa3878b1","Type":"ContainerStarted","Data":"7b1e69cb5430caa0f4534576837e526978c20be43d7ad8fdb42c2b61a6f3ab0d"}
Sep 16 14:33:25.396200 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:33:25.396178    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-927f98-74b8f5ddcf-wstvh" event={"ID":"b6c6d408-ef8b-4e6a-83db-22c6834a59c3","Type":"ContainerStarted","Data":"b22e08a69b4939582ac78c02c766dbbdebf313e6694fb6ee0966dbc0bd150f47"}
Sep 16 14:33:25.396325 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:33:25.396212    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-927f98-74b8f5ddcf-wstvh" event={"ID":"b6c6d408-ef8b-4e6a-83db-22c6834a59c3","Type":"ContainerStarted","Data":"c74293373fdc501771abdc285d554ed690b1c64d8ddd56d5dc8ab036153a6c2e"}
Sep 16 14:33:25.396325 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:33:25.396231    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-927f98-74b8f5ddcf-wstvh"
Sep 16 14:33:25.431669 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:33:25.431346    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-927f98-cd4ff9fc4-5xmv5" podStartSLOduration=1.431325652 podStartE2EDuration="1.431325652s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:33:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:33:25.425976588 +0000 UTC m=+1738.441086452" watchObservedRunningTime="2026-09-16 14:33:25.431325652 +0000 UTC m=+1738.446435515"
Sep 16 14:33:25.455839 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:33:25.455777    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-927f98-74b8f5ddcf-wstvh" podStartSLOduration=1.4557593 podStartE2EDuration="1.4557593s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:33:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:33:25.452757602 +0000 UTC m=+1738.467867461" watchObservedRunningTime="2026-09-16 14:33:25.4557593 +0000 UTC m=+1738.470869162"
Sep 16 14:33:26.373136 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:33:26.373104    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7eeba8-656566c466-gw78b"
Sep 16 14:33:36.404999 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:33:36.404965    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-927f98-cd4ff9fc4-5xmv5"
Sep 16 14:33:36.405373 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:33:36.405069    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-927f98-74b8f5ddcf-wstvh"
Sep 16 14:34:05.515394 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:05.515357    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-7b98f7-75d74965fc-fqpc6"]
Sep 16 14:34:05.518265 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:05.516954    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-7b98f7-75d74965fc-fqpc6" podUID="188c78a5-068c-44da-a90d-45917ba5de3b" containerName="payload-processing" containerID="cri-o://f97d8a8db167adac5b5641cb761af51d2d531ce739e577fad92ece29617480fa" gracePeriod=30
Sep 16 14:34:05.779139 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:05.779113    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-7b98f7-75d74965fc-fqpc6"
Sep 16 14:34:05.793846 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:05.793826    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5sjm4\" (UniqueName: \"kubernetes.io/projected/188c78a5-068c-44da-a90d-45917ba5de3b-kube-api-access-5sjm4\") pod \"188c78a5-068c-44da-a90d-45917ba5de3b\" (UID: \"188c78a5-068c-44da-a90d-45917ba5de3b\") "
Sep 16 14:34:05.793947 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:05.793891    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/188c78a5-068c-44da-a90d-45917ba5de3b-plugins-config-volume\") pod \"188c78a5-068c-44da-a90d-45917ba5de3b\" (UID: \"188c78a5-068c-44da-a90d-45917ba5de3b\") "
Sep 16 14:34:05.794235 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:05.794207    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/188c78a5-068c-44da-a90d-45917ba5de3b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "188c78a5-068c-44da-a90d-45917ba5de3b" (UID: "188c78a5-068c-44da-a90d-45917ba5de3b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:34:05.795878 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:05.795857    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/188c78a5-068c-44da-a90d-45917ba5de3b-kube-api-access-5sjm4" (OuterVolumeSpecName: "kube-api-access-5sjm4") pod "188c78a5-068c-44da-a90d-45917ba5de3b" (UID: "188c78a5-068c-44da-a90d-45917ba5de3b"). InnerVolumeSpecName "kube-api-access-5sjm4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:34:05.895285 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:05.895251    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5sjm4\" (UniqueName: \"kubernetes.io/projected/188c78a5-068c-44da-a90d-45917ba5de3b-kube-api-access-5sjm4\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:34:05.895285 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:05.895286    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/188c78a5-068c-44da-a90d-45917ba5de3b-plugins-config-volume\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:34:06.635947 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:06.635911    2580 generic.go:356] "Generic (PLEG): container finished" podID="188c78a5-068c-44da-a90d-45917ba5de3b" containerID="f97d8a8db167adac5b5641cb761af51d2d531ce739e577fad92ece29617480fa" exitCode=0
Sep 16 14:34:06.636353 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:06.635959    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-7b98f7-75d74965fc-fqpc6"
Sep 16 14:34:06.636353 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:06.636008    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-7b98f7-75d74965fc-fqpc6" event={"ID":"188c78a5-068c-44da-a90d-45917ba5de3b","Type":"ContainerDied","Data":"f97d8a8db167adac5b5641cb761af51d2d531ce739e577fad92ece29617480fa"}
Sep 16 14:34:06.636353 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:06.636059    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-7b98f7-75d74965fc-fqpc6" event={"ID":"188c78a5-068c-44da-a90d-45917ba5de3b","Type":"ContainerDied","Data":"fbbe764ce807a4f776f379f8664709213f430c1f3a82b8a57137f14c24fea7b4"}
Sep 16 14:34:06.636353 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:06.636082    2580 scope.go:117] "RemoveContainer" containerID="f97d8a8db167adac5b5641cb761af51d2d531ce739e577fad92ece29617480fa"
Sep 16 14:34:06.646011 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:06.645989    2580 scope.go:117] "RemoveContainer" containerID="f97d8a8db167adac5b5641cb761af51d2d531ce739e577fad92ece29617480fa"
Sep 16 14:34:06.646288 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:34:06.646268    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f97d8a8db167adac5b5641cb761af51d2d531ce739e577fad92ece29617480fa\": container with ID starting with f97d8a8db167adac5b5641cb761af51d2d531ce739e577fad92ece29617480fa not found: ID does not exist" containerID="f97d8a8db167adac5b5641cb761af51d2d531ce739e577fad92ece29617480fa"
Sep 16 14:34:06.646333 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:06.646295    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f97d8a8db167adac5b5641cb761af51d2d531ce739e577fad92ece29617480fa"} err="failed to get container status \"f97d8a8db167adac5b5641cb761af51d2d531ce739e577fad92ece29617480fa\": rpc error: code = NotFound desc = could not find container \"f97d8a8db167adac5b5641cb761af51d2d531ce739e577fad92ece29617480fa\": container with ID starting with f97d8a8db167adac5b5641cb761af51d2d531ce739e577fad92ece29617480fa not found: ID does not exist"
Sep 16 14:34:06.658684 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:06.658651    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-7b98f7-75d74965fc-fqpc6"]
Sep 16 14:34:06.663325 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:06.663303    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-7b98f7-75d74965fc-fqpc6"]
Sep 16 14:34:07.430829 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:07.430801    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="188c78a5-068c-44da-a90d-45917ba5de3b" path="/var/lib/kubelet/pods/188c78a5-068c-44da-a90d-45917ba5de3b/volumes"
Sep 16 14:34:29.498452 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:29.498418    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-927f98-74b8f5ddcf-wstvh"]
Sep 16 14:34:29.498948 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:29.498692    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-927f98-74b8f5ddcf-wstvh" podUID="b6c6d408-ef8b-4e6a-83db-22c6834a59c3" containerName="payload-processing" containerID="cri-o://b22e08a69b4939582ac78c02c766dbbdebf313e6694fb6ee0966dbc0bd150f47" gracePeriod=30
Sep 16 14:34:29.506812 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:29.506780    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-927f98-cd4ff9fc4-5xmv5"]
Sep 16 14:34:29.507364 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:29.507052    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-927f98-cd4ff9fc4-5xmv5" podUID="c56923cb-d5e0-4c17-8d6f-5f3cfa3878b1" containerName="payload-pre-processing" containerID="cri-o://7ca0e2a5b34bc2566a3169bffc8ec1d33f2f3015a9a550e9696f80f0d715a4dd" gracePeriod=30
Sep 16 14:34:29.769065 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:29.769033    2580 generic.go:356] "Generic (PLEG): container finished" podID="c56923cb-d5e0-4c17-8d6f-5f3cfa3878b1" containerID="7ca0e2a5b34bc2566a3169bffc8ec1d33f2f3015a9a550e9696f80f0d715a4dd" exitCode=0
Sep 16 14:34:29.769189 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:29.769071    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-927f98-cd4ff9fc4-5xmv5" event={"ID":"c56923cb-d5e0-4c17-8d6f-5f3cfa3878b1","Type":"ContainerDied","Data":"7ca0e2a5b34bc2566a3169bffc8ec1d33f2f3015a9a550e9696f80f0d715a4dd"}
Sep 16 14:34:29.770123 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:29.770102    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-927f98-74b8f5ddcf-wstvh"
Sep 16 14:34:29.770467 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:29.770446    2580 generic.go:356] "Generic (PLEG): container finished" podID="b6c6d408-ef8b-4e6a-83db-22c6834a59c3" containerID="b22e08a69b4939582ac78c02c766dbbdebf313e6694fb6ee0966dbc0bd150f47" exitCode=0
Sep 16 14:34:29.770561 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:29.770509    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-927f98-74b8f5ddcf-wstvh" event={"ID":"b6c6d408-ef8b-4e6a-83db-22c6834a59c3","Type":"ContainerDied","Data":"b22e08a69b4939582ac78c02c766dbbdebf313e6694fb6ee0966dbc0bd150f47"}
Sep 16 14:34:29.770561 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:29.770551    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-927f98-74b8f5ddcf-wstvh" event={"ID":"b6c6d408-ef8b-4e6a-83db-22c6834a59c3","Type":"ContainerDied","Data":"c74293373fdc501771abdc285d554ed690b1c64d8ddd56d5dc8ab036153a6c2e"}
Sep 16 14:34:29.770638 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:29.770574    2580 scope.go:117] "RemoveContainer" containerID="b22e08a69b4939582ac78c02c766dbbdebf313e6694fb6ee0966dbc0bd150f47"
Sep 16 14:34:29.779647 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:29.779631    2580 scope.go:117] "RemoveContainer" containerID="b22e08a69b4939582ac78c02c766dbbdebf313e6694fb6ee0966dbc0bd150f47"
Sep 16 14:34:29.779927 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:34:29.779907    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b22e08a69b4939582ac78c02c766dbbdebf313e6694fb6ee0966dbc0bd150f47\": container with ID starting with b22e08a69b4939582ac78c02c766dbbdebf313e6694fb6ee0966dbc0bd150f47 not found: ID does not exist" containerID="b22e08a69b4939582ac78c02c766dbbdebf313e6694fb6ee0966dbc0bd150f47"
Sep 16 14:34:29.779993 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:29.779936    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b22e08a69b4939582ac78c02c766dbbdebf313e6694fb6ee0966dbc0bd150f47"} err="failed to get container status \"b22e08a69b4939582ac78c02c766dbbdebf313e6694fb6ee0966dbc0bd150f47\": rpc error: code = NotFound desc = could not find container \"b22e08a69b4939582ac78c02c766dbbdebf313e6694fb6ee0966dbc0bd150f47\": container with ID starting with b22e08a69b4939582ac78c02c766dbbdebf313e6694fb6ee0966dbc0bd150f47 not found: ID does not exist"
Sep 16 14:34:29.812015 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:29.811995    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-927f98-cd4ff9fc4-5xmv5"
Sep 16 14:34:29.919352 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:29.919322    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6c6d408-ef8b-4e6a-83db-22c6834a59c3-plugins-config-volume\") pod \"b6c6d408-ef8b-4e6a-83db-22c6834a59c3\" (UID: \"b6c6d408-ef8b-4e6a-83db-22c6834a59c3\") "
Sep 16 14:34:29.919545 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:29.919366    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jdnm7\" (UniqueName: \"kubernetes.io/projected/c56923cb-d5e0-4c17-8d6f-5f3cfa3878b1-kube-api-access-jdnm7\") pod \"c56923cb-d5e0-4c17-8d6f-5f3cfa3878b1\" (UID: \"c56923cb-d5e0-4c17-8d6f-5f3cfa3878b1\") "
Sep 16 14:34:29.919545 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:29.919418    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c56923cb-d5e0-4c17-8d6f-5f3cfa3878b1-plugins-config-volume\") pod \"c56923cb-d5e0-4c17-8d6f-5f3cfa3878b1\" (UID: \"c56923cb-d5e0-4c17-8d6f-5f3cfa3878b1\") "
Sep 16 14:34:29.919545 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:29.919437    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tct28\" (UniqueName: \"kubernetes.io/projected/b6c6d408-ef8b-4e6a-83db-22c6834a59c3-kube-api-access-tct28\") pod \"b6c6d408-ef8b-4e6a-83db-22c6834a59c3\" (UID: \"b6c6d408-ef8b-4e6a-83db-22c6834a59c3\") "
Sep 16 14:34:29.919836 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:29.919808    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6c6d408-ef8b-4e6a-83db-22c6834a59c3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b6c6d408-ef8b-4e6a-83db-22c6834a59c3" (UID: "b6c6d408-ef8b-4e6a-83db-22c6834a59c3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:34:29.919880 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:29.919834    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c56923cb-d5e0-4c17-8d6f-5f3cfa3878b1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c56923cb-d5e0-4c17-8d6f-5f3cfa3878b1" (UID: "c56923cb-d5e0-4c17-8d6f-5f3cfa3878b1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:34:29.921581 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:29.921553    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c56923cb-d5e0-4c17-8d6f-5f3cfa3878b1-kube-api-access-jdnm7" (OuterVolumeSpecName: "kube-api-access-jdnm7") pod "c56923cb-d5e0-4c17-8d6f-5f3cfa3878b1" (UID: "c56923cb-d5e0-4c17-8d6f-5f3cfa3878b1"). InnerVolumeSpecName "kube-api-access-jdnm7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:34:29.921889 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:29.921869    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6c6d408-ef8b-4e6a-83db-22c6834a59c3-kube-api-access-tct28" (OuterVolumeSpecName: "kube-api-access-tct28") pod "b6c6d408-ef8b-4e6a-83db-22c6834a59c3" (UID: "b6c6d408-ef8b-4e6a-83db-22c6834a59c3"). InnerVolumeSpecName "kube-api-access-tct28". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:34:30.021044 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:30.020960    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6c6d408-ef8b-4e6a-83db-22c6834a59c3-plugins-config-volume\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:34:30.021044 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:30.020994    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jdnm7\" (UniqueName: \"kubernetes.io/projected/c56923cb-d5e0-4c17-8d6f-5f3cfa3878b1-kube-api-access-jdnm7\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:34:30.021044 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:30.021003    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c56923cb-d5e0-4c17-8d6f-5f3cfa3878b1-plugins-config-volume\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:34:30.021044 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:30.021012    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tct28\" (UniqueName: \"kubernetes.io/projected/b6c6d408-ef8b-4e6a-83db-22c6834a59c3-kube-api-access-tct28\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:34:30.777384 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:30.777299    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-927f98-cd4ff9fc4-5xmv5"
Sep 16 14:34:30.777384 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:30.777332    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-927f98-cd4ff9fc4-5xmv5" event={"ID":"c56923cb-d5e0-4c17-8d6f-5f3cfa3878b1","Type":"ContainerDied","Data":"7b1e69cb5430caa0f4534576837e526978c20be43d7ad8fdb42c2b61a6f3ab0d"}
Sep 16 14:34:30.777884 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:30.777401    2580 scope.go:117] "RemoveContainer" containerID="7ca0e2a5b34bc2566a3169bffc8ec1d33f2f3015a9a550e9696f80f0d715a4dd"
Sep 16 14:34:30.778406 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:30.778389    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-927f98-74b8f5ddcf-wstvh"
Sep 16 14:34:30.824071 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:30.824048    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-927f98-74b8f5ddcf-wstvh"]
Sep 16 14:34:30.826348 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:30.826329    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-927f98-74b8f5ddcf-wstvh"]
Sep 16 14:34:30.834848 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:30.834828    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-927f98-cd4ff9fc4-5xmv5"]
Sep 16 14:34:30.846627 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:30.846607    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-927f98-cd4ff9fc4-5xmv5"]
Sep 16 14:34:31.440679 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:31.440647    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6c6d408-ef8b-4e6a-83db-22c6834a59c3" path="/var/lib/kubelet/pods/b6c6d408-ef8b-4e6a-83db-22c6834a59c3/volumes"
Sep 16 14:34:31.441030 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:31.441016    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c56923cb-d5e0-4c17-8d6f-5f3cfa3878b1" path="/var/lib/kubelet/pods/c56923cb-d5e0-4c17-8d6f-5f3cfa3878b1/volumes"
Sep 16 14:34:45.410222 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:45.410194    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-80d46a-76c9675489-8s4zj"]
Sep 16 14:34:45.410690 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:45.410639    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c56923cb-d5e0-4c17-8d6f-5f3cfa3878b1" containerName="payload-pre-processing"
Sep 16 14:34:45.410690 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:45.410655    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c56923cb-d5e0-4c17-8d6f-5f3cfa3878b1" containerName="payload-pre-processing"
Sep 16 14:34:45.410690 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:45.410664    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="188c78a5-068c-44da-a90d-45917ba5de3b" containerName="payload-processing"
Sep 16 14:34:45.410690 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:45.410671    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="188c78a5-068c-44da-a90d-45917ba5de3b" containerName="payload-processing"
Sep 16 14:34:45.410690 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:45.410687    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6c6d408-ef8b-4e6a-83db-22c6834a59c3" containerName="payload-processing"
Sep 16 14:34:45.410690 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:45.410692    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6c6d408-ef8b-4e6a-83db-22c6834a59c3" containerName="payload-processing"
Sep 16 14:34:45.410886 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:45.410758    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c56923cb-d5e0-4c17-8d6f-5f3cfa3878b1" containerName="payload-pre-processing"
Sep 16 14:34:45.410886 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:45.410768    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="188c78a5-068c-44da-a90d-45917ba5de3b" containerName="payload-processing"
Sep 16 14:34:45.410886 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:45.410775    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6c6d408-ef8b-4e6a-83db-22c6834a59c3" containerName="payload-processing"
Sep 16 14:34:45.415364 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:45.415344    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-80d46a-76c9675489-8s4zj"
Sep 16 14:34:45.418974 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:45.418945    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-80d46a-serving-cert\""
Sep 16 14:34:45.434228 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:45.434202    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-80d46a-76c9675489-8s4zj"]
Sep 16 14:34:45.470703 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:45.470675    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2c2ca51-46ee-4a99-b200-1b101c636511-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-80d46a-76c9675489-8s4zj\" (UID: \"a2c2ca51-46ee-4a99-b200-1b101c636511\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-80d46a-76c9675489-8s4zj"
Sep 16 14:34:45.470703 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:45.470705    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a2c2ca51-46ee-4a99-b200-1b101c636511-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-80d46a-76c9675489-8s4zj\" (UID: \"a2c2ca51-46ee-4a99-b200-1b101c636511\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-80d46a-76c9675489-8s4zj"
Sep 16 14:34:45.470988 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:45.470777    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlswq\" (UniqueName: \"kubernetes.io/projected/a2c2ca51-46ee-4a99-b200-1b101c636511-kube-api-access-tlswq\") pod \"maas-api-e2e-ipp-cleanup-80d46a-76c9675489-8s4zj\" (UID: \"a2c2ca51-46ee-4a99-b200-1b101c636511\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-80d46a-76c9675489-8s4zj"
Sep 16 14:34:45.470988 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:45.470869    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2c2ca51-46ee-4a99-b200-1b101c636511-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-80d46a-76c9675489-8s4zj\" (UID: \"a2c2ca51-46ee-4a99-b200-1b101c636511\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-80d46a-76c9675489-8s4zj"
Sep 16 14:34:45.571749 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:45.571713    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2c2ca51-46ee-4a99-b200-1b101c636511-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-80d46a-76c9675489-8s4zj\" (UID: \"a2c2ca51-46ee-4a99-b200-1b101c636511\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-80d46a-76c9675489-8s4zj"
Sep 16 14:34:45.571749 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:45.571749    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a2c2ca51-46ee-4a99-b200-1b101c636511-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-80d46a-76c9675489-8s4zj\" (UID: \"a2c2ca51-46ee-4a99-b200-1b101c636511\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-80d46a-76c9675489-8s4zj"
Sep 16 14:34:45.572018 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:45.571855    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlswq\" (UniqueName: \"kubernetes.io/projected/a2c2ca51-46ee-4a99-b200-1b101c636511-kube-api-access-tlswq\") pod \"maas-api-e2e-ipp-cleanup-80d46a-76c9675489-8s4zj\" (UID: \"a2c2ca51-46ee-4a99-b200-1b101c636511\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-80d46a-76c9675489-8s4zj"
Sep 16 14:34:45.572018 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:34:45.571889    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-80d46a-serving-cert: secret "maas-api-e2e-ipp-cleanup-80d46a-serving-cert" not found
Sep 16 14:34:45.572018 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:45.571911    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2c2ca51-46ee-4a99-b200-1b101c636511-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-80d46a-76c9675489-8s4zj\" (UID: \"a2c2ca51-46ee-4a99-b200-1b101c636511\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-80d46a-76c9675489-8s4zj"
Sep 16 14:34:45.572018 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:34:45.572003    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2c2ca51-46ee-4a99-b200-1b101c636511-maas-api-tls podName:a2c2ca51-46ee-4a99-b200-1b101c636511 nodeName:}" failed. No retries permitted until 2026-09-16 14:34:46.071980907 +0000 UTC m=+1819.087090763 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a2c2ca51-46ee-4a99-b200-1b101c636511-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-80d46a-76c9675489-8s4zj" (UID: "a2c2ca51-46ee-4a99-b200-1b101c636511") : secret "maas-api-e2e-ipp-cleanup-80d46a-serving-cert" not found
Sep 16 14:34:45.572630 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:45.572611    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2c2ca51-46ee-4a99-b200-1b101c636511-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-80d46a-76c9675489-8s4zj\" (UID: \"a2c2ca51-46ee-4a99-b200-1b101c636511\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-80d46a-76c9675489-8s4zj"
Sep 16 14:34:45.574390 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:45.574362    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a2c2ca51-46ee-4a99-b200-1b101c636511-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-80d46a-76c9675489-8s4zj\" (UID: \"a2c2ca51-46ee-4a99-b200-1b101c636511\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-80d46a-76c9675489-8s4zj"
Sep 16 14:34:45.582185 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:45.582161    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlswq\" (UniqueName: \"kubernetes.io/projected/a2c2ca51-46ee-4a99-b200-1b101c636511-kube-api-access-tlswq\") pod \"maas-api-e2e-ipp-cleanup-80d46a-76c9675489-8s4zj\" (UID: \"a2c2ca51-46ee-4a99-b200-1b101c636511\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-80d46a-76c9675489-8s4zj"
Sep 16 14:34:46.076710 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:46.076678    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2c2ca51-46ee-4a99-b200-1b101c636511-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-80d46a-76c9675489-8s4zj\" (UID: \"a2c2ca51-46ee-4a99-b200-1b101c636511\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-80d46a-76c9675489-8s4zj"
Sep 16 14:34:46.079137 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:46.079114    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2c2ca51-46ee-4a99-b200-1b101c636511-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-80d46a-76c9675489-8s4zj\" (UID: \"a2c2ca51-46ee-4a99-b200-1b101c636511\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-80d46a-76c9675489-8s4zj"
Sep 16 14:34:46.325360 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:46.325316    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-80d46a-76c9675489-8s4zj"
Sep 16 14:34:46.602569 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:46.602538    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-80d46a-76c9675489-8s4zj"]
Sep 16 14:34:46.603389 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:34:46.603359    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2c2ca51_46ee_4a99_b200_1b101c636511.slice/crio-5b0a3c8c7d73f881379e26b8cee9bc9ab8c7fa771f69586d80a4752173594fcb WatchSource:0}: Error finding container 5b0a3c8c7d73f881379e26b8cee9bc9ab8c7fa771f69586d80a4752173594fcb: Status 404 returned error can't find the container with id 5b0a3c8c7d73f881379e26b8cee9bc9ab8c7fa771f69586d80a4752173594fcb
Sep 16 14:34:46.870990 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:46.870902    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-80d46a-76c9675489-8s4zj" event={"ID":"a2c2ca51-46ee-4a99-b200-1b101c636511","Type":"ContainerStarted","Data":"5b0a3c8c7d73f881379e26b8cee9bc9ab8c7fa771f69586d80a4752173594fcb"}
Sep 16 14:34:47.878472 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:47.878441    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-80d46a-76c9675489-8s4zj" event={"ID":"a2c2ca51-46ee-4a99-b200-1b101c636511","Type":"ContainerStarted","Data":"ace3c7ccf9b774491ed2209b79179dda04cf60aa0521bb55e194c50ac7bc751b"}
Sep 16 14:34:47.878857 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:47.878525    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-80d46a-76c9675489-8s4zj"
Sep 16 14:34:47.904755 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:47.904702    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-80d46a-76c9675489-8s4zj" podStartSLOduration=2.521501707 podStartE2EDuration="2.904686821s" totalImagesPullingTime="383.185114ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:34:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:34:47.901423639 +0000 UTC m=+1820.916533514" watchObservedRunningTime="2026-09-16 14:34:47.904686821 +0000 UTC m=+1820.919796681"
Sep 16 14:34:48.362515 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:48.362466    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-2f9be3-96bb88d79-nvqx7"]
Sep 16 14:34:48.362728 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:48.362696    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-2f9be3-96bb88d79-nvqx7" podUID="ac9b3cd6-0c4e-4aa5-96d8-4d92646548ca" containerName="maas-api" containerID="cri-o://62569fb35ab1aa1a7784a79e989790492e1a8b02fc0c43a2628c7771f865e1b6" gracePeriod=30
Sep 16 14:34:48.457338 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:48.457308    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-2f9be3-b566b5c74-6m2r7"]
Sep 16 14:34:48.457577 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:48.457550    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-2f9be3-b566b5c74-6m2r7" podUID="fa6c5500-4785-4955-bb2b-ceec12a2ea84" containerName="payload-pre-processing" containerID="cri-o://391c9023067ea2f0efb23da5ccac9461072bf09d200977d6a02a7303e2b3e491" gracePeriod=30
Sep 16 14:34:48.720782 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:48.720750    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-2f9be3-96bb88d79-nvqx7"
Sep 16 14:34:48.748717 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:48.748691    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-2f9be3-b566b5c74-6m2r7"
Sep 16 14:34:48.804093 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:48.804065    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-djwwg\" (UniqueName: \"kubernetes.io/projected/ac9b3cd6-0c4e-4aa5-96d8-4d92646548ca-kube-api-access-djwwg\") pod \"ac9b3cd6-0c4e-4aa5-96d8-4d92646548ca\" (UID: \"ac9b3cd6-0c4e-4aa5-96d8-4d92646548ca\") "
Sep 16 14:34:48.804247 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:48.804108    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac9b3cd6-0c4e-4aa5-96d8-4d92646548ca-maas-api-tls\") pod \"ac9b3cd6-0c4e-4aa5-96d8-4d92646548ca\" (UID: \"ac9b3cd6-0c4e-4aa5-96d8-4d92646548ca\") "
Sep 16 14:34:48.804247 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:48.804133    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4hnjw\" (UniqueName: \"kubernetes.io/projected/fa6c5500-4785-4955-bb2b-ceec12a2ea84-kube-api-access-4hnjw\") pod \"fa6c5500-4785-4955-bb2b-ceec12a2ea84\" (UID: \"fa6c5500-4785-4955-bb2b-ceec12a2ea84\") "
Sep 16 14:34:48.804247 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:48.804170    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa6c5500-4785-4955-bb2b-ceec12a2ea84-plugins-config-volume\") pod \"fa6c5500-4785-4955-bb2b-ceec12a2ea84\" (UID: \"fa6c5500-4785-4955-bb2b-ceec12a2ea84\") "
Sep 16 14:34:48.804247 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:48.804188    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac9b3cd6-0c4e-4aa5-96d8-4d92646548ca-openshift-service-ca\") pod \"ac9b3cd6-0c4e-4aa5-96d8-4d92646548ca\" (UID: \"ac9b3cd6-0c4e-4aa5-96d8-4d92646548ca\") "
Sep 16 14:34:48.804247 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:48.804213    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac9b3cd6-0c4e-4aa5-96d8-4d92646548ca-metrics-certs\") pod \"ac9b3cd6-0c4e-4aa5-96d8-4d92646548ca\" (UID: \"ac9b3cd6-0c4e-4aa5-96d8-4d92646548ca\") "
Sep 16 14:34:48.804655 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:48.804612    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa6c5500-4785-4955-bb2b-ceec12a2ea84-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fa6c5500-4785-4955-bb2b-ceec12a2ea84" (UID: "fa6c5500-4785-4955-bb2b-ceec12a2ea84"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:34:48.804750 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:48.804661    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac9b3cd6-0c4e-4aa5-96d8-4d92646548ca-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ac9b3cd6-0c4e-4aa5-96d8-4d92646548ca" (UID: "ac9b3cd6-0c4e-4aa5-96d8-4d92646548ca"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:34:48.806806 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:48.806576    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac9b3cd6-0c4e-4aa5-96d8-4d92646548ca-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ac9b3cd6-0c4e-4aa5-96d8-4d92646548ca" (UID: "ac9b3cd6-0c4e-4aa5-96d8-4d92646548ca"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:34:48.806806 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:48.806753    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa6c5500-4785-4955-bb2b-ceec12a2ea84-kube-api-access-4hnjw" (OuterVolumeSpecName: "kube-api-access-4hnjw") pod "fa6c5500-4785-4955-bb2b-ceec12a2ea84" (UID: "fa6c5500-4785-4955-bb2b-ceec12a2ea84"). InnerVolumeSpecName "kube-api-access-4hnjw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:34:48.806806 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:48.806762    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac9b3cd6-0c4e-4aa5-96d8-4d92646548ca-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ac9b3cd6-0c4e-4aa5-96d8-4d92646548ca" (UID: "ac9b3cd6-0c4e-4aa5-96d8-4d92646548ca"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:34:48.806806 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:48.806787    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac9b3cd6-0c4e-4aa5-96d8-4d92646548ca-kube-api-access-djwwg" (OuterVolumeSpecName: "kube-api-access-djwwg") pod "ac9b3cd6-0c4e-4aa5-96d8-4d92646548ca" (UID: "ac9b3cd6-0c4e-4aa5-96d8-4d92646548ca"). InnerVolumeSpecName "kube-api-access-djwwg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:34:48.885274 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:48.885194    2580 generic.go:356] "Generic (PLEG): container finished" podID="ac9b3cd6-0c4e-4aa5-96d8-4d92646548ca" containerID="62569fb35ab1aa1a7784a79e989790492e1a8b02fc0c43a2628c7771f865e1b6" exitCode=0
Sep 16 14:34:48.885274 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:48.885245    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-2f9be3-96bb88d79-nvqx7"
Sep 16 14:34:48.885784 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:48.885277    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-2f9be3-96bb88d79-nvqx7" event={"ID":"ac9b3cd6-0c4e-4aa5-96d8-4d92646548ca","Type":"ContainerDied","Data":"62569fb35ab1aa1a7784a79e989790492e1a8b02fc0c43a2628c7771f865e1b6"}
Sep 16 14:34:48.885784 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:48.885333    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-2f9be3-96bb88d79-nvqx7" event={"ID":"ac9b3cd6-0c4e-4aa5-96d8-4d92646548ca","Type":"ContainerDied","Data":"cfc844f8d94cf0ed6be2eaa94367ba426f8b045715e101493e53f7f4ab389666"}
Sep 16 14:34:48.885784 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:48.885357    2580 scope.go:117] "RemoveContainer" containerID="62569fb35ab1aa1a7784a79e989790492e1a8b02fc0c43a2628c7771f865e1b6"
Sep 16 14:34:48.886654 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:48.886628    2580 generic.go:356] "Generic (PLEG): container finished" podID="fa6c5500-4785-4955-bb2b-ceec12a2ea84" containerID="391c9023067ea2f0efb23da5ccac9461072bf09d200977d6a02a7303e2b3e491" exitCode=0
Sep 16 14:34:48.886766 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:48.886661    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-2f9be3-b566b5c74-6m2r7"
Sep 16 14:34:48.886766 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:48.886706    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-2f9be3-b566b5c74-6m2r7" event={"ID":"fa6c5500-4785-4955-bb2b-ceec12a2ea84","Type":"ContainerDied","Data":"391c9023067ea2f0efb23da5ccac9461072bf09d200977d6a02a7303e2b3e491"}
Sep 16 14:34:48.886766 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:48.886762    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-2f9be3-b566b5c74-6m2r7" event={"ID":"fa6c5500-4785-4955-bb2b-ceec12a2ea84","Type":"ContainerDied","Data":"007c361b64c301e3d6644a1943102a39db195f3039837e8b6ab335ce9c3c5c16"}
Sep 16 14:34:48.895701 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:48.895691    2580 scope.go:117] "RemoveContainer" containerID="62569fb35ab1aa1a7784a79e989790492e1a8b02fc0c43a2628c7771f865e1b6"
Sep 16 14:34:48.895969 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:34:48.895931    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62569fb35ab1aa1a7784a79e989790492e1a8b02fc0c43a2628c7771f865e1b6\": container with ID starting with 62569fb35ab1aa1a7784a79e989790492e1a8b02fc0c43a2628c7771f865e1b6 not found: ID does not exist" containerID="62569fb35ab1aa1a7784a79e989790492e1a8b02fc0c43a2628c7771f865e1b6"
Sep 16 14:34:48.895969 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:48.895956    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62569fb35ab1aa1a7784a79e989790492e1a8b02fc0c43a2628c7771f865e1b6"} err="failed to get container status \"62569fb35ab1aa1a7784a79e989790492e1a8b02fc0c43a2628c7771f865e1b6\": rpc error: code = NotFound desc = could not find container \"62569fb35ab1aa1a7784a79e989790492e1a8b02fc0c43a2628c7771f865e1b6\": container with ID starting with 62569fb35ab1aa1a7784a79e989790492e1a8b02fc0c43a2628c7771f865e1b6 not found: ID does not exist"
Sep 16 14:34:48.896101 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:48.895974    2580 scope.go:117] "RemoveContainer" containerID="391c9023067ea2f0efb23da5ccac9461072bf09d200977d6a02a7303e2b3e491"
Sep 16 14:34:48.903816 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:48.903802    2580 scope.go:117] "RemoveContainer" containerID="391c9023067ea2f0efb23da5ccac9461072bf09d200977d6a02a7303e2b3e491"
Sep 16 14:34:48.904086 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:34:48.904071    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"391c9023067ea2f0efb23da5ccac9461072bf09d200977d6a02a7303e2b3e491\": container with ID starting with 391c9023067ea2f0efb23da5ccac9461072bf09d200977d6a02a7303e2b3e491 not found: ID does not exist" containerID="391c9023067ea2f0efb23da5ccac9461072bf09d200977d6a02a7303e2b3e491"
Sep 16 14:34:48.904125 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:48.904091    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"391c9023067ea2f0efb23da5ccac9461072bf09d200977d6a02a7303e2b3e491"} err="failed to get container status \"391c9023067ea2f0efb23da5ccac9461072bf09d200977d6a02a7303e2b3e491\": rpc error: code = NotFound desc = could not find container \"391c9023067ea2f0efb23da5ccac9461072bf09d200977d6a02a7303e2b3e491\": container with ID starting with 391c9023067ea2f0efb23da5ccac9461072bf09d200977d6a02a7303e2b3e491 not found: ID does not exist"
Sep 16 14:34:48.905470 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:48.905455    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-djwwg\" (UniqueName: \"kubernetes.io/projected/ac9b3cd6-0c4e-4aa5-96d8-4d92646548ca-kube-api-access-djwwg\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:34:48.905539 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:48.905474    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac9b3cd6-0c4e-4aa5-96d8-4d92646548ca-maas-api-tls\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:34:48.905539 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:48.905484    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4hnjw\" (UniqueName: \"kubernetes.io/projected/fa6c5500-4785-4955-bb2b-ceec12a2ea84-kube-api-access-4hnjw\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:34:48.905539 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:48.905528    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa6c5500-4785-4955-bb2b-ceec12a2ea84-plugins-config-volume\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:34:48.905685 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:48.905545    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac9b3cd6-0c4e-4aa5-96d8-4d92646548ca-openshift-service-ca\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:34:48.905685 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:48.905558    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac9b3cd6-0c4e-4aa5-96d8-4d92646548ca-metrics-certs\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:34:48.911782 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:48.911735    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-2f9be3-96bb88d79-nvqx7"]
Sep 16 14:34:48.915687 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:48.915663    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-2f9be3-96bb88d79-nvqx7"]
Sep 16 14:34:48.921036 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:48.921017    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-2f9be3-b566b5c74-6m2r7"]
Sep 16 14:34:48.927127 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:48.927104    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-2f9be3-b566b5c74-6m2r7"]
Sep 16 14:34:49.430298 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:49.430259    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac9b3cd6-0c4e-4aa5-96d8-4d92646548ca" path="/var/lib/kubelet/pods/ac9b3cd6-0c4e-4aa5-96d8-4d92646548ca/volumes"
Sep 16 14:34:49.430807 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:49.430789    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa6c5500-4785-4955-bb2b-ceec12a2ea84" path="/var/lib/kubelet/pods/fa6c5500-4785-4955-bb2b-ceec12a2ea84/volumes"
Sep 16 14:34:53.892032 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:34:53.892001    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-80d46a-76c9675489-8s4zj"
Sep 16 14:35:07.897864 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:07.897828    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-cbeb11-69d98645fc-bwhqw"]
Sep 16 14:35:07.898288 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:07.898052    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-cbeb11-69d98645fc-bwhqw" podUID="26764d22-8352-4c06-98c1-1097e10c7e7d" containerName="maas-api" containerID="cri-o://ef2e7e91490f915656d47f7cca2cafbdc2985b543617ab7b872a3d0a6c30d51e" gracePeriod=30
Sep 16 14:35:07.957465 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:07.957437    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-cbeb11-29826150-zjnb4"]
Sep 16 14:35:07.960848 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:07.960823    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-cbeb11-29826150-zjnb4"]
Sep 16 14:35:07.987925 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:07.987896    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-cbeb11-549d695445-rztrz"]
Sep 16 14:35:07.988149 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:07.988123    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-cbeb11-549d695445-rztrz" podUID="d25388e0-508a-4169-b297-82e555257170" containerName="payload-processing" containerID="cri-o://1263deecacb26c6e62233ca31250cf922072c50f5238b518f11f983f261cc659" gracePeriod=30
Sep 16 14:35:08.276302 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:08.276279    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-cbeb11-69d98645fc-bwhqw"
Sep 16 14:35:08.278936 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:08.278913    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-cbeb11-549d695445-rztrz"
Sep 16 14:35:08.391207 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:08.391174    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/26764d22-8352-4c06-98c1-1097e10c7e7d-maas-api-tls\") pod \"26764d22-8352-4c06-98c1-1097e10c7e7d\" (UID: \"26764d22-8352-4c06-98c1-1097e10c7e7d\") "
Sep 16 14:35:08.391207 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:08.391208    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-956ws\" (UniqueName: \"kubernetes.io/projected/26764d22-8352-4c06-98c1-1097e10c7e7d-kube-api-access-956ws\") pod \"26764d22-8352-4c06-98c1-1097e10c7e7d\" (UID: \"26764d22-8352-4c06-98c1-1097e10c7e7d\") "
Sep 16 14:35:08.391459 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:08.391262    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d25388e0-508a-4169-b297-82e555257170-plugins-config-volume\") pod \"d25388e0-508a-4169-b297-82e555257170\" (UID: \"d25388e0-508a-4169-b297-82e555257170\") "
Sep 16 14:35:08.391459 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:08.391305    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/26764d22-8352-4c06-98c1-1097e10c7e7d-openshift-service-ca\") pod \"26764d22-8352-4c06-98c1-1097e10c7e7d\" (UID: \"26764d22-8352-4c06-98c1-1097e10c7e7d\") "
Sep 16 14:35:08.391459 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:08.391343    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26764d22-8352-4c06-98c1-1097e10c7e7d-metrics-certs\") pod \"26764d22-8352-4c06-98c1-1097e10c7e7d\" (UID: \"26764d22-8352-4c06-98c1-1097e10c7e7d\") "
Sep 16 14:35:08.391459 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:08.391388    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9md5v\" (UniqueName: \"kubernetes.io/projected/d25388e0-508a-4169-b297-82e555257170-kube-api-access-9md5v\") pod \"d25388e0-508a-4169-b297-82e555257170\" (UID: \"d25388e0-508a-4169-b297-82e555257170\") "
Sep 16 14:35:08.391718 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:08.391658    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26764d22-8352-4c06-98c1-1097e10c7e7d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "26764d22-8352-4c06-98c1-1097e10c7e7d" (UID: "26764d22-8352-4c06-98c1-1097e10c7e7d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:35:08.391718 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:08.391701    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d25388e0-508a-4169-b297-82e555257170-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d25388e0-508a-4169-b297-82e555257170" (UID: "d25388e0-508a-4169-b297-82e555257170"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:35:08.391833 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:08.391772    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d25388e0-508a-4169-b297-82e555257170-plugins-config-volume\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:35:08.391833 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:08.391791    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/26764d22-8352-4c06-98c1-1097e10c7e7d-openshift-service-ca\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:35:08.393537 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:08.393508    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d25388e0-508a-4169-b297-82e555257170-kube-api-access-9md5v" (OuterVolumeSpecName: "kube-api-access-9md5v") pod "d25388e0-508a-4169-b297-82e555257170" (UID: "d25388e0-508a-4169-b297-82e555257170"). InnerVolumeSpecName "kube-api-access-9md5v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:35:08.393638 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:08.393545    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/26764d22-8352-4c06-98c1-1097e10c7e7d-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "26764d22-8352-4c06-98c1-1097e10c7e7d" (UID: "26764d22-8352-4c06-98c1-1097e10c7e7d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:35:08.393638 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:08.393583    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/26764d22-8352-4c06-98c1-1097e10c7e7d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "26764d22-8352-4c06-98c1-1097e10c7e7d" (UID: "26764d22-8352-4c06-98c1-1097e10c7e7d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:35:08.393638 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:08.393620    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26764d22-8352-4c06-98c1-1097e10c7e7d-kube-api-access-956ws" (OuterVolumeSpecName: "kube-api-access-956ws") pod "26764d22-8352-4c06-98c1-1097e10c7e7d" (UID: "26764d22-8352-4c06-98c1-1097e10c7e7d"). InnerVolumeSpecName "kube-api-access-956ws". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:35:08.492734 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:08.492700    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9md5v\" (UniqueName: \"kubernetes.io/projected/d25388e0-508a-4169-b297-82e555257170-kube-api-access-9md5v\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:35:08.492734 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:08.492727    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/26764d22-8352-4c06-98c1-1097e10c7e7d-maas-api-tls\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:35:08.492734 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:08.492739    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-956ws\" (UniqueName: \"kubernetes.io/projected/26764d22-8352-4c06-98c1-1097e10c7e7d-kube-api-access-956ws\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:35:08.493050 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:08.492753    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26764d22-8352-4c06-98c1-1097e10c7e7d-metrics-certs\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:35:09.019456 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:09.019397    2580 generic.go:356] "Generic (PLEG): container finished" podID="26764d22-8352-4c06-98c1-1097e10c7e7d" containerID="ef2e7e91490f915656d47f7cca2cafbdc2985b543617ab7b872a3d0a6c30d51e" exitCode=0
Sep 16 14:35:09.019456 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:09.019438    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-cbeb11-69d98645fc-bwhqw"
Sep 16 14:35:09.019456 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:09.019477    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-cbeb11-69d98645fc-bwhqw" event={"ID":"26764d22-8352-4c06-98c1-1097e10c7e7d","Type":"ContainerDied","Data":"ef2e7e91490f915656d47f7cca2cafbdc2985b543617ab7b872a3d0a6c30d51e"}
Sep 16 14:35:09.020091 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:09.019547    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-cbeb11-69d98645fc-bwhqw" event={"ID":"26764d22-8352-4c06-98c1-1097e10c7e7d","Type":"ContainerDied","Data":"c154cb268489bcceb616c2522a01583ba683c854268ad153aa295ba6b01dbfd2"}
Sep 16 14:35:09.020091 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:09.019578    2580 scope.go:117] "RemoveContainer" containerID="ef2e7e91490f915656d47f7cca2cafbdc2985b543617ab7b872a3d0a6c30d51e"
Sep 16 14:35:09.021169 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:09.021148    2580 generic.go:356] "Generic (PLEG): container finished" podID="d25388e0-508a-4169-b297-82e555257170" containerID="1263deecacb26c6e62233ca31250cf922072c50f5238b518f11f983f261cc659" exitCode=0
Sep 16 14:35:09.021250 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:09.021178    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-cbeb11-549d695445-rztrz" event={"ID":"d25388e0-508a-4169-b297-82e555257170","Type":"ContainerDied","Data":"1263deecacb26c6e62233ca31250cf922072c50f5238b518f11f983f261cc659"}
Sep 16 14:35:09.021250 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:09.021201    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-cbeb11-549d695445-rztrz"
Sep 16 14:35:09.021250 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:09.021236    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-cbeb11-549d695445-rztrz" event={"ID":"d25388e0-508a-4169-b297-82e555257170","Type":"ContainerDied","Data":"86e6eb2eae532f0ac374ee8ffd8b59d9d3dff0a746209de86e0cab116ecb385c"}
Sep 16 14:35:09.030359 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:09.030339    2580 scope.go:117] "RemoveContainer" containerID="ef2e7e91490f915656d47f7cca2cafbdc2985b543617ab7b872a3d0a6c30d51e"
Sep 16 14:35:09.030745 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:35:09.030630    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef2e7e91490f915656d47f7cca2cafbdc2985b543617ab7b872a3d0a6c30d51e\": container with ID starting with ef2e7e91490f915656d47f7cca2cafbdc2985b543617ab7b872a3d0a6c30d51e not found: ID does not exist" containerID="ef2e7e91490f915656d47f7cca2cafbdc2985b543617ab7b872a3d0a6c30d51e"
Sep 16 14:35:09.030745 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:09.030656    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef2e7e91490f915656d47f7cca2cafbdc2985b543617ab7b872a3d0a6c30d51e"} err="failed to get container status \"ef2e7e91490f915656d47f7cca2cafbdc2985b543617ab7b872a3d0a6c30d51e\": rpc error: code = NotFound desc = could not find container \"ef2e7e91490f915656d47f7cca2cafbdc2985b543617ab7b872a3d0a6c30d51e\": container with ID starting with ef2e7e91490f915656d47f7cca2cafbdc2985b543617ab7b872a3d0a6c30d51e not found: ID does not exist"
Sep 16 14:35:09.030745 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:09.030671    2580 scope.go:117] "RemoveContainer" containerID="1263deecacb26c6e62233ca31250cf922072c50f5238b518f11f983f261cc659"
Sep 16 14:35:09.039197 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:09.039177    2580 scope.go:117] "RemoveContainer" containerID="1263deecacb26c6e62233ca31250cf922072c50f5238b518f11f983f261cc659"
Sep 16 14:35:09.039469 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:35:09.039452    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1263deecacb26c6e62233ca31250cf922072c50f5238b518f11f983f261cc659\": container with ID starting with 1263deecacb26c6e62233ca31250cf922072c50f5238b518f11f983f261cc659 not found: ID does not exist" containerID="1263deecacb26c6e62233ca31250cf922072c50f5238b518f11f983f261cc659"
Sep 16 14:35:09.039549 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:09.039476    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1263deecacb26c6e62233ca31250cf922072c50f5238b518f11f983f261cc659"} err="failed to get container status \"1263deecacb26c6e62233ca31250cf922072c50f5238b518f11f983f261cc659\": rpc error: code = NotFound desc = could not find container \"1263deecacb26c6e62233ca31250cf922072c50f5238b518f11f983f261cc659\": container with ID starting with 1263deecacb26c6e62233ca31250cf922072c50f5238b518f11f983f261cc659 not found: ID does not exist"
Sep 16 14:35:09.047307 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:09.047287    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-cbeb11-69d98645fc-bwhqw"]
Sep 16 14:35:09.051068 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:09.051048    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-cbeb11-69d98645fc-bwhqw"]
Sep 16 14:35:09.063061 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:09.063041    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-cbeb11-549d695445-rztrz"]
Sep 16 14:35:09.069057 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:09.069038    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-cbeb11-549d695445-rztrz"]
Sep 16 14:35:09.430919 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:09.430888    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26764d22-8352-4c06-98c1-1097e10c7e7d" path="/var/lib/kubelet/pods/26764d22-8352-4c06-98c1-1097e10c7e7d/volumes"
Sep 16 14:35:09.431214 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:09.431201    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67048f3c-d226-46d0-8535-04f5db62ea5f" path="/var/lib/kubelet/pods/67048f3c-d226-46d0-8535-04f5db62ea5f/volumes"
Sep 16 14:35:09.431485 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:09.431474    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d25388e0-508a-4169-b297-82e555257170" path="/var/lib/kubelet/pods/d25388e0-508a-4169-b297-82e555257170/volumes"
Sep 16 14:35:40.942148 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:40.942112    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv"]
Sep 16 14:35:40.942710 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:40.942347    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv" podUID="e0f4134f-f3be-492c-921f-a371c8d204a4" containerName="istio-proxy" containerID="cri-o://e21dad2079bdc022ebad1238cd803d1156aa95936899e202ab4458b0974777fd" gracePeriod=30
Sep 16 14:35:41.734890 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:41.734853    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-80d46a-76c9675489-8s4zj"]
Sep 16 14:35:41.735170 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:41.735117    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-80d46a-76c9675489-8s4zj" podUID="a2c2ca51-46ee-4a99-b200-1b101c636511" containerName="maas-api" containerID="cri-o://ace3c7ccf9b774491ed2209b79179dda04cf60aa0521bb55e194c50ac7bc751b" gracePeriod=30
Sep 16 14:35:41.988172 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:41.988118    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-80d46a-76c9675489-8s4zj"
Sep 16 14:35:42.085314 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:42.085279    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2c2ca51-46ee-4a99-b200-1b101c636511-maas-api-tls\") pod \"a2c2ca51-46ee-4a99-b200-1b101c636511\" (UID: \"a2c2ca51-46ee-4a99-b200-1b101c636511\") "
Sep 16 14:35:42.085536 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:42.085447    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tlswq\" (UniqueName: \"kubernetes.io/projected/a2c2ca51-46ee-4a99-b200-1b101c636511-kube-api-access-tlswq\") pod \"a2c2ca51-46ee-4a99-b200-1b101c636511\" (UID: \"a2c2ca51-46ee-4a99-b200-1b101c636511\") "
Sep 16 14:35:42.085536 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:42.085532    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2c2ca51-46ee-4a99-b200-1b101c636511-openshift-service-ca\") pod \"a2c2ca51-46ee-4a99-b200-1b101c636511\" (UID: \"a2c2ca51-46ee-4a99-b200-1b101c636511\") "
Sep 16 14:35:42.085671 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:42.085561    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a2c2ca51-46ee-4a99-b200-1b101c636511-metrics-certs\") pod \"a2c2ca51-46ee-4a99-b200-1b101c636511\" (UID: \"a2c2ca51-46ee-4a99-b200-1b101c636511\") "
Sep 16 14:35:42.085855 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:42.085831    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2c2ca51-46ee-4a99-b200-1b101c636511-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a2c2ca51-46ee-4a99-b200-1b101c636511" (UID: "a2c2ca51-46ee-4a99-b200-1b101c636511"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:35:42.087442 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:42.087413    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2c2ca51-46ee-4a99-b200-1b101c636511-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a2c2ca51-46ee-4a99-b200-1b101c636511" (UID: "a2c2ca51-46ee-4a99-b200-1b101c636511"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:35:42.087582 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:42.087561    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2c2ca51-46ee-4a99-b200-1b101c636511-kube-api-access-tlswq" (OuterVolumeSpecName: "kube-api-access-tlswq") pod "a2c2ca51-46ee-4a99-b200-1b101c636511" (UID: "a2c2ca51-46ee-4a99-b200-1b101c636511"). InnerVolumeSpecName "kube-api-access-tlswq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:35:42.087668 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:42.087649    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2c2ca51-46ee-4a99-b200-1b101c636511-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a2c2ca51-46ee-4a99-b200-1b101c636511" (UID: "a2c2ca51-46ee-4a99-b200-1b101c636511"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:35:42.186878 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:42.186843    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2c2ca51-46ee-4a99-b200-1b101c636511-openshift-service-ca\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:35:42.186878 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:42.186872    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a2c2ca51-46ee-4a99-b200-1b101c636511-metrics-certs\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:35:42.186878 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:42.186882    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2c2ca51-46ee-4a99-b200-1b101c636511-maas-api-tls\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:35:42.187120 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:42.186892    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tlswq\" (UniqueName: \"kubernetes.io/projected/a2c2ca51-46ee-4a99-b200-1b101c636511-kube-api-access-tlswq\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:35:42.205632 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:42.205555    2580 generic.go:356] "Generic (PLEG): container finished" podID="a2c2ca51-46ee-4a99-b200-1b101c636511" containerID="ace3c7ccf9b774491ed2209b79179dda04cf60aa0521bb55e194c50ac7bc751b" exitCode=0
Sep 16 14:35:42.205632 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:42.205599    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-80d46a-76c9675489-8s4zj" event={"ID":"a2c2ca51-46ee-4a99-b200-1b101c636511","Type":"ContainerDied","Data":"ace3c7ccf9b774491ed2209b79179dda04cf60aa0521bb55e194c50ac7bc751b"}
Sep 16 14:35:42.205632 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:42.205620    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-80d46a-76c9675489-8s4zj"
Sep 16 14:35:42.205891 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:42.205659    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-80d46a-76c9675489-8s4zj" event={"ID":"a2c2ca51-46ee-4a99-b200-1b101c636511","Type":"ContainerDied","Data":"5b0a3c8c7d73f881379e26b8cee9bc9ab8c7fa771f69586d80a4752173594fcb"}
Sep 16 14:35:42.205891 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:42.205690    2580 scope.go:117] "RemoveContainer" containerID="ace3c7ccf9b774491ed2209b79179dda04cf60aa0521bb55e194c50ac7bc751b"
Sep 16 14:35:42.218837 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:42.218818    2580 scope.go:117] "RemoveContainer" containerID="ace3c7ccf9b774491ed2209b79179dda04cf60aa0521bb55e194c50ac7bc751b"
Sep 16 14:35:42.219149 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:35:42.219125    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ace3c7ccf9b774491ed2209b79179dda04cf60aa0521bb55e194c50ac7bc751b\": container with ID starting with ace3c7ccf9b774491ed2209b79179dda04cf60aa0521bb55e194c50ac7bc751b not found: ID does not exist" containerID="ace3c7ccf9b774491ed2209b79179dda04cf60aa0521bb55e194c50ac7bc751b"
Sep 16 14:35:42.219203 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:42.219162    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ace3c7ccf9b774491ed2209b79179dda04cf60aa0521bb55e194c50ac7bc751b"} err="failed to get container status \"ace3c7ccf9b774491ed2209b79179dda04cf60aa0521bb55e194c50ac7bc751b\": rpc error: code = NotFound desc = could not find container \"ace3c7ccf9b774491ed2209b79179dda04cf60aa0521bb55e194c50ac7bc751b\": container with ID starting with ace3c7ccf9b774491ed2209b79179dda04cf60aa0521bb55e194c50ac7bc751b not found: ID does not exist"
Sep 16 14:35:42.232439 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:42.232411    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-80d46a-76c9675489-8s4zj"]
Sep 16 14:35:42.238927 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:42.238869    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-80d46a-76c9675489-8s4zj"]
Sep 16 14:35:43.430554 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:43.430513    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2c2ca51-46ee-4a99-b200-1b101c636511" path="/var/lib/kubelet/pods/a2c2ca51-46ee-4a99-b200-1b101c636511/volumes"
Sep 16 14:35:46.186927 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:46.186903    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv"
Sep 16 14:35:46.225917 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:46.225855    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0f4134f-f3be-492c-921f-a371c8d204a4-istiod-ca-cert\") pod \"e0f4134f-f3be-492c-921f-a371c8d204a4\" (UID: \"e0f4134f-f3be-492c-921f-a371c8d204a4\") "
Sep 16 14:35:46.225917 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:46.225897    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wtv4b\" (UniqueName: \"kubernetes.io/projected/e0f4134f-f3be-492c-921f-a371c8d204a4-kube-api-access-wtv4b\") pod \"e0f4134f-f3be-492c-921f-a371c8d204a4\" (UID: \"e0f4134f-f3be-492c-921f-a371c8d204a4\") "
Sep 16 14:35:46.226116 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:46.225949    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0f4134f-f3be-492c-921f-a371c8d204a4-istio-token\") pod \"e0f4134f-f3be-492c-921f-a371c8d204a4\" (UID: \"e0f4134f-f3be-492c-921f-a371c8d204a4\") "
Sep 16 14:35:46.226116 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:46.225973    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0f4134f-f3be-492c-921f-a371c8d204a4-credential-socket\") pod \"e0f4134f-f3be-492c-921f-a371c8d204a4\" (UID: \"e0f4134f-f3be-492c-921f-a371c8d204a4\") "
Sep 16 14:35:46.226116 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:46.226002    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0f4134f-f3be-492c-921f-a371c8d204a4-istio-data\") pod \"e0f4134f-f3be-492c-921f-a371c8d204a4\" (UID: \"e0f4134f-f3be-492c-921f-a371c8d204a4\") "
Sep 16 14:35:46.226116 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:46.226039    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0f4134f-f3be-492c-921f-a371c8d204a4-istio-envoy\") pod \"e0f4134f-f3be-492c-921f-a371c8d204a4\" (UID: \"e0f4134f-f3be-492c-921f-a371c8d204a4\") "
Sep 16 14:35:46.226116 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:46.226052    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0f4134f-f3be-492c-921f-a371c8d204a4-workload-socket\") pod \"e0f4134f-f3be-492c-921f-a371c8d204a4\" (UID: \"e0f4134f-f3be-492c-921f-a371c8d204a4\") "
Sep 16 14:35:46.226116 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:46.226114    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0f4134f-f3be-492c-921f-a371c8d204a4-workload-certs\") pod \"e0f4134f-f3be-492c-921f-a371c8d204a4\" (UID: \"e0f4134f-f3be-492c-921f-a371c8d204a4\") "
Sep 16 14:35:46.226539 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:46.226142    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0f4134f-f3be-492c-921f-a371c8d204a4-istio-podinfo\") pod \"e0f4134f-f3be-492c-921f-a371c8d204a4\" (UID: \"e0f4134f-f3be-492c-921f-a371c8d204a4\") "
Sep 16 14:35:46.226539 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:46.226229    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e0f4134f-f3be-492c-921f-a371c8d204a4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e0f4134f-f3be-492c-921f-a371c8d204a4" (UID: "e0f4134f-f3be-492c-921f-a371c8d204a4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:35:46.226539 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:46.226396    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0f4134f-f3be-492c-921f-a371c8d204a4-istiod-ca-cert\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:35:46.226539 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:46.226452    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0f4134f-f3be-492c-921f-a371c8d204a4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e0f4134f-f3be-492c-921f-a371c8d204a4" (UID: "e0f4134f-f3be-492c-921f-a371c8d204a4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:35:46.226822 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:46.226621    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0f4134f-f3be-492c-921f-a371c8d204a4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e0f4134f-f3be-492c-921f-a371c8d204a4" (UID: "e0f4134f-f3be-492c-921f-a371c8d204a4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:35:46.226822 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:46.226642    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0f4134f-f3be-492c-921f-a371c8d204a4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e0f4134f-f3be-492c-921f-a371c8d204a4" (UID: "e0f4134f-f3be-492c-921f-a371c8d204a4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:35:46.226926 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:46.226820    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0f4134f-f3be-492c-921f-a371c8d204a4-istio-data" (OuterVolumeSpecName: "istio-data") pod "e0f4134f-f3be-492c-921f-a371c8d204a4" (UID: "e0f4134f-f3be-492c-921f-a371c8d204a4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:35:46.228765 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:46.228705    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e0f4134f-f3be-492c-921f-a371c8d204a4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e0f4134f-f3be-492c-921f-a371c8d204a4" (UID: "e0f4134f-f3be-492c-921f-a371c8d204a4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 14:35:46.228884 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:46.228818    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0f4134f-f3be-492c-921f-a371c8d204a4-kube-api-access-wtv4b" (OuterVolumeSpecName: "kube-api-access-wtv4b") pod "e0f4134f-f3be-492c-921f-a371c8d204a4" (UID: "e0f4134f-f3be-492c-921f-a371c8d204a4"). InnerVolumeSpecName "kube-api-access-wtv4b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:35:46.229039 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:46.229016    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0f4134f-f3be-492c-921f-a371c8d204a4-istio-token" (OuterVolumeSpecName: "istio-token") pod "e0f4134f-f3be-492c-921f-a371c8d204a4" (UID: "e0f4134f-f3be-492c-921f-a371c8d204a4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:35:46.229101 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:46.229027    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0f4134f-f3be-492c-921f-a371c8d204a4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e0f4134f-f3be-492c-921f-a371c8d204a4" (UID: "e0f4134f-f3be-492c-921f-a371c8d204a4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:35:46.233120 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:46.233093    2580 generic.go:356] "Generic (PLEG): container finished" podID="e0f4134f-f3be-492c-921f-a371c8d204a4" containerID="e21dad2079bdc022ebad1238cd803d1156aa95936899e202ab4458b0974777fd" exitCode=0
Sep 16 14:35:46.233227 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:46.233137    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv"
Sep 16 14:35:46.233227 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:46.233167    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv" event={"ID":"e0f4134f-f3be-492c-921f-a371c8d204a4","Type":"ContainerDied","Data":"e21dad2079bdc022ebad1238cd803d1156aa95936899e202ab4458b0974777fd"}
Sep 16 14:35:46.233227 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:46.233217    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv" event={"ID":"e0f4134f-f3be-492c-921f-a371c8d204a4","Type":"ContainerDied","Data":"96eb3bc906e1a28d2020a92cc885edf57e145922c6c08596f4c45612c67c3cce"}
Sep 16 14:35:46.233384 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:46.233244    2580 scope.go:117] "RemoveContainer" containerID="e21dad2079bdc022ebad1238cd803d1156aa95936899e202ab4458b0974777fd"
Sep 16 14:35:46.251050 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:46.251030    2580 scope.go:117] "RemoveContainer" containerID="866add6938b768b31ac90398c44051bacc61dee65be546d7094ccbc4dd5b0468"
Sep 16 14:35:46.258831 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:46.258815    2580 scope.go:117] "RemoveContainer" containerID="e21dad2079bdc022ebad1238cd803d1156aa95936899e202ab4458b0974777fd"
Sep 16 14:35:46.259096 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:35:46.259077    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e21dad2079bdc022ebad1238cd803d1156aa95936899e202ab4458b0974777fd\": container with ID starting with e21dad2079bdc022ebad1238cd803d1156aa95936899e202ab4458b0974777fd not found: ID does not exist" containerID="e21dad2079bdc022ebad1238cd803d1156aa95936899e202ab4458b0974777fd"
Sep 16 14:35:46.259148 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:46.259103    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e21dad2079bdc022ebad1238cd803d1156aa95936899e202ab4458b0974777fd"} err="failed to get container status \"e21dad2079bdc022ebad1238cd803d1156aa95936899e202ab4458b0974777fd\": rpc error: code = NotFound desc = could not find container \"e21dad2079bdc022ebad1238cd803d1156aa95936899e202ab4458b0974777fd\": container with ID starting with e21dad2079bdc022ebad1238cd803d1156aa95936899e202ab4458b0974777fd not found: ID does not exist"
Sep 16 14:35:46.259148 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:46.259119    2580 scope.go:117] "RemoveContainer" containerID="866add6938b768b31ac90398c44051bacc61dee65be546d7094ccbc4dd5b0468"
Sep 16 14:35:46.259349 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:35:46.259332    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"866add6938b768b31ac90398c44051bacc61dee65be546d7094ccbc4dd5b0468\": container with ID starting with 866add6938b768b31ac90398c44051bacc61dee65be546d7094ccbc4dd5b0468 not found: ID does not exist" containerID="866add6938b768b31ac90398c44051bacc61dee65be546d7094ccbc4dd5b0468"
Sep 16 14:35:46.259409 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:46.259358    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"866add6938b768b31ac90398c44051bacc61dee65be546d7094ccbc4dd5b0468"} err="failed to get container status \"866add6938b768b31ac90398c44051bacc61dee65be546d7094ccbc4dd5b0468\": rpc error: code = NotFound desc = could not find container \"866add6938b768b31ac90398c44051bacc61dee65be546d7094ccbc4dd5b0468\": container with ID starting with 866add6938b768b31ac90398c44051bacc61dee65be546d7094ccbc4dd5b0468 not found: ID does not exist"
Sep 16 14:35:46.265009 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:46.264985    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv"]
Sep 16 14:35:46.270263 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:46.270245    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-cbeb11-openshift-default-5597f6bb8b-sb5jv"]
Sep 16 14:35:46.327601 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:46.327582    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0f4134f-f3be-492c-921f-a371c8d204a4-istio-token\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:35:46.327601 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:46.327599    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0f4134f-f3be-492c-921f-a371c8d204a4-credential-socket\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:35:46.327716 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:46.327609    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0f4134f-f3be-492c-921f-a371c8d204a4-istio-data\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:35:46.327716 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:46.327618    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0f4134f-f3be-492c-921f-a371c8d204a4-istio-envoy\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:35:46.327716 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:46.327626    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0f4134f-f3be-492c-921f-a371c8d204a4-workload-socket\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:35:46.327716 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:46.327634    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0f4134f-f3be-492c-921f-a371c8d204a4-workload-certs\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:35:46.327716 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:46.327642    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0f4134f-f3be-492c-921f-a371c8d204a4-istio-podinfo\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:35:46.327716 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:46.327651    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wtv4b\" (UniqueName: \"kubernetes.io/projected/e0f4134f-f3be-492c-921f-a371c8d204a4-kube-api-access-wtv4b\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:35:47.431071 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:35:47.431040    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0f4134f-f3be-492c-921f-a371c8d204a4" path="/var/lib/kubelet/pods/e0f4134f-f3be-492c-921f-a371c8d204a4/volumes"
Sep 16 14:36:08.321795 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:36:08.321705    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-b77fde-68ddfb7b5d-4wjc8"]
Sep 16 14:36:08.322263 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:36:08.321968    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b77fde-68ddfb7b5d-4wjc8" podUID="ef0b01c7-42dc-466d-bb9a-0d6918e83d91" containerName="payload-pre-processing" containerID="cri-o://c4e15bed0ccb72903e831891c511f89188bfc89a8a2cd30eec4ca23f6d27637a" gracePeriod=30
Sep 16 14:36:08.580160 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:36:08.580096    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b77fde-68ddfb7b5d-4wjc8"
Sep 16 14:36:08.735246 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:36:08.735216    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef0b01c7-42dc-466d-bb9a-0d6918e83d91-plugins-config-volume\") pod \"ef0b01c7-42dc-466d-bb9a-0d6918e83d91\" (UID: \"ef0b01c7-42dc-466d-bb9a-0d6918e83d91\") "
Sep 16 14:36:08.735443 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:36:08.735348    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r2fsh\" (UniqueName: \"kubernetes.io/projected/ef0b01c7-42dc-466d-bb9a-0d6918e83d91-kube-api-access-r2fsh\") pod \"ef0b01c7-42dc-466d-bb9a-0d6918e83d91\" (UID: \"ef0b01c7-42dc-466d-bb9a-0d6918e83d91\") "
Sep 16 14:36:08.735697 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:36:08.735668    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef0b01c7-42dc-466d-bb9a-0d6918e83d91-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ef0b01c7-42dc-466d-bb9a-0d6918e83d91" (UID: "ef0b01c7-42dc-466d-bb9a-0d6918e83d91"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:36:08.737525 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:36:08.737472    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef0b01c7-42dc-466d-bb9a-0d6918e83d91-kube-api-access-r2fsh" (OuterVolumeSpecName: "kube-api-access-r2fsh") pod "ef0b01c7-42dc-466d-bb9a-0d6918e83d91" (UID: "ef0b01c7-42dc-466d-bb9a-0d6918e83d91"). InnerVolumeSpecName "kube-api-access-r2fsh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:36:08.836980 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:36:08.836898    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r2fsh\" (UniqueName: \"kubernetes.io/projected/ef0b01c7-42dc-466d-bb9a-0d6918e83d91-kube-api-access-r2fsh\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:36:08.836980 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:36:08.836929    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef0b01c7-42dc-466d-bb9a-0d6918e83d91-plugins-config-volume\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:36:09.364060 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:36:09.364023    2580 generic.go:356] "Generic (PLEG): container finished" podID="ef0b01c7-42dc-466d-bb9a-0d6918e83d91" containerID="c4e15bed0ccb72903e831891c511f89188bfc89a8a2cd30eec4ca23f6d27637a" exitCode=0
Sep 16 14:36:09.364449 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:36:09.364090    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b77fde-68ddfb7b5d-4wjc8"
Sep 16 14:36:09.364449 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:36:09.364109    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b77fde-68ddfb7b5d-4wjc8" event={"ID":"ef0b01c7-42dc-466d-bb9a-0d6918e83d91","Type":"ContainerDied","Data":"c4e15bed0ccb72903e831891c511f89188bfc89a8a2cd30eec4ca23f6d27637a"}
Sep 16 14:36:09.364449 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:36:09.364181    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b77fde-68ddfb7b5d-4wjc8" event={"ID":"ef0b01c7-42dc-466d-bb9a-0d6918e83d91","Type":"ContainerDied","Data":"ad404017651ac51b0acdf6b64426b8fe4c3186044e78fd2b104927384db12cfe"}
Sep 16 14:36:09.364449 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:36:09.364204    2580 scope.go:117] "RemoveContainer" containerID="c4e15bed0ccb72903e831891c511f89188bfc89a8a2cd30eec4ca23f6d27637a"
Sep 16 14:36:09.372983 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:36:09.372968    2580 scope.go:117] "RemoveContainer" containerID="c4e15bed0ccb72903e831891c511f89188bfc89a8a2cd30eec4ca23f6d27637a"
Sep 16 14:36:09.373248 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:36:09.373231    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4e15bed0ccb72903e831891c511f89188bfc89a8a2cd30eec4ca23f6d27637a\": container with ID starting with c4e15bed0ccb72903e831891c511f89188bfc89a8a2cd30eec4ca23f6d27637a not found: ID does not exist" containerID="c4e15bed0ccb72903e831891c511f89188bfc89a8a2cd30eec4ca23f6d27637a"
Sep 16 14:36:09.373291 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:36:09.373257    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4e15bed0ccb72903e831891c511f89188bfc89a8a2cd30eec4ca23f6d27637a"} err="failed to get container status \"c4e15bed0ccb72903e831891c511f89188bfc89a8a2cd30eec4ca23f6d27637a\": rpc error: code = NotFound desc = could not find container \"c4e15bed0ccb72903e831891c511f89188bfc89a8a2cd30eec4ca23f6d27637a\": container with ID starting with c4e15bed0ccb72903e831891c511f89188bfc89a8a2cd30eec4ca23f6d27637a not found: ID does not exist"
Sep 16 14:36:09.386926 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:36:09.386902    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-b77fde-68ddfb7b5d-4wjc8"]
Sep 16 14:36:09.391104 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:36:09.391082    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-b77fde-68ddfb7b5d-4wjc8"]
Sep 16 14:36:09.430716 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:36:09.430686    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef0b01c7-42dc-466d-bb9a-0d6918e83d91" path="/var/lib/kubelet/pods/ef0b01c7-42dc-466d-bb9a-0d6918e83d91/volumes"
Sep 16 14:36:27.665718 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:36:27.665692    2580 scope.go:117] "RemoveContainer" containerID="41536a673d68eee8f227933d9c08a115e15ec1401c1500d368ca63647b499164"
Sep 16 14:37:10.227586 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:10.227487    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-7eeba8-656566c466-gw78b"]
Sep 16 14:37:10.228050 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:10.227766    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7eeba8-656566c466-gw78b" podUID="71ad7864-113f-434c-aaa7-998dd7774203" containerName="maas-api" containerID="cri-o://a9fdc7c99f4ae4f1a9cb0d669ce28011e11fbd6f8ea607ff0db9bb9c2241a299" gracePeriod=30
Sep 16 14:37:10.471404 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:10.471376    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7eeba8-656566c466-gw78b"
Sep 16 14:37:10.574174 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:10.574099    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71ad7864-113f-434c-aaa7-998dd7774203-maas-api-tls\") pod \"71ad7864-113f-434c-aaa7-998dd7774203\" (UID: \"71ad7864-113f-434c-aaa7-998dd7774203\") "
Sep 16 14:37:10.574174 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:10.574167    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71ad7864-113f-434c-aaa7-998dd7774203-openshift-service-ca\") pod \"71ad7864-113f-434c-aaa7-998dd7774203\" (UID: \"71ad7864-113f-434c-aaa7-998dd7774203\") "
Sep 16 14:37:10.574398 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:10.574190    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g4mb8\" (UniqueName: \"kubernetes.io/projected/71ad7864-113f-434c-aaa7-998dd7774203-kube-api-access-g4mb8\") pod \"71ad7864-113f-434c-aaa7-998dd7774203\" (UID: \"71ad7864-113f-434c-aaa7-998dd7774203\") "
Sep 16 14:37:10.574398 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:10.574205    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/71ad7864-113f-434c-aaa7-998dd7774203-metrics-certs\") pod \"71ad7864-113f-434c-aaa7-998dd7774203\" (UID: \"71ad7864-113f-434c-aaa7-998dd7774203\") "
Sep 16 14:37:10.574587 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:10.574555    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71ad7864-113f-434c-aaa7-998dd7774203-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "71ad7864-113f-434c-aaa7-998dd7774203" (UID: "71ad7864-113f-434c-aaa7-998dd7774203"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:37:10.576539 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:10.576489    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/71ad7864-113f-434c-aaa7-998dd7774203-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "71ad7864-113f-434c-aaa7-998dd7774203" (UID: "71ad7864-113f-434c-aaa7-998dd7774203"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:37:10.576646 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:10.576588    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71ad7864-113f-434c-aaa7-998dd7774203-kube-api-access-g4mb8" (OuterVolumeSpecName: "kube-api-access-g4mb8") pod "71ad7864-113f-434c-aaa7-998dd7774203" (UID: "71ad7864-113f-434c-aaa7-998dd7774203"). InnerVolumeSpecName "kube-api-access-g4mb8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:37:10.576692 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:10.576642    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/71ad7864-113f-434c-aaa7-998dd7774203-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "71ad7864-113f-434c-aaa7-998dd7774203" (UID: "71ad7864-113f-434c-aaa7-998dd7774203"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:37:10.675558 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:10.675522    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71ad7864-113f-434c-aaa7-998dd7774203-maas-api-tls\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:37:10.675558 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:10.675551    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71ad7864-113f-434c-aaa7-998dd7774203-openshift-service-ca\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:37:10.675558 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:10.675562    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g4mb8\" (UniqueName: \"kubernetes.io/projected/71ad7864-113f-434c-aaa7-998dd7774203-kube-api-access-g4mb8\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:37:10.675789 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:10.675572    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/71ad7864-113f-434c-aaa7-998dd7774203-metrics-certs\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:37:10.684786 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:10.684747    2580 generic.go:356] "Generic (PLEG): container finished" podID="71ad7864-113f-434c-aaa7-998dd7774203" containerID="a9fdc7c99f4ae4f1a9cb0d669ce28011e11fbd6f8ea607ff0db9bb9c2241a299" exitCode=0
Sep 16 14:37:10.684997 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:10.684814    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7eeba8-656566c466-gw78b"
Sep 16 14:37:10.684997 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:10.684851    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7eeba8-656566c466-gw78b" event={"ID":"71ad7864-113f-434c-aaa7-998dd7774203","Type":"ContainerDied","Data":"a9fdc7c99f4ae4f1a9cb0d669ce28011e11fbd6f8ea607ff0db9bb9c2241a299"}
Sep 16 14:37:10.684997 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:10.684914    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7eeba8-656566c466-gw78b" event={"ID":"71ad7864-113f-434c-aaa7-998dd7774203","Type":"ContainerDied","Data":"88c1964c2fe1a91a0a17c73b7923163942117a206613fd14a5cde505c0bd166f"}
Sep 16 14:37:10.684997 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:10.684946    2580 scope.go:117] "RemoveContainer" containerID="a9fdc7c99f4ae4f1a9cb0d669ce28011e11fbd6f8ea607ff0db9bb9c2241a299"
Sep 16 14:37:10.696865 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:10.696835    2580 scope.go:117] "RemoveContainer" containerID="a9fdc7c99f4ae4f1a9cb0d669ce28011e11fbd6f8ea607ff0db9bb9c2241a299"
Sep 16 14:37:10.697178 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:37:10.697155    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9fdc7c99f4ae4f1a9cb0d669ce28011e11fbd6f8ea607ff0db9bb9c2241a299\": container with ID starting with a9fdc7c99f4ae4f1a9cb0d669ce28011e11fbd6f8ea607ff0db9bb9c2241a299 not found: ID does not exist" containerID="a9fdc7c99f4ae4f1a9cb0d669ce28011e11fbd6f8ea607ff0db9bb9c2241a299"
Sep 16 14:37:10.697298 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:10.697193    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9fdc7c99f4ae4f1a9cb0d669ce28011e11fbd6f8ea607ff0db9bb9c2241a299"} err="failed to get container status \"a9fdc7c99f4ae4f1a9cb0d669ce28011e11fbd6f8ea607ff0db9bb9c2241a299\": rpc error: code = NotFound desc = could not find container \"a9fdc7c99f4ae4f1a9cb0d669ce28011e11fbd6f8ea607ff0db9bb9c2241a299\": container with ID starting with a9fdc7c99f4ae4f1a9cb0d669ce28011e11fbd6f8ea607ff0db9bb9c2241a299 not found: ID does not exist"
Sep 16 14:37:10.712113 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:10.712085    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-7eeba8-656566c466-gw78b"]
Sep 16 14:37:10.719215 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:10.719193    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-7eeba8-656566c466-gw78b"]
Sep 16 14:37:11.430428 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:11.430396    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71ad7864-113f-434c-aaa7-998dd7774203" path="/var/lib/kubelet/pods/71ad7864-113f-434c-aaa7-998dd7774203/volumes"
Sep 16 14:37:29.385676 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:29.385639    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-50b57c-57b44f6f78-5d29x"]
Sep 16 14:37:29.386085 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:29.385915    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-50b57c-57b44f6f78-5d29x" podUID="7ab77b2d-c006-43b0-b94f-ec5fbfda3f62" containerName="payload-pre-processing" containerID="cri-o://4f3699b916166cb2ab2dbb579183c3542a2c8e788049de1ed48c1cf21177b244" gracePeriod=30
Sep 16 14:37:29.635189 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:29.635165    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-50b57c-57b44f6f78-5d29x"
Sep 16 14:37:29.746284 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:29.746257    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fr775\" (UniqueName: \"kubernetes.io/projected/7ab77b2d-c006-43b0-b94f-ec5fbfda3f62-kube-api-access-fr775\") pod \"7ab77b2d-c006-43b0-b94f-ec5fbfda3f62\" (UID: \"7ab77b2d-c006-43b0-b94f-ec5fbfda3f62\") "
Sep 16 14:37:29.746457 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:29.746309    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ab77b2d-c006-43b0-b94f-ec5fbfda3f62-plugins-config-volume\") pod \"7ab77b2d-c006-43b0-b94f-ec5fbfda3f62\" (UID: \"7ab77b2d-c006-43b0-b94f-ec5fbfda3f62\") "
Sep 16 14:37:29.746713 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:29.746684    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ab77b2d-c006-43b0-b94f-ec5fbfda3f62-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7ab77b2d-c006-43b0-b94f-ec5fbfda3f62" (UID: "7ab77b2d-c006-43b0-b94f-ec5fbfda3f62"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:37:29.748540 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:29.748492    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ab77b2d-c006-43b0-b94f-ec5fbfda3f62-kube-api-access-fr775" (OuterVolumeSpecName: "kube-api-access-fr775") pod "7ab77b2d-c006-43b0-b94f-ec5fbfda3f62" (UID: "7ab77b2d-c006-43b0-b94f-ec5fbfda3f62"). InnerVolumeSpecName "kube-api-access-fr775". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:37:29.787543 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:29.787516    2580 generic.go:356] "Generic (PLEG): container finished" podID="7ab77b2d-c006-43b0-b94f-ec5fbfda3f62" containerID="4f3699b916166cb2ab2dbb579183c3542a2c8e788049de1ed48c1cf21177b244" exitCode=0
Sep 16 14:37:29.787667 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:29.787558    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-50b57c-57b44f6f78-5d29x"
Sep 16 14:37:29.787667 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:29.787605    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-50b57c-57b44f6f78-5d29x" event={"ID":"7ab77b2d-c006-43b0-b94f-ec5fbfda3f62","Type":"ContainerDied","Data":"4f3699b916166cb2ab2dbb579183c3542a2c8e788049de1ed48c1cf21177b244"}
Sep 16 14:37:29.787667 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:29.787651    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-50b57c-57b44f6f78-5d29x" event={"ID":"7ab77b2d-c006-43b0-b94f-ec5fbfda3f62","Type":"ContainerDied","Data":"be404a21e1384be38084d1816e4618706b5b65b60bf52b3186d09f18a66f0a76"}
Sep 16 14:37:29.787798 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:29.787672    2580 scope.go:117] "RemoveContainer" containerID="4f3699b916166cb2ab2dbb579183c3542a2c8e788049de1ed48c1cf21177b244"
Sep 16 14:37:29.799410 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:29.799392    2580 scope.go:117] "RemoveContainer" containerID="4f3699b916166cb2ab2dbb579183c3542a2c8e788049de1ed48c1cf21177b244"
Sep 16 14:37:29.799753 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:37:29.799734    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f3699b916166cb2ab2dbb579183c3542a2c8e788049de1ed48c1cf21177b244\": container with ID starting with 4f3699b916166cb2ab2dbb579183c3542a2c8e788049de1ed48c1cf21177b244 not found: ID does not exist" containerID="4f3699b916166cb2ab2dbb579183c3542a2c8e788049de1ed48c1cf21177b244"
Sep 16 14:37:29.799846 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:29.799758    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f3699b916166cb2ab2dbb579183c3542a2c8e788049de1ed48c1cf21177b244"} err="failed to get container status \"4f3699b916166cb2ab2dbb579183c3542a2c8e788049de1ed48c1cf21177b244\": rpc error: code = NotFound desc = could not find container \"4f3699b916166cb2ab2dbb579183c3542a2c8e788049de1ed48c1cf21177b244\": container with ID starting with 4f3699b916166cb2ab2dbb579183c3542a2c8e788049de1ed48c1cf21177b244 not found: ID does not exist"
Sep 16 14:37:29.822593 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:29.822562    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-50b57c-57b44f6f78-5d29x"]
Sep 16 14:37:29.846683 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:29.846654    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-50b57c-57b44f6f78-5d29x"]
Sep 16 14:37:29.847053 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:29.847029    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fr775\" (UniqueName: \"kubernetes.io/projected/7ab77b2d-c006-43b0-b94f-ec5fbfda3f62-kube-api-access-fr775\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:37:29.847100 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:29.847057    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ab77b2d-c006-43b0-b94f-ec5fbfda3f62-plugins-config-volume\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:37:31.430334 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:31.430301    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ab77b2d-c006-43b0-b94f-ec5fbfda3f62" path="/var/lib/kubelet/pods/7ab77b2d-c006-43b0-b94f-ec5fbfda3f62/volumes"
Sep 16 14:37:48.214373 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:48.214332    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-f72402-5f595b4b77-j97vl"]
Sep 16 14:37:48.214978 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:48.214880    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f72402-5f595b4b77-j97vl" podUID="118c3a58-d101-4668-9da2-e9a824500df4" containerName="payload-processing" containerID="cri-o://29ddb00408e01bb5b82992408c531b3ece71abd93e693f5575c641cef28dff3b" gracePeriod=30
Sep 16 14:37:48.471795 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:48.471739    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f72402-5f595b4b77-j97vl"
Sep 16 14:37:48.618760 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:48.618718    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6fxcf\" (UniqueName: \"kubernetes.io/projected/118c3a58-d101-4668-9da2-e9a824500df4-kube-api-access-6fxcf\") pod \"118c3a58-d101-4668-9da2-e9a824500df4\" (UID: \"118c3a58-d101-4668-9da2-e9a824500df4\") "
Sep 16 14:37:48.618938 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:48.618881    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/118c3a58-d101-4668-9da2-e9a824500df4-plugins-config-volume\") pod \"118c3a58-d101-4668-9da2-e9a824500df4\" (UID: \"118c3a58-d101-4668-9da2-e9a824500df4\") "
Sep 16 14:37:48.619400 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:48.619268    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/118c3a58-d101-4668-9da2-e9a824500df4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "118c3a58-d101-4668-9da2-e9a824500df4" (UID: "118c3a58-d101-4668-9da2-e9a824500df4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:37:48.621395 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:48.621369    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/118c3a58-d101-4668-9da2-e9a824500df4-kube-api-access-6fxcf" (OuterVolumeSpecName: "kube-api-access-6fxcf") pod "118c3a58-d101-4668-9da2-e9a824500df4" (UID: "118c3a58-d101-4668-9da2-e9a824500df4"). InnerVolumeSpecName "kube-api-access-6fxcf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:37:48.719902 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:48.719867    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/118c3a58-d101-4668-9da2-e9a824500df4-plugins-config-volume\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:37:48.719902 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:48.719899    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6fxcf\" (UniqueName: \"kubernetes.io/projected/118c3a58-d101-4668-9da2-e9a824500df4-kube-api-access-6fxcf\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:37:48.893199 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:48.893118    2580 generic.go:356] "Generic (PLEG): container finished" podID="118c3a58-d101-4668-9da2-e9a824500df4" containerID="29ddb00408e01bb5b82992408c531b3ece71abd93e693f5575c641cef28dff3b" exitCode=0
Sep 16 14:37:48.893199 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:48.893175    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f72402-5f595b4b77-j97vl"
Sep 16 14:37:48.893402 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:48.893206    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f72402-5f595b4b77-j97vl" event={"ID":"118c3a58-d101-4668-9da2-e9a824500df4","Type":"ContainerDied","Data":"29ddb00408e01bb5b82992408c531b3ece71abd93e693f5575c641cef28dff3b"}
Sep 16 14:37:48.893402 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:48.893257    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f72402-5f595b4b77-j97vl" event={"ID":"118c3a58-d101-4668-9da2-e9a824500df4","Type":"ContainerDied","Data":"0b0d8f69f691fd283906962059e3847e61407e49b39325682a235bb2dd7d407b"}
Sep 16 14:37:48.893402 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:48.893281    2580 scope.go:117] "RemoveContainer" containerID="29ddb00408e01bb5b82992408c531b3ece71abd93e693f5575c641cef28dff3b"
Sep 16 14:37:48.902264 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:48.902247    2580 scope.go:117] "RemoveContainer" containerID="29ddb00408e01bb5b82992408c531b3ece71abd93e693f5575c641cef28dff3b"
Sep 16 14:37:48.902526 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:37:48.902487    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29ddb00408e01bb5b82992408c531b3ece71abd93e693f5575c641cef28dff3b\": container with ID starting with 29ddb00408e01bb5b82992408c531b3ece71abd93e693f5575c641cef28dff3b not found: ID does not exist" containerID="29ddb00408e01bb5b82992408c531b3ece71abd93e693f5575c641cef28dff3b"
Sep 16 14:37:48.902576 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:48.902536    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29ddb00408e01bb5b82992408c531b3ece71abd93e693f5575c641cef28dff3b"} err="failed to get container status \"29ddb00408e01bb5b82992408c531b3ece71abd93e693f5575c641cef28dff3b\": rpc error: code = NotFound desc = could not find container \"29ddb00408e01bb5b82992408c531b3ece71abd93e693f5575c641cef28dff3b\": container with ID starting with 29ddb00408e01bb5b82992408c531b3ece71abd93e693f5575c641cef28dff3b not found: ID does not exist"
Sep 16 14:37:48.915969 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:48.915948    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-f72402-5f595b4b77-j97vl"]
Sep 16 14:37:48.921141 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:48.921117    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-f72402-5f595b4b77-j97vl"]
Sep 16 14:37:49.432248 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:49.432211    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="118c3a58-d101-4668-9da2-e9a824500df4" path="/var/lib/kubelet/pods/118c3a58-d101-4668-9da2-e9a824500df4/volumes"
Sep 16 14:37:57.879549 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:57.879515    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-658475d98d-6stb8"]
Sep 16 14:37:57.879990 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:57.879756    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-658475d98d-6stb8" podUID="1f34b361-5fad-40f6-8f95-14b71f83a416" containerName="manager" containerID="cri-o://1b870f4f8496e50685648b98d709898e0694635a67a69bd4f551dbbf683b5603" gracePeriod=30
Sep 16 14:37:58.123022 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:58.122993    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-658475d98d-6stb8"
Sep 16 14:37:58.197577 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:58.197546    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1f34b361-5fad-40f6-8f95-14b71f83a416-metrics-certs\") pod \"1f34b361-5fad-40f6-8f95-14b71f83a416\" (UID: \"1f34b361-5fad-40f6-8f95-14b71f83a416\") "
Sep 16 14:37:58.197778 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:58.197603    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1f34b361-5fad-40f6-8f95-14b71f83a416-webhook-cert\") pod \"1f34b361-5fad-40f6-8f95-14b71f83a416\" (UID: \"1f34b361-5fad-40f6-8f95-14b71f83a416\") "
Sep 16 14:37:58.197778 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:58.197659    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wp2kw\" (UniqueName: \"kubernetes.io/projected/1f34b361-5fad-40f6-8f95-14b71f83a416-kube-api-access-wp2kw\") pod \"1f34b361-5fad-40f6-8f95-14b71f83a416\" (UID: \"1f34b361-5fad-40f6-8f95-14b71f83a416\") "
Sep 16 14:37:58.199765 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:58.199732    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f34b361-5fad-40f6-8f95-14b71f83a416-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "1f34b361-5fad-40f6-8f95-14b71f83a416" (UID: "1f34b361-5fad-40f6-8f95-14b71f83a416"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:37:58.199898 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:58.199792    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f34b361-5fad-40f6-8f95-14b71f83a416-kube-api-access-wp2kw" (OuterVolumeSpecName: "kube-api-access-wp2kw") pod "1f34b361-5fad-40f6-8f95-14b71f83a416" (UID: "1f34b361-5fad-40f6-8f95-14b71f83a416"). InnerVolumeSpecName "kube-api-access-wp2kw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:37:58.199898 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:58.199882    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f34b361-5fad-40f6-8f95-14b71f83a416-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "1f34b361-5fad-40f6-8f95-14b71f83a416" (UID: "1f34b361-5fad-40f6-8f95-14b71f83a416"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:37:58.299148 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:58.299113    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1f34b361-5fad-40f6-8f95-14b71f83a416-metrics-certs\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:37:58.299148 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:58.299142    2580 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1f34b361-5fad-40f6-8f95-14b71f83a416-webhook-cert\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:37:58.299148 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:58.299151    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wp2kw\" (UniqueName: \"kubernetes.io/projected/1f34b361-5fad-40f6-8f95-14b71f83a416-kube-api-access-wp2kw\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:37:58.947153 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:58.947118    2580 generic.go:356] "Generic (PLEG): container finished" podID="1f34b361-5fad-40f6-8f95-14b71f83a416" containerID="1b870f4f8496e50685648b98d709898e0694635a67a69bd4f551dbbf683b5603" exitCode=0
Sep 16 14:37:58.947565 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:58.947166    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-658475d98d-6stb8"
Sep 16 14:37:58.947565 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:58.947203    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-658475d98d-6stb8" event={"ID":"1f34b361-5fad-40f6-8f95-14b71f83a416","Type":"ContainerDied","Data":"1b870f4f8496e50685648b98d709898e0694635a67a69bd4f551dbbf683b5603"}
Sep 16 14:37:58.947565 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:58.947260    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-658475d98d-6stb8" event={"ID":"1f34b361-5fad-40f6-8f95-14b71f83a416","Type":"ContainerDied","Data":"9d1f2a6a2a729580354b9a8989fdbcc36916edb93b45c6c88c6689cdb9b68d41"}
Sep 16 14:37:58.947565 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:58.947287    2580 scope.go:117] "RemoveContainer" containerID="1b870f4f8496e50685648b98d709898e0694635a67a69bd4f551dbbf683b5603"
Sep 16 14:37:58.957276 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:58.957258    2580 scope.go:117] "RemoveContainer" containerID="1b870f4f8496e50685648b98d709898e0694635a67a69bd4f551dbbf683b5603"
Sep 16 14:37:58.957562 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:37:58.957534    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b870f4f8496e50685648b98d709898e0694635a67a69bd4f551dbbf683b5603\": container with ID starting with 1b870f4f8496e50685648b98d709898e0694635a67a69bd4f551dbbf683b5603 not found: ID does not exist" containerID="1b870f4f8496e50685648b98d709898e0694635a67a69bd4f551dbbf683b5603"
Sep 16 14:37:58.957643 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:58.957570    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b870f4f8496e50685648b98d709898e0694635a67a69bd4f551dbbf683b5603"} err="failed to get container status \"1b870f4f8496e50685648b98d709898e0694635a67a69bd4f551dbbf683b5603\": rpc error: code = NotFound desc = could not find container \"1b870f4f8496e50685648b98d709898e0694635a67a69bd4f551dbbf683b5603\": container with ID starting with 1b870f4f8496e50685648b98d709898e0694635a67a69bd4f551dbbf683b5603 not found: ID does not exist"
Sep 16 14:37:58.968533 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:58.968493    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-658475d98d-6stb8"]
Sep 16 14:37:58.974609 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:58.974592    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-658475d98d-6stb8"]
Sep 16 14:37:59.433578 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:37:59.433548    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f34b361-5fad-40f6-8f95-14b71f83a416" path="/var/lib/kubelet/pods/1f34b361-5fad-40f6-8f95-14b71f83a416/volumes"
Sep 16 14:38:59.893282 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:38:59.893249    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-658475d98d-pf9g7"]
Sep 16 14:38:59.893770 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:38:59.893598    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa6c5500-4785-4955-bb2b-ceec12a2ea84" containerName="payload-pre-processing"
Sep 16 14:38:59.893770 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:38:59.893609    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa6c5500-4785-4955-bb2b-ceec12a2ea84" containerName="payload-pre-processing"
Sep 16 14:38:59.893770 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:38:59.893621    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0f4134f-f3be-492c-921f-a371c8d204a4" containerName="istio-proxy"
Sep 16 14:38:59.893770 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:38:59.893626    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0f4134f-f3be-492c-921f-a371c8d204a4" containerName="istio-proxy"
Sep 16 14:38:59.893770 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:38:59.893634    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26764d22-8352-4c06-98c1-1097e10c7e7d" containerName="maas-api"
Sep 16 14:38:59.893770 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:38:59.893639    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="26764d22-8352-4c06-98c1-1097e10c7e7d" containerName="maas-api"
Sep 16 14:38:59.893770 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:38:59.893647    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac9b3cd6-0c4e-4aa5-96d8-4d92646548ca" containerName="maas-api"
Sep 16 14:38:59.893770 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:38:59.893652    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac9b3cd6-0c4e-4aa5-96d8-4d92646548ca" containerName="maas-api"
Sep 16 14:38:59.893770 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:38:59.893661    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d25388e0-508a-4169-b297-82e555257170" containerName="payload-processing"
Sep 16 14:38:59.893770 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:38:59.893666    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d25388e0-508a-4169-b297-82e555257170" containerName="payload-processing"
Sep 16 14:38:59.893770 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:38:59.893675    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ab77b2d-c006-43b0-b94f-ec5fbfda3f62" containerName="payload-pre-processing"
Sep 16 14:38:59.893770 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:38:59.893681    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ab77b2d-c006-43b0-b94f-ec5fbfda3f62" containerName="payload-pre-processing"
Sep 16 14:38:59.893770 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:38:59.893688    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71ad7864-113f-434c-aaa7-998dd7774203" containerName="maas-api"
Sep 16 14:38:59.893770 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:38:59.893693    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="71ad7864-113f-434c-aaa7-998dd7774203" containerName="maas-api"
Sep 16 14:38:59.893770 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:38:59.893700    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="118c3a58-d101-4668-9da2-e9a824500df4" containerName="payload-processing"
Sep 16 14:38:59.893770 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:38:59.893705    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="118c3a58-d101-4668-9da2-e9a824500df4" containerName="payload-processing"
Sep 16 14:38:59.893770 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:38:59.893715    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f34b361-5fad-40f6-8f95-14b71f83a416" containerName="manager"
Sep 16 14:38:59.893770 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:38:59.893720    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f34b361-5fad-40f6-8f95-14b71f83a416" containerName="manager"
Sep 16 14:38:59.893770 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:38:59.893727    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef0b01c7-42dc-466d-bb9a-0d6918e83d91" containerName="payload-pre-processing"
Sep 16 14:38:59.893770 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:38:59.893731    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef0b01c7-42dc-466d-bb9a-0d6918e83d91" containerName="payload-pre-processing"
Sep 16 14:38:59.893770 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:38:59.893737    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2c2ca51-46ee-4a99-b200-1b101c636511" containerName="maas-api"
Sep 16 14:38:59.893770 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:38:59.893741    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2c2ca51-46ee-4a99-b200-1b101c636511" containerName="maas-api"
Sep 16 14:38:59.893770 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:38:59.893749    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0f4134f-f3be-492c-921f-a371c8d204a4" containerName="istio-proxy"
Sep 16 14:38:59.893770 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:38:59.893754    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0f4134f-f3be-492c-921f-a371c8d204a4" containerName="istio-proxy"
Sep 16 14:38:59.894547 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:38:59.893805    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d25388e0-508a-4169-b297-82e555257170" containerName="payload-processing"
Sep 16 14:38:59.894547 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:38:59.893813    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2c2ca51-46ee-4a99-b200-1b101c636511" containerName="maas-api"
Sep 16 14:38:59.894547 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:38:59.893819    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac9b3cd6-0c4e-4aa5-96d8-4d92646548ca" containerName="maas-api"
Sep 16 14:38:59.894547 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:38:59.893826    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa6c5500-4785-4955-bb2b-ceec12a2ea84" containerName="payload-pre-processing"
Sep 16 14:38:59.894547 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:38:59.893830    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="26764d22-8352-4c06-98c1-1097e10c7e7d" containerName="maas-api"
Sep 16 14:38:59.894547 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:38:59.893835    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="71ad7864-113f-434c-aaa7-998dd7774203" containerName="maas-api"
Sep 16 14:38:59.894547 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:38:59.893841    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0f4134f-f3be-492c-921f-a371c8d204a4" containerName="istio-proxy"
Sep 16 14:38:59.894547 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:38:59.893845    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="118c3a58-d101-4668-9da2-e9a824500df4" containerName="payload-processing"
Sep 16 14:38:59.894547 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:38:59.893850    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0f4134f-f3be-492c-921f-a371c8d204a4" containerName="istio-proxy"
Sep 16 14:38:59.894547 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:38:59.893854    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef0b01c7-42dc-466d-bb9a-0d6918e83d91" containerName="payload-pre-processing"
Sep 16 14:38:59.894547 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:38:59.893859    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f34b361-5fad-40f6-8f95-14b71f83a416" containerName="manager"
Sep 16 14:38:59.894547 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:38:59.893864    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ab77b2d-c006-43b0-b94f-ec5fbfda3f62" containerName="payload-pre-processing"
Sep 16 14:38:59.896853 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:38:59.896836    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-658475d98d-pf9g7"
Sep 16 14:38:59.900344 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:38:59.900318    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 16 14:38:59.900344 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:38:59.900328    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 16 14:38:59.900548 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:38:59.900332    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 16 14:38:59.900662 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:38:59.900649    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-25lqs\""
Sep 16 14:38:59.908378 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:38:59.908356    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-658475d98d-pf9g7"]
Sep 16 14:39:00.051730 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:39:00.051698    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f711841-a789-4f57-8383-9892b3c6f75f-metrics-certs\") pod \"maas-controller-658475d98d-pf9g7\" (UID: \"6f711841-a789-4f57-8383-9892b3c6f75f\") " pod="opendatahub/maas-controller-658475d98d-pf9g7"
Sep 16 14:39:00.051730 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:39:00.051740    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfszd\" (UniqueName: \"kubernetes.io/projected/6f711841-a789-4f57-8383-9892b3c6f75f-kube-api-access-bfszd\") pod \"maas-controller-658475d98d-pf9g7\" (UID: \"6f711841-a789-4f57-8383-9892b3c6f75f\") " pod="opendatahub/maas-controller-658475d98d-pf9g7"
Sep 16 14:39:00.051974 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:39:00.051811    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6f711841-a789-4f57-8383-9892b3c6f75f-webhook-cert\") pod \"maas-controller-658475d98d-pf9g7\" (UID: \"6f711841-a789-4f57-8383-9892b3c6f75f\") " pod="opendatahub/maas-controller-658475d98d-pf9g7"
Sep 16 14:39:00.153076 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:39:00.152990    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6f711841-a789-4f57-8383-9892b3c6f75f-webhook-cert\") pod \"maas-controller-658475d98d-pf9g7\" (UID: \"6f711841-a789-4f57-8383-9892b3c6f75f\") " pod="opendatahub/maas-controller-658475d98d-pf9g7"
Sep 16 14:39:00.153233 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:39:00.153091    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f711841-a789-4f57-8383-9892b3c6f75f-metrics-certs\") pod \"maas-controller-658475d98d-pf9g7\" (UID: \"6f711841-a789-4f57-8383-9892b3c6f75f\") " pod="opendatahub/maas-controller-658475d98d-pf9g7"
Sep 16 14:39:00.153233 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:39:00.153124    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfszd\" (UniqueName: \"kubernetes.io/projected/6f711841-a789-4f57-8383-9892b3c6f75f-kube-api-access-bfszd\") pod \"maas-controller-658475d98d-pf9g7\" (UID: \"6f711841-a789-4f57-8383-9892b3c6f75f\") " pod="opendatahub/maas-controller-658475d98d-pf9g7"
Sep 16 14:39:00.155491 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:39:00.155465    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f711841-a789-4f57-8383-9892b3c6f75f-metrics-certs\") pod \"maas-controller-658475d98d-pf9g7\" (UID: \"6f711841-a789-4f57-8383-9892b3c6f75f\") " pod="opendatahub/maas-controller-658475d98d-pf9g7"
Sep 16 14:39:00.155491 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:39:00.155480    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6f711841-a789-4f57-8383-9892b3c6f75f-webhook-cert\") pod \"maas-controller-658475d98d-pf9g7\" (UID: \"6f711841-a789-4f57-8383-9892b3c6f75f\") " pod="opendatahub/maas-controller-658475d98d-pf9g7"
Sep 16 14:39:00.161962 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:39:00.161941    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfszd\" (UniqueName: \"kubernetes.io/projected/6f711841-a789-4f57-8383-9892b3c6f75f-kube-api-access-bfszd\") pod \"maas-controller-658475d98d-pf9g7\" (UID: \"6f711841-a789-4f57-8383-9892b3c6f75f\") " pod="opendatahub/maas-controller-658475d98d-pf9g7"
Sep 16 14:39:00.207230 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:39:00.207201    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-658475d98d-pf9g7"
Sep 16 14:39:00.481277 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:39:00.481248    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-658475d98d-pf9g7"]
Sep 16 14:39:00.482571 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:39:00.482546    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f711841_a789_4f57_8383_9892b3c6f75f.slice/crio-5e53f7b67c1724a03d393bfe436cf0455cf0d69d108c185176b9d5dc91ec3959 WatchSource:0}: Error finding container 5e53f7b67c1724a03d393bfe436cf0455cf0d69d108c185176b9d5dc91ec3959: Status 404 returned error can't find the container with id 5e53f7b67c1724a03d393bfe436cf0455cf0d69d108c185176b9d5dc91ec3959
Sep 16 14:39:00.483926 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:39:00.483907    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 14:39:01.258227 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:39:01.258193    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-658475d98d-pf9g7" event={"ID":"6f711841-a789-4f57-8383-9892b3c6f75f","Type":"ContainerStarted","Data":"5e53f7b67c1724a03d393bfe436cf0455cf0d69d108c185176b9d5dc91ec3959"}
Sep 16 14:39:02.265256 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:39:02.265222    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-658475d98d-pf9g7" event={"ID":"6f711841-a789-4f57-8383-9892b3c6f75f","Type":"ContainerStarted","Data":"62ffd99e968a71ee934d25588aa3b91cc9c86ac7d32eb703cc985f99bda5d346"}
Sep 16 14:39:02.265661 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:39:02.265309    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-658475d98d-pf9g7"
Sep 16 14:39:02.285401 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:39:02.285360    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-658475d98d-pf9g7" podStartSLOduration=2.545042471 podStartE2EDuration="3.285345927s" totalImagesPullingTime="740.303456ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:38:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:39:02.283727289 +0000 UTC m=+2075.298837150" watchObservedRunningTime="2026-09-16 14:39:02.285345927 +0000 UTC m=+2075.300455812"
Sep 16 14:39:13.272991 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:39:13.272962    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-658475d98d-pf9g7"
Sep 16 14:39:26.340645 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:39:26.340611    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-658475d98d-pf9g7"]
Sep 16 14:39:26.341043 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:39:26.340834    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-658475d98d-pf9g7" podUID="6f711841-a789-4f57-8383-9892b3c6f75f" containerName="manager" containerID="cri-o://62ffd99e968a71ee934d25588aa3b91cc9c86ac7d32eb703cc985f99bda5d346" gracePeriod=30
Sep 16 14:39:26.583027 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:39:26.583002    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-658475d98d-pf9g7"
Sep 16 14:39:26.670405 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:39:26.670323    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f711841-a789-4f57-8383-9892b3c6f75f-metrics-certs\") pod \"6f711841-a789-4f57-8383-9892b3c6f75f\" (UID: \"6f711841-a789-4f57-8383-9892b3c6f75f\") "
Sep 16 14:39:26.670405 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:39:26.670358    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6f711841-a789-4f57-8383-9892b3c6f75f-webhook-cert\") pod \"6f711841-a789-4f57-8383-9892b3c6f75f\" (UID: \"6f711841-a789-4f57-8383-9892b3c6f75f\") "
Sep 16 14:39:26.670405 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:39:26.670391    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bfszd\" (UniqueName: \"kubernetes.io/projected/6f711841-a789-4f57-8383-9892b3c6f75f-kube-api-access-bfszd\") pod \"6f711841-a789-4f57-8383-9892b3c6f75f\" (UID: \"6f711841-a789-4f57-8383-9892b3c6f75f\") "
Sep 16 14:39:26.672525 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:39:26.672472    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f711841-a789-4f57-8383-9892b3c6f75f-kube-api-access-bfszd" (OuterVolumeSpecName: "kube-api-access-bfszd") pod "6f711841-a789-4f57-8383-9892b3c6f75f" (UID: "6f711841-a789-4f57-8383-9892b3c6f75f"). InnerVolumeSpecName "kube-api-access-bfszd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:39:26.672638 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:39:26.672534    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f711841-a789-4f57-8383-9892b3c6f75f-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "6f711841-a789-4f57-8383-9892b3c6f75f" (UID: "6f711841-a789-4f57-8383-9892b3c6f75f"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:39:26.672698 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:39:26.672646    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f711841-a789-4f57-8383-9892b3c6f75f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "6f711841-a789-4f57-8383-9892b3c6f75f" (UID: "6f711841-a789-4f57-8383-9892b3c6f75f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:39:26.771564 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:39:26.771534    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f711841-a789-4f57-8383-9892b3c6f75f-metrics-certs\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:39:26.771564 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:39:26.771559    2580 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6f711841-a789-4f57-8383-9892b3c6f75f-webhook-cert\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:39:26.771564 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:39:26.771568    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bfszd\" (UniqueName: \"kubernetes.io/projected/6f711841-a789-4f57-8383-9892b3c6f75f-kube-api-access-bfszd\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:39:27.395112 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:39:27.395077    2580 generic.go:356] "Generic (PLEG): container finished" podID="6f711841-a789-4f57-8383-9892b3c6f75f" containerID="62ffd99e968a71ee934d25588aa3b91cc9c86ac7d32eb703cc985f99bda5d346" exitCode=0
Sep 16 14:39:27.395519 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:39:27.395120    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-658475d98d-pf9g7"
Sep 16 14:39:27.395519 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:39:27.395165    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-658475d98d-pf9g7" event={"ID":"6f711841-a789-4f57-8383-9892b3c6f75f","Type":"ContainerDied","Data":"62ffd99e968a71ee934d25588aa3b91cc9c86ac7d32eb703cc985f99bda5d346"}
Sep 16 14:39:27.395519 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:39:27.395216    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-658475d98d-pf9g7" event={"ID":"6f711841-a789-4f57-8383-9892b3c6f75f","Type":"ContainerDied","Data":"5e53f7b67c1724a03d393bfe436cf0455cf0d69d108c185176b9d5dc91ec3959"}
Sep 16 14:39:27.395519 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:39:27.395231    2580 scope.go:117] "RemoveContainer" containerID="62ffd99e968a71ee934d25588aa3b91cc9c86ac7d32eb703cc985f99bda5d346"
Sep 16 14:39:27.406809 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:39:27.406784    2580 scope.go:117] "RemoveContainer" containerID="62ffd99e968a71ee934d25588aa3b91cc9c86ac7d32eb703cc985f99bda5d346"
Sep 16 14:39:27.407125 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:39:27.407103    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62ffd99e968a71ee934d25588aa3b91cc9c86ac7d32eb703cc985f99bda5d346\": container with ID starting with 62ffd99e968a71ee934d25588aa3b91cc9c86ac7d32eb703cc985f99bda5d346 not found: ID does not exist" containerID="62ffd99e968a71ee934d25588aa3b91cc9c86ac7d32eb703cc985f99bda5d346"
Sep 16 14:39:27.407206 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:39:27.407132    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62ffd99e968a71ee934d25588aa3b91cc9c86ac7d32eb703cc985f99bda5d346"} err="failed to get container status \"62ffd99e968a71ee934d25588aa3b91cc9c86ac7d32eb703cc985f99bda5d346\": rpc error: code = NotFound desc = could not find container \"62ffd99e968a71ee934d25588aa3b91cc9c86ac7d32eb703cc985f99bda5d346\": container with ID starting with 62ffd99e968a71ee934d25588aa3b91cc9c86ac7d32eb703cc985f99bda5d346 not found: ID does not exist"
Sep 16 14:39:27.422328 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:39:27.422297    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-658475d98d-pf9g7"]
Sep 16 14:39:27.426958 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:39:27.426921    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-658475d98d-pf9g7"]
Sep 16 14:39:27.435411 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:39:27.435388    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f711841-a789-4f57-8383-9892b3c6f75f" path="/var/lib/kubelet/pods/6f711841-a789-4f57-8383-9892b3c6f75f/volumes"
Sep 16 14:39:28.360738 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:39:28.360697    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-658475d98d-76vtt"]
Sep 16 14:39:28.361146 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:39:28.361128    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f711841-a789-4f57-8383-9892b3c6f75f" containerName="manager"
Sep 16 14:39:28.361237 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:39:28.361148    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f711841-a789-4f57-8383-9892b3c6f75f" containerName="manager"
Sep 16 14:39:28.361291 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:39:28.361255    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f711841-a789-4f57-8383-9892b3c6f75f" containerName="manager"
Sep 16 14:39:28.365537 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:39:28.365491    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-658475d98d-76vtt"
Sep 16 14:39:28.368737 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:39:28.368713    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-25lqs\""
Sep 16 14:39:28.368872 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:39:28.368797    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 16 14:39:28.369198 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:39:28.369185    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 16 14:39:28.369306 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:39:28.369289    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 16 14:39:28.376237 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:39:28.376215    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-658475d98d-76vtt"]
Sep 16 14:39:28.381894 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:39:28.381875    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c2748250-f330-42fb-9c89-195312b617bf-webhook-cert\") pod \"maas-controller-658475d98d-76vtt\" (UID: \"c2748250-f330-42fb-9c89-195312b617bf\") " pod="opendatahub/maas-controller-658475d98d-76vtt"
Sep 16 14:39:28.381990 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:39:28.381908    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c2748250-f330-42fb-9c89-195312b617bf-metrics-certs\") pod \"maas-controller-658475d98d-76vtt\" (UID: \"c2748250-f330-42fb-9c89-195312b617bf\") " pod="opendatahub/maas-controller-658475d98d-76vtt"
Sep 16 14:39:28.382037 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:39:28.381991    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdm7p\" (UniqueName: \"kubernetes.io/projected/c2748250-f330-42fb-9c89-195312b617bf-kube-api-access-gdm7p\") pod \"maas-controller-658475d98d-76vtt\" (UID: \"c2748250-f330-42fb-9c89-195312b617bf\") " pod="opendatahub/maas-controller-658475d98d-76vtt"
Sep 16 14:39:28.483278 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:39:28.483238    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdm7p\" (UniqueName: \"kubernetes.io/projected/c2748250-f330-42fb-9c89-195312b617bf-kube-api-access-gdm7p\") pod \"maas-controller-658475d98d-76vtt\" (UID: \"c2748250-f330-42fb-9c89-195312b617bf\") " pod="opendatahub/maas-controller-658475d98d-76vtt"
Sep 16 14:39:28.483771 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:39:28.483295    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c2748250-f330-42fb-9c89-195312b617bf-webhook-cert\") pod \"maas-controller-658475d98d-76vtt\" (UID: \"c2748250-f330-42fb-9c89-195312b617bf\") " pod="opendatahub/maas-controller-658475d98d-76vtt"
Sep 16 14:39:28.483771 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:39:28.483318    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c2748250-f330-42fb-9c89-195312b617bf-metrics-certs\") pod \"maas-controller-658475d98d-76vtt\" (UID: \"c2748250-f330-42fb-9c89-195312b617bf\") " pod="opendatahub/maas-controller-658475d98d-76vtt"
Sep 16 14:39:28.485808 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:39:28.485777    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c2748250-f330-42fb-9c89-195312b617bf-metrics-certs\") pod \"maas-controller-658475d98d-76vtt\" (UID: \"c2748250-f330-42fb-9c89-195312b617bf\") " pod="opendatahub/maas-controller-658475d98d-76vtt"
Sep 16 14:39:28.485902 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:39:28.485838    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c2748250-f330-42fb-9c89-195312b617bf-webhook-cert\") pod \"maas-controller-658475d98d-76vtt\" (UID: \"c2748250-f330-42fb-9c89-195312b617bf\") " pod="opendatahub/maas-controller-658475d98d-76vtt"
Sep 16 14:39:28.492436 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:39:28.492413    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdm7p\" (UniqueName: \"kubernetes.io/projected/c2748250-f330-42fb-9c89-195312b617bf-kube-api-access-gdm7p\") pod \"maas-controller-658475d98d-76vtt\" (UID: \"c2748250-f330-42fb-9c89-195312b617bf\") " pod="opendatahub/maas-controller-658475d98d-76vtt"
Sep 16 14:39:28.676017 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:39:28.675930    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-658475d98d-76vtt"
Sep 16 14:39:28.951872 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:39:28.951835    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-658475d98d-76vtt"]
Sep 16 14:39:28.952142 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:39:28.952119    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2748250_f330_42fb_9c89_195312b617bf.slice/crio-3bdd7b53b7c3f5b09d2392ea938d08f44c9e498feb2800b170bf3efb19433ba1 WatchSource:0}: Error finding container 3bdd7b53b7c3f5b09d2392ea938d08f44c9e498feb2800b170bf3efb19433ba1: Status 404 returned error can't find the container with id 3bdd7b53b7c3f5b09d2392ea938d08f44c9e498feb2800b170bf3efb19433ba1
Sep 16 14:39:29.408140 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:39:29.408109    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-658475d98d-76vtt" event={"ID":"c2748250-f330-42fb-9c89-195312b617bf","Type":"ContainerStarted","Data":"9408d4fc86ea7006afb6f677333552acfea8ffad1092d3ace73cf9d40a42cbfc"}
Sep 16 14:39:29.408358 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:39:29.408165    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-658475d98d-76vtt" event={"ID":"c2748250-f330-42fb-9c89-195312b617bf","Type":"ContainerStarted","Data":"3bdd7b53b7c3f5b09d2392ea938d08f44c9e498feb2800b170bf3efb19433ba1"}
Sep 16 14:39:29.408358 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:39:29.408221    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-658475d98d-76vtt"
Sep 16 14:39:29.428546 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:39:29.428480    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-658475d98d-76vtt" podStartSLOduration=1.05103815 podStartE2EDuration="1.428464437s" totalImagesPullingTime="377.426287ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:39:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:39:29.426440744 +0000 UTC m=+2102.441550606" watchObservedRunningTime="2026-09-16 14:39:29.428464437 +0000 UTC m=+2102.443574296"
Sep 16 14:39:40.415857 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:39:40.415831    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-658475d98d-76vtt"
Sep 16 14:40:47.054939 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:40:47.054856    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zf8st"]
Sep 16 14:40:47.055482 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:40:47.055061    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zf8st" podUID="dd3f16be-5f14-4ca7-98d2-dc26179a97a7" containerName="manager" containerID="cri-o://5d44c1ddeabfba0f37528e40d2009b5351406c3e0921262e09e707a1875d18c0" gracePeriod=10
Sep 16 14:40:47.307553 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:40:47.307473    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zf8st"
Sep 16 14:40:47.464310 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:40:47.464276    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dd3f16be-5f14-4ca7-98d2-dc26179a97a7-extensions-socket-volume\") pod \"dd3f16be-5f14-4ca7-98d2-dc26179a97a7\" (UID: \"dd3f16be-5f14-4ca7-98d2-dc26179a97a7\") "
Sep 16 14:40:47.464489 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:40:47.464339    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nz4qg\" (UniqueName: \"kubernetes.io/projected/dd3f16be-5f14-4ca7-98d2-dc26179a97a7-kube-api-access-nz4qg\") pod \"dd3f16be-5f14-4ca7-98d2-dc26179a97a7\" (UID: \"dd3f16be-5f14-4ca7-98d2-dc26179a97a7\") "
Sep 16 14:40:47.464646 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:40:47.464625    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd3f16be-5f14-4ca7-98d2-dc26179a97a7-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "dd3f16be-5f14-4ca7-98d2-dc26179a97a7" (UID: "dd3f16be-5f14-4ca7-98d2-dc26179a97a7"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:40:47.466373 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:40:47.466347    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd3f16be-5f14-4ca7-98d2-dc26179a97a7-kube-api-access-nz4qg" (OuterVolumeSpecName: "kube-api-access-nz4qg") pod "dd3f16be-5f14-4ca7-98d2-dc26179a97a7" (UID: "dd3f16be-5f14-4ca7-98d2-dc26179a97a7"). InnerVolumeSpecName "kube-api-access-nz4qg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:40:47.565701 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:40:47.565621    2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dd3f16be-5f14-4ca7-98d2-dc26179a97a7-extensions-socket-volume\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:40:47.565701 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:40:47.565649    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nz4qg\" (UniqueName: \"kubernetes.io/projected/dd3f16be-5f14-4ca7-98d2-dc26179a97a7-kube-api-access-nz4qg\") on node \"ip-10-0-129-95.ec2.internal\" DevicePath \"\""
Sep 16 14:40:47.803641 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:40:47.803605    2580 generic.go:356] "Generic (PLEG): container finished" podID="dd3f16be-5f14-4ca7-98d2-dc26179a97a7" containerID="5d44c1ddeabfba0f37528e40d2009b5351406c3e0921262e09e707a1875d18c0" exitCode=0
Sep 16 14:40:47.803817 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:40:47.803649    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zf8st"
Sep 16 14:40:47.803817 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:40:47.803689    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zf8st" event={"ID":"dd3f16be-5f14-4ca7-98d2-dc26179a97a7","Type":"ContainerDied","Data":"5d44c1ddeabfba0f37528e40d2009b5351406c3e0921262e09e707a1875d18c0"}
Sep 16 14:40:47.803817 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:40:47.803734    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zf8st" event={"ID":"dd3f16be-5f14-4ca7-98d2-dc26179a97a7","Type":"ContainerDied","Data":"12f198a2024d6fb64c9626694799980ce1deb464c9af237c585bab9b57c61b11"}
Sep 16 14:40:47.803817 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:40:47.803755    2580 scope.go:117] "RemoveContainer" containerID="5d44c1ddeabfba0f37528e40d2009b5351406c3e0921262e09e707a1875d18c0"
Sep 16 14:40:47.813016 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:40:47.813000    2580 scope.go:117] "RemoveContainer" containerID="5d44c1ddeabfba0f37528e40d2009b5351406c3e0921262e09e707a1875d18c0"
Sep 16 14:40:47.813274 ip-10-0-129-95 kubenswrapper[2580]: E0916 14:40:47.813256    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d44c1ddeabfba0f37528e40d2009b5351406c3e0921262e09e707a1875d18c0\": container with ID starting with 5d44c1ddeabfba0f37528e40d2009b5351406c3e0921262e09e707a1875d18c0 not found: ID does not exist" containerID="5d44c1ddeabfba0f37528e40d2009b5351406c3e0921262e09e707a1875d18c0"
Sep 16 14:40:47.813315 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:40:47.813284    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d44c1ddeabfba0f37528e40d2009b5351406c3e0921262e09e707a1875d18c0"} err="failed to get container status \"5d44c1ddeabfba0f37528e40d2009b5351406c3e0921262e09e707a1875d18c0\": rpc error: code = NotFound desc = could not find container \"5d44c1ddeabfba0f37528e40d2009b5351406c3e0921262e09e707a1875d18c0\": container with ID starting with 5d44c1ddeabfba0f37528e40d2009b5351406c3e0921262e09e707a1875d18c0 not found: ID does not exist"
Sep 16 14:40:47.827137 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:40:47.827089    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zf8st"]
Sep 16 14:40:47.831949 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:40:47.831931    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zf8st"]
Sep 16 14:40:49.429795 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:40:49.429765    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd3f16be-5f14-4ca7-98d2-dc26179a97a7" path="/var/lib/kubelet/pods/dd3f16be-5f14-4ca7-98d2-dc26179a97a7/volumes"
Sep 16 14:41:47.853441 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:41:47.853405    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2rn2h"]
Sep 16 14:41:47.853903 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:41:47.853830    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd3f16be-5f14-4ca7-98d2-dc26179a97a7" containerName="manager"
Sep 16 14:41:47.853903 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:41:47.853844    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd3f16be-5f14-4ca7-98d2-dc26179a97a7" containerName="manager"
Sep 16 14:41:47.853979 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:41:47.853920    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd3f16be-5f14-4ca7-98d2-dc26179a97a7" containerName="manager"
Sep 16 14:41:47.856771 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:41:47.856757    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2rn2h"
Sep 16 14:41:47.858760 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:41:47.858737    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-l98d8\""
Sep 16 14:41:47.872658 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:41:47.872633    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2rn2h"]
Sep 16 14:41:47.977468 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:41:47.977436    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pg92\" (UniqueName: \"kubernetes.io/projected/540e504e-a0e3-4c3c-9a45-d323776bda32-kube-api-access-2pg92\") pod \"kuadrant-operator-controller-manager-5769f99d9f-2rn2h\" (UID: \"540e504e-a0e3-4c3c-9a45-d323776bda32\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2rn2h"
Sep 16 14:41:47.977652 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:41:47.977488    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/540e504e-a0e3-4c3c-9a45-d323776bda32-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-2rn2h\" (UID: \"540e504e-a0e3-4c3c-9a45-d323776bda32\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2rn2h"
Sep 16 14:41:48.078851 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:41:48.078820    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pg92\" (UniqueName: \"kubernetes.io/projected/540e504e-a0e3-4c3c-9a45-d323776bda32-kube-api-access-2pg92\") pod \"kuadrant-operator-controller-manager-5769f99d9f-2rn2h\" (UID: \"540e504e-a0e3-4c3c-9a45-d323776bda32\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2rn2h"
Sep 16 14:41:48.079017 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:41:48.078885    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/540e504e-a0e3-4c3c-9a45-d323776bda32-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-2rn2h\" (UID: \"540e504e-a0e3-4c3c-9a45-d323776bda32\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2rn2h"
Sep 16 14:41:48.079284 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:41:48.079266    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/540e504e-a0e3-4c3c-9a45-d323776bda32-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-2rn2h\" (UID: \"540e504e-a0e3-4c3c-9a45-d323776bda32\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2rn2h"
Sep 16 14:41:48.087974 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:41:48.087945    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pg92\" (UniqueName: \"kubernetes.io/projected/540e504e-a0e3-4c3c-9a45-d323776bda32-kube-api-access-2pg92\") pod \"kuadrant-operator-controller-manager-5769f99d9f-2rn2h\" (UID: \"540e504e-a0e3-4c3c-9a45-d323776bda32\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2rn2h"
Sep 16 14:41:48.166434 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:41:48.166365    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2rn2h"
Sep 16 14:41:48.451466 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:41:48.451444    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2rn2h"]
Sep 16 14:41:48.453759 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:41:48.453733    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod540e504e_a0e3_4c3c_9a45_d323776bda32.slice/crio-02323b021bc15ea32183ae0ca4df85a3f4802a768949f381d42c08d1d48c7b05 WatchSource:0}: Error finding container 02323b021bc15ea32183ae0ca4df85a3f4802a768949f381d42c08d1d48c7b05: Status 404 returned error can't find the container with id 02323b021bc15ea32183ae0ca4df85a3f4802a768949f381d42c08d1d48c7b05
Sep 16 14:41:49.110209 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:41:49.110178    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2rn2h" event={"ID":"540e504e-a0e3-4c3c-9a45-d323776bda32","Type":"ContainerStarted","Data":"4b0d3c1cfe23e8b414c9bd632d6e3e39e15e06ba7a7feb6ed1eee7913bf2a609"}
Sep 16 14:41:49.110593 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:41:49.110230    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2rn2h" event={"ID":"540e504e-a0e3-4c3c-9a45-d323776bda32","Type":"ContainerStarted","Data":"02323b021bc15ea32183ae0ca4df85a3f4802a768949f381d42c08d1d48c7b05"}
Sep 16 14:41:49.110593 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:41:49.110273    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2rn2h"
Sep 16 14:41:49.133473 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:41:49.133429    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2rn2h" podStartSLOduration=2.133414389 podStartE2EDuration="2.133414389s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:41:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:41:49.130451821 +0000 UTC m=+2242.145561680" watchObservedRunningTime="2026-09-16 14:41:49.133414389 +0000 UTC m=+2242.148524248"
Sep 16 14:42:00.118634 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:42:00.118552    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2rn2h"
Sep 16 14:47:18.529376 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:18.529340    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-wg92s_832cddc0-66b7-45a7-8cad-2a8609131f31/global-pull-secret-syncer/0.log"
Sep 16 14:47:18.658347 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:18.658317    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-cxcmv_ffa9d62b-620c-46d1-ab3a-73a2fdb84817/konnectivity-agent/0.log"
Sep 16 14:47:18.675050 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:18.675021    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-95.ec2.internal_38046ded5fcb5dec29d3f7bce0f86c40/haproxy/0.log"
Sep 16 14:47:23.371227 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:23.371194    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7867f86b4c-s2hnz_c3ae2ea0-bde3-4337-98cb-96555908f89a/authorino/0.log"
Sep 16 14:47:23.399388 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:23.399357    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-j4pg9_b244a479-4fb3-4bdf-a42a-fad2ff97f288/manager/0.log"
Sep 16 14:47:23.430763 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:23.430739    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-2phrz_f0ebc0cd-7c60-4324-8820-d5858eb98477/manager/0.log"
Sep 16 14:47:23.504016 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:23.503984    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-2rn2h_540e504e-a0e3-4c3c-9a45-d323776bda32/manager/0.log"
Sep 16 14:47:23.587271 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:23.587237    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-2262r_af8a3a0e-cec2-4fc6-b552-6e5fd93004b0/manager/0.log"
Sep 16 14:47:25.402600 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:25.402572    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-v5khd_375147d4-f89a-4678-9f71-70a295e903a1/node-exporter/0.log"
Sep 16 14:47:25.426431 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:25.426406    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-v5khd_375147d4-f89a-4678-9f71-70a295e903a1/kube-rbac-proxy/0.log"
Sep 16 14:47:25.447065 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:25.447042    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-v5khd_375147d4-f89a-4678-9f71-70a295e903a1/init-textfile/0.log"
Sep 16 14:47:25.995894 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:25.995867    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6c77cb6fc8-vg979_25d3799f-6859-473b-af72-56716dfa3dbb/telemeter-client/0.log"
Sep 16 14:47:26.017490 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:26.017470    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6c77cb6fc8-vg979_25d3799f-6859-473b-af72-56716dfa3dbb/reload/0.log"
Sep 16 14:47:26.040656 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:26.040632    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6c77cb6fc8-vg979_25d3799f-6859-473b-af72-56716dfa3dbb/kube-rbac-proxy/0.log"
Sep 16 14:47:28.704420 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:28.704386    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-74mws/perf-node-gather-daemonset-mqhzm"]
Sep 16 14:47:28.708142 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:28.708122    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-74mws/perf-node-gather-daemonset-mqhzm"
Sep 16 14:47:28.710249 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:28.710218    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-74mws\"/\"openshift-service-ca.crt\""
Sep 16 14:47:28.710366 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:28.710256    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-74mws\"/\"default-dockercfg-cq4jd\""
Sep 16 14:47:28.710711 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:28.710696    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-74mws\"/\"kube-root-ca.crt\""
Sep 16 14:47:28.717730 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:28.717707    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-74mws/perf-node-gather-daemonset-mqhzm"]
Sep 16 14:47:28.763040 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:28.763011    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cd8b09d7-4b14-4748-8983-0a60c9b5c669-proc\") pod \"perf-node-gather-daemonset-mqhzm\" (UID: \"cd8b09d7-4b14-4748-8983-0a60c9b5c669\") " pod="openshift-must-gather-74mws/perf-node-gather-daemonset-mqhzm"
Sep 16 14:47:28.763040 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:28.763039    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cd8b09d7-4b14-4748-8983-0a60c9b5c669-lib-modules\") pod \"perf-node-gather-daemonset-mqhzm\" (UID: \"cd8b09d7-4b14-4748-8983-0a60c9b5c669\") " pod="openshift-must-gather-74mws/perf-node-gather-daemonset-mqhzm"
Sep 16 14:47:28.763224 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:28.763068    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cd8b09d7-4b14-4748-8983-0a60c9b5c669-podres\") pod \"perf-node-gather-daemonset-mqhzm\" (UID: \"cd8b09d7-4b14-4748-8983-0a60c9b5c669\") " pod="openshift-must-gather-74mws/perf-node-gather-daemonset-mqhzm"
Sep 16 14:47:28.763224 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:28.763153    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cd8b09d7-4b14-4748-8983-0a60c9b5c669-sys\") pod \"perf-node-gather-daemonset-mqhzm\" (UID: \"cd8b09d7-4b14-4748-8983-0a60c9b5c669\") " pod="openshift-must-gather-74mws/perf-node-gather-daemonset-mqhzm"
Sep 16 14:47:28.763224 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:28.763189    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnwqc\" (UniqueName: \"kubernetes.io/projected/cd8b09d7-4b14-4748-8983-0a60c9b5c669-kube-api-access-rnwqc\") pod \"perf-node-gather-daemonset-mqhzm\" (UID: \"cd8b09d7-4b14-4748-8983-0a60c9b5c669\") " pod="openshift-must-gather-74mws/perf-node-gather-daemonset-mqhzm"
Sep 16 14:47:28.864053 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:28.864015    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cd8b09d7-4b14-4748-8983-0a60c9b5c669-proc\") pod \"perf-node-gather-daemonset-mqhzm\" (UID: \"cd8b09d7-4b14-4748-8983-0a60c9b5c669\") " pod="openshift-must-gather-74mws/perf-node-gather-daemonset-mqhzm"
Sep 16 14:47:28.864053 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:28.864050    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cd8b09d7-4b14-4748-8983-0a60c9b5c669-lib-modules\") pod \"perf-node-gather-daemonset-mqhzm\" (UID: \"cd8b09d7-4b14-4748-8983-0a60c9b5c669\") " pod="openshift-must-gather-74mws/perf-node-gather-daemonset-mqhzm"
Sep 16 14:47:28.864261 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:28.864083    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cd8b09d7-4b14-4748-8983-0a60c9b5c669-podres\") pod \"perf-node-gather-daemonset-mqhzm\" (UID: \"cd8b09d7-4b14-4748-8983-0a60c9b5c669\") " pod="openshift-must-gather-74mws/perf-node-gather-daemonset-mqhzm"
Sep 16 14:47:28.864261 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:28.864131    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cd8b09d7-4b14-4748-8983-0a60c9b5c669-sys\") pod \"perf-node-gather-daemonset-mqhzm\" (UID: \"cd8b09d7-4b14-4748-8983-0a60c9b5c669\") " pod="openshift-must-gather-74mws/perf-node-gather-daemonset-mqhzm"
Sep 16 14:47:28.864261 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:28.864139    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cd8b09d7-4b14-4748-8983-0a60c9b5c669-proc\") pod \"perf-node-gather-daemonset-mqhzm\" (UID: \"cd8b09d7-4b14-4748-8983-0a60c9b5c669\") " pod="openshift-must-gather-74mws/perf-node-gather-daemonset-mqhzm"
Sep 16 14:47:28.864261 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:28.864167    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnwqc\" (UniqueName: \"kubernetes.io/projected/cd8b09d7-4b14-4748-8983-0a60c9b5c669-kube-api-access-rnwqc\") pod \"perf-node-gather-daemonset-mqhzm\" (UID: \"cd8b09d7-4b14-4748-8983-0a60c9b5c669\") " pod="openshift-must-gather-74mws/perf-node-gather-daemonset-mqhzm"
Sep 16 14:47:28.864261 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:28.864186    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cd8b09d7-4b14-4748-8983-0a60c9b5c669-lib-modules\") pod \"perf-node-gather-daemonset-mqhzm\" (UID: \"cd8b09d7-4b14-4748-8983-0a60c9b5c669\") " pod="openshift-must-gather-74mws/perf-node-gather-daemonset-mqhzm"
Sep 16 14:47:28.864261 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:28.864200    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cd8b09d7-4b14-4748-8983-0a60c9b5c669-sys\") pod \"perf-node-gather-daemonset-mqhzm\" (UID: \"cd8b09d7-4b14-4748-8983-0a60c9b5c669\") " pod="openshift-must-gather-74mws/perf-node-gather-daemonset-mqhzm"
Sep 16 14:47:28.864261 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:28.864248    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cd8b09d7-4b14-4748-8983-0a60c9b5c669-podres\") pod \"perf-node-gather-daemonset-mqhzm\" (UID: \"cd8b09d7-4b14-4748-8983-0a60c9b5c669\") " pod="openshift-must-gather-74mws/perf-node-gather-daemonset-mqhzm"
Sep 16 14:47:28.874846 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:28.874821    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnwqc\" (UniqueName: \"kubernetes.io/projected/cd8b09d7-4b14-4748-8983-0a60c9b5c669-kube-api-access-rnwqc\") pod \"perf-node-gather-daemonset-mqhzm\" (UID: \"cd8b09d7-4b14-4748-8983-0a60c9b5c669\") " pod="openshift-must-gather-74mws/perf-node-gather-daemonset-mqhzm"
Sep 16 14:47:29.018716 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:29.018622    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-74mws/perf-node-gather-daemonset-mqhzm"
Sep 16 14:47:29.286667 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:29.286641    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-74mws/perf-node-gather-daemonset-mqhzm"]
Sep 16 14:47:29.288365 ip-10-0-129-95 kubenswrapper[2580]: W0916 14:47:29.288339    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podcd8b09d7_4b14_4748_8983_0a60c9b5c669.slice/crio-e41fe2f84994decd0f1cd7b81ab2ba587b5dfc86ceff15643be5c9f78cc806f6 WatchSource:0}: Error finding container e41fe2f84994decd0f1cd7b81ab2ba587b5dfc86ceff15643be5c9f78cc806f6: Status 404 returned error can't find the container with id e41fe2f84994decd0f1cd7b81ab2ba587b5dfc86ceff15643be5c9f78cc806f6
Sep 16 14:47:29.828884 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:29.828850    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-74mws/perf-node-gather-daemonset-mqhzm" event={"ID":"cd8b09d7-4b14-4748-8983-0a60c9b5c669","Type":"ContainerStarted","Data":"adddb1732f4c4d619c57ddc2a91b4e8b8bfb4a638db26d6b6d8c9fc11dae92f8"}
Sep 16 14:47:29.829293 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:29.828898    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-74mws/perf-node-gather-daemonset-mqhzm" event={"ID":"cd8b09d7-4b14-4748-8983-0a60c9b5c669","Type":"ContainerStarted","Data":"e41fe2f84994decd0f1cd7b81ab2ba587b5dfc86ceff15643be5c9f78cc806f6"}
Sep 16 14:47:29.829293 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:29.828955    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-74mws/perf-node-gather-daemonset-mqhzm"
Sep 16 14:47:29.846921 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:29.846876    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-74mws/perf-node-gather-daemonset-mqhzm" podStartSLOduration=1.8468610399999998 podStartE2EDuration="1.84686104s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 14:47:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:47:29.844148772 +0000 UTC m=+2582.859258634" watchObservedRunningTime="2026-09-16 14:47:29.84686104 +0000 UTC m=+2582.861970900"
Sep 16 14:47:30.847707 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:30.847680    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5zzjr_39a375e4-748f-4cc1-b445-f5c5a32a2652/dns/0.log"
Sep 16 14:47:30.865912 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:30.865889    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5zzjr_39a375e4-748f-4cc1-b445-f5c5a32a2652/kube-rbac-proxy/0.log"
Sep 16 14:47:31.007169 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:31.007142    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-s544c_e046d878-c8a6-4278-bde8-238f63fb96b2/dns-node-resolver/0.log"
Sep 16 14:47:31.602634 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:31.602599    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-r9k4l_fa3e87c7-91e7-4767-b43d-6f00261f1d56/node-ca/0.log"
Sep 16 14:47:32.659957 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:32.659926    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-4t6lk_c6a422ff-e208-4686-9ea5-d7e75901d878/istio-proxy/0.log"
Sep 16 14:47:33.326133 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:33.326102    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b574975b8-rkf8t_d8dbea9a-e715-4bd9-88d2-7aea66a83a63/kube-auth-proxy/0.log"
Sep 16 14:47:33.353075 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:33.353048    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-jfn4c_aedbe6b8-69e8-4213-b355-e4cdcd452d49/istio-proxy/0.log"
Sep 16 14:47:33.355459 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:33.355438    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-jfn4c_aedbe6b8-69e8-4213-b355-e4cdcd452d49/istio-proxy/1.log"
Sep 16 14:47:34.289808 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:34.289777    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-szx9c_4b7b253f-9622-4e91-8def-d4c654bb1f14/serve-healthcheck-canary/0.log"
Sep 16 14:47:34.911626 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:34.911597    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lhttb_ef29ef06-9374-482a-a3fc-77032fbf6537/kube-rbac-proxy/0.log"
Sep 16 14:47:34.928444 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:34.928416    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lhttb_ef29ef06-9374-482a-a3fc-77032fbf6537/exporter/0.log"
Sep 16 14:47:34.945262 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:34.945238    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lhttb_ef29ef06-9374-482a-a3fc-77032fbf6537/extractor/0.log"
Sep 16 14:47:35.843799 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:35.843766    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-74mws/perf-node-gather-daemonset-mqhzm"
Sep 16 14:47:37.105443 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:37.105415    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-x6xb8_c91ff34d-ce75-4dbf-9fb3-15c057742185/manager/0.log"
Sep 16 14:47:37.329290 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:37.329262    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-ktsml_0b1a49bc-75d4-4e7a-baae-7283af813749/manager/0.log"
Sep 16 14:47:37.371357 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:37.371283    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-658475d98d-76vtt_c2748250-f330-42fb-9c89-195312b617bf/manager/0.log"
Sep 16 14:47:37.464338 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:37.464307    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-snvvv_2bb4393a-0f0a-4213-a212-19ae51c42bb3/manager/0.log"
Sep 16 14:47:37.533948 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:37.533921    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-77f6d9877-g8lwc_52f3de3b-2520-41d1-9d34-0ad44c966111/manager/0.log"
Sep 16 14:47:37.553666 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:37.553648    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-77f6d9877-g8lwc_52f3de3b-2520-41d1-9d34-0ad44c966111/copy-manifests/0.log"
Sep 16 14:47:38.720428 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:38.720396    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-4t6lk_c6a422ff-e208-4686-9ea5-d7e75901d878/istio-proxy/0.log"
Sep 16 14:47:39.410605 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:39.410579    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b574975b8-rkf8t_d8dbea9a-e715-4bd9-88d2-7aea66a83a63/kube-auth-proxy/0.log"
Sep 16 14:47:39.431531 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:39.431493    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-jfn4c_aedbe6b8-69e8-4213-b355-e4cdcd452d49/istio-proxy/0.log"
Sep 16 14:47:39.433753 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:39.433731    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-jfn4c_aedbe6b8-69e8-4213-b355-e4cdcd452d49/istio-proxy/1.log"
Sep 16 14:47:40.968312 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:40.968280    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-4t6lk_c6a422ff-e208-4686-9ea5-d7e75901d878/istio-proxy/0.log"
Sep 16 14:47:41.652415 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:41.652388    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b574975b8-rkf8t_d8dbea9a-e715-4bd9-88d2-7aea66a83a63/kube-auth-proxy/0.log"
Sep 16 14:47:41.669137 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:41.669115    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-jfn4c_aedbe6b8-69e8-4213-b355-e4cdcd452d49/istio-proxy/0.log"
Sep 16 14:47:41.671589 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:41.671570    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-jfn4c_aedbe6b8-69e8-4213-b355-e4cdcd452d49/istio-proxy/1.log"
Sep 16 14:47:45.580256 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:45.580222    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-7xg75_d89c07ec-f200-4033-9fe1-c8dab7ef1d48/kube-multus/0.log"
Sep 16 14:47:45.600326 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:45.600305    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-c5xjg_9840e56c-3a33-49c0-a917-70f641aabf3b/kube-multus-additional-cni-plugins/0.log"
Sep 16 14:47:45.618725 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:45.618700    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-c5xjg_9840e56c-3a33-49c0-a917-70f641aabf3b/egress-router-binary-copy/0.log"
Sep 16 14:47:45.635902 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:45.635883    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-c5xjg_9840e56c-3a33-49c0-a917-70f641aabf3b/cni-plugins/0.log"
Sep 16 14:47:45.652302 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:45.652283    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-c5xjg_9840e56c-3a33-49c0-a917-70f641aabf3b/bond-cni-plugin/0.log"
Sep 16 14:47:45.670755 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:45.670739    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-c5xjg_9840e56c-3a33-49c0-a917-70f641aabf3b/routeoverride-cni/0.log"
Sep 16 14:47:45.687545 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:45.687515    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-c5xjg_9840e56c-3a33-49c0-a917-70f641aabf3b/whereabouts-cni-bincopy/0.log"
Sep 16 14:47:45.702813 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:45.702795    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-c5xjg_9840e56c-3a33-49c0-a917-70f641aabf3b/whereabouts-cni/0.log"
Sep 16 14:47:46.152967 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:46.152937    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-pxv9d_f0346ba8-7f08-4472-933c-7238392887b6/network-metrics-daemon/0.log"
Sep 16 14:47:46.168062 ip-10-0-129-95 kubenswrapper[2580]: I0916 14:47:46.168041    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-pxv9d_f0346ba8-7f08-4472-933c-7238392887b6/kube-rbac-proxy/0.log"