Jul 31 22:55:10.384467 ip-10-0-128-243 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 31 22:55:10.384477 ip-10-0-128-243 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 31 22:55:10.384484 ip-10-0-128-243 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 31 22:55:10.384779 ip-10-0-128-243 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 31 22:55:20.525520 ip-10-0-128-243 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 31 22:55:20.525539 ip-10-0-128-243 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 8581e0637c904e399f5e8a6aaccefc80 --
Jul 31 22:56:56.227006 ip-10-0-128-243 systemd[1]: Starting Kubernetes Kubelet...
Jul 31 22:56:56.667609 ip-10-0-128-243 kubenswrapper[2584]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 31 22:56:56.668315 ip-10-0-128-243 kubenswrapper[2584]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 31 22:56:56.668315 ip-10-0-128-243 kubenswrapper[2584]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 31 22:56:56.668315 ip-10-0-128-243 kubenswrapper[2584]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 31 22:56:56.668315 ip-10-0-128-243 kubenswrapper[2584]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 31 22:56:56.669086 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.668998    2584 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 31 22:56:56.673055 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673033    2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 31 22:56:56.673055 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673049    2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 31 22:56:56.673055 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673055    2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 31 22:56:56.673055 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673059    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 31 22:56:56.673055 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673063    2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 31 22:56:56.673335 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673068    2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 31 22:56:56.673335 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673072    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 31 22:56:56.673335 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673077    2584 feature_gate.go:328] unrecognized feature gate: Example2
Jul 31 22:56:56.673335 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673083    2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 31 22:56:56.673335 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673088    2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 31 22:56:56.673335 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673092    2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 31 22:56:56.673335 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673096    2584 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 31 22:56:56.673335 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673101    2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 31 22:56:56.673335 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673105    2584 feature_gate.go:328] unrecognized feature gate: Example
Jul 31 22:56:56.673335 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673109    2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 31 22:56:56.673335 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673112    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 31 22:56:56.673335 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673116    2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 31 22:56:56.673335 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673120    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 31 22:56:56.673335 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673124    2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 31 22:56:56.673335 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673128    2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 31 22:56:56.673335 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673132    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 31 22:56:56.673335 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673136    2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 31 22:56:56.673335 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673140    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 31 22:56:56.673335 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673144    2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 31 22:56:56.674117 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673151    2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 31 22:56:56.674117 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673156    2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 31 22:56:56.674117 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673161    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 31 22:56:56.674117 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673165    2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 31 22:56:56.674117 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673170    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 31 22:56:56.674117 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673183    2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 31 22:56:56.674117 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673189    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 31 22:56:56.674117 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673193    2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 31 22:56:56.674117 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673197    2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 31 22:56:56.674117 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673201    2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 31 22:56:56.674117 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673206    2584 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 31 22:56:56.674117 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673210    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 31 22:56:56.674117 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673214    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 31 22:56:56.674117 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673218    2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 31 22:56:56.674117 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673223    2584 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 31 22:56:56.674117 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673227    2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 31 22:56:56.674117 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673232    2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 31 22:56:56.674117 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673236    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 31 22:56:56.674117 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673240    2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 31 22:56:56.674117 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673244    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 31 22:56:56.674976 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673248    2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 31 22:56:56.674976 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673252    2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 31 22:56:56.674976 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673256    2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 31 22:56:56.674976 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673260    2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 31 22:56:56.674976 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673264    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 31 22:56:56.674976 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673269    2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 31 22:56:56.674976 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673273    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 31 22:56:56.674976 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673277    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 31 22:56:56.674976 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673281    2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 31 22:56:56.674976 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673285    2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 31 22:56:56.674976 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673289    2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 31 22:56:56.674976 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673296    2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 31 22:56:56.674976 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673302    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 31 22:56:56.674976 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673309    2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 31 22:56:56.674976 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673313    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 31 22:56:56.674976 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673318    2584 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 31 22:56:56.674976 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673322    2584 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 31 22:56:56.674976 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673327    2584 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 31 22:56:56.674976 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673331    2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 31 22:56:56.674976 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673335    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 31 22:56:56.675826 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673341    2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 31 22:56:56.675826 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673345    2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 31 22:56:56.675826 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673350    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 31 22:56:56.675826 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673354    2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 31 22:56:56.675826 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673358    2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 31 22:56:56.675826 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673362    2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 31 22:56:56.675826 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673367    2584 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 31 22:56:56.675826 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673372    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 31 22:56:56.675826 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673376    2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 31 22:56:56.675826 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673380    2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 31 22:56:56.675826 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673384    2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 31 22:56:56.675826 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673389    2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 31 22:56:56.675826 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673393    2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 31 22:56:56.675826 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673398    2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 31 22:56:56.675826 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673402    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 31 22:56:56.675826 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673406    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 31 22:56:56.675826 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673412    2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 31 22:56:56.675826 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673416    2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 31 22:56:56.675826 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673420    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 31 22:56:56.675826 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673425    2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 31 22:56:56.677471 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673429    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 31 22:56:56.677471 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673434    2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 31 22:56:56.677471 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673438    2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 31 22:56:56.677471 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673442    2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 31 22:56:56.677471 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673446    2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 31 22:56:56.677471 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673450    2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 31 22:56:56.677471 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673455    2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 31 22:56:56.677471 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673459    2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 31 22:56:56.677471 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673463    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 31 22:56:56.677471 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673467    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 31 22:56:56.677471 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673471    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 31 22:56:56.677471 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673475    2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 31 22:56:56.677471 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673480    2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 31 22:56:56.677471 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673484    2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 31 22:56:56.677471 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.673488    2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 31 22:56:56.677471 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.674675    2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 31 22:56:56.677471 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.674685    2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 31 22:56:56.677471 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.674691    2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 31 22:56:56.677471 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.674716    2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 31 22:56:56.677471 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.674721    2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 31 22:56:56.678051 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.674726    2584 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 31 22:56:56.678051 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.674731    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 31 22:56:56.678051 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.674736    2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 31 22:56:56.678051 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.674740    2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 31 22:56:56.678051 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.674745    2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 31 22:56:56.678051 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.674749    2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 31 22:56:56.678051 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.674754    2584 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 31 22:56:56.678051 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.674758    2584 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 31 22:56:56.678051 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.674763    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 31 22:56:56.678051 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.674768    2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 31 22:56:56.678051 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.674773    2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 31 22:56:56.678051 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.674778    2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 31 22:56:56.678051 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.674783    2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 31 22:56:56.678051 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.674788    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 31 22:56:56.678051 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.674792    2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 31 22:56:56.678051 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.674797    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 31 22:56:56.678051 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.674801    2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 31 22:56:56.678051 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.674805    2584 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 31 22:56:56.678051 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.674810    2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 31 22:56:56.678051 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.674814    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 31 22:56:56.678802 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.674818    2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 31 22:56:56.678802 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.674822    2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 31 22:56:56.678802 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.674826    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 31 22:56:56.678802 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.674833    2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 31 22:56:56.678802 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.674839    2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 31 22:56:56.678802 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.674843    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 31 22:56:56.678802 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.674848    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 31 22:56:56.678802 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.674852    2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 31 22:56:56.678802 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.674856    2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 31 22:56:56.678802 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.674860    2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 31 22:56:56.678802 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.674865    2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 31 22:56:56.678802 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.674870    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 31 22:56:56.678802 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.674874    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 31 22:56:56.678802 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.674879    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 31 22:56:56.678802 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.674883    2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 31 22:56:56.678802 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.674887    2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 31 22:56:56.678802 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.674891    2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 31 22:56:56.678802 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.674895    2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 31 22:56:56.678802 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.674899    2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 31 22:56:56.679278 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.674904    2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 31 22:56:56.679278 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.674908    2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 31 22:56:56.679278 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.674912    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 31 22:56:56.679278 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.674919    2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 31 22:56:56.679278 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.674923    2584 feature_gate.go:328] unrecognized feature gate: Example2
Jul 31 22:56:56.679278 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.674928    2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 31 22:56:56.679278 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.674932    2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 31 22:56:56.679278 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.674936    2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 31 22:56:56.679278 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.674943    2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 31 22:56:56.679278 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.674950    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 31 22:56:56.679278 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.674955    2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 31 22:56:56.679278 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.674959    2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 31 22:56:56.679278 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.674964    2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 31 22:56:56.679278 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.674968    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 31 22:56:56.679278 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.674972    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 31 22:56:56.679278 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.674976    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 31 22:56:56.679278 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.674980    2584 feature_gate.go:328] unrecognized feature gate: Example
Jul 31 22:56:56.679278 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.674984    2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 31 22:56:56.679278 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.674988    2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 31 22:56:56.679769 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.674992    2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 31 22:56:56.679769 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.674996    2584 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 31 22:56:56.679769 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675001    2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 31 22:56:56.679769 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675005    2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 31 22:56:56.679769 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675010    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 31 22:56:56.679769 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675014    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 31 22:56:56.679769 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675018    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 31 22:56:56.679769 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675023    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 31 22:56:56.679769 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675027    2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 31 22:56:56.679769 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675031    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 31 22:56:56.679769 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675036    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 31 22:56:56.679769 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675041    2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 31 22:56:56.679769 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675045    2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 31 22:56:56.679769 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675049    2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 31 22:56:56.679769 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675053    2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 31 22:56:56.679769 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675057    2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 31 22:56:56.679769 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675062    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 31 22:56:56.679769 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675067    2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 31 22:56:56.679769 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675071    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 31 22:56:56.679769 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675076    2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 31 22:56:56.680359 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675080    2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 31 22:56:56.680359 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675084    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 31 22:56:56.680359 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675089    2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 31 22:56:56.680359 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675093    2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 31 22:56:56.680359 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675097    2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 31 22:56:56.680359 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675102    2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 31 22:56:56.680359 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675106    2584 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 31 22:56:56.680359 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675110    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 31 22:56:56.680359 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675114    2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 31 22:56:56.680359 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675119    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 31 22:56:56.680359 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675123    2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 31 22:56:56.680359 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675127    2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 31 22:56:56.680359 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675133    2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 31 22:56:56.680359 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675140    2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 31 22:56:56.680359 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675144    2584 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 31 22:56:56.680359 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675149    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 31 22:56:56.680359 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675791    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 31 22:56:56.680359 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675800    2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 31 22:56:56.680359 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675806    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 31 22:56:56.680359 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675811    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 31 22:56:56.680876 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675815    2584 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 31 22:56:56.680876 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675820    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 31 22:56:56.680876 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675824    2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 31 22:56:56.680876 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675828    2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 31 22:56:56.680876 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675832    2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 31 22:56:56.680876 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675836    2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 31 22:56:56.680876 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675840    2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 31 22:56:56.680876 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675846    2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 31 22:56:56.680876 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675852    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 31 22:56:56.680876 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675858    2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 31 22:56:56.680876 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675863    2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 31 22:56:56.680876 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675869    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 31 22:56:56.680876 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675873    2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 31 22:56:56.680876 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675878    2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 31 22:56:56.680876 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675882    2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 31 22:56:56.680876 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675886    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 31 22:56:56.680876 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675890    2584 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 31 22:56:56.680876 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675895    2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 31 22:56:56.680876 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675899    2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 31 22:56:56.680876 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675903    2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 31 22:56:56.681396 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675907    2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 31 22:56:56.681396 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675911    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 31 22:56:56.681396 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675915    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 31 22:56:56.681396 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675920    2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 31 22:56:56.681396 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675924    2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 31 22:56:56.681396 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675928    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 31 22:56:56.681396 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675932    2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 31 22:56:56.681396 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675936    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 31 22:56:56.681396 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675943    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 31 22:56:56.681396 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675947    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 31 22:56:56.681396 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675952    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 31 22:56:56.681396 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675958    2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 31 22:56:56.681396 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675962    2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 31 22:56:56.681396 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675968    2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 31 22:56:56.681396 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675974    2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 31 22:56:56.681396 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675978    2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 31 22:56:56.681396 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675982    2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 31 22:56:56.681396 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675988    2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 31 22:56:56.681848 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675995    2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 31 22:56:56.681848 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.675999    2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 31 22:56:56.681848 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.676003    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 31 22:56:56.681848 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.676007    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 31 22:56:56.681848 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.676014    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 31 22:56:56.681848 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.676019    2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 31 22:56:56.681848 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.676022    2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 31 22:56:56.681848 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.676027    2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 31 22:56:56.681848 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.676031    2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 31 22:56:56.681848 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.676035    2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 31 22:56:56.681848 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.676039    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 31 22:56:56.681848 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.676043    2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 31 22:56:56.681848 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.676048    2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 31 22:56:56.681848 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.676052    2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 31 22:56:56.681848 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.676056    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 31 22:56:56.681848 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.676060    2584 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 31 22:56:56.681848 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.676063    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 31 22:56:56.681848 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.676068    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 31 22:56:56.681848 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.676072    2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 31 22:56:56.681848 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.676076    2584 feature_gate.go:328] unrecognized feature gate: Example2
Jul 31 22:56:56.682335 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.676080    2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 31 22:56:56.682335 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.676084    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 31 22:56:56.682335 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.676093    2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 31 22:56:56.682335 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.676097    2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 31 22:56:56.682335 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.676101    2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 31 22:56:56.682335 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.676105    2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 31 22:56:56.682335 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.676109    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 31 22:56:56.682335 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.676114    2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 31 22:56:56.682335 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.676118    2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 31 22:56:56.682335 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.676122    2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 31 22:56:56.682335 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.676126    2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 31 22:56:56.682335 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.676130    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 31 22:56:56.682335 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.676134    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 31 22:56:56.682335 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.676139    2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 31 22:56:56.682335 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.676142    2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 31 22:56:56.682335 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.676146    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 31 22:56:56.682335 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.676152    2584 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 31 22:56:56.682335 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.676157    2584 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 31 22:56:56.682335 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.676162    2584 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 31 22:56:56.682335 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.676166    2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 31 22:56:56.682859 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.676171    2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 31 22:56:56.682859 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.676175    2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 31 22:56:56.682859 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.676179    2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 31 22:56:56.682859 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.676183    2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 31 22:56:56.682859 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.676187    2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 31 22:56:56.682859 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.676191    2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 31 22:56:56.682859 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.676196    2584 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 31 22:56:56.682859 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.676200    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 31 22:56:56.682859 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.676204    2584 feature_gate.go:328] unrecognized feature gate: Example
Jul 31 22:56:56.682859 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.676208    2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 31 22:56:56.682859 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.676212    2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 31 22:56:56.682859 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.676216    2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 31 22:56:56.682859 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.676220    2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 31 22:56:56.682859 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.676224    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 31 22:56:56.682859 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.676230    2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 31 22:56:56.682859 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.676234    2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 31 22:56:56.682859 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.676238    2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 31 22:56:56.682859 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676339    2584 flags.go:64] FLAG: --address="0.0.0.0"
Jul 31 22:56:56.682859 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676350    2584 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 31 22:56:56.682859 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676361    2584 flags.go:64] FLAG: --anonymous-auth="true"
Jul 31 22:56:56.682859 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676367    2584 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 31 22:56:56.683431 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676373    2584 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 31 22:56:56.683431 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676378    2584 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 31 22:56:56.683431 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676382    2584 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 31 22:56:56.683431 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676387    2584 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 31 22:56:56.683431 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676390    2584 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 31 22:56:56.683431 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676393    2584 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 31 22:56:56.683431 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676396    2584 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 31 22:56:56.683431 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676402    2584 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 31 22:56:56.683431 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676405    2584 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 31 22:56:56.683431 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676409    2584 flags.go:64] FLAG: --cgroup-root=""
Jul 31 22:56:56.683431 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676411    2584 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 31 22:56:56.683431 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676414    2584 flags.go:64] FLAG: --client-ca-file=""
Jul 31 22:56:56.683431 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676417    2584 flags.go:64] FLAG: --cloud-provider="external"
Jul 31 22:56:56.683431 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676420    2584 flags.go:64] FLAG: --cluster-dns="[]"
Jul 31 22:56:56.683431 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676424    2584 flags.go:64] FLAG: --cluster-domain=""
Jul 31 22:56:56.683431 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676427    2584 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 31 22:56:56.683431 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676430    2584 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 31 22:56:56.683431 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676433    2584 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 31 22:56:56.683431 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676436    2584 flags.go:64] FLAG: --container-log-max-files="5"
Jul 31 22:56:56.683431 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676440    2584 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 31 22:56:56.683431 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676443    2584 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 31 22:56:56.683431 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676446    2584 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 31 22:56:56.683431 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676449    2584 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 31 22:56:56.683992 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676452    2584 flags.go:64] FLAG: --contention-profiling="false"
Jul 31 22:56:56.683992 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676454    2584 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 31 22:56:56.683992 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676459    2584 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 31 22:56:56.683992 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676462    2584 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 31 22:56:56.683992 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676465    2584 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 31 22:56:56.683992 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676469    2584 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 31 22:56:56.683992 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676472    2584 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 31 22:56:56.683992 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676475    2584 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 31 22:56:56.683992 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676478    2584 flags.go:64] FLAG: --enable-load-reader="false"
Jul 31 22:56:56.683992 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676480    2584 flags.go:64] FLAG: --enable-server="true"
Jul 31 22:56:56.683992 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676483    2584 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 31 22:56:56.683992 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676488    2584 flags.go:64] FLAG: --event-burst="100"
Jul 31 22:56:56.683992 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676491    2584 flags.go:64] FLAG: --event-qps="50"
Jul 31 22:56:56.683992 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676494    2584 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 31 22:56:56.683992 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676497    2584 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 31 22:56:56.683992 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676500    2584 flags.go:64] FLAG: --eviction-hard=""
Jul 31 22:56:56.683992 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676505    2584 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 31 22:56:56.683992 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676508    2584 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 31 22:56:56.683992 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676511    2584 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 31 22:56:56.683992 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676514    2584 flags.go:64] FLAG: --eviction-soft=""
Jul 31 22:56:56.683992 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676517    2584 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 31 22:56:56.683992 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676520    2584 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 31 22:56:56.683992 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676523    2584 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 31 22:56:56.683992 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676525    2584 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 31 22:56:56.683992 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676528    2584 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 31 22:56:56.684611 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676531    2584 flags.go:64] FLAG: --fail-swap-on="true"
Jul 31 22:56:56.684611 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676533    2584 flags.go:64] FLAG: --feature-gates=""
Jul 31 22:56:56.684611 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676537    2584 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 31 22:56:56.684611 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676540    2584 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 31 22:56:56.684611 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676543    2584 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 31 22:56:56.684611 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676546    2584 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 31 22:56:56.684611 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676549    2584 flags.go:64] FLAG: --healthz-port="10248"
Jul 31 22:56:56.684611 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676552    2584 flags.go:64] FLAG: --help="false"
Jul 31 22:56:56.684611 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676555    2584 flags.go:64] FLAG: --hostname-override="ip-10-0-128-243.ec2.internal"
Jul 31 22:56:56.684611 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676559    2584 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 31 22:56:56.684611 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676562    2584 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 31 22:56:56.684611 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676565    2584 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 31 22:56:56.684611 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676568    2584 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 31 22:56:56.684611 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676572    2584 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 31 22:56:56.684611 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676575    2584 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 31 22:56:56.684611 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676577    2584 flags.go:64] FLAG: --image-service-endpoint=""
Jul 31 22:56:56.684611 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676580    2584 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 31 22:56:56.684611 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676583    2584 flags.go:64] FLAG: --kube-api-burst="100"
Jul 31 22:56:56.684611 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676585    2584 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 31 22:56:56.684611 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676588    2584 flags.go:64] FLAG: --kube-api-qps="50"
Jul 31 22:56:56.684611 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676591    2584 flags.go:64] FLAG: --kube-reserved=""
Jul 31 22:56:56.684611 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676594    2584 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 31 22:56:56.684611 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676597    2584 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 31 22:56:56.684611 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676601    2584 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 31 22:56:56.685227 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676603    2584 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 31 22:56:56.685227 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676606    2584 flags.go:64] FLAG: --lock-file=""
Jul 31 22:56:56.685227 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676609    2584 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 31 22:56:56.685227 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676612    2584 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 31 22:56:56.685227 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676615    2584 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 31 22:56:56.685227 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676621    2584 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 31 22:56:56.685227 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676624    2584 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 31 22:56:56.685227 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676626    2584 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 31 22:56:56.685227 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676629    2584 flags.go:64] FLAG: --logging-format="text"
Jul 31 22:56:56.685227 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676632    2584 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 31 22:56:56.685227 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676635    2584 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 31 22:56:56.685227 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676638    2584 flags.go:64] FLAG: --manifest-url=""
Jul 31 22:56:56.685227 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676640    2584 flags.go:64] FLAG: --manifest-url-header=""
Jul 31 22:56:56.685227 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676644    2584 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 31 22:56:56.685227 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676647    2584 flags.go:64] FLAG: --max-open-files="1000000"
Jul 31 22:56:56.685227 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676651    2584 flags.go:64] FLAG: --max-pods="110"
Jul 31 22:56:56.685227 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676654    2584 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 31 22:56:56.685227 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676658    2584 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 31 22:56:56.685227 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676661    2584 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 31 22:56:56.685227 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676664    2584 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 31 22:56:56.685227 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676667    2584 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 31 22:56:56.685227 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676670    2584 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 31 22:56:56.685227 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676673    2584 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 31 22:56:56.685227 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676679    2584 flags.go:64] FLAG: --node-status-max-images="50"
Jul 31 22:56:56.685227 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676682    2584 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 31 22:56:56.685857 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676685    2584 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 31 22:56:56.685857 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676688    2584 flags.go:64] FLAG: --pod-cidr=""
Jul 31 22:56:56.685857 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676690    2584 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:267c8c33005f5dc3d4121af550b1c396e30e2928a13624d7362adfbfb7f440f8"
Jul 31 22:56:56.685857 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676711    2584 flags.go:64] FLAG: --pod-manifest-path=""
Jul 31 22:56:56.685857 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676716    2584 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 31 22:56:56.685857 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676720    2584 flags.go:64] FLAG: --pods-per-core="0"
Jul 31 22:56:56.685857 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676725    2584 flags.go:64] FLAG: --port="10250"
Jul 31 22:56:56.685857 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676728    2584 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 31 22:56:56.685857 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676731    2584 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0bf0bd5ee265f78af"
Jul 31 22:56:56.685857 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676734    2584 flags.go:64] FLAG: --qos-reserved=""
Jul 31 22:56:56.685857 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676737    2584 flags.go:64] FLAG: --read-only-port="10255"
Jul 31 22:56:56.685857 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676741    2584 flags.go:64] FLAG: --register-node="true"
Jul 31 22:56:56.685857 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676744    2584 flags.go:64] FLAG: --register-with-taints=""
Jul 31 22:56:56.685857 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676748    2584 flags.go:64] FLAG: --registry-burst="10"
Jul 31 22:56:56.685857 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676750    2584 flags.go:64] FLAG: --registry-qps="5"
Jul 31 22:56:56.685857 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676753    2584 flags.go:64] FLAG: --reserved-cpus=""
Jul 31 22:56:56.685857 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676756    2584 flags.go:64] FLAG: --reserved-memory=""
Jul 31 22:56:56.685857 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676759    2584 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 31 22:56:56.685857 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676762    2584 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 31 22:56:56.685857 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676766    2584 flags.go:64] FLAG: --rotate-certificates="false"
Jul 31 22:56:56.685857 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676769    2584 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 31 22:56:56.685857 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676772    2584 flags.go:64] FLAG: --runonce="false"
Jul 31 22:56:56.685857 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676774    2584 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 31 22:56:56.685857 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676778    2584 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 31 22:56:56.685857 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676780    2584 flags.go:64] FLAG: --seccomp-default="false"
Jul 31 22:56:56.686492 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676785    2584 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 31 22:56:56.686492 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676788    2584 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 31 22:56:56.686492 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676791    2584 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 31 22:56:56.686492 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676794    2584 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 31 22:56:56.686492 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676797    2584 flags.go:64] FLAG: --storage-driver-password="root"
Jul 31 22:56:56.686492 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676800    2584 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 31 22:56:56.686492 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676802    2584 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 31 22:56:56.686492 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676812    2584 flags.go:64] FLAG: --storage-driver-user="root"
Jul 31 22:56:56.686492 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676815    2584 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 31 22:56:56.686492 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676818    2584 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 31 22:56:56.686492 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676821    2584 flags.go:64] FLAG: --system-cgroups=""
Jul 31 22:56:56.686492 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676824    2584 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 31 22:56:56.686492 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676829    2584 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 31 22:56:56.686492 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676834    2584 flags.go:64] FLAG: --tls-cert-file=""
Jul 31 22:56:56.686492 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676837    2584 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 31 22:56:56.686492 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676841    2584 flags.go:64] FLAG: --tls-min-version=""
Jul 31 22:56:56.686492 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676844    2584 flags.go:64] FLAG: --tls-private-key-file=""
Jul 31 22:56:56.686492 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676847    2584 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 31 22:56:56.686492 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676850    2584 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 31 22:56:56.686492 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676853    2584 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 31 22:56:56.686492 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676856    2584 flags.go:64] FLAG: --v="2"
Jul 31 22:56:56.686492 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676860    2584 flags.go:64] FLAG: --version="false"
Jul 31 22:56:56.686492 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676870    2584 flags.go:64] FLAG: --vmodule=""
Jul 31 22:56:56.686492 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676874    2584 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 31 22:56:56.686492 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.676877    2584 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 31 22:56:56.687121 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.676990    2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 31 22:56:56.687121 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.676994    2584 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 31 22:56:56.687121 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.676997    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 31 22:56:56.687121 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677000    2584 feature_gate.go:328] unrecognized feature gate: Example2
Jul 31 22:56:56.687121 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677002    2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 31 22:56:56.687121 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677007    2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 31 22:56:56.687121 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677010    2584 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 31 22:56:56.687121 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677015    2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 31 22:56:56.687121 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677018    2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 31 22:56:56.687121 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677020    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 31 22:56:56.687121 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677023    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 31 22:56:56.687121 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677026    2584 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 31 22:56:56.687121 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677028    2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 31 22:56:56.687121 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677031    2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 31 22:56:56.687121 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677034    2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 31 22:56:56.687121 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677037    2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 31 22:56:56.687121 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677040    2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 31 22:56:56.687121 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677043    2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 31 22:56:56.687121 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677045    2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 31 22:56:56.687121 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677048    2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 31 22:56:56.687603 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677052    2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 31 22:56:56.687603 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677054    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 31 22:56:56.687603 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677057    2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 31 22:56:56.687603 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677059    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 31 22:56:56.687603 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677062    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 31 22:56:56.687603 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677065    2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 31 22:56:56.687603 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677068    2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 31 22:56:56.687603 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677072    2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 31 22:56:56.687603 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677075    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 31 22:56:56.687603 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677078    2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 31 22:56:56.687603 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677081    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 31 22:56:56.687603 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677083    2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 31 22:56:56.687603 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677086    2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 31 22:56:56.687603 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677089    2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 31 22:56:56.687603 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677091    2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 31 22:56:56.687603 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677095    2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 31 22:56:56.687603 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677100    2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 31 22:56:56.687603 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677104    2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 31 22:56:56.687603 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677108    2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 31 22:56:56.688079 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677114    2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 31 22:56:56.688079 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677118    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 31 22:56:56.688079 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677122    2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 31 22:56:56.688079 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677127    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 31 22:56:56.688079 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677131    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 31 22:56:56.688079 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677135    2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 31 22:56:56.688079 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677139    2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 31 22:56:56.688079 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677143    2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 31 22:56:56.688079 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677147    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 31 22:56:56.688079 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677151    2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 31 22:56:56.688079 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677155    2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 31 22:56:56.688079 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677160    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 31 22:56:56.688079 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677164    2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 31 22:56:56.688079 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677170    2584 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 31 22:56:56.688079 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677173    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 31 22:56:56.688079 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677177    2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 31 22:56:56.688079 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677181    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 31 22:56:56.688079 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677185    2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 31 22:56:56.688079 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677189    2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 31 22:56:56.688549 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677193    2584 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 31 22:56:56.688549 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677197    2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 31 22:56:56.688549 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677203    2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 31 22:56:56.688549 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677208    2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 31 22:56:56.688549 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677212    2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 31 22:56:56.688549 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677216    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 31 22:56:56.688549 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677221    2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 31 22:56:56.688549 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677225    2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 31 22:56:56.688549 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677230    2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 31 22:56:56.688549 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677234    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 31 22:56:56.688549 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677238    2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 31 22:56:56.688549 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677243    2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 31 22:56:56.688549 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677248    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 31 22:56:56.688549 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677253    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 31 22:56:56.688549 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677257    2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 31 22:56:56.688549 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677262    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 31 22:56:56.688549 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677266    2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 31 22:56:56.688549 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677270    2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 31 22:56:56.688549 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677274    2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 31 22:56:56.688549 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677278    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 31 22:56:56.689055 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677283    2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 31 22:56:56.689055 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677287    2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 31 22:56:56.689055 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677292    2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 31 22:56:56.689055 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677296    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 31 22:56:56.689055 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677300    2584 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 31 22:56:56.689055 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677304    2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 31 22:56:56.689055 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677311    2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 31 22:56:56.689055 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677315    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 31 22:56:56.689055 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677319    2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 31 22:56:56.689055 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677323    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 31 22:56:56.689055 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677328    2584 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 31 22:56:56.689055 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677332    2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 31 22:56:56.689055 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677337    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 31 22:56:56.689055 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677341    2584 feature_gate.go:328] unrecognized feature gate: Example
Jul 31 22:56:56.689055 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677346    2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 31 22:56:56.689055 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677352    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 31 22:56:56.689055 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677356    2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 31 22:56:56.689055 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677361    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 31 22:56:56.689055 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677365    2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 31 22:56:56.689055 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677369    2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 31 22:56:56.689609 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.677373    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 31 22:56:56.689609 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.677386    2584 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 31 22:56:56.689609 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.684977    2584 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 31 22:56:56.689609 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.684992    2584 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 31 22:56:56.689609 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.685011    2584 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 31 22:56:56.689609 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.685017    2584 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 31 22:56:56.689609 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685071    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 31 22:56:56.689609 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685076    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 31 22:56:56.689609 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685080    2584 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 31 22:56:56.689609 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685084    2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 31 22:56:56.689609 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685088    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 31 22:56:56.689609 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685092    2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 31 22:56:56.689609 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685095    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 31 22:56:56.689609 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685098    2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 31 22:56:56.689976 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685102    2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 31 22:56:56.689976 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685106    2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 31 22:56:56.689976 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685109    2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 31 22:56:56.689976 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685112    2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 31 22:56:56.689976 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685115    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 31 22:56:56.689976 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685118    2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 31 22:56:56.689976 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685121    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 31 22:56:56.689976 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685123    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 31 22:56:56.689976 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685126    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 31 22:56:56.689976 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685129    2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 31 22:56:56.689976 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685132    2584 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 31 22:56:56.689976 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685134    2584 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 31 22:56:56.689976 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685137    2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 31 22:56:56.689976 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685140    2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 31 22:56:56.689976 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685142    2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 31 22:56:56.689976 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685145    2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 31 22:56:56.689976 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685147    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 31 22:56:56.689976 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685150    2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 31 22:56:56.689976 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685153    2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 31 22:56:56.690424 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685155    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 31 22:56:56.690424 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685158    2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 31 22:56:56.690424 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685162    2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 31 22:56:56.690424 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685165    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 31 22:56:56.690424 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685168    2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 31 22:56:56.690424 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685171    2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 31 22:56:56.690424 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685174    2584 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 31 22:56:56.690424 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685176    2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 31 22:56:56.690424 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685179    2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 31 22:56:56.690424 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685181    2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 31 22:56:56.690424 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685184    2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 31 22:56:56.690424 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685186    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 31 22:56:56.690424 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685189    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 31 22:56:56.690424 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685192    2584 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 31 22:56:56.690424 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685194    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 31 22:56:56.690424 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685197    2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 31 22:56:56.690424 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685200    2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 31 22:56:56.690424 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685202    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 31 22:56:56.690424 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685205    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 31 22:56:56.690424 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685207    2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 31 22:56:56.690914 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685210    2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 31 22:56:56.690914 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685213    2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 31 22:56:56.690914 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685215    2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 31 22:56:56.690914 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685218    2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 31 22:56:56.690914 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685220    2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 31 22:56:56.690914 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685223    2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 31 22:56:56.690914 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685225    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 31 22:56:56.690914 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685228    2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 31 22:56:56.690914 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685230    2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 31 22:56:56.690914 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685233    2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 31 22:56:56.690914 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685236    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 31 22:56:56.690914 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685238    2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 31 22:56:56.690914 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685241    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 31 22:56:56.690914 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685244    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 31 22:56:56.690914 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685247    2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 31 22:56:56.690914 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685251    2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 31 22:56:56.690914 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685254    2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 31 22:56:56.690914 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685256    2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 31 22:56:56.690914 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685259    2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 31 22:56:56.690914 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685261    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 31 22:56:56.691384 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685264    2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 31 22:56:56.691384 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685266    2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 31 22:56:56.691384 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685269    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 31 22:56:56.691384 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685271    2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 31 22:56:56.691384 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685274    2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 31 22:56:56.691384 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685276    2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 31 22:56:56.691384 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685279    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 31 22:56:56.691384 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685281    2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 31 22:56:56.691384 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685284    2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 31 22:56:56.691384 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685286    2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 31 22:56:56.691384 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685289    2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 31 22:56:56.691384 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685291    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 31 22:56:56.691384 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685294    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 31 22:56:56.691384 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685296    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 31 22:56:56.691384 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685299    2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 31 22:56:56.691384 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685301    2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 31 22:56:56.691384 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685304    2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 31 22:56:56.691384 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685306    2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 31 22:56:56.691384 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685309    2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 31 22:56:56.691384 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685311    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 31 22:56:56.691934 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685314    2584 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 31 22:56:56.691934 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685316    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 31 22:56:56.691934 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685320    2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 31 22:56:56.691934 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685324    2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 31 22:56:56.691934 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685327    2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 31 22:56:56.691934 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685329    2584 feature_gate.go:328] unrecognized feature gate: Example2
Jul 31 22:56:56.691934 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685333    2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 31 22:56:56.691934 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685336    2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 31 22:56:56.691934 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685340    2584 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 31 22:56:56.691934 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685342    2584 feature_gate.go:328] unrecognized feature gate: Example
Jul 31 22:56:56.691934 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685345    2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 31 22:56:56.691934 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685348    2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 31 22:56:56.691934 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.685353    2584 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 31 22:56:56.691934 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685462    2584 feature_gate.go:328] unrecognized feature gate: Example2
Jul 31 22:56:56.691934 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685467    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 31 22:56:56.692324 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685470    2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 31 22:56:56.692324 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685472    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 31 22:56:56.692324 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685475    2584 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 31 22:56:56.692324 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685478    2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 31 22:56:56.692324 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685481    2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 31 22:56:56.692324 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685483    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 31 22:56:56.692324 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685486    2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 31 22:56:56.692324 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685488    2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 31 22:56:56.692324 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685491    2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 31 22:56:56.692324 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685493    2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 31 22:56:56.692324 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685496    2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 31 22:56:56.692324 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685498    2584 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 31 22:56:56.692324 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685502    2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 31 22:56:56.692324 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685524    2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 31 22:56:56.692324 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685528    2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 31 22:56:56.692324 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685531    2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 31 22:56:56.692324 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685534    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 31 22:56:56.692324 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685536    2584 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 31 22:56:56.692324 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685539    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 31 22:56:56.692797 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685542    2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 31 22:56:56.692797 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685546    2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 31 22:56:56.692797 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685549    2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 31 22:56:56.692797 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685552    2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 31 22:56:56.692797 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685555    2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 31 22:56:56.692797 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685558    2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 31 22:56:56.692797 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685560    2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 31 22:56:56.692797 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685563    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 31 22:56:56.692797 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685566    2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 31 22:56:56.692797 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685569    2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 31 22:56:56.692797 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685572    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 31 22:56:56.692797 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685574    2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 31 22:56:56.692797 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685577    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 31 22:56:56.692797 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685579    2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 31 22:56:56.692797 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685581    2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 31 22:56:56.692797 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685584    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 31 22:56:56.692797 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685586    2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 31 22:56:56.692797 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685589    2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 31 22:56:56.692797 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685592    2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 31 22:56:56.693266 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685594    2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 31 22:56:56.693266 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685597    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 31 22:56:56.693266 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685599    2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 31 22:56:56.693266 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685602    2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 31 22:56:56.693266 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685604    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 31 22:56:56.693266 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685607    2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 31 22:56:56.693266 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685609    2584 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 31 22:56:56.693266 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685612    2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 31 22:56:56.693266 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685614    2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 31 22:56:56.693266 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685617    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 31 22:56:56.693266 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685619    2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 31 22:56:56.693266 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685622    2584 feature_gate.go:328] unrecognized feature gate: Example
Jul 31 22:56:56.693266 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685624    2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 31 22:56:56.693266 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685626    2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 31 22:56:56.693266 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685629    2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 31 22:56:56.693266 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685631    2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 31 22:56:56.693266 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685634    2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 31 22:56:56.693266 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685637    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 31 22:56:56.693266 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685640    2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 31 22:56:56.693746 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685642    2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 31 22:56:56.693746 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685645    2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 31 22:56:56.693746 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685647    2584 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 31 22:56:56.693746 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685650    2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 31 22:56:56.693746 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685652    2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 31 22:56:56.693746 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685655    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 31 22:56:56.693746 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685658    2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 31 22:56:56.693746 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685660    2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 31 22:56:56.693746 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685663    2584 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 31 22:56:56.693746 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685665    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 31 22:56:56.693746 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685667    2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 31 22:56:56.693746 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685670    2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 31 22:56:56.693746 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685672    2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 31 22:56:56.693746 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685675    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 31 22:56:56.693746 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685677    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 31 22:56:56.693746 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685680    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 31 22:56:56.693746 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685682    2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 31 22:56:56.693746 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685685    2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 31 22:56:56.693746 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685687    2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 31 22:56:56.693746 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685690    2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 31 22:56:56.694470 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685692    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 31 22:56:56.694470 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685695    2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 31 22:56:56.694470 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685713    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 31 22:56:56.694470 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685715    2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 31 22:56:56.694470 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685718    2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 31 22:56:56.694470 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685721    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 31 22:56:56.694470 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685723    2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 31 22:56:56.694470 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685726    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 31 22:56:56.694470 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685728    2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 31 22:56:56.694470 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685731    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 31 22:56:56.694470 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685735    2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 31 22:56:56.694470 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685738    2584 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 31 22:56:56.694470 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685741    2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 31 22:56:56.694470 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685744    2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 31 22:56:56.694470 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685747    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 31 22:56:56.694470 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685750    2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 31 22:56:56.694470 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685753    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 31 22:56:56.694470 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685755    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 31 22:56:56.694470 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685758    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 31 22:56:56.695130 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:56.685760    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 31 22:56:56.695130 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.685765    2584 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 31 22:56:56.695130 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.685870    2584 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 31 22:56:56.695130 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.688130    2584 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 31 22:56:56.695130 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.689728    2584 server.go:1019] "Starting client certificate rotation"
Jul 31 22:56:56.695130 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.689825    2584 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 31 22:56:56.695130 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.689859    2584 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 31 22:56:56.695130 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.693485    2584 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 31 22:56:56.699567 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.699546    2584 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 31 22:56:56.707588 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.707567    2584 log.go:25] "Validated CRI v1 runtime API"
Jul 31 22:56:56.709803 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.709787    2584 log.go:25] "Validated CRI v1 image API"
Jul 31 22:56:56.710683 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.710663    2584 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 31 22:56:56.715131 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.715112    2584 fs.go:135] Filesystem UUIDs: map[371620a1-ed90-411b-8565-44d835838734:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 dc345b5b-3b64-4b02-8587-05d0ae551b61:/dev/nvme0n1p4]
Jul 31 22:56:56.715197 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.715131    2584 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 31 22:56:56.721183 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.721063    2584 manager.go:216] Machine: {Timestamp:2026-07-31 22:56:56.719592978 +0000 UTC m=+0.365379278 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3093168 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2914da45e14e3aeee5d2d2c3737f0c SystemUUID:ec2914da-45e1-4e3a-eee5-d2d2c3737f0c BootID:8581e063-7c90-4e39-9f5e-8a6aaccefc80 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:31:d1:ed:9f:9d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:31:d1:ed:9f:9d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:5e:b3:b2:a4:79:4f Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 31 22:56:56.721183 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.721175    2584 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 31 22:56:56.721291 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.721262    2584 manager.go:232] Version: {KernelVersion:5.14.0-570.131.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260727-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 31 22:56:56.725057 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.725033    2584 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 31 22:56:56.725195 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.725060    2584 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-243.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 31 22:56:56.725976 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.725965    2584 topology_manager.go:138] "Creating topology manager with none policy"
Jul 31 22:56:56.726018 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.725978    2584 container_manager_linux.go:309] "Creating device plugin manager"
Jul 31 22:56:56.726580 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.726570    2584 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 31 22:56:56.727331 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.727320    2584 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 31 22:56:56.728521 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.728511    2584 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 31 22:56:56.731730 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.731712    2584 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 31 22:56:56.733591 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.733578    2584 state_mem.go:36] "Initialized new in-memory state store"
Jul 31 22:56:56.733717 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.733690    2584 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 31 22:56:56.733778 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.733756    2584 kubelet.go:486] "Attempting to sync node with API server"
Jul 31 22:56:56.733778 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.733774    2584 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 31 22:56:56.733851 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.733790    2584 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 31 22:56:56.733851 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.733809    2584 kubelet.go:398] "Adding apiserver pod source"
Jul 31 22:56:56.733851 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.733822    2584 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 31 22:56:56.734255 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.734241    2584 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 31 22:56:56.734294 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.734270    2584 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 31 22:56:56.736983 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.736967    2584 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 31 22:56:56.738896 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.738879    2584 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 31 22:56:56.738955 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.738912    2584 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 31 22:56:56.740357 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.740344    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 31 22:56:56.740357 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.740360    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 31 22:56:56.740457 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.740366    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 31 22:56:56.740457 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.740372    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 31 22:56:56.740457 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.740380    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 31 22:56:56.740457 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.740386    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 31 22:56:56.740457 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.740397    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 31 22:56:56.740457 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.740411    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 31 22:56:56.740457 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.740424    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 31 22:56:56.740457 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.740429    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 31 22:56:56.740457 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.740441    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 31 22:56:56.740457 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.740447    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 31 22:56:56.742110 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.742099    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 31 22:56:56.742110 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.742110    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 31 22:56:56.744459 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.744420    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-243.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 31 22:56:56.744583 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:56:56.744562    2584 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-243.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 31 22:56:56.744630 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:56:56.744564    2584 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 31 22:56:56.746794 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.746780    2584 server.go:1268] "Started kubelet"
Jul 31 22:56:56.746953 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.746899    2584 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 31 22:56:56.747014 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.747006    2584 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 31 22:56:56.747058 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.747010    2584 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 31 22:56:56.747760 ip-10-0-128-243 systemd[1]: Started Kubernetes Kubelet.
Jul 31 22:56:56.748601 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.748581    2584 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 31 22:56:56.749404 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.749350    2584 server.go:310] "Adding debug handlers to kubelet server"
Jul 31 22:56:56.753296 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.753255    2584 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-b7gs5"
Jul 31 22:56:56.753885 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:56:56.753868    2584 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 31 22:56:56.757010 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:56:56.755878    2584 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-243.ec2.internal.18c781e967ed28ce  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-243.ec2.internal,UID:ip-10-0-128-243.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-243.ec2.internal,},FirstTimestamp:2026-07-31 22:56:56.746649806 +0000 UTC m=+0.392436107,LastTimestamp:2026-07-31 22:56:56.746649806 +0000 UTC m=+0.392436107,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-243.ec2.internal,}"
Jul 31 22:56:56.758798 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.758782    2584 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-b7gs5"
Jul 31 22:56:56.759601 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.759588    2584 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 31 22:56:56.759660 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.759588    2584 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 31 22:56:56.759723 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:56:56.759690    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-243.ec2.internal\" not found"
Jul 31 22:56:56.760337 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.760284    2584 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 31 22:56:56.760337 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.760306    2584 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 31 22:56:56.760526 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.760390    2584 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 31 22:56:56.760526 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.760401    2584 factory.go:55] Registering systemd factory
Jul 31 22:56:56.760526 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.760407    2584 factory.go:223] Registration of the systemd container factory successfully
Jul 31 22:56:56.760526 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.760415    2584 reconstruct.go:97] "Volume reconstruction finished"
Jul 31 22:56:56.760526 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.760426    2584 reconciler.go:29] "Reconciler: start to sync state"
Jul 31 22:56:56.760749 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.760561    2584 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 31 22:56:56.760749 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.760668    2584 factory.go:153] Registering CRI-O factory
Jul 31 22:56:56.760749 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.760692    2584 factory.go:223] Registration of the crio container factory successfully
Jul 31 22:56:56.760749 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.760727    2584 factory.go:103] Registering Raw factory
Jul 31 22:56:56.760749 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.760739    2584 manager.go:1196] Started watching for new ooms in manager
Jul 31 22:56:56.761194 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.761183    2584 manager.go:349] Starting recovery of all containers
Jul 31 22:56:56.769034 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.769007    2584 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 31 22:56:56.771749 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:56:56.771729    2584 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-128-243.ec2.internal\" not found" node="ip-10-0-128-243.ec2.internal"
Jul 31 22:56:56.774154 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.774101    2584 manager.go:354] Recovery completed
Jul 31 22:56:56.778376 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.778362    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 31 22:56:56.780104 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.780089    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-243.ec2.internal" event="NodeHasSufficientMemory"
Jul 31 22:56:56.780188 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.780120    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-243.ec2.internal" event="NodeHasNoDiskPressure"
Jul 31 22:56:56.780188 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.780134    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-243.ec2.internal" event="NodeHasSufficientPID"
Jul 31 22:56:56.780716 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.780684    2584 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 31 22:56:56.780716 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.780710    2584 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 31 22:56:56.780837 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.780729    2584 state_mem.go:36] "Initialized new in-memory state store"
Jul 31 22:56:56.780837 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.780758    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 31 22:56:56.781427 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.781414    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-243.ec2.internal" event="NodeHasSufficientMemory"
Jul 31 22:56:56.781475 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.781441    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-243.ec2.internal" event="NodeHasNoDiskPressure"
Jul 31 22:56:56.781475 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.781454    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-243.ec2.internal" event="NodeHasSufficientPID"
Jul 31 22:56:56.782933 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.782919    2584 policy_none.go:49] "None policy: Start"
Jul 31 22:56:56.782995 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.782938    2584 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 31 22:56:56.782995 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.782951    2584 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 31 22:56:56.785051 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.785037    2584 policy_none.go:47] "Start"
Jul 31 22:56:56.835444 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.826579    2584 manager.go:337] "Starting Device Plugin manager"
Jul 31 22:56:56.835444 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:56:56.826614    2584 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 31 22:56:56.835444 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.826625    2584 server.go:87] "Starting device plugin registration server"
Jul 31 22:56:56.835444 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.826921    2584 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 31 22:56:56.835444 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.826933    2584 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 31 22:56:56.835444 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.827022    2584 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 31 22:56:56.835444 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.827114    2584 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 31 22:56:56.835444 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.827124    2584 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 31 22:56:56.835444 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:56:56.827663    2584 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 31 22:56:56.835444 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:56:56.827725    2584 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-243.ec2.internal\" not found"
Jul 31 22:56:56.883468 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.883436    2584 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 31 22:56:56.884565 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.884538    2584 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 31 22:56:56.884675 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.884575    2584 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 31 22:56:56.884675 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.884596    2584 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 31 22:56:56.884675 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:56:56.884631    2584 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 31 22:56:56.887203 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.887179    2584 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 31 22:56:56.928091 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.928044    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 31 22:56:56.928887 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.928870    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-243.ec2.internal" event="NodeHasSufficientMemory"
Jul 31 22:56:56.928961 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.928900    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-243.ec2.internal" event="NodeHasNoDiskPressure"
Jul 31 22:56:56.928961 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.928914    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-243.ec2.internal" event="NodeHasSufficientPID"
Jul 31 22:56:56.928961 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.928943    2584 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-243.ec2.internal"
Jul 31 22:56:56.937565 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.937423    2584 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-243.ec2.internal"
Jul 31 22:56:56.937565 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:56:56.937566    2584 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-243.ec2.internal\": node \"ip-10-0-128-243.ec2.internal\" not found"
Jul 31 22:56:56.948518 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:56:56.948495    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-243.ec2.internal\" not found"
Jul 31 22:56:56.985741 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.985721    2584 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-243.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-243.ec2.internal"]
Jul 31 22:56:56.985802 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.985775    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 31 22:56:56.986552 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.986534    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-243.ec2.internal" event="NodeHasSufficientMemory"
Jul 31 22:56:56.986622 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.986560    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-243.ec2.internal" event="NodeHasNoDiskPressure"
Jul 31 22:56:56.986622 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.986570    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-243.ec2.internal" event="NodeHasSufficientPID"
Jul 31 22:56:56.988803 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.988792    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 31 22:56:56.988941 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.988925    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-243.ec2.internal"
Jul 31 22:56:56.988981 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.988961    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 31 22:56:56.990068 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.990051    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-243.ec2.internal" event="NodeHasSufficientMemory"
Jul 31 22:56:56.990152 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.990081    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-243.ec2.internal" event="NodeHasNoDiskPressure"
Jul 31 22:56:56.990152 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.990092    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-243.ec2.internal" event="NodeHasSufficientPID"
Jul 31 22:56:56.990152 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.990053    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-243.ec2.internal" event="NodeHasSufficientMemory"
Jul 31 22:56:56.990256 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.990155    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-243.ec2.internal" event="NodeHasNoDiskPressure"
Jul 31 22:56:56.990256 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.990168    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-243.ec2.internal" event="NodeHasSufficientPID"
Jul 31 22:56:56.992354 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.992339    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-243.ec2.internal"
Jul 31 22:56:56.992402 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.992370    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 31 22:56:56.993557 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.993542    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-243.ec2.internal" event="NodeHasSufficientMemory"
Jul 31 22:56:56.993638 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.993569    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-243.ec2.internal" event="NodeHasNoDiskPressure"
Jul 31 22:56:56.993638 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:56.993583    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-243.ec2.internal" event="NodeHasSufficientPID"
Jul 31 22:56:57.016232 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:56:57.016206    2584 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-243.ec2.internal\" not found" node="ip-10-0-128-243.ec2.internal"
Jul 31 22:56:57.019521 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:56:57.019504    2584 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-243.ec2.internal\" not found" node="ip-10-0-128-243.ec2.internal"
Jul 31 22:56:57.048777 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:56:57.048750    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-243.ec2.internal\" not found"
Jul 31 22:56:57.062532 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.062511    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/59797420703085b3d07401083a2a0810-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-243.ec2.internal\" (UID: \"59797420703085b3d07401083a2a0810\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-243.ec2.internal"
Jul 31 22:56:57.062609 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.062542    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/59797420703085b3d07401083a2a0810-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-243.ec2.internal\" (UID: \"59797420703085b3d07401083a2a0810\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-243.ec2.internal"
Jul 31 22:56:57.062609 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.062561    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bf2bb11c3449d46ed43b4ff9c8ef237d-config\") pod \"kube-apiserver-proxy-ip-10-0-128-243.ec2.internal\" (UID: \"bf2bb11c3449d46ed43b4ff9c8ef237d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-243.ec2.internal"
Jul 31 22:56:57.149848 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:56:57.149812    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-243.ec2.internal\" not found"
Jul 31 22:56:57.163212 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.163192    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/59797420703085b3d07401083a2a0810-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-243.ec2.internal\" (UID: \"59797420703085b3d07401083a2a0810\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-243.ec2.internal"
Jul 31 22:56:57.163282 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.163218    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/59797420703085b3d07401083a2a0810-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-243.ec2.internal\" (UID: \"59797420703085b3d07401083a2a0810\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-243.ec2.internal"
Jul 31 22:56:57.163282 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.163235    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bf2bb11c3449d46ed43b4ff9c8ef237d-config\") pod \"kube-apiserver-proxy-ip-10-0-128-243.ec2.internal\" (UID: \"bf2bb11c3449d46ed43b4ff9c8ef237d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-243.ec2.internal"
Jul 31 22:56:57.163379 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.163280    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bf2bb11c3449d46ed43b4ff9c8ef237d-config\") pod \"kube-apiserver-proxy-ip-10-0-128-243.ec2.internal\" (UID: \"bf2bb11c3449d46ed43b4ff9c8ef237d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-243.ec2.internal"
Jul 31 22:56:57.163379 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.163311    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/59797420703085b3d07401083a2a0810-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-243.ec2.internal\" (UID: \"59797420703085b3d07401083a2a0810\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-243.ec2.internal"
Jul 31 22:56:57.163379 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.163318    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/59797420703085b3d07401083a2a0810-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-243.ec2.internal\" (UID: \"59797420703085b3d07401083a2a0810\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-243.ec2.internal"
Jul 31 22:56:57.250717 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:56:57.250633    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-243.ec2.internal\" not found"
Jul 31 22:56:57.319135 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.319113    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-243.ec2.internal"
Jul 31 22:56:57.322301 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.322274    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-243.ec2.internal"
Jul 31 22:56:57.351754 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:56:57.351725    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-243.ec2.internal\" not found"
Jul 31 22:56:57.453315 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:56:57.453267    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-243.ec2.internal\" not found"
Jul 31 22:56:57.554863 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:56:57.554774    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-243.ec2.internal\" not found"
Jul 31 22:56:57.629690 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.629656    2584 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 31 22:56:57.660793 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.660767    2584 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-243.ec2.internal"
Jul 31 22:56:57.671535 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.671516    2584 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 31 22:56:57.673482 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.673467    2584 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-243.ec2.internal"
Jul 31 22:56:57.687348 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.687326    2584 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 31 22:56:57.688506 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.688488    2584 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 31 22:56:57.688624 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.688607    2584 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 31 22:56:57.688663 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.688648    2584 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 31 22:56:57.688697 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.688654    2584 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 31 22:56:57.734019 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.733997    2584 apiserver.go:52] "Watching apiserver"
Jul 31 22:56:57.752416 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.752395    2584 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 31 22:56:57.753953 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.753487    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-mqwlt","kube-system/konnectivity-agent-shjgx","kube-system/kube-apiserver-proxy-ip-10-0-128-243.ec2.internal","openshift-cluster-node-tuning-operator/tuned-9wt4q","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-243.ec2.internal","openshift-multus/multus-pbslg","openshift-multus/network-metrics-daemon-mz5wz","openshift-ovn-kubernetes/ovnkube-node-w96jg","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bk57","openshift-dns/node-resolver-llmj8","openshift-image-registry/node-ca-t8qkd","openshift-multus/multus-additional-cni-plugins-2trlm","openshift-network-diagnostics/network-check-target-84css","openshift-network-operator/iptables-alerter-qchlw"]
Jul 31 22:56:57.756351 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.756332    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mz5wz"
Jul 31 22:56:57.756447 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:56:57.756398    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mz5wz" podUID="b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945"
Jul 31 22:56:57.759130 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.759109    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-t8qkd"
Jul 31 22:56:57.759829 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.759775    2584 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 31 22:56:57.761667 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.761650    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 31 22:56:57.761802 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.761784    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 31 22:56:57.761913 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.761894    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 31 22:56:57.762131 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.762115    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qchlw"
Jul 31 22:56:57.762333 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.762312    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-xvtzz\""
Jul 31 22:56:57.764525 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.764507    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-57n9m\""
Jul 31 22:56:57.764627 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.764613    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 31 22:56:57.764627 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.764619    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 31 22:56:57.764745 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.764641    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 31 22:56:57.764875 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.764839    2584 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-30 22:51:57 +0000 UTC" deadline="2028-01-14 21:10:46.42345534 +0000 UTC"
Jul 31 22:56:57.764928 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.764883    2584 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12766h13m48.658578318s"
Jul 31 22:56:57.764961 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.764935    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9wt4q"
Jul 31 22:56:57.766643 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.766623    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945-metrics-certs\") pod \"network-metrics-daemon-mz5wz\" (UID: \"b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945\") " pod="openshift-multus/network-metrics-daemon-mz5wz"
Jul 31 22:56:57.766789 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.766650    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0bb4d7d9-ff22-41eb-8883-b7c1942767fb-serviceca\") pod \"node-ca-t8qkd\" (UID: \"0bb4d7d9-ff22-41eb-8883-b7c1942767fb\") " pod="openshift-image-registry/node-ca-t8qkd"
Jul 31 22:56:57.766789 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.766681    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44r6z\" (UniqueName: \"kubernetes.io/projected/0bb4d7d9-ff22-41eb-8883-b7c1942767fb-kube-api-access-44r6z\") pod \"node-ca-t8qkd\" (UID: \"0bb4d7d9-ff22-41eb-8883-b7c1942767fb\") " pod="openshift-image-registry/node-ca-t8qkd"
Jul 31 22:56:57.766789 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.766709    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9a9cc748-9a60-4656-9236-34d75e5cb22a-host-slash\") pod \"iptables-alerter-qchlw\" (UID: \"9a9cc748-9a60-4656-9236-34d75e5cb22a\") " pod="openshift-network-operator/iptables-alerter-qchlw"
Jul 31 22:56:57.766789 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.766753    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pps5d\" (UniqueName: \"kubernetes.io/projected/9a9cc748-9a60-4656-9236-34d75e5cb22a-kube-api-access-pps5d\") pod \"iptables-alerter-qchlw\" (UID: \"9a9cc748-9a60-4656-9236-34d75e5cb22a\") " pod="openshift-network-operator/iptables-alerter-qchlw"
Jul 31 22:56:57.766947 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.766804    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfnz4\" (UniqueName: \"kubernetes.io/projected/b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945-kube-api-access-lfnz4\") pod \"network-metrics-daemon-mz5wz\" (UID: \"b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945\") " pod="openshift-multus/network-metrics-daemon-mz5wz"
Jul 31 22:56:57.766947 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.766837    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0bb4d7d9-ff22-41eb-8883-b7c1942767fb-host\") pod \"node-ca-t8qkd\" (UID: \"0bb4d7d9-ff22-41eb-8883-b7c1942767fb\") " pod="openshift-image-registry/node-ca-t8qkd"
Jul 31 22:56:57.766947 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.766858    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9a9cc748-9a60-4656-9236-34d75e5cb22a-iptables-alerter-script\") pod \"iptables-alerter-qchlw\" (UID: \"9a9cc748-9a60-4656-9236-34d75e5cb22a\") " pod="openshift-network-operator/iptables-alerter-qchlw"
Jul 31 22:56:57.767464 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.767450    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-56w8s\""
Jul 31 22:56:57.767750 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.767724    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 31 22:56:57.767829 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.767778    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 31 22:56:57.768041 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.768029    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-w96jg"
Jul 31 22:56:57.770360 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.770342    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 31 22:56:57.770455 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.770427    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-f5q7j\""
Jul 31 22:56:57.770520 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.770462    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 31 22:56:57.770588 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.770516    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 31 22:56:57.770664 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.770648    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 31 22:56:57.770787 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.770758    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 31 22:56:57.770876 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.770860    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-pbslg"
Jul 31 22:56:57.771098 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.771079    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 31 22:56:57.773506 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.773488    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 31 22:56:57.773609 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.773509    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-6g9hb\""
Jul 31 22:56:57.773609 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.773534    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 31 22:56:57.773609 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.773511    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 31 22:56:57.773800 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.773772    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bk57"
Jul 31 22:56:57.774031 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.774011    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 31 22:56:57.776613 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.776593    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-84css"
Jul 31 22:56:57.776737 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:56:57.776647    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-84css" podUID="b80644e6-82bb-4c59-86ed-b60bd24e6a05"
Jul 31 22:56:57.776867 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.776848    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 31 22:56:57.776921 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.776910    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 31 22:56:57.777002 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.776981    2584 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 31 22:56:57.777375 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.777350    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 31 22:56:57.777512 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.777496    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-2wmdp\""
Jul 31 22:56:57.779619 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.779604    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-shjgx"
Jul 31 22:56:57.781870 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.781851    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 31 22:56:57.784309 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.782201    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-g8hxx\""
Jul 31 22:56:57.784309 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.782205    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 31 22:56:57.784309 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.783923    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-llmj8"
Jul 31 22:56:57.786274 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.786231    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mqwlt"
Jul 31 22:56:57.786274 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.786253    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 31 22:56:57.786470 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:56:57.786289    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mqwlt" podUID="b85a1443-4bc2-48a0-8d29-35f46cbc3d57"
Jul 31 22:56:57.786606 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.786589    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-qwzbq\""
Jul 31 22:56:57.786669 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.786637    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 31 22:56:57.789096 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.789075    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2trlm"
Jul 31 22:56:57.791822 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.791801    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-zr898\""
Jul 31 22:56:57.791929 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.791867    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 31 22:56:57.791929 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.791890    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 31 22:56:57.802850 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.802831    2584 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-pvf4t"
Jul 31 22:56:57.811756 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.811713    2584 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-pvf4t"
Jul 31 22:56:57.861870 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.861848    2584 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 31 22:56:57.867076 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.867059    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/87c830a4-dd74-43c5-84d0-45cdd2dbe9a3-host-run-netns\") pod \"ovnkube-node-w96jg\" (UID: \"87c830a4-dd74-43c5-84d0-45cdd2dbe9a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w96jg"
Jul 31 22:56:57.867166 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.867089    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/639fb18d-e947-41cc-ac53-d4616b7cc85b-multus-socket-dir-parent\") pod \"multus-pbslg\" (UID: \"639fb18d-e947-41cc-ac53-d4616b7cc85b\") " pod="openshift-multus/multus-pbslg"
Jul 31 22:56:57.867166 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.867116    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1d0f0d49-bc8d-489a-932a-de4c94edb6a4-lib-modules\") pod \"tuned-9wt4q\" (UID: \"1d0f0d49-bc8d-489a-932a-de4c94edb6a4\") " pod="openshift-cluster-node-tuning-operator/tuned-9wt4q"
Jul 31 22:56:57.867166 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.867149    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3f91895a-1a8c-4ef9-8f27-0cfed935fae8-registration-dir\") pod \"aws-ebs-csi-driver-node-7bk57\" (UID: \"3f91895a-1a8c-4ef9-8f27-0cfed935fae8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bk57"
Jul 31 22:56:57.867324 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.867170    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b85a1443-4bc2-48a0-8d29-35f46cbc3d57-dbus\") pod \"global-pull-secret-syncer-mqwlt\" (UID: \"b85a1443-4bc2-48a0-8d29-35f46cbc3d57\") " pod="kube-system/global-pull-secret-syncer-mqwlt"
Jul 31 22:56:57.867324 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.867232    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/eb4e0ba5-fe8b-4bf7-a86d-15db1a2532b2-agent-certs\") pod \"konnectivity-agent-shjgx\" (UID: \"eb4e0ba5-fe8b-4bf7-a86d-15db1a2532b2\") " pod="kube-system/konnectivity-agent-shjgx"
Jul 31 22:56:57.867324 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.867254    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/eb4e0ba5-fe8b-4bf7-a86d-15db1a2532b2-konnectivity-ca\") pod \"konnectivity-agent-shjgx\" (UID: \"eb4e0ba5-fe8b-4bf7-a86d-15db1a2532b2\") " pod="kube-system/konnectivity-agent-shjgx"
Jul 31 22:56:57.867324 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.867280    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1d0f0d49-bc8d-489a-932a-de4c94edb6a4-etc-systemd\") pod \"tuned-9wt4q\" (UID: \"1d0f0d49-bc8d-489a-932a-de4c94edb6a4\") " pod="openshift-cluster-node-tuning-operator/tuned-9wt4q"
Jul 31 22:56:57.867324 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.867310    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/87c830a4-dd74-43c5-84d0-45cdd2dbe9a3-run-systemd\") pod \"ovnkube-node-w96jg\" (UID: \"87c830a4-dd74-43c5-84d0-45cdd2dbe9a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w96jg"
Jul 31 22:56:57.867591 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.867327    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87c830a4-dd74-43c5-84d0-45cdd2dbe9a3-run-openvswitch\") pod \"ovnkube-node-w96jg\" (UID: \"87c830a4-dd74-43c5-84d0-45cdd2dbe9a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w96jg"
Jul 31 22:56:57.867591 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.867346    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/639fb18d-e947-41cc-ac53-d4616b7cc85b-host-var-lib-cni-bin\") pod \"multus-pbslg\" (UID: \"639fb18d-e947-41cc-ac53-d4616b7cc85b\") " pod="openshift-multus/multus-pbslg"
Jul 31 22:56:57.867591 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.867372    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/148fa8ca-4d3b-4ae1-8be5-30ea87aa412e-hosts-file\") pod \"node-resolver-llmj8\" (UID: \"148fa8ca-4d3b-4ae1-8be5-30ea87aa412e\") " pod="openshift-dns/node-resolver-llmj8"
Jul 31 22:56:57.867591 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.867403    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1d0f0d49-bc8d-489a-932a-de4c94edb6a4-host\") pod \"tuned-9wt4q\" (UID: \"1d0f0d49-bc8d-489a-932a-de4c94edb6a4\") " pod="openshift-cluster-node-tuning-operator/tuned-9wt4q"
Jul 31 22:56:57.867591 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.867428    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b85a1443-4bc2-48a0-8d29-35f46cbc3d57-original-pull-secret\") pod \"global-pull-secret-syncer-mqwlt\" (UID: \"b85a1443-4bc2-48a0-8d29-35f46cbc3d57\") " pod="kube-system/global-pull-secret-syncer-mqwlt"
Jul 31 22:56:57.867591 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.867450    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfnz4\" (UniqueName: \"kubernetes.io/projected/b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945-kube-api-access-lfnz4\") pod \"network-metrics-daemon-mz5wz\" (UID: \"b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945\") " pod="openshift-multus/network-metrics-daemon-mz5wz"
Jul 31 22:56:57.867591 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.867476    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9a9cc748-9a60-4656-9236-34d75e5cb22a-iptables-alerter-script\") pod \"iptables-alerter-qchlw\" (UID: \"9a9cc748-9a60-4656-9236-34d75e5cb22a\") " pod="openshift-network-operator/iptables-alerter-qchlw"
Jul 31 22:56:57.867591 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.867575    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/639fb18d-e947-41cc-ac53-d4616b7cc85b-system-cni-dir\") pod \"multus-pbslg\" (UID: \"639fb18d-e947-41cc-ac53-d4616b7cc85b\") " pod="openshift-multus/multus-pbslg"
Jul 31 22:56:57.867591 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.867594    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/639fb18d-e947-41cc-ac53-d4616b7cc85b-multus-cni-dir\") pod \"multus-pbslg\" (UID: \"639fb18d-e947-41cc-ac53-d4616b7cc85b\") " pod="openshift-multus/multus-pbslg"
Jul 31 22:56:57.868196 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.867611    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/639fb18d-e947-41cc-ac53-d4616b7cc85b-cnibin\") pod \"multus-pbslg\" (UID: \"639fb18d-e947-41cc-ac53-d4616b7cc85b\") " pod="openshift-multus/multus-pbslg"
Jul 31 22:56:57.868196 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.867626    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/639fb18d-e947-41cc-ac53-d4616b7cc85b-multus-conf-dir\") pod \"multus-pbslg\" (UID: \"639fb18d-e947-41cc-ac53-d4616b7cc85b\") " pod="openshift-multus/multus-pbslg"
Jul 31 22:56:57.868196 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.867645    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1d0f0d49-bc8d-489a-932a-de4c94edb6a4-run\") pod \"tuned-9wt4q\" (UID: \"1d0f0d49-bc8d-489a-932a-de4c94edb6a4\") " pod="openshift-cluster-node-tuning-operator/tuned-9wt4q"
Jul 31 22:56:57.868196 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.867667    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1d0f0d49-bc8d-489a-932a-de4c94edb6a4-var-lib-kubelet\") pod \"tuned-9wt4q\" (UID: \"1d0f0d49-bc8d-489a-932a-de4c94edb6a4\") " pod="openshift-cluster-node-tuning-operator/tuned-9wt4q"
Jul 31 22:56:57.868196 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.867692    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1d0f0d49-bc8d-489a-932a-de4c94edb6a4-etc-kubernetes\") pod \"tuned-9wt4q\" (UID: \"1d0f0d49-bc8d-489a-932a-de4c94edb6a4\") " pod="openshift-cluster-node-tuning-operator/tuned-9wt4q"
Jul 31 22:56:57.868196 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.867742    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0bb4d7d9-ff22-41eb-8883-b7c1942767fb-serviceca\") pod \"node-ca-t8qkd\" (UID: \"0bb4d7d9-ff22-41eb-8883-b7c1942767fb\") " pod="openshift-image-registry/node-ca-t8qkd"
Jul 31 22:56:57.868196 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.867800    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/87c830a4-dd74-43c5-84d0-45cdd2dbe9a3-ovn-node-metrics-cert\") pod \"ovnkube-node-w96jg\" (UID: \"87c830a4-dd74-43c5-84d0-45cdd2dbe9a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w96jg"
Jul 31 22:56:57.868196 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.867839    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/639fb18d-e947-41cc-ac53-d4616b7cc85b-host-var-lib-kubelet\") pod \"multus-pbslg\" (UID: \"639fb18d-e947-41cc-ac53-d4616b7cc85b\") " pod="openshift-multus/multus-pbslg"
Jul 31 22:56:57.868196 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.867874    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/639fb18d-e947-41cc-ac53-d4616b7cc85b-hostroot\") pod \"multus-pbslg\" (UID: \"639fb18d-e947-41cc-ac53-d4616b7cc85b\") " pod="openshift-multus/multus-pbslg"
Jul 31 22:56:57.868196 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.867899    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/639fb18d-e947-41cc-ac53-d4616b7cc85b-multus-daemon-config\") pod \"multus-pbslg\" (UID: \"639fb18d-e947-41cc-ac53-d4616b7cc85b\") " pod="openshift-multus/multus-pbslg"
Jul 31 22:56:57.868196 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.867947    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2m5pb\" (UniqueName: \"kubernetes.io/projected/639fb18d-e947-41cc-ac53-d4616b7cc85b-kube-api-access-2m5pb\") pod \"multus-pbslg\" (UID: \"639fb18d-e947-41cc-ac53-d4616b7cc85b\") " pod="openshift-multus/multus-pbslg"
Jul 31 22:56:57.868196 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.867974    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3f91895a-1a8c-4ef9-8f27-0cfed935fae8-sys-fs\") pod \"aws-ebs-csi-driver-node-7bk57\" (UID: \"3f91895a-1a8c-4ef9-8f27-0cfed935fae8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bk57"
Jul 31 22:56:57.868196 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.868000    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/87c830a4-dd74-43c5-84d0-45cdd2dbe9a3-host-slash\") pod \"ovnkube-node-w96jg\" (UID: \"87c830a4-dd74-43c5-84d0-45cdd2dbe9a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w96jg"
Jul 31 22:56:57.868196 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.868048    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/87c830a4-dd74-43c5-84d0-45cdd2dbe9a3-host-cni-bin\") pod \"ovnkube-node-w96jg\" (UID: \"87c830a4-dd74-43c5-84d0-45cdd2dbe9a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w96jg"
Jul 31 22:56:57.868196 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.868077    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4c7c83f2-713b-4651-b1da-f65c84f8edc9-cni-binary-copy\") pod \"multus-additional-cni-plugins-2trlm\" (UID: \"4c7c83f2-713b-4651-b1da-f65c84f8edc9\") " pod="openshift-multus/multus-additional-cni-plugins-2trlm"
Jul 31 22:56:57.868196 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.868104    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4c7c83f2-713b-4651-b1da-f65c84f8edc9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2trlm\" (UID: \"4c7c83f2-713b-4651-b1da-f65c84f8edc9\") " pod="openshift-multus/multus-additional-cni-plugins-2trlm"
Jul 31 22:56:57.868196 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.868141    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9a9cc748-9a60-4656-9236-34d75e5cb22a-iptables-alerter-script\") pod \"iptables-alerter-qchlw\" (UID: \"9a9cc748-9a60-4656-9236-34d75e5cb22a\") " pod="openshift-network-operator/iptables-alerter-qchlw"
Jul 31 22:56:57.868902 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.868175    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/87c830a4-dd74-43c5-84d0-45cdd2dbe9a3-host-kubelet\") pod \"ovnkube-node-w96jg\" (UID: \"87c830a4-dd74-43c5-84d0-45cdd2dbe9a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w96jg"
Jul 31 22:56:57.868902 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.868196    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0bb4d7d9-ff22-41eb-8883-b7c1942767fb-serviceca\") pod \"node-ca-t8qkd\" (UID: \"0bb4d7d9-ff22-41eb-8883-b7c1942767fb\") " pod="openshift-image-registry/node-ca-t8qkd"
Jul 31 22:56:57.868902 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.868200    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87c830a4-dd74-43c5-84d0-45cdd2dbe9a3-host-run-ovn-kubernetes\") pod \"ovnkube-node-w96jg\" (UID: \"87c830a4-dd74-43c5-84d0-45cdd2dbe9a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w96jg"
Jul 31 22:56:57.868902 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.868255    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3f91895a-1a8c-4ef9-8f27-0cfed935fae8-device-dir\") pod \"aws-ebs-csi-driver-node-7bk57\" (UID: \"3f91895a-1a8c-4ef9-8f27-0cfed935fae8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bk57"
Jul 31 22:56:57.868902 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.868284    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwbjs\" (UniqueName: \"kubernetes.io/projected/b80644e6-82bb-4c59-86ed-b60bd24e6a05-kube-api-access-bwbjs\") pod \"network-check-target-84css\" (UID: \"b80644e6-82bb-4c59-86ed-b60bd24e6a05\") " pod="openshift-network-diagnostics/network-check-target-84css"
Jul 31 22:56:57.868902 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.868314    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4c7c83f2-713b-4651-b1da-f65c84f8edc9-system-cni-dir\") pod \"multus-additional-cni-plugins-2trlm\" (UID: \"4c7c83f2-713b-4651-b1da-f65c84f8edc9\") " pod="openshift-multus/multus-additional-cni-plugins-2trlm"
Jul 31 22:56:57.868902 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.868339    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4c7c83f2-713b-4651-b1da-f65c84f8edc9-cnibin\") pod \"multus-additional-cni-plugins-2trlm\" (UID: \"4c7c83f2-713b-4651-b1da-f65c84f8edc9\") " pod="openshift-multus/multus-additional-cni-plugins-2trlm"
Jul 31 22:56:57.868902 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.868366    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0bb4d7d9-ff22-41eb-8883-b7c1942767fb-host\") pod \"node-ca-t8qkd\" (UID: \"0bb4d7d9-ff22-41eb-8883-b7c1942767fb\") " pod="openshift-image-registry/node-ca-t8qkd"
Jul 31 22:56:57.868902 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.868400    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/639fb18d-e947-41cc-ac53-d4616b7cc85b-cni-binary-copy\") pod \"multus-pbslg\" (UID: \"639fb18d-e947-41cc-ac53-d4616b7cc85b\") " pod="openshift-multus/multus-pbslg"
Jul 31 22:56:57.868902 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.868431    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/639fb18d-e947-41cc-ac53-d4616b7cc85b-host-run-multus-certs\") pod \"multus-pbslg\" (UID: \"639fb18d-e947-41cc-ac53-d4616b7cc85b\") " pod="openshift-multus/multus-pbslg"
Jul 31 22:56:57.868902 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.868451    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0bb4d7d9-ff22-41eb-8883-b7c1942767fb-host\") pod \"node-ca-t8qkd\" (UID: \"0bb4d7d9-ff22-41eb-8883-b7c1942767fb\") " pod="openshift-image-registry/node-ca-t8qkd"
Jul 31 22:56:57.868902 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.868456    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/87c830a4-dd74-43c5-84d0-45cdd2dbe9a3-run-ovn\") pod \"ovnkube-node-w96jg\" (UID: \"87c830a4-dd74-43c5-84d0-45cdd2dbe9a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w96jg"
Jul 31 22:56:57.868902 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.868501    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1d0f0d49-bc8d-489a-932a-de4c94edb6a4-tmp\") pod \"tuned-9wt4q\" (UID: \"1d0f0d49-bc8d-489a-932a-de4c94edb6a4\") " pod="openshift-cluster-node-tuning-operator/tuned-9wt4q"
Jul 31 22:56:57.868902 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.868523    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1d0f0d49-bc8d-489a-932a-de4c94edb6a4-etc-sysconfig\") pod \"tuned-9wt4q\" (UID: \"1d0f0d49-bc8d-489a-932a-de4c94edb6a4\") " pod="openshift-cluster-node-tuning-operator/tuned-9wt4q"
Jul 31 22:56:57.868902 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.868548    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/87c830a4-dd74-43c5-84d0-45cdd2dbe9a3-env-overrides\") pod \"ovnkube-node-w96jg\" (UID: \"87c830a4-dd74-43c5-84d0-45cdd2dbe9a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w96jg"
Jul 31 22:56:57.868902 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.868588    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87c830a4-dd74-43c5-84d0-45cdd2dbe9a3-var-lib-openvswitch\") pod \"ovnkube-node-w96jg\" (UID: \"87c830a4-dd74-43c5-84d0-45cdd2dbe9a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w96jg"
Jul 31 22:56:57.868902 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.868618    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/87c830a4-dd74-43c5-84d0-45cdd2dbe9a3-host-cni-netd\") pod \"ovnkube-node-w96jg\" (UID: \"87c830a4-dd74-43c5-84d0-45cdd2dbe9a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w96jg"
Jul 31 22:56:57.869544 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.868643    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/148fa8ca-4d3b-4ae1-8be5-30ea87aa412e-tmp-dir\") pod \"node-resolver-llmj8\" (UID: \"148fa8ca-4d3b-4ae1-8be5-30ea87aa412e\") " pod="openshift-dns/node-resolver-llmj8"
Jul 31 22:56:57.869544 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.868664    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1d0f0d49-bc8d-489a-932a-de4c94edb6a4-etc-sysctl-d\") pod \"tuned-9wt4q\" (UID: \"1d0f0d49-bc8d-489a-932a-de4c94edb6a4\") " pod="openshift-cluster-node-tuning-operator/tuned-9wt4q"
Jul 31 22:56:57.869544 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.868687    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1d0f0d49-bc8d-489a-932a-de4c94edb6a4-etc-sysctl-conf\") pod \"tuned-9wt4q\" (UID: \"1d0f0d49-bc8d-489a-932a-de4c94edb6a4\") " pod="openshift-cluster-node-tuning-operator/tuned-9wt4q"
Jul 31 22:56:57.869544 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.868727    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1d0f0d49-bc8d-489a-932a-de4c94edb6a4-etc-modprobe-d\") pod \"tuned-9wt4q\" (UID: \"1d0f0d49-bc8d-489a-932a-de4c94edb6a4\") " pod="openshift-cluster-node-tuning-operator/tuned-9wt4q"
Jul 31 22:56:57.869544 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.868756    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pps5d\" (UniqueName: \"kubernetes.io/projected/9a9cc748-9a60-4656-9236-34d75e5cb22a-kube-api-access-pps5d\") pod \"iptables-alerter-qchlw\" (UID: \"9a9cc748-9a60-4656-9236-34d75e5cb22a\") " pod="openshift-network-operator/iptables-alerter-qchlw"
Jul 31 22:56:57.869544 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.868779    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/87c830a4-dd74-43c5-84d0-45cdd2dbe9a3-node-log\") pod \"ovnkube-node-w96jg\" (UID: \"87c830a4-dd74-43c5-84d0-45cdd2dbe9a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w96jg"
Jul 31 22:56:57.869544 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.868802    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/87c830a4-dd74-43c5-84d0-45cdd2dbe9a3-ovnkube-script-lib\") pod \"ovnkube-node-w96jg\" (UID: \"87c830a4-dd74-43c5-84d0-45cdd2dbe9a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w96jg"
Jul 31 22:56:57.869544 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.868853    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/639fb18d-e947-41cc-ac53-d4616b7cc85b-etc-kubernetes\") pod \"multus-pbslg\" (UID: \"639fb18d-e947-41cc-ac53-d4616b7cc85b\") " pod="openshift-multus/multus-pbslg"
Jul 31 22:56:57.869544 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.868889    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klbxv\" (UniqueName: \"kubernetes.io/projected/3f91895a-1a8c-4ef9-8f27-0cfed935fae8-kube-api-access-klbxv\") pod \"aws-ebs-csi-driver-node-7bk57\" (UID: \"3f91895a-1a8c-4ef9-8f27-0cfed935fae8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bk57"
Jul 31 22:56:57.869544 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.868923    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4sqv4\" (UniqueName: \"kubernetes.io/projected/4c7c83f2-713b-4651-b1da-f65c84f8edc9-kube-api-access-4sqv4\") pod \"multus-additional-cni-plugins-2trlm\" (UID: \"4c7c83f2-713b-4651-b1da-f65c84f8edc9\") " pod="openshift-multus/multus-additional-cni-plugins-2trlm"
Jul 31 22:56:57.869544 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.868969    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wzc4\" (UniqueName: \"kubernetes.io/projected/148fa8ca-4d3b-4ae1-8be5-30ea87aa412e-kube-api-access-5wzc4\") pod \"node-resolver-llmj8\" (UID: \"148fa8ca-4d3b-4ae1-8be5-30ea87aa412e\") " pod="openshift-dns/node-resolver-llmj8"
Jul 31 22:56:57.869544 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.868997    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/87c830a4-dd74-43c5-84d0-45cdd2dbe9a3-ovnkube-config\") pod \"ovnkube-node-w96jg\" (UID: \"87c830a4-dd74-43c5-84d0-45cdd2dbe9a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w96jg"
Jul 31 22:56:57.869544 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.869025    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/639fb18d-e947-41cc-ac53-d4616b7cc85b-host-run-k8s-cni-cncf-io\") pod \"multus-pbslg\" (UID: \"639fb18d-e947-41cc-ac53-d4616b7cc85b\") " pod="openshift-multus/multus-pbslg"
Jul 31 22:56:57.869544 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.869062    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1d0f0d49-bc8d-489a-932a-de4c94edb6a4-etc-tuned\") pod \"tuned-9wt4q\" (UID: \"1d0f0d49-bc8d-489a-932a-de4c94edb6a4\") " pod="openshift-cluster-node-tuning-operator/tuned-9wt4q"
Jul 31 22:56:57.869544 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.869080    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4c7c83f2-713b-4651-b1da-f65c84f8edc9-os-release\") pod \"multus-additional-cni-plugins-2trlm\" (UID: \"4c7c83f2-713b-4651-b1da-f65c84f8edc9\") " pod="openshift-multus/multus-additional-cni-plugins-2trlm"
Jul 31 22:56:57.869544 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.869102    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945-metrics-certs\") pod \"network-metrics-daemon-mz5wz\" (UID: \"b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945\") " pod="openshift-multus/network-metrics-daemon-mz5wz"
Jul 31 22:56:57.870322 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.869119    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/87c830a4-dd74-43c5-84d0-45cdd2dbe9a3-log-socket\") pod \"ovnkube-node-w96jg\" (UID: \"87c830a4-dd74-43c5-84d0-45cdd2dbe9a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w96jg"
Jul 31 22:56:57.870322 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.869134    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1d0f0d49-bc8d-489a-932a-de4c94edb6a4-sys\") pod \"tuned-9wt4q\" (UID: \"1d0f0d49-bc8d-489a-932a-de4c94edb6a4\") " pod="openshift-cluster-node-tuning-operator/tuned-9wt4q"
Jul 31 22:56:57.870322 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.869153    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bvbb\" (UniqueName: \"kubernetes.io/projected/1d0f0d49-bc8d-489a-932a-de4c94edb6a4-kube-api-access-9bvbb\") pod \"tuned-9wt4q\" (UID: \"1d0f0d49-bc8d-489a-932a-de4c94edb6a4\") " pod="openshift-cluster-node-tuning-operator/tuned-9wt4q"
Jul 31 22:56:57.870322 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.869172    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3f91895a-1a8c-4ef9-8f27-0cfed935fae8-etc-selinux\") pod \"aws-ebs-csi-driver-node-7bk57\" (UID: \"3f91895a-1a8c-4ef9-8f27-0cfed935fae8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bk57"
Jul 31 22:56:57.870322 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:56:57.869199    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 31 22:56:57.870322 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.869205    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b85a1443-4bc2-48a0-8d29-35f46cbc3d57-kubelet-config\") pod \"global-pull-secret-syncer-mqwlt\" (UID: \"b85a1443-4bc2-48a0-8d29-35f46cbc3d57\") " pod="kube-system/global-pull-secret-syncer-mqwlt"
Jul 31 22:56:57.870322 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:56:57.869276    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945-metrics-certs podName:b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945 nodeName:}" failed. No retries permitted until 2026-07-31 22:56:58.369250405 +0000 UTC m=+2.015036708 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945-metrics-certs") pod "network-metrics-daemon-mz5wz" (UID: "b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 31 22:56:57.870322 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.869295    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/87c830a4-dd74-43c5-84d0-45cdd2dbe9a3-systemd-units\") pod \"ovnkube-node-w96jg\" (UID: \"87c830a4-dd74-43c5-84d0-45cdd2dbe9a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w96jg"
Jul 31 22:56:57.870322 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.869326    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87c830a4-dd74-43c5-84d0-45cdd2dbe9a3-etc-openvswitch\") pod \"ovnkube-node-w96jg\" (UID: \"87c830a4-dd74-43c5-84d0-45cdd2dbe9a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w96jg"
Jul 31 22:56:57.870322 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.869366    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9wk5\" (UniqueName: \"kubernetes.io/projected/87c830a4-dd74-43c5-84d0-45cdd2dbe9a3-kube-api-access-d9wk5\") pod \"ovnkube-node-w96jg\" (UID: \"87c830a4-dd74-43c5-84d0-45cdd2dbe9a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w96jg"
Jul 31 22:56:57.870322 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.869404    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/639fb18d-e947-41cc-ac53-d4616b7cc85b-os-release\") pod \"multus-pbslg\" (UID: \"639fb18d-e947-41cc-ac53-d4616b7cc85b\") " pod="openshift-multus/multus-pbslg"
Jul 31 22:56:57.870322 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.869446    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/639fb18d-e947-41cc-ac53-d4616b7cc85b-host-run-netns\") pod \"multus-pbslg\" (UID: \"639fb18d-e947-41cc-ac53-d4616b7cc85b\") " pod="openshift-multus/multus-pbslg"
Jul 31 22:56:57.870322 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.869477    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3f91895a-1a8c-4ef9-8f27-0cfed935fae8-socket-dir\") pod \"aws-ebs-csi-driver-node-7bk57\" (UID: \"3f91895a-1a8c-4ef9-8f27-0cfed935fae8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bk57"
Jul 31 22:56:57.870322 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.869494    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4c7c83f2-713b-4651-b1da-f65c84f8edc9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2trlm\" (UID: \"4c7c83f2-713b-4651-b1da-f65c84f8edc9\") " pod="openshift-multus/multus-additional-cni-plugins-2trlm"
Jul 31 22:56:57.870322 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.869513    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44r6z\" (UniqueName: \"kubernetes.io/projected/0bb4d7d9-ff22-41eb-8883-b7c1942767fb-kube-api-access-44r6z\") pod \"node-ca-t8qkd\" (UID: \"0bb4d7d9-ff22-41eb-8883-b7c1942767fb\") " pod="openshift-image-registry/node-ca-t8qkd"
Jul 31 22:56:57.870322 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.869531    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9a9cc748-9a60-4656-9236-34d75e5cb22a-host-slash\") pod \"iptables-alerter-qchlw\" (UID: \"9a9cc748-9a60-4656-9236-34d75e5cb22a\") " pod="openshift-network-operator/iptables-alerter-qchlw"
Jul 31 22:56:57.871094 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.869547    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3f91895a-1a8c-4ef9-8f27-0cfed935fae8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7bk57\" (UID: \"3f91895a-1a8c-4ef9-8f27-0cfed935fae8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bk57"
Jul 31 22:56:57.871094 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.869570    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87c830a4-dd74-43c5-84d0-45cdd2dbe9a3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-w96jg\" (UID: \"87c830a4-dd74-43c5-84d0-45cdd2dbe9a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w96jg"
Jul 31 22:56:57.871094 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.869581    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9a9cc748-9a60-4656-9236-34d75e5cb22a-host-slash\") pod \"iptables-alerter-qchlw\" (UID: \"9a9cc748-9a60-4656-9236-34d75e5cb22a\") " pod="openshift-network-operator/iptables-alerter-qchlw"
Jul 31 22:56:57.871094 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.869587    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/639fb18d-e947-41cc-ac53-d4616b7cc85b-host-var-lib-cni-multus\") pod \"multus-pbslg\" (UID: \"639fb18d-e947-41cc-ac53-d4616b7cc85b\") " pod="openshift-multus/multus-pbslg"
Jul 31 22:56:57.871094 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.869621    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4c7c83f2-713b-4651-b1da-f65c84f8edc9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2trlm\" (UID: \"4c7c83f2-713b-4651-b1da-f65c84f8edc9\") " pod="openshift-multus/multus-additional-cni-plugins-2trlm"
Jul 31 22:56:57.876122 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.876077    2584 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 31 22:56:57.879916 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.879888    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-44r6z\" (UniqueName: \"kubernetes.io/projected/0bb4d7d9-ff22-41eb-8883-b7c1942767fb-kube-api-access-44r6z\") pod \"node-ca-t8qkd\" (UID: \"0bb4d7d9-ff22-41eb-8883-b7c1942767fb\") " pod="openshift-image-registry/node-ca-t8qkd"
Jul 31 22:56:57.880007 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.879935    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfnz4\" (UniqueName: \"kubernetes.io/projected/b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945-kube-api-access-lfnz4\") pod \"network-metrics-daemon-mz5wz\" (UID: \"b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945\") " pod="openshift-multus/network-metrics-daemon-mz5wz"
Jul 31 22:56:57.880007 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.879952    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pps5d\" (UniqueName: \"kubernetes.io/projected/9a9cc748-9a60-4656-9236-34d75e5cb22a-kube-api-access-pps5d\") pod \"iptables-alerter-qchlw\" (UID: \"9a9cc748-9a60-4656-9236-34d75e5cb22a\") " pod="openshift-network-operator/iptables-alerter-qchlw"
Jul 31 22:56:57.922495 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:57.922461    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf2bb11c3449d46ed43b4ff9c8ef237d.slice/crio-f1075e4f8bf1fb142a40a5d00d4643c353b508c698ca3a31baafe316187a9bce WatchSource:0}: Error finding container f1075e4f8bf1fb142a40a5d00d4643c353b508c698ca3a31baafe316187a9bce: Status 404 returned error can't find the container with id f1075e4f8bf1fb142a40a5d00d4643c353b508c698ca3a31baafe316187a9bce
Jul 31 22:56:57.922771 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:57.922739    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59797420703085b3d07401083a2a0810.slice/crio-cb38daa36f2224d21c1297d162427cfb61ef6f334ef26e22ce5d8123e2508b06 WatchSource:0}: Error finding container cb38daa36f2224d21c1297d162427cfb61ef6f334ef26e22ce5d8123e2508b06: Status 404 returned error can't find the container with id cb38daa36f2224d21c1297d162427cfb61ef6f334ef26e22ce5d8123e2508b06
Jul 31 22:56:57.930183 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.930165    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 31 22:56:57.970411 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.970378    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/eb4e0ba5-fe8b-4bf7-a86d-15db1a2532b2-agent-certs\") pod \"konnectivity-agent-shjgx\" (UID: \"eb4e0ba5-fe8b-4bf7-a86d-15db1a2532b2\") " pod="kube-system/konnectivity-agent-shjgx"
Jul 31 22:56:57.970411 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.970413    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/eb4e0ba5-fe8b-4bf7-a86d-15db1a2532b2-konnectivity-ca\") pod \"konnectivity-agent-shjgx\" (UID: \"eb4e0ba5-fe8b-4bf7-a86d-15db1a2532b2\") " pod="kube-system/konnectivity-agent-shjgx"
Jul 31 22:56:57.970620 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.970431    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1d0f0d49-bc8d-489a-932a-de4c94edb6a4-etc-systemd\") pod \"tuned-9wt4q\" (UID: \"1d0f0d49-bc8d-489a-932a-de4c94edb6a4\") " pod="openshift-cluster-node-tuning-operator/tuned-9wt4q"
Jul 31 22:56:57.970620 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.970459    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/87c830a4-dd74-43c5-84d0-45cdd2dbe9a3-run-systemd\") pod \"ovnkube-node-w96jg\" (UID: \"87c830a4-dd74-43c5-84d0-45cdd2dbe9a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w96jg"
Jul 31 22:56:57.970620 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.970482    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87c830a4-dd74-43c5-84d0-45cdd2dbe9a3-run-openvswitch\") pod \"ovnkube-node-w96jg\" (UID: \"87c830a4-dd74-43c5-84d0-45cdd2dbe9a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w96jg"
Jul 31 22:56:57.970620 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.970505    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/639fb18d-e947-41cc-ac53-d4616b7cc85b-host-var-lib-cni-bin\") pod \"multus-pbslg\" (UID: \"639fb18d-e947-41cc-ac53-d4616b7cc85b\") " pod="openshift-multus/multus-pbslg"
Jul 31 22:56:57.970620 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.970546    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/87c830a4-dd74-43c5-84d0-45cdd2dbe9a3-run-systemd\") pod \"ovnkube-node-w96jg\" (UID: \"87c830a4-dd74-43c5-84d0-45cdd2dbe9a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w96jg"
Jul 31 22:56:57.970620 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.970550    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87c830a4-dd74-43c5-84d0-45cdd2dbe9a3-run-openvswitch\") pod \"ovnkube-node-w96jg\" (UID: \"87c830a4-dd74-43c5-84d0-45cdd2dbe9a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w96jg"
Jul 31 22:56:57.970620 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.970546    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1d0f0d49-bc8d-489a-932a-de4c94edb6a4-etc-systemd\") pod \"tuned-9wt4q\" (UID: \"1d0f0d49-bc8d-489a-932a-de4c94edb6a4\") " pod="openshift-cluster-node-tuning-operator/tuned-9wt4q"
Jul 31 22:56:57.970620 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.970555    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/639fb18d-e947-41cc-ac53-d4616b7cc85b-host-var-lib-cni-bin\") pod \"multus-pbslg\" (UID: \"639fb18d-e947-41cc-ac53-d4616b7cc85b\") " pod="openshift-multus/multus-pbslg"
Jul 31 22:56:57.970620 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.970600    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/148fa8ca-4d3b-4ae1-8be5-30ea87aa412e-hosts-file\") pod \"node-resolver-llmj8\" (UID: \"148fa8ca-4d3b-4ae1-8be5-30ea87aa412e\") " pod="openshift-dns/node-resolver-llmj8"
Jul 31 22:56:57.970620 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.970624    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1d0f0d49-bc8d-489a-932a-de4c94edb6a4-host\") pod \"tuned-9wt4q\" (UID: \"1d0f0d49-bc8d-489a-932a-de4c94edb6a4\") " pod="openshift-cluster-node-tuning-operator/tuned-9wt4q"
Jul 31 22:56:57.971170 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.970634    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/148fa8ca-4d3b-4ae1-8be5-30ea87aa412e-hosts-file\") pod \"node-resolver-llmj8\" (UID: \"148fa8ca-4d3b-4ae1-8be5-30ea87aa412e\") " pod="openshift-dns/node-resolver-llmj8"
Jul 31 22:56:57.971170 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.970656    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b85a1443-4bc2-48a0-8d29-35f46cbc3d57-original-pull-secret\") pod \"global-pull-secret-syncer-mqwlt\" (UID: \"b85a1443-4bc2-48a0-8d29-35f46cbc3d57\") " pod="kube-system/global-pull-secret-syncer-mqwlt"
Jul 31 22:56:57.971170 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.970670    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1d0f0d49-bc8d-489a-932a-de4c94edb6a4-host\") pod \"tuned-9wt4q\" (UID: \"1d0f0d49-bc8d-489a-932a-de4c94edb6a4\") " pod="openshift-cluster-node-tuning-operator/tuned-9wt4q"
Jul 31 22:56:57.971170 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.970683    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/639fb18d-e947-41cc-ac53-d4616b7cc85b-system-cni-dir\") pod \"multus-pbslg\" (UID: \"639fb18d-e947-41cc-ac53-d4616b7cc85b\") " pod="openshift-multus/multus-pbslg"
Jul 31 22:56:57.971170 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.970743    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/639fb18d-e947-41cc-ac53-d4616b7cc85b-system-cni-dir\") pod \"multus-pbslg\" (UID: \"639fb18d-e947-41cc-ac53-d4616b7cc85b\") " pod="openshift-multus/multus-pbslg"
Jul 31 22:56:57.971170 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.970743    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/639fb18d-e947-41cc-ac53-d4616b7cc85b-multus-cni-dir\") pod \"multus-pbslg\" (UID: \"639fb18d-e947-41cc-ac53-d4616b7cc85b\") " pod="openshift-multus/multus-pbslg"
Jul 31 22:56:57.971170 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.970771    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/639fb18d-e947-41cc-ac53-d4616b7cc85b-cnibin\") pod \"multus-pbslg\" (UID: \"639fb18d-e947-41cc-ac53-d4616b7cc85b\") " pod="openshift-multus/multus-pbslg"
Jul 31 22:56:57.971170 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.970794    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/639fb18d-e947-41cc-ac53-d4616b7cc85b-multus-conf-dir\") pod \"multus-pbslg\" (UID: \"639fb18d-e947-41cc-ac53-d4616b7cc85b\") " pod="openshift-multus/multus-pbslg"
Jul 31 22:56:57.971170 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:56:57.970809    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 31 22:56:57.971170 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.970833    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/639fb18d-e947-41cc-ac53-d4616b7cc85b-cnibin\") pod \"multus-pbslg\" (UID: \"639fb18d-e947-41cc-ac53-d4616b7cc85b\") " pod="openshift-multus/multus-pbslg"
Jul 31 22:56:57.971170 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.970813    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1d0f0d49-bc8d-489a-932a-de4c94edb6a4-run\") pod \"tuned-9wt4q\" (UID: \"1d0f0d49-bc8d-489a-932a-de4c94edb6a4\") " pod="openshift-cluster-node-tuning-operator/tuned-9wt4q"
Jul 31 22:56:57.971170 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:56:57.970880    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b85a1443-4bc2-48a0-8d29-35f46cbc3d57-original-pull-secret podName:b85a1443-4bc2-48a0-8d29-35f46cbc3d57 nodeName:}" failed. No retries permitted until 2026-07-31 22:56:58.470859304 +0000 UTC m=+2.116645594 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b85a1443-4bc2-48a0-8d29-35f46cbc3d57-original-pull-secret") pod "global-pull-secret-syncer-mqwlt" (UID: "b85a1443-4bc2-48a0-8d29-35f46cbc3d57") : object "kube-system"/"original-pull-secret" not registered
Jul 31 22:56:57.971170 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.970881    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/639fb18d-e947-41cc-ac53-d4616b7cc85b-multus-conf-dir\") pod \"multus-pbslg\" (UID: \"639fb18d-e947-41cc-ac53-d4616b7cc85b\") " pod="openshift-multus/multus-pbslg"
Jul 31 22:56:57.971170 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.970913    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1d0f0d49-bc8d-489a-932a-de4c94edb6a4-run\") pod \"tuned-9wt4q\" (UID: \"1d0f0d49-bc8d-489a-932a-de4c94edb6a4\") " pod="openshift-cluster-node-tuning-operator/tuned-9wt4q"
Jul 31 22:56:57.971170 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.970923    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1d0f0d49-bc8d-489a-932a-de4c94edb6a4-var-lib-kubelet\") pod \"tuned-9wt4q\" (UID: \"1d0f0d49-bc8d-489a-932a-de4c94edb6a4\") " pod="openshift-cluster-node-tuning-operator/tuned-9wt4q"
Jul 31 22:56:57.971170 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.970926    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/639fb18d-e947-41cc-ac53-d4616b7cc85b-multus-cni-dir\") pod \"multus-pbslg\" (UID: \"639fb18d-e947-41cc-ac53-d4616b7cc85b\") " pod="openshift-multus/multus-pbslg"
Jul 31 22:56:57.971170 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.970960    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1d0f0d49-bc8d-489a-932a-de4c94edb6a4-etc-kubernetes\") pod \"tuned-9wt4q\" (UID: \"1d0f0d49-bc8d-489a-932a-de4c94edb6a4\") " pod="openshift-cluster-node-tuning-operator/tuned-9wt4q"
Jul 31 22:56:57.971170 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.970971    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1d0f0d49-bc8d-489a-932a-de4c94edb6a4-var-lib-kubelet\") pod \"tuned-9wt4q\" (UID: \"1d0f0d49-bc8d-489a-932a-de4c94edb6a4\") " pod="openshift-cluster-node-tuning-operator/tuned-9wt4q"
Jul 31 22:56:57.972131 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.970980    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/87c830a4-dd74-43c5-84d0-45cdd2dbe9a3-ovn-node-metrics-cert\") pod \"ovnkube-node-w96jg\" (UID: \"87c830a4-dd74-43c5-84d0-45cdd2dbe9a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w96jg"
Jul 31 22:56:57.972131 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.971026    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/639fb18d-e947-41cc-ac53-d4616b7cc85b-host-var-lib-kubelet\") pod \"multus-pbslg\" (UID: \"639fb18d-e947-41cc-ac53-d4616b7cc85b\") " pod="openshift-multus/multus-pbslg"
Jul 31 22:56:57.972131 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.971016    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1d0f0d49-bc8d-489a-932a-de4c94edb6a4-etc-kubernetes\") pod \"tuned-9wt4q\" (UID: \"1d0f0d49-bc8d-489a-932a-de4c94edb6a4\") " pod="openshift-cluster-node-tuning-operator/tuned-9wt4q"
Jul 31 22:56:57.972131 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.971051    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/639fb18d-e947-41cc-ac53-d4616b7cc85b-hostroot\") pod \"multus-pbslg\" (UID: \"639fb18d-e947-41cc-ac53-d4616b7cc85b\") " pod="openshift-multus/multus-pbslg"
Jul 31 22:56:57.972131 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.971067    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/639fb18d-e947-41cc-ac53-d4616b7cc85b-multus-daemon-config\") pod \"multus-pbslg\" (UID: \"639fb18d-e947-41cc-ac53-d4616b7cc85b\") " pod="openshift-multus/multus-pbslg"
Jul 31 22:56:57.972131 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.971088    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/639fb18d-e947-41cc-ac53-d4616b7cc85b-host-var-lib-kubelet\") pod \"multus-pbslg\" (UID: \"639fb18d-e947-41cc-ac53-d4616b7cc85b\") " pod="openshift-multus/multus-pbslg"
Jul 31 22:56:57.972131 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.971090    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2m5pb\" (UniqueName: \"kubernetes.io/projected/639fb18d-e947-41cc-ac53-d4616b7cc85b-kube-api-access-2m5pb\") pod \"multus-pbslg\" (UID: \"639fb18d-e947-41cc-ac53-d4616b7cc85b\") " pod="openshift-multus/multus-pbslg"
Jul 31 22:56:57.972131 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.971095    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/eb4e0ba5-fe8b-4bf7-a86d-15db1a2532b2-konnectivity-ca\") pod \"konnectivity-agent-shjgx\" (UID: \"eb4e0ba5-fe8b-4bf7-a86d-15db1a2532b2\") " pod="kube-system/konnectivity-agent-shjgx"
Jul 31 22:56:57.972131 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.971122    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3f91895a-1a8c-4ef9-8f27-0cfed935fae8-sys-fs\") pod \"aws-ebs-csi-driver-node-7bk57\" (UID: \"3f91895a-1a8c-4ef9-8f27-0cfed935fae8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bk57"
Jul 31 22:56:57.972131 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.971133    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/639fb18d-e947-41cc-ac53-d4616b7cc85b-hostroot\") pod \"multus-pbslg\" (UID: \"639fb18d-e947-41cc-ac53-d4616b7cc85b\") " pod="openshift-multus/multus-pbslg"
Jul 31 22:56:57.972131 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.971149    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/87c830a4-dd74-43c5-84d0-45cdd2dbe9a3-host-slash\") pod \"ovnkube-node-w96jg\" (UID: \"87c830a4-dd74-43c5-84d0-45cdd2dbe9a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w96jg"
Jul 31 22:56:57.972131 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.971164    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3f91895a-1a8c-4ef9-8f27-0cfed935fae8-sys-fs\") pod \"aws-ebs-csi-driver-node-7bk57\" (UID: \"3f91895a-1a8c-4ef9-8f27-0cfed935fae8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bk57"
Jul 31 22:56:57.972131 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.971176    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/87c830a4-dd74-43c5-84d0-45cdd2dbe9a3-host-cni-bin\") pod \"ovnkube-node-w96jg\" (UID: \"87c830a4-dd74-43c5-84d0-45cdd2dbe9a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w96jg"
Jul 31 22:56:57.972131 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.971204    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4c7c83f2-713b-4651-b1da-f65c84f8edc9-cni-binary-copy\") pod \"multus-additional-cni-plugins-2trlm\" (UID: \"4c7c83f2-713b-4651-b1da-f65c84f8edc9\") " pod="openshift-multus/multus-additional-cni-plugins-2trlm"
Jul 31 22:56:57.972131 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.971212    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/87c830a4-dd74-43c5-84d0-45cdd2dbe9a3-host-slash\") pod \"ovnkube-node-w96jg\" (UID: \"87c830a4-dd74-43c5-84d0-45cdd2dbe9a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w96jg"
Jul 31 22:56:57.972131 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.971238    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4c7c83f2-713b-4651-b1da-f65c84f8edc9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2trlm\" (UID: \"4c7c83f2-713b-4651-b1da-f65c84f8edc9\") " pod="openshift-multus/multus-additional-cni-plugins-2trlm"
Jul 31 22:56:57.972131 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.971245    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/87c830a4-dd74-43c5-84d0-45cdd2dbe9a3-host-cni-bin\") pod \"ovnkube-node-w96jg\" (UID: \"87c830a4-dd74-43c5-84d0-45cdd2dbe9a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w96jg"
Jul 31 22:56:57.973083 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.971266    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/87c830a4-dd74-43c5-84d0-45cdd2dbe9a3-host-kubelet\") pod \"ovnkube-node-w96jg\" (UID: \"87c830a4-dd74-43c5-84d0-45cdd2dbe9a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w96jg"
Jul 31 22:56:57.973083 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.971291    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87c830a4-dd74-43c5-84d0-45cdd2dbe9a3-host-run-ovn-kubernetes\") pod \"ovnkube-node-w96jg\" (UID: \"87c830a4-dd74-43c5-84d0-45cdd2dbe9a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w96jg"
Jul 31 22:56:57.973083 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.971317    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3f91895a-1a8c-4ef9-8f27-0cfed935fae8-device-dir\") pod \"aws-ebs-csi-driver-node-7bk57\" (UID: \"3f91895a-1a8c-4ef9-8f27-0cfed935fae8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bk57"
Jul 31 22:56:57.973083 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.971331    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/87c830a4-dd74-43c5-84d0-45cdd2dbe9a3-host-kubelet\") pod \"ovnkube-node-w96jg\" (UID: \"87c830a4-dd74-43c5-84d0-45cdd2dbe9a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w96jg"
Jul 31 22:56:57.973083 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.971331    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87c830a4-dd74-43c5-84d0-45cdd2dbe9a3-host-run-ovn-kubernetes\") pod \"ovnkube-node-w96jg\" (UID: \"87c830a4-dd74-43c5-84d0-45cdd2dbe9a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w96jg"
Jul 31 22:56:57.973083 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.971422    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3f91895a-1a8c-4ef9-8f27-0cfed935fae8-device-dir\") pod \"aws-ebs-csi-driver-node-7bk57\" (UID: \"3f91895a-1a8c-4ef9-8f27-0cfed935fae8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bk57"
Jul 31 22:56:57.973083 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.971340    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwbjs\" (UniqueName: \"kubernetes.io/projected/b80644e6-82bb-4c59-86ed-b60bd24e6a05-kube-api-access-bwbjs\") pod \"network-check-target-84css\" (UID: \"b80644e6-82bb-4c59-86ed-b60bd24e6a05\") " pod="openshift-network-diagnostics/network-check-target-84css"
Jul 31 22:56:57.973083 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.971471    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4c7c83f2-713b-4651-b1da-f65c84f8edc9-system-cni-dir\") pod \"multus-additional-cni-plugins-2trlm\" (UID: \"4c7c83f2-713b-4651-b1da-f65c84f8edc9\") " pod="openshift-multus/multus-additional-cni-plugins-2trlm"
Jul 31 22:56:57.973083 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.971497    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4c7c83f2-713b-4651-b1da-f65c84f8edc9-cnibin\") pod \"multus-additional-cni-plugins-2trlm\" (UID: \"4c7c83f2-713b-4651-b1da-f65c84f8edc9\") " pod="openshift-multus/multus-additional-cni-plugins-2trlm"
Jul 31 22:56:57.973083 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.971530    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/639fb18d-e947-41cc-ac53-d4616b7cc85b-cni-binary-copy\") pod \"multus-pbslg\" (UID: \"639fb18d-e947-41cc-ac53-d4616b7cc85b\") " pod="openshift-multus/multus-pbslg"
Jul 31 22:56:57.973083 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.971545    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4c7c83f2-713b-4651-b1da-f65c84f8edc9-system-cni-dir\") pod \"multus-additional-cni-plugins-2trlm\" (UID: \"4c7c83f2-713b-4651-b1da-f65c84f8edc9\") " pod="openshift-multus/multus-additional-cni-plugins-2trlm"
Jul 31 22:56:57.973083 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.971555    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/639fb18d-e947-41cc-ac53-d4616b7cc85b-host-run-multus-certs\") pod \"multus-pbslg\" (UID: \"639fb18d-e947-41cc-ac53-d4616b7cc85b\") " pod="openshift-multus/multus-pbslg"
Jul 31 22:56:57.973083 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.971567    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4c7c83f2-713b-4651-b1da-f65c84f8edc9-cnibin\") pod \"multus-additional-cni-plugins-2trlm\" (UID: \"4c7c83f2-713b-4651-b1da-f65c84f8edc9\") " pod="openshift-multus/multus-additional-cni-plugins-2trlm"
Jul 31 22:56:57.973083 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.971581    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/87c830a4-dd74-43c5-84d0-45cdd2dbe9a3-run-ovn\") pod \"ovnkube-node-w96jg\" (UID: \"87c830a4-dd74-43c5-84d0-45cdd2dbe9a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w96jg"
Jul 31 22:56:57.973083 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.971605    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1d0f0d49-bc8d-489a-932a-de4c94edb6a4-tmp\") pod \"tuned-9wt4q\" (UID: \"1d0f0d49-bc8d-489a-932a-de4c94edb6a4\") " pod="openshift-cluster-node-tuning-operator/tuned-9wt4q"
Jul 31 22:56:57.973083 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.971621    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/639fb18d-e947-41cc-ac53-d4616b7cc85b-host-run-multus-certs\") pod \"multus-pbslg\" (UID: \"639fb18d-e947-41cc-ac53-d4616b7cc85b\") " pod="openshift-multus/multus-pbslg"
Jul 31 22:56:57.973083 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.971631    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1d0f0d49-bc8d-489a-932a-de4c94edb6a4-etc-sysconfig\") pod \"tuned-9wt4q\" (UID: \"1d0f0d49-bc8d-489a-932a-de4c94edb6a4\") " pod="openshift-cluster-node-tuning-operator/tuned-9wt4q"
Jul 31 22:56:57.973955 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.971665    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/87c830a4-dd74-43c5-84d0-45cdd2dbe9a3-env-overrides\") pod \"ovnkube-node-w96jg\" (UID: \"87c830a4-dd74-43c5-84d0-45cdd2dbe9a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w96jg"
Jul 31 22:56:57.973955 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.971694    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87c830a4-dd74-43c5-84d0-45cdd2dbe9a3-var-lib-openvswitch\") pod \"ovnkube-node-w96jg\" (UID: \"87c830a4-dd74-43c5-84d0-45cdd2dbe9a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w96jg"
Jul 31 22:56:57.973955 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.971722    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/87c830a4-dd74-43c5-84d0-45cdd2dbe9a3-host-cni-netd\") pod \"ovnkube-node-w96jg\" (UID: \"87c830a4-dd74-43c5-84d0-45cdd2dbe9a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w96jg"
Jul 31 22:56:57.973955 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.971748    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/148fa8ca-4d3b-4ae1-8be5-30ea87aa412e-tmp-dir\") pod \"node-resolver-llmj8\" (UID: \"148fa8ca-4d3b-4ae1-8be5-30ea87aa412e\") " pod="openshift-dns/node-resolver-llmj8"
Jul 31 22:56:57.973955 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.971790    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1d0f0d49-bc8d-489a-932a-de4c94edb6a4-etc-sysctl-d\") pod \"tuned-9wt4q\" (UID: \"1d0f0d49-bc8d-489a-932a-de4c94edb6a4\") " pod="openshift-cluster-node-tuning-operator/tuned-9wt4q"
Jul 31 22:56:57.973955 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.971816    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1d0f0d49-bc8d-489a-932a-de4c94edb6a4-etc-sysctl-conf\") pod \"tuned-9wt4q\" (UID: \"1d0f0d49-bc8d-489a-932a-de4c94edb6a4\") " pod="openshift-cluster-node-tuning-operator/tuned-9wt4q"
Jul 31 22:56:57.973955 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.971834    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4c7c83f2-713b-4651-b1da-f65c84f8edc9-cni-binary-copy\") pod \"multus-additional-cni-plugins-2trlm\" (UID: \"4c7c83f2-713b-4651-b1da-f65c84f8edc9\") " pod="openshift-multus/multus-additional-cni-plugins-2trlm"
Jul 31 22:56:57.973955 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.971863    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1d0f0d49-bc8d-489a-932a-de4c94edb6a4-etc-modprobe-d\") pod \"tuned-9wt4q\" (UID: \"1d0f0d49-bc8d-489a-932a-de4c94edb6a4\") " pod="openshift-cluster-node-tuning-operator/tuned-9wt4q"
Jul 31 22:56:57.973955 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.971891    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/87c830a4-dd74-43c5-84d0-45cdd2dbe9a3-node-log\") pod \"ovnkube-node-w96jg\" (UID: \"87c830a4-dd74-43c5-84d0-45cdd2dbe9a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w96jg"
Jul 31 22:56:57.973955 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.971902    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87c830a4-dd74-43c5-84d0-45cdd2dbe9a3-var-lib-openvswitch\") pod \"ovnkube-node-w96jg\" (UID: \"87c830a4-dd74-43c5-84d0-45cdd2dbe9a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w96jg"
Jul 31 22:56:57.973955 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.971917    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/87c830a4-dd74-43c5-84d0-45cdd2dbe9a3-ovnkube-script-lib\") pod \"ovnkube-node-w96jg\" (UID: \"87c830a4-dd74-43c5-84d0-45cdd2dbe9a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w96jg"
Jul 31 22:56:57.973955 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.971918    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4c7c83f2-713b-4651-b1da-f65c84f8edc9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2trlm\" (UID: \"4c7c83f2-713b-4651-b1da-f65c84f8edc9\") " pod="openshift-multus/multus-additional-cni-plugins-2trlm"
Jul 31 22:56:57.973955 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.971952    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1d0f0d49-bc8d-489a-932a-de4c94edb6a4-etc-sysconfig\") pod \"tuned-9wt4q\" (UID: \"1d0f0d49-bc8d-489a-932a-de4c94edb6a4\") " pod="openshift-cluster-node-tuning-operator/tuned-9wt4q"
Jul 31 22:56:57.973955 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.971667    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/87c830a4-dd74-43c5-84d0-45cdd2dbe9a3-run-ovn\") pod \"ovnkube-node-w96jg\" (UID: \"87c830a4-dd74-43c5-84d0-45cdd2dbe9a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w96jg"
Jul 31 22:56:57.973955 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.971951    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/639fb18d-e947-41cc-ac53-d4616b7cc85b-etc-kubernetes\") pod \"multus-pbslg\" (UID: \"639fb18d-e947-41cc-ac53-d4616b7cc85b\") " pod="openshift-multus/multus-pbslg"
Jul 31 22:56:57.973955 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.972006    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klbxv\" (UniqueName: \"kubernetes.io/projected/3f91895a-1a8c-4ef9-8f27-0cfed935fae8-kube-api-access-klbxv\") pod \"aws-ebs-csi-driver-node-7bk57\" (UID: \"3f91895a-1a8c-4ef9-8f27-0cfed935fae8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bk57"
Jul 31 22:56:57.973955 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.972006    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/639fb18d-e947-41cc-ac53-d4616b7cc85b-etc-kubernetes\") pod \"multus-pbslg\" (UID: \"639fb18d-e947-41cc-ac53-d4616b7cc85b\") " pod="openshift-multus/multus-pbslg"
Jul 31 22:56:57.974459 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.972044    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/87c830a4-dd74-43c5-84d0-45cdd2dbe9a3-host-cni-netd\") pod \"ovnkube-node-w96jg\" (UID: \"87c830a4-dd74-43c5-84d0-45cdd2dbe9a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w96jg"
Jul 31 22:56:57.974459 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.972081    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4sqv4\" (UniqueName: \"kubernetes.io/projected/4c7c83f2-713b-4651-b1da-f65c84f8edc9-kube-api-access-4sqv4\") pod \"multus-additional-cni-plugins-2trlm\" (UID: \"4c7c83f2-713b-4651-b1da-f65c84f8edc9\") " pod="openshift-multus/multus-additional-cni-plugins-2trlm"
Jul 31 22:56:57.974459 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.972108    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wzc4\" (UniqueName: \"kubernetes.io/projected/148fa8ca-4d3b-4ae1-8be5-30ea87aa412e-kube-api-access-5wzc4\") pod \"node-resolver-llmj8\" (UID: \"148fa8ca-4d3b-4ae1-8be5-30ea87aa412e\") " pod="openshift-dns/node-resolver-llmj8"
Jul 31 22:56:57.974459 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.972134    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/87c830a4-dd74-43c5-84d0-45cdd2dbe9a3-ovnkube-config\") pod \"ovnkube-node-w96jg\" (UID: \"87c830a4-dd74-43c5-84d0-45cdd2dbe9a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w96jg"
Jul 31 22:56:57.974459 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.972163    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/639fb18d-e947-41cc-ac53-d4616b7cc85b-host-run-k8s-cni-cncf-io\") pod \"multus-pbslg\" (UID: \"639fb18d-e947-41cc-ac53-d4616b7cc85b\") " pod="openshift-multus/multus-pbslg"
Jul 31 22:56:57.974459 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.972188    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1d0f0d49-bc8d-489a-932a-de4c94edb6a4-etc-tuned\") pod \"tuned-9wt4q\" (UID: \"1d0f0d49-bc8d-489a-932a-de4c94edb6a4\") " pod="openshift-cluster-node-tuning-operator/tuned-9wt4q"
Jul 31 22:56:57.974459 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.972208    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/639fb18d-e947-41cc-ac53-d4616b7cc85b-cni-binary-copy\") pod \"multus-pbslg\" (UID: \"639fb18d-e947-41cc-ac53-d4616b7cc85b\") " pod="openshift-multus/multus-pbslg"
Jul 31 22:56:57.974459 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.972213    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4c7c83f2-713b-4651-b1da-f65c84f8edc9-os-release\") pod \"multus-additional-cni-plugins-2trlm\" (UID: \"4c7c83f2-713b-4651-b1da-f65c84f8edc9\") " pod="openshift-multus/multus-additional-cni-plugins-2trlm"
Jul 31 22:56:57.974459 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.972278    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4c7c83f2-713b-4651-b1da-f65c84f8edc9-os-release\") pod \"multus-additional-cni-plugins-2trlm\" (UID: \"4c7c83f2-713b-4651-b1da-f65c84f8edc9\") " pod="openshift-multus/multus-additional-cni-plugins-2trlm"
Jul 31 22:56:57.974459 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.972315    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/87c830a4-dd74-43c5-84d0-45cdd2dbe9a3-log-socket\") pod \"ovnkube-node-w96jg\" (UID: \"87c830a4-dd74-43c5-84d0-45cdd2dbe9a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w96jg"
Jul 31 22:56:57.974459 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.972342    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1d0f0d49-bc8d-489a-932a-de4c94edb6a4-sys\") pod \"tuned-9wt4q\" (UID: \"1d0f0d49-bc8d-489a-932a-de4c94edb6a4\") " pod="openshift-cluster-node-tuning-operator/tuned-9wt4q"
Jul 31 22:56:57.974459 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.972366    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bvbb\" (UniqueName: \"kubernetes.io/projected/1d0f0d49-bc8d-489a-932a-de4c94edb6a4-kube-api-access-9bvbb\") pod \"tuned-9wt4q\" (UID: \"1d0f0d49-bc8d-489a-932a-de4c94edb6a4\") " pod="openshift-cluster-node-tuning-operator/tuned-9wt4q"
Jul 31 22:56:57.974459 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.972393    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3f91895a-1a8c-4ef9-8f27-0cfed935fae8-etc-selinux\") pod \"aws-ebs-csi-driver-node-7bk57\" (UID: \"3f91895a-1a8c-4ef9-8f27-0cfed935fae8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bk57"
Jul 31 22:56:57.974459 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.972402    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/87c830a4-dd74-43c5-84d0-45cdd2dbe9a3-env-overrides\") pod \"ovnkube-node-w96jg\" (UID: \"87c830a4-dd74-43c5-84d0-45cdd2dbe9a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w96jg"
Jul 31 22:56:57.974459 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.972420    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b85a1443-4bc2-48a0-8d29-35f46cbc3d57-kubelet-config\") pod \"global-pull-secret-syncer-mqwlt\" (UID: \"b85a1443-4bc2-48a0-8d29-35f46cbc3d57\") " pod="kube-system/global-pull-secret-syncer-mqwlt"
Jul 31 22:56:57.974459 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.972451    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/87c830a4-dd74-43c5-84d0-45cdd2dbe9a3-systemd-units\") pod \"ovnkube-node-w96jg\" (UID: \"87c830a4-dd74-43c5-84d0-45cdd2dbe9a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w96jg"
Jul 31 22:56:57.974459 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.972470    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/87c830a4-dd74-43c5-84d0-45cdd2dbe9a3-log-socket\") pod \"ovnkube-node-w96jg\" (UID: \"87c830a4-dd74-43c5-84d0-45cdd2dbe9a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w96jg"
Jul 31 22:56:57.974976 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.972478    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87c830a4-dd74-43c5-84d0-45cdd2dbe9a3-etc-openvswitch\") pod \"ovnkube-node-w96jg\" (UID: \"87c830a4-dd74-43c5-84d0-45cdd2dbe9a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w96jg"
Jul 31 22:56:57.974976 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.972524    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/639fb18d-e947-41cc-ac53-d4616b7cc85b-host-run-k8s-cni-cncf-io\") pod \"multus-pbslg\" (UID: \"639fb18d-e947-41cc-ac53-d4616b7cc85b\") " pod="openshift-multus/multus-pbslg"
Jul 31 22:56:57.974976 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.972532    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9wk5\" (UniqueName: \"kubernetes.io/projected/87c830a4-dd74-43c5-84d0-45cdd2dbe9a3-kube-api-access-d9wk5\") pod \"ovnkube-node-w96jg\" (UID: \"87c830a4-dd74-43c5-84d0-45cdd2dbe9a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w96jg"
Jul 31 22:56:57.974976 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.972562    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/639fb18d-e947-41cc-ac53-d4616b7cc85b-os-release\") pod \"multus-pbslg\" (UID: \"639fb18d-e947-41cc-ac53-d4616b7cc85b\") " pod="openshift-multus/multus-pbslg"
Jul 31 22:56:57.974976 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.972587    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/639fb18d-e947-41cc-ac53-d4616b7cc85b-host-run-netns\") pod \"multus-pbslg\" (UID: \"639fb18d-e947-41cc-ac53-d4616b7cc85b\") " pod="openshift-multus/multus-pbslg"
Jul 31 22:56:57.974976 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.972615    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3f91895a-1a8c-4ef9-8f27-0cfed935fae8-socket-dir\") pod \"aws-ebs-csi-driver-node-7bk57\" (UID: \"3f91895a-1a8c-4ef9-8f27-0cfed935fae8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bk57"
Jul 31 22:56:57.974976 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.972643    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4c7c83f2-713b-4651-b1da-f65c84f8edc9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2trlm\" (UID: \"4c7c83f2-713b-4651-b1da-f65c84f8edc9\") " pod="openshift-multus/multus-additional-cni-plugins-2trlm"
Jul 31 22:56:57.974976 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.972691    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3f91895a-1a8c-4ef9-8f27-0cfed935fae8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7bk57\" (UID: \"3f91895a-1a8c-4ef9-8f27-0cfed935fae8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bk57"
Jul 31 22:56:57.974976 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.972723    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87c830a4-dd74-43c5-84d0-45cdd2dbe9a3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-w96jg\" (UID: \"87c830a4-dd74-43c5-84d0-45cdd2dbe9a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w96jg"
Jul 31 22:56:57.974976 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.972726    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/87c830a4-dd74-43c5-84d0-45cdd2dbe9a3-ovnkube-script-lib\") pod \"ovnkube-node-w96jg\" (UID: \"87c830a4-dd74-43c5-84d0-45cdd2dbe9a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w96jg"
Jul 31 22:56:57.974976 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.972748    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/639fb18d-e947-41cc-ac53-d4616b7cc85b-host-var-lib-cni-multus\") pod \"multus-pbslg\" (UID: \"639fb18d-e947-41cc-ac53-d4616b7cc85b\") " pod="openshift-multus/multus-pbslg"
Jul 31 22:56:57.974976 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.972861    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4c7c83f2-713b-4651-b1da-f65c84f8edc9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2trlm\" (UID: \"4c7c83f2-713b-4651-b1da-f65c84f8edc9\") " pod="openshift-multus/multus-additional-cni-plugins-2trlm"
Jul 31 22:56:57.974976 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.972889    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/87c830a4-dd74-43c5-84d0-45cdd2dbe9a3-host-run-netns\") pod \"ovnkube-node-w96jg\" (UID: \"87c830a4-dd74-43c5-84d0-45cdd2dbe9a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w96jg"
Jul 31 22:56:57.974976 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.972916    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/639fb18d-e947-41cc-ac53-d4616b7cc85b-multus-socket-dir-parent\") pod \"multus-pbslg\" (UID: \"639fb18d-e947-41cc-ac53-d4616b7cc85b\") " pod="openshift-multus/multus-pbslg"
Jul 31 22:56:57.974976 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.972941    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1d0f0d49-bc8d-489a-932a-de4c94edb6a4-lib-modules\") pod \"tuned-9wt4q\" (UID: \"1d0f0d49-bc8d-489a-932a-de4c94edb6a4\") " pod="openshift-cluster-node-tuning-operator/tuned-9wt4q"
Jul 31 22:56:57.974976 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.972966    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3f91895a-1a8c-4ef9-8f27-0cfed935fae8-registration-dir\") pod \"aws-ebs-csi-driver-node-7bk57\" (UID: \"3f91895a-1a8c-4ef9-8f27-0cfed935fae8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bk57"
Jul 31 22:56:57.974976 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.973005    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b85a1443-4bc2-48a0-8d29-35f46cbc3d57-dbus\") pod \"global-pull-secret-syncer-mqwlt\" (UID: \"b85a1443-4bc2-48a0-8d29-35f46cbc3d57\") " pod="kube-system/global-pull-secret-syncer-mqwlt"
Jul 31 22:56:57.975486 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.973048    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/87c830a4-dd74-43c5-84d0-45cdd2dbe9a3-ovnkube-config\") pod \"ovnkube-node-w96jg\" (UID: \"87c830a4-dd74-43c5-84d0-45cdd2dbe9a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w96jg"
Jul 31 22:56:57.975486 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.973135    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b85a1443-4bc2-48a0-8d29-35f46cbc3d57-dbus\") pod \"global-pull-secret-syncer-mqwlt\" (UID: \"b85a1443-4bc2-48a0-8d29-35f46cbc3d57\") " pod="kube-system/global-pull-secret-syncer-mqwlt"
Jul 31 22:56:57.975486 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.973178    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1d0f0d49-bc8d-489a-932a-de4c94edb6a4-etc-sysctl-d\") pod \"tuned-9wt4q\" (UID: \"1d0f0d49-bc8d-489a-932a-de4c94edb6a4\") " pod="openshift-cluster-node-tuning-operator/tuned-9wt4q"
Jul 31 22:56:57.975486 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.973259    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1d0f0d49-bc8d-489a-932a-de4c94edb6a4-etc-modprobe-d\") pod \"tuned-9wt4q\" (UID: \"1d0f0d49-bc8d-489a-932a-de4c94edb6a4\") " pod="openshift-cluster-node-tuning-operator/tuned-9wt4q"
Jul 31 22:56:57.975486 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.973053    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/148fa8ca-4d3b-4ae1-8be5-30ea87aa412e-tmp-dir\") pod \"node-resolver-llmj8\" (UID: \"148fa8ca-4d3b-4ae1-8be5-30ea87aa412e\") " pod="openshift-dns/node-resolver-llmj8"
Jul 31 22:56:57.975486 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.973363    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3f91895a-1a8c-4ef9-8f27-0cfed935fae8-etc-selinux\") pod \"aws-ebs-csi-driver-node-7bk57\" (UID: \"3f91895a-1a8c-4ef9-8f27-0cfed935fae8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bk57"
Jul 31 22:56:57.975486 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.973409    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1d0f0d49-bc8d-489a-932a-de4c94edb6a4-etc-sysctl-conf\") pod \"tuned-9wt4q\" (UID: \"1d0f0d49-bc8d-489a-932a-de4c94edb6a4\") " pod="openshift-cluster-node-tuning-operator/tuned-9wt4q"
Jul 31 22:56:57.975486 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.973451    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3f91895a-1a8c-4ef9-8f27-0cfed935fae8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7bk57\" (UID: \"3f91895a-1a8c-4ef9-8f27-0cfed935fae8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bk57"
Jul 31 22:56:57.975486 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.973457    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4c7c83f2-713b-4651-b1da-f65c84f8edc9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2trlm\" (UID: \"4c7c83f2-713b-4651-b1da-f65c84f8edc9\") " pod="openshift-multus/multus-additional-cni-plugins-2trlm"
Jul 31 22:56:57.975486 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.973476    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87c830a4-dd74-43c5-84d0-45cdd2dbe9a3-etc-openvswitch\") pod \"ovnkube-node-w96jg\" (UID: \"87c830a4-dd74-43c5-84d0-45cdd2dbe9a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w96jg"
Jul 31 22:56:57.975486 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.973499    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87c830a4-dd74-43c5-84d0-45cdd2dbe9a3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-w96jg\" (UID: \"87c830a4-dd74-43c5-84d0-45cdd2dbe9a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w96jg"
Jul 31 22:56:57.975486 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.973506    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/639fb18d-e947-41cc-ac53-d4616b7cc85b-host-run-netns\") pod \"multus-pbslg\" (UID: \"639fb18d-e947-41cc-ac53-d4616b7cc85b\") " pod="openshift-multus/multus-pbslg"
Jul 31 22:56:57.975486 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.973549    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/639fb18d-e947-41cc-ac53-d4616b7cc85b-host-var-lib-cni-multus\") pod \"multus-pbslg\" (UID: \"639fb18d-e947-41cc-ac53-d4616b7cc85b\") " pod="openshift-multus/multus-pbslg"
Jul 31 22:56:57.975486 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.973565    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/639fb18d-e947-41cc-ac53-d4616b7cc85b-os-release\") pod \"multus-pbslg\" (UID: \"639fb18d-e947-41cc-ac53-d4616b7cc85b\") " pod="openshift-multus/multus-pbslg"
Jul 31 22:56:57.975486 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.973563    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/eb4e0ba5-fe8b-4bf7-a86d-15db1a2532b2-agent-certs\") pod \"konnectivity-agent-shjgx\" (UID: \"eb4e0ba5-fe8b-4bf7-a86d-15db1a2532b2\") " pod="kube-system/konnectivity-agent-shjgx"
Jul 31 22:56:57.975486 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.973603    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3f91895a-1a8c-4ef9-8f27-0cfed935fae8-socket-dir\") pod \"aws-ebs-csi-driver-node-7bk57\" (UID: \"3f91895a-1a8c-4ef9-8f27-0cfed935fae8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bk57"
Jul 31 22:56:57.975486 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.973629    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3f91895a-1a8c-4ef9-8f27-0cfed935fae8-registration-dir\") pod \"aws-ebs-csi-driver-node-7bk57\" (UID: \"3f91895a-1a8c-4ef9-8f27-0cfed935fae8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bk57"
Jul 31 22:56:57.975486 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.973633    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/87c830a4-dd74-43c5-84d0-45cdd2dbe9a3-host-run-netns\") pod \"ovnkube-node-w96jg\" (UID: \"87c830a4-dd74-43c5-84d0-45cdd2dbe9a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w96jg"
Jul 31 22:56:57.976031 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.973648    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/87c830a4-dd74-43c5-84d0-45cdd2dbe9a3-node-log\") pod \"ovnkube-node-w96jg\" (UID: \"87c830a4-dd74-43c5-84d0-45cdd2dbe9a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w96jg"
Jul 31 22:56:57.976031 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.973686    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/639fb18d-e947-41cc-ac53-d4616b7cc85b-multus-socket-dir-parent\") pod \"multus-pbslg\" (UID: \"639fb18d-e947-41cc-ac53-d4616b7cc85b\") " pod="openshift-multus/multus-pbslg"
Jul 31 22:56:57.976031 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.973721    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/87c830a4-dd74-43c5-84d0-45cdd2dbe9a3-systemd-units\") pod \"ovnkube-node-w96jg\" (UID: \"87c830a4-dd74-43c5-84d0-45cdd2dbe9a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w96jg"
Jul 31 22:56:57.976031 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.973724    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1d0f0d49-bc8d-489a-932a-de4c94edb6a4-lib-modules\") pod \"tuned-9wt4q\" (UID: \"1d0f0d49-bc8d-489a-932a-de4c94edb6a4\") " pod="openshift-cluster-node-tuning-operator/tuned-9wt4q"
Jul 31 22:56:57.976031 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.973722    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1d0f0d49-bc8d-489a-932a-de4c94edb6a4-sys\") pod \"tuned-9wt4q\" (UID: \"1d0f0d49-bc8d-489a-932a-de4c94edb6a4\") " pod="openshift-cluster-node-tuning-operator/tuned-9wt4q"
Jul 31 22:56:57.976031 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.973755    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b85a1443-4bc2-48a0-8d29-35f46cbc3d57-kubelet-config\") pod \"global-pull-secret-syncer-mqwlt\" (UID: \"b85a1443-4bc2-48a0-8d29-35f46cbc3d57\") " pod="kube-system/global-pull-secret-syncer-mqwlt"
Jul 31 22:56:57.976031 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.973938    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/87c830a4-dd74-43c5-84d0-45cdd2dbe9a3-ovn-node-metrics-cert\") pod \"ovnkube-node-w96jg\" (UID: \"87c830a4-dd74-43c5-84d0-45cdd2dbe9a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w96jg"
Jul 31 22:56:57.976031 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.974104    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4c7c83f2-713b-4651-b1da-f65c84f8edc9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2trlm\" (UID: \"4c7c83f2-713b-4651-b1da-f65c84f8edc9\") " pod="openshift-multus/multus-additional-cni-plugins-2trlm"
Jul 31 22:56:57.976031 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.974331    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1d0f0d49-bc8d-489a-932a-de4c94edb6a4-tmp\") pod \"tuned-9wt4q\" (UID: \"1d0f0d49-bc8d-489a-932a-de4c94edb6a4\") " pod="openshift-cluster-node-tuning-operator/tuned-9wt4q"
Jul 31 22:56:57.976031 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.974891    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1d0f0d49-bc8d-489a-932a-de4c94edb6a4-etc-tuned\") pod \"tuned-9wt4q\" (UID: \"1d0f0d49-bc8d-489a-932a-de4c94edb6a4\") " pod="openshift-cluster-node-tuning-operator/tuned-9wt4q"
Jul 31 22:56:57.976031 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.974910    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/639fb18d-e947-41cc-ac53-d4616b7cc85b-multus-daemon-config\") pod \"multus-pbslg\" (UID: \"639fb18d-e947-41cc-ac53-d4616b7cc85b\") " pod="openshift-multus/multus-pbslg"
Jul 31 22:56:57.978718 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:56:57.978700    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 31 22:56:57.978756 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:56:57.978722    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 31 22:56:57.978756 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:56:57.978734    2584 projected.go:196] Error preparing data for projected volume kube-api-access-bwbjs for pod openshift-network-diagnostics/network-check-target-84css: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 31 22:56:57.978846 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:56:57.978803    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b80644e6-82bb-4c59-86ed-b60bd24e6a05-kube-api-access-bwbjs podName:b80644e6-82bb-4c59-86ed-b60bd24e6a05 nodeName:}" failed. No retries permitted until 2026-07-31 22:56:58.478783652 +0000 UTC m=+2.124569955 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-bwbjs" (UniqueName: "kubernetes.io/projected/b80644e6-82bb-4c59-86ed-b60bd24e6a05-kube-api-access-bwbjs") pod "network-check-target-84css" (UID: "b80644e6-82bb-4c59-86ed-b60bd24e6a05") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 31 22:56:57.980973 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.980953    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2m5pb\" (UniqueName: \"kubernetes.io/projected/639fb18d-e947-41cc-ac53-d4616b7cc85b-kube-api-access-2m5pb\") pod \"multus-pbslg\" (UID: \"639fb18d-e947-41cc-ac53-d4616b7cc85b\") " pod="openshift-multus/multus-pbslg"
Jul 31 22:56:57.982022 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.981999    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-klbxv\" (UniqueName: \"kubernetes.io/projected/3f91895a-1a8c-4ef9-8f27-0cfed935fae8-kube-api-access-klbxv\") pod \"aws-ebs-csi-driver-node-7bk57\" (UID: \"3f91895a-1a8c-4ef9-8f27-0cfed935fae8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bk57"
Jul 31 22:56:57.982747 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.982729    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9wk5\" (UniqueName: \"kubernetes.io/projected/87c830a4-dd74-43c5-84d0-45cdd2dbe9a3-kube-api-access-d9wk5\") pod \"ovnkube-node-w96jg\" (UID: \"87c830a4-dd74-43c5-84d0-45cdd2dbe9a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w96jg"
Jul 31 22:56:57.983002 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.982985    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wzc4\" (UniqueName: \"kubernetes.io/projected/148fa8ca-4d3b-4ae1-8be5-30ea87aa412e-kube-api-access-5wzc4\") pod \"node-resolver-llmj8\" (UID: \"148fa8ca-4d3b-4ae1-8be5-30ea87aa412e\") " pod="openshift-dns/node-resolver-llmj8"
Jul 31 22:56:57.983462 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.983437    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4sqv4\" (UniqueName: \"kubernetes.io/projected/4c7c83f2-713b-4651-b1da-f65c84f8edc9-kube-api-access-4sqv4\") pod \"multus-additional-cni-plugins-2trlm\" (UID: \"4c7c83f2-713b-4651-b1da-f65c84f8edc9\") " pod="openshift-multus/multus-additional-cni-plugins-2trlm"
Jul 31 22:56:57.983536 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:57.983462    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bvbb\" (UniqueName: \"kubernetes.io/projected/1d0f0d49-bc8d-489a-932a-de4c94edb6a4-kube-api-access-9bvbb\") pod \"tuned-9wt4q\" (UID: \"1d0f0d49-bc8d-489a-932a-de4c94edb6a4\") " pod="openshift-cluster-node-tuning-operator/tuned-9wt4q"
Jul 31 22:56:58.090232 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:58.090116    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-t8qkd"
Jul 31 22:56:58.100400 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:58.100371    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bb4d7d9_ff22_41eb_8883_b7c1942767fb.slice/crio-0ae91e71b636f4cb3bd13555a06771c553a7e1ff9b15b23b92ec34a96cb60430 WatchSource:0}: Error finding container 0ae91e71b636f4cb3bd13555a06771c553a7e1ff9b15b23b92ec34a96cb60430: Status 404 returned error can't find the container with id 0ae91e71b636f4cb3bd13555a06771c553a7e1ff9b15b23b92ec34a96cb60430
Jul 31 22:56:58.102446 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:58.102403    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qchlw"
Jul 31 22:56:58.110867 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:58.110846    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a9cc748_9a60_4656_9236_34d75e5cb22a.slice/crio-ab03a23b272a3ce008ea980580ec88c6c62383f5396af40655cb7f7f5a386309 WatchSource:0}: Error finding container ab03a23b272a3ce008ea980580ec88c6c62383f5396af40655cb7f7f5a386309: Status 404 returned error can't find the container with id ab03a23b272a3ce008ea980580ec88c6c62383f5396af40655cb7f7f5a386309
Jul 31 22:56:58.112728 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:58.112706    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9wt4q"
Jul 31 22:56:58.118236 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:58.118219    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-w96jg"
Jul 31 22:56:58.120998 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:58.120976    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d0f0d49_bc8d_489a_932a_de4c94edb6a4.slice/crio-6698449e621bc45d3d86632c11f2fdaab82ff7473b6ba4444ffd63f7d0b9d574 WatchSource:0}: Error finding container 6698449e621bc45d3d86632c11f2fdaab82ff7473b6ba4444ffd63f7d0b9d574: Status 404 returned error can't find the container with id 6698449e621bc45d3d86632c11f2fdaab82ff7473b6ba4444ffd63f7d0b9d574
Jul 31 22:56:58.127229 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:58.127201    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87c830a4_dd74_43c5_84d0_45cdd2dbe9a3.slice/crio-026ffb16c85b6fbb667ea84d88805bcde1063f884f65bd14871054cba37239e1 WatchSource:0}: Error finding container 026ffb16c85b6fbb667ea84d88805bcde1063f884f65bd14871054cba37239e1: Status 404 returned error can't find the container with id 026ffb16c85b6fbb667ea84d88805bcde1063f884f65bd14871054cba37239e1
Jul 31 22:56:58.150829 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:58.150803    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-pbslg"
Jul 31 22:56:58.160096 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:58.160074    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod639fb18d_e947_41cc_ac53_d4616b7cc85b.slice/crio-717d5c4b012a8fad4a7e9399aff1c486b8a109e655b3b13dcd02aa99e369972e WatchSource:0}: Error finding container 717d5c4b012a8fad4a7e9399aff1c486b8a109e655b3b13dcd02aa99e369972e: Status 404 returned error can't find the container with id 717d5c4b012a8fad4a7e9399aff1c486b8a109e655b3b13dcd02aa99e369972e
Jul 31 22:56:58.165450 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:58.165434    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bk57"
Jul 31 22:56:58.175000 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:58.174978    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f91895a_1a8c_4ef9_8f27_0cfed935fae8.slice/crio-0f70bad58fad441a0a80d68f4273c37fc58e067430097a1774f069f688c68c04 WatchSource:0}: Error finding container 0f70bad58fad441a0a80d68f4273c37fc58e067430097a1774f069f688c68c04: Status 404 returned error can't find the container with id 0f70bad58fad441a0a80d68f4273c37fc58e067430097a1774f069f688c68c04
Jul 31 22:56:58.180267 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:58.180244    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-shjgx"
Jul 31 22:56:58.188187 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:58.188161    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb4e0ba5_fe8b_4bf7_a86d_15db1a2532b2.slice/crio-17be9d1781b3c198810855102bee4850d900320d0b988bd0e6f92903401e2f15 WatchSource:0}: Error finding container 17be9d1781b3c198810855102bee4850d900320d0b988bd0e6f92903401e2f15: Status 404 returned error can't find the container with id 17be9d1781b3c198810855102bee4850d900320d0b988bd0e6f92903401e2f15
Jul 31 22:56:58.196540 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:58.196522    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-llmj8"
Jul 31 22:56:58.201552 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:58.201534    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2trlm"
Jul 31 22:56:58.206880 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:58.206857    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod148fa8ca_4d3b_4ae1_8be5_30ea87aa412e.slice/crio-e02049cc2fe9bdf81809d4d1fa7445c591ab24f9470be03fea3cce236b7032c4 WatchSource:0}: Error finding container e02049cc2fe9bdf81809d4d1fa7445c591ab24f9470be03fea3cce236b7032c4: Status 404 returned error can't find the container with id e02049cc2fe9bdf81809d4d1fa7445c591ab24f9470be03fea3cce236b7032c4
Jul 31 22:56:58.211065 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:56:58.211044    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c7c83f2_713b_4651_b1da_f65c84f8edc9.slice/crio-9be21cd2a85ac1322c422fe3c200b6e7f0394a142700e9c32078c23971b83d36 WatchSource:0}: Error finding container 9be21cd2a85ac1322c422fe3c200b6e7f0394a142700e9c32078c23971b83d36: Status 404 returned error can't find the container with id 9be21cd2a85ac1322c422fe3c200b6e7f0394a142700e9c32078c23971b83d36
Jul 31 22:56:58.298927 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:58.298900    2584 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 31 22:56:58.375752 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:58.375651    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945-metrics-certs\") pod \"network-metrics-daemon-mz5wz\" (UID: \"b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945\") " pod="openshift-multus/network-metrics-daemon-mz5wz"
Jul 31 22:56:58.375900 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:56:58.375808    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 31 22:56:58.375900 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:56:58.375866    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945-metrics-certs podName:b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945 nodeName:}" failed. No retries permitted until 2026-07-31 22:56:59.37585089 +0000 UTC m=+3.021637181 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945-metrics-certs") pod "network-metrics-daemon-mz5wz" (UID: "b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 31 22:56:58.477057 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:58.477019    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b85a1443-4bc2-48a0-8d29-35f46cbc3d57-original-pull-secret\") pod \"global-pull-secret-syncer-mqwlt\" (UID: \"b85a1443-4bc2-48a0-8d29-35f46cbc3d57\") " pod="kube-system/global-pull-secret-syncer-mqwlt"
Jul 31 22:56:58.477243 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:56:58.477183    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 31 22:56:58.477305 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:56:58.477243    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b85a1443-4bc2-48a0-8d29-35f46cbc3d57-original-pull-secret podName:b85a1443-4bc2-48a0-8d29-35f46cbc3d57 nodeName:}" failed. No retries permitted until 2026-07-31 22:56:59.477224953 +0000 UTC m=+3.123011263 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b85a1443-4bc2-48a0-8d29-35f46cbc3d57-original-pull-secret") pod "global-pull-secret-syncer-mqwlt" (UID: "b85a1443-4bc2-48a0-8d29-35f46cbc3d57") : object "kube-system"/"original-pull-secret" not registered
Jul 31 22:56:58.578235 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:58.578183    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwbjs\" (UniqueName: \"kubernetes.io/projected/b80644e6-82bb-4c59-86ed-b60bd24e6a05-kube-api-access-bwbjs\") pod \"network-check-target-84css\" (UID: \"b80644e6-82bb-4c59-86ed-b60bd24e6a05\") " pod="openshift-network-diagnostics/network-check-target-84css"
Jul 31 22:56:58.578401 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:56:58.578369    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 31 22:56:58.578401 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:56:58.578389    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 31 22:56:58.578401 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:56:58.578402    2584 projected.go:196] Error preparing data for projected volume kube-api-access-bwbjs for pod openshift-network-diagnostics/network-check-target-84css: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 31 22:56:58.578582 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:56:58.578460    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b80644e6-82bb-4c59-86ed-b60bd24e6a05-kube-api-access-bwbjs podName:b80644e6-82bb-4c59-86ed-b60bd24e6a05 nodeName:}" failed. No retries permitted until 2026-07-31 22:56:59.578440352 +0000 UTC m=+3.224226655 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-bwbjs" (UniqueName: "kubernetes.io/projected/b80644e6-82bb-4c59-86ed-b60bd24e6a05-kube-api-access-bwbjs") pod "network-check-target-84css" (UID: "b80644e6-82bb-4c59-86ed-b60bd24e6a05") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 31 22:56:58.815422 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:58.815261    2584 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-30 22:51:58 +0000 UTC" deadline="2028-01-12 01:15:05.407915212 +0000 UTC"
Jul 31 22:56:58.815422 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:58.815321    2584 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12698h18m6.592599473s"
Jul 31 22:56:58.933335 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:58.933266    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2trlm" event={"ID":"4c7c83f2-713b-4651-b1da-f65c84f8edc9","Type":"ContainerStarted","Data":"9be21cd2a85ac1322c422fe3c200b6e7f0394a142700e9c32078c23971b83d36"}
Jul 31 22:56:58.955903 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:58.955820    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-llmj8" event={"ID":"148fa8ca-4d3b-4ae1-8be5-30ea87aa412e","Type":"ContainerStarted","Data":"e02049cc2fe9bdf81809d4d1fa7445c591ab24f9470be03fea3cce236b7032c4"}
Jul 31 22:56:58.962330 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:58.962291    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w96jg" event={"ID":"87c830a4-dd74-43c5-84d0-45cdd2dbe9a3","Type":"ContainerStarted","Data":"026ffb16c85b6fbb667ea84d88805bcde1063f884f65bd14871054cba37239e1"}
Jul 31 22:56:58.971088 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:58.971058    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-243.ec2.internal" event={"ID":"59797420703085b3d07401083a2a0810","Type":"ContainerStarted","Data":"cb38daa36f2224d21c1297d162427cfb61ef6f334ef26e22ce5d8123e2508b06"}
Jul 31 22:56:58.982071 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:58.982037    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-shjgx" event={"ID":"eb4e0ba5-fe8b-4bf7-a86d-15db1a2532b2","Type":"ContainerStarted","Data":"17be9d1781b3c198810855102bee4850d900320d0b988bd0e6f92903401e2f15"}
Jul 31 22:56:59.002880 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:59.002844    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bk57" event={"ID":"3f91895a-1a8c-4ef9-8f27-0cfed935fae8","Type":"ContainerStarted","Data":"0f70bad58fad441a0a80d68f4273c37fc58e067430097a1774f069f688c68c04"}
Jul 31 22:56:59.014084 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:59.014049    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-pbslg" event={"ID":"639fb18d-e947-41cc-ac53-d4616b7cc85b","Type":"ContainerStarted","Data":"717d5c4b012a8fad4a7e9399aff1c486b8a109e655b3b13dcd02aa99e369972e"}
Jul 31 22:56:59.034982 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:59.033952    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9wt4q" event={"ID":"1d0f0d49-bc8d-489a-932a-de4c94edb6a4","Type":"ContainerStarted","Data":"6698449e621bc45d3d86632c11f2fdaab82ff7473b6ba4444ffd63f7d0b9d574"}
Jul 31 22:56:59.052819 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:59.048549    2584 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 31 22:56:59.052819 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:59.049814    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qchlw" event={"ID":"9a9cc748-9a60-4656-9236-34d75e5cb22a","Type":"ContainerStarted","Data":"ab03a23b272a3ce008ea980580ec88c6c62383f5396af40655cb7f7f5a386309"}
Jul 31 22:56:59.063445 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:59.063358    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-t8qkd" event={"ID":"0bb4d7d9-ff22-41eb-8883-b7c1942767fb","Type":"ContainerStarted","Data":"0ae91e71b636f4cb3bd13555a06771c553a7e1ff9b15b23b92ec34a96cb60430"}
Jul 31 22:56:59.090925 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:59.090793    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-243.ec2.internal" event={"ID":"bf2bb11c3449d46ed43b4ff9c8ef237d","Type":"ContainerStarted","Data":"f1075e4f8bf1fb142a40a5d00d4643c353b508c698ca3a31baafe316187a9bce"}
Jul 31 22:56:59.279380 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:59.279302    2584 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 31 22:56:59.385135 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:59.385052    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945-metrics-certs\") pod \"network-metrics-daemon-mz5wz\" (UID: \"b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945\") " pod="openshift-multus/network-metrics-daemon-mz5wz"
Jul 31 22:56:59.385299 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:56:59.385236    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 31 22:56:59.385360 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:56:59.385302    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945-metrics-certs podName:b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945 nodeName:}" failed. No retries permitted until 2026-07-31 22:57:01.385283165 +0000 UTC m=+5.031069457 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945-metrics-certs") pod "network-metrics-daemon-mz5wz" (UID: "b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 31 22:56:59.487438 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:59.487316    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b85a1443-4bc2-48a0-8d29-35f46cbc3d57-original-pull-secret\") pod \"global-pull-secret-syncer-mqwlt\" (UID: \"b85a1443-4bc2-48a0-8d29-35f46cbc3d57\") " pod="kube-system/global-pull-secret-syncer-mqwlt"
Jul 31 22:56:59.487636 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:56:59.487500    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 31 22:56:59.487636 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:56:59.487579    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b85a1443-4bc2-48a0-8d29-35f46cbc3d57-original-pull-secret podName:b85a1443-4bc2-48a0-8d29-35f46cbc3d57 nodeName:}" failed. No retries permitted until 2026-07-31 22:57:01.487557454 +0000 UTC m=+5.133343747 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b85a1443-4bc2-48a0-8d29-35f46cbc3d57-original-pull-secret") pod "global-pull-secret-syncer-mqwlt" (UID: "b85a1443-4bc2-48a0-8d29-35f46cbc3d57") : object "kube-system"/"original-pull-secret" not registered
Jul 31 22:56:59.589028 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:59.588343    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwbjs\" (UniqueName: \"kubernetes.io/projected/b80644e6-82bb-4c59-86ed-b60bd24e6a05-kube-api-access-bwbjs\") pod \"network-check-target-84css\" (UID: \"b80644e6-82bb-4c59-86ed-b60bd24e6a05\") " pod="openshift-network-diagnostics/network-check-target-84css"
Jul 31 22:56:59.589028 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:56:59.588508    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 31 22:56:59.589028 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:56:59.588527    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 31 22:56:59.589028 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:56:59.588541    2584 projected.go:196] Error preparing data for projected volume kube-api-access-bwbjs for pod openshift-network-diagnostics/network-check-target-84css: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 31 22:56:59.589028 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:56:59.588604    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b80644e6-82bb-4c59-86ed-b60bd24e6a05-kube-api-access-bwbjs podName:b80644e6-82bb-4c59-86ed-b60bd24e6a05 nodeName:}" failed. No retries permitted until 2026-07-31 22:57:01.588584748 +0000 UTC m=+5.234371042 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-bwbjs" (UniqueName: "kubernetes.io/projected/b80644e6-82bb-4c59-86ed-b60bd24e6a05-kube-api-access-bwbjs") pod "network-check-target-84css" (UID: "b80644e6-82bb-4c59-86ed-b60bd24e6a05") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 31 22:56:59.816046 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:59.815955    2584 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-30 22:51:58 +0000 UTC" deadline="2028-05-13 20:40:29.757029362 +0000 UTC"
Jul 31 22:56:59.816046 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:59.816023    2584 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15645h43m29.941040179s"
Jul 31 22:56:59.885273 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:59.885230    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mz5wz"
Jul 31 22:56:59.885478 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:56:59.885410    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mz5wz" podUID="b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945"
Jul 31 22:56:59.885906 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:59.885883    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-84css"
Jul 31 22:56:59.886022 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:56:59.885988    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-84css" podUID="b80644e6-82bb-4c59-86ed-b60bd24e6a05"
Jul 31 22:56:59.886082 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:56:59.886052    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mqwlt"
Jul 31 22:56:59.886189 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:56:59.886138    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mqwlt" podUID="b85a1443-4bc2-48a0-8d29-35f46cbc3d57"
Jul 31 22:57:00.774354 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:00.774125    2584 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 31 22:57:01.405645 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:01.405603    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945-metrics-certs\") pod \"network-metrics-daemon-mz5wz\" (UID: \"b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945\") " pod="openshift-multus/network-metrics-daemon-mz5wz"
Jul 31 22:57:01.406288 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:01.405847    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 31 22:57:01.406288 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:01.405919    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945-metrics-certs podName:b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945 nodeName:}" failed. No retries permitted until 2026-07-31 22:57:05.405898665 +0000 UTC m=+9.051684967 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945-metrics-certs") pod "network-metrics-daemon-mz5wz" (UID: "b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 31 22:57:01.506797 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:01.506330    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b85a1443-4bc2-48a0-8d29-35f46cbc3d57-original-pull-secret\") pod \"global-pull-secret-syncer-mqwlt\" (UID: \"b85a1443-4bc2-48a0-8d29-35f46cbc3d57\") " pod="kube-system/global-pull-secret-syncer-mqwlt"
Jul 31 22:57:01.506797 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:01.506537    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 31 22:57:01.506797 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:01.506610    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b85a1443-4bc2-48a0-8d29-35f46cbc3d57-original-pull-secret podName:b85a1443-4bc2-48a0-8d29-35f46cbc3d57 nodeName:}" failed. No retries permitted until 2026-07-31 22:57:05.506590345 +0000 UTC m=+9.152376635 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b85a1443-4bc2-48a0-8d29-35f46cbc3d57-original-pull-secret") pod "global-pull-secret-syncer-mqwlt" (UID: "b85a1443-4bc2-48a0-8d29-35f46cbc3d57") : object "kube-system"/"original-pull-secret" not registered
Jul 31 22:57:01.608174 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:01.607463    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwbjs\" (UniqueName: \"kubernetes.io/projected/b80644e6-82bb-4c59-86ed-b60bd24e6a05-kube-api-access-bwbjs\") pod \"network-check-target-84css\" (UID: \"b80644e6-82bb-4c59-86ed-b60bd24e6a05\") " pod="openshift-network-diagnostics/network-check-target-84css"
Jul 31 22:57:01.608174 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:01.607694    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 31 22:57:01.608174 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:01.607718    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 31 22:57:01.608174 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:01.607732    2584 projected.go:196] Error preparing data for projected volume kube-api-access-bwbjs for pod openshift-network-diagnostics/network-check-target-84css: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 31 22:57:01.608174 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:01.607815    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b80644e6-82bb-4c59-86ed-b60bd24e6a05-kube-api-access-bwbjs podName:b80644e6-82bb-4c59-86ed-b60bd24e6a05 nodeName:}" failed. No retries permitted until 2026-07-31 22:57:05.607794079 +0000 UTC m=+9.253580368 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-bwbjs" (UniqueName: "kubernetes.io/projected/b80644e6-82bb-4c59-86ed-b60bd24e6a05-kube-api-access-bwbjs") pod "network-check-target-84css" (UID: "b80644e6-82bb-4c59-86ed-b60bd24e6a05") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 31 22:57:01.885056 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:01.885023    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-84css"
Jul 31 22:57:01.885264 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:01.885066    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mqwlt"
Jul 31 22:57:01.885264 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:01.885023    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mz5wz"
Jul 31 22:57:01.885264 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:01.885209    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-84css" podUID="b80644e6-82bb-4c59-86ed-b60bd24e6a05"
Jul 31 22:57:01.885439 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:01.885360    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mz5wz" podUID="b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945"
Jul 31 22:57:01.885497 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:01.885450    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mqwlt" podUID="b85a1443-4bc2-48a0-8d29-35f46cbc3d57"
Jul 31 22:57:03.885568 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:03.885020    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mz5wz"
Jul 31 22:57:03.885568 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:03.885192    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mz5wz" podUID="b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945"
Jul 31 22:57:03.885568 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:03.885262    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-84css"
Jul 31 22:57:03.885568 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:03.885382    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-84css" podUID="b80644e6-82bb-4c59-86ed-b60bd24e6a05"
Jul 31 22:57:03.885568 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:03.885414    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mqwlt"
Jul 31 22:57:03.885568 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:03.885530    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mqwlt" podUID="b85a1443-4bc2-48a0-8d29-35f46cbc3d57"
Jul 31 22:57:05.443841 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:05.443694    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945-metrics-certs\") pod \"network-metrics-daemon-mz5wz\" (UID: \"b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945\") " pod="openshift-multus/network-metrics-daemon-mz5wz"
Jul 31 22:57:05.444347 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:05.443845    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 31 22:57:05.444347 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:05.443933    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945-metrics-certs podName:b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945 nodeName:}" failed. No retries permitted until 2026-07-31 22:57:13.443907528 +0000 UTC m=+17.089693825 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945-metrics-certs") pod "network-metrics-daemon-mz5wz" (UID: "b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 31 22:57:05.544781 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:05.544677    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b85a1443-4bc2-48a0-8d29-35f46cbc3d57-original-pull-secret\") pod \"global-pull-secret-syncer-mqwlt\" (UID: \"b85a1443-4bc2-48a0-8d29-35f46cbc3d57\") " pod="kube-system/global-pull-secret-syncer-mqwlt"
Jul 31 22:57:05.544978 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:05.544872    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 31 22:57:05.544978 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:05.544956    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b85a1443-4bc2-48a0-8d29-35f46cbc3d57-original-pull-secret podName:b85a1443-4bc2-48a0-8d29-35f46cbc3d57 nodeName:}" failed. No retries permitted until 2026-07-31 22:57:13.544934089 +0000 UTC m=+17.190720384 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b85a1443-4bc2-48a0-8d29-35f46cbc3d57-original-pull-secret") pod "global-pull-secret-syncer-mqwlt" (UID: "b85a1443-4bc2-48a0-8d29-35f46cbc3d57") : object "kube-system"/"original-pull-secret" not registered
Jul 31 22:57:05.645512 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:05.645342    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwbjs\" (UniqueName: \"kubernetes.io/projected/b80644e6-82bb-4c59-86ed-b60bd24e6a05-kube-api-access-bwbjs\") pod \"network-check-target-84css\" (UID: \"b80644e6-82bb-4c59-86ed-b60bd24e6a05\") " pod="openshift-network-diagnostics/network-check-target-84css"
Jul 31 22:57:05.645697 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:05.645586    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 31 22:57:05.645697 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:05.645616    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 31 22:57:05.645697 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:05.645631    2584 projected.go:196] Error preparing data for projected volume kube-api-access-bwbjs for pod openshift-network-diagnostics/network-check-target-84css: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 31 22:57:05.645697 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:05.645697    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b80644e6-82bb-4c59-86ed-b60bd24e6a05-kube-api-access-bwbjs podName:b80644e6-82bb-4c59-86ed-b60bd24e6a05 nodeName:}" failed. No retries permitted until 2026-07-31 22:57:13.645676068 +0000 UTC m=+17.291462379 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-bwbjs" (UniqueName: "kubernetes.io/projected/b80644e6-82bb-4c59-86ed-b60bd24e6a05-kube-api-access-bwbjs") pod "network-check-target-84css" (UID: "b80644e6-82bb-4c59-86ed-b60bd24e6a05") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 31 22:57:05.885691 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:05.885576    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mz5wz"
Jul 31 22:57:05.885900 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:05.885750    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mz5wz" podUID="b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945"
Jul 31 22:57:05.886136 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:05.886115    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mqwlt"
Jul 31 22:57:05.886411 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:05.886234    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mqwlt" podUID="b85a1443-4bc2-48a0-8d29-35f46cbc3d57"
Jul 31 22:57:05.886411 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:05.886281    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-84css"
Jul 31 22:57:05.886411 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:05.886351    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-84css" podUID="b80644e6-82bb-4c59-86ed-b60bd24e6a05"
Jul 31 22:57:07.885126 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:07.885087    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-84css"
Jul 31 22:57:07.885126 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:07.885120    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mz5wz"
Jul 31 22:57:07.885677 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:07.885136    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mqwlt"
Jul 31 22:57:07.885677 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:07.885237    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-84css" podUID="b80644e6-82bb-4c59-86ed-b60bd24e6a05"
Jul 31 22:57:07.885677 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:07.885345    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mz5wz" podUID="b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945"
Jul 31 22:57:07.885677 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:07.885444    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mqwlt" podUID="b85a1443-4bc2-48a0-8d29-35f46cbc3d57"
Jul 31 22:57:09.885072 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:09.885036    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-84css"
Jul 31 22:57:09.885589 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:09.885147    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mz5wz"
Jul 31 22:57:09.885589 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:09.885161    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-84css" podUID="b80644e6-82bb-4c59-86ed-b60bd24e6a05"
Jul 31 22:57:09.885589 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:09.885176    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mqwlt"
Jul 31 22:57:09.885589 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:09.885261    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mz5wz" podUID="b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945"
Jul 31 22:57:09.885589 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:09.885330    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mqwlt" podUID="b85a1443-4bc2-48a0-8d29-35f46cbc3d57"
Jul 31 22:57:11.884906 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:11.884860    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-84css"
Jul 31 22:57:11.884906 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:11.884898    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mz5wz"
Jul 31 22:57:11.885380 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:11.884867    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mqwlt"
Jul 31 22:57:11.885380 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:11.885011    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mz5wz" podUID="b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945"
Jul 31 22:57:11.885380 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:11.885067    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mqwlt" podUID="b85a1443-4bc2-48a0-8d29-35f46cbc3d57"
Jul 31 22:57:11.885380 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:11.885092    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-84css" podUID="b80644e6-82bb-4c59-86ed-b60bd24e6a05"
Jul 31 22:57:13.508298 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:13.508258    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945-metrics-certs\") pod \"network-metrics-daemon-mz5wz\" (UID: \"b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945\") " pod="openshift-multus/network-metrics-daemon-mz5wz"
Jul 31 22:57:13.508804 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:13.508440    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 31 22:57:13.508804 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:13.508511    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945-metrics-certs podName:b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945 nodeName:}" failed. No retries permitted until 2026-07-31 22:57:29.508495401 +0000 UTC m=+33.154281692 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945-metrics-certs") pod "network-metrics-daemon-mz5wz" (UID: "b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 31 22:57:13.609420 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:13.609376    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b85a1443-4bc2-48a0-8d29-35f46cbc3d57-original-pull-secret\") pod \"global-pull-secret-syncer-mqwlt\" (UID: \"b85a1443-4bc2-48a0-8d29-35f46cbc3d57\") " pod="kube-system/global-pull-secret-syncer-mqwlt"
Jul 31 22:57:13.609608 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:13.609561    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 31 22:57:13.609676 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:13.609643    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b85a1443-4bc2-48a0-8d29-35f46cbc3d57-original-pull-secret podName:b85a1443-4bc2-48a0-8d29-35f46cbc3d57 nodeName:}" failed. No retries permitted until 2026-07-31 22:57:29.609621154 +0000 UTC m=+33.255407454 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b85a1443-4bc2-48a0-8d29-35f46cbc3d57-original-pull-secret") pod "global-pull-secret-syncer-mqwlt" (UID: "b85a1443-4bc2-48a0-8d29-35f46cbc3d57") : object "kube-system"/"original-pull-secret" not registered
Jul 31 22:57:13.710138 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:13.710101    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwbjs\" (UniqueName: \"kubernetes.io/projected/b80644e6-82bb-4c59-86ed-b60bd24e6a05-kube-api-access-bwbjs\") pod \"network-check-target-84css\" (UID: \"b80644e6-82bb-4c59-86ed-b60bd24e6a05\") " pod="openshift-network-diagnostics/network-check-target-84css"
Jul 31 22:57:13.710303 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:13.710265    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 31 22:57:13.710303 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:13.710288    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 31 22:57:13.710303 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:13.710301    2584 projected.go:196] Error preparing data for projected volume kube-api-access-bwbjs for pod openshift-network-diagnostics/network-check-target-84css: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 31 22:57:13.710473 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:13.710372    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b80644e6-82bb-4c59-86ed-b60bd24e6a05-kube-api-access-bwbjs podName:b80644e6-82bb-4c59-86ed-b60bd24e6a05 nodeName:}" failed. No retries permitted until 2026-07-31 22:57:29.710351746 +0000 UTC m=+33.356138045 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-bwbjs" (UniqueName: "kubernetes.io/projected/b80644e6-82bb-4c59-86ed-b60bd24e6a05-kube-api-access-bwbjs") pod "network-check-target-84css" (UID: "b80644e6-82bb-4c59-86ed-b60bd24e6a05") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 31 22:57:13.885190 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:13.885108    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-84css"
Jul 31 22:57:13.885342 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:13.885214    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mqwlt"
Jul 31 22:57:13.885342 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:13.885237    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mz5wz"
Jul 31 22:57:13.885342 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:13.885226    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-84css" podUID="b80644e6-82bb-4c59-86ed-b60bd24e6a05"
Jul 31 22:57:13.885342 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:13.885328    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mqwlt" podUID="b85a1443-4bc2-48a0-8d29-35f46cbc3d57"
Jul 31 22:57:13.885557 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:13.885410    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mz5wz" podUID="b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945"
Jul 31 22:57:15.885812 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:15.885780    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-84css"
Jul 31 22:57:15.885812 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:15.885821    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mz5wz"
Jul 31 22:57:15.886334 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:15.885792    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mqwlt"
Jul 31 22:57:15.886334 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:15.885891    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-84css" podUID="b80644e6-82bb-4c59-86ed-b60bd24e6a05"
Jul 31 22:57:15.886334 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:15.885989    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mz5wz" podUID="b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945"
Jul 31 22:57:15.886334 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:15.886070    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mqwlt" podUID="b85a1443-4bc2-48a0-8d29-35f46cbc3d57"
Jul 31 22:57:17.885815 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:17.885773    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mz5wz"
Jul 31 22:57:17.886267 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:17.885787    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-84css"
Jul 31 22:57:17.886267 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:17.885896    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mqwlt"
Jul 31 22:57:17.886267 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:17.885889    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mz5wz" podUID="b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945"
Jul 31 22:57:17.886267 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:17.885986    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-84css" podUID="b80644e6-82bb-4c59-86ed-b60bd24e6a05"
Jul 31 22:57:17.886267 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:17.886074    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mqwlt" podUID="b85a1443-4bc2-48a0-8d29-35f46cbc3d57"
Jul 31 22:57:19.141804 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:19.141602    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bk57" event={"ID":"3f91895a-1a8c-4ef9-8f27-0cfed935fae8","Type":"ContainerStarted","Data":"2104e96272c94d24d95c660ba876924962a43a6435d67ec6a7566c3346426953"}
Jul 31 22:57:19.143286 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:19.143239    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-pbslg" event={"ID":"639fb18d-e947-41cc-ac53-d4616b7cc85b","Type":"ContainerStarted","Data":"7ec83590bd8a6562e17e5bf80b42f662dcb44d55ce07c95b5f51523e21638981"}
Jul 31 22:57:19.144983 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:19.144948    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9wt4q" event={"ID":"1d0f0d49-bc8d-489a-932a-de4c94edb6a4","Type":"ContainerStarted","Data":"a271fa303e61a4da87814d01fbfb55b4d4be3a2038d05b6dc67d226ce12c5819"}
Jul 31 22:57:19.146577 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:19.146472    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-t8qkd" event={"ID":"0bb4d7d9-ff22-41eb-8883-b7c1942767fb","Type":"ContainerStarted","Data":"3315a18c0e42d8eb1cb7093376946f0f312c4690731dd174b9f811b647ccaff1"}
Jul 31 22:57:19.148065 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:19.148041    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-243.ec2.internal" event={"ID":"bf2bb11c3449d46ed43b4ff9c8ef237d","Type":"ContainerStarted","Data":"b4b6b05a17e64e042235bd57b3f98cef80763928b8001b63b6d0d0b83d2a1fbe"}
Jul 31 22:57:19.149559 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:19.149538    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2trlm" event={"ID":"4c7c83f2-713b-4651-b1da-f65c84f8edc9","Type":"ContainerStarted","Data":"5604a7c1bb2d6c9a8f88c422d3d5a966c621321a315ccca7c1533914e26a563e"}
Jul 31 22:57:19.151078 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:19.151059    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-llmj8" event={"ID":"148fa8ca-4d3b-4ae1-8be5-30ea87aa412e","Type":"ContainerStarted","Data":"e3e7921705c9e269548a76d180dc05c1c77a50ec22bc74601a33d4d680ee64b7"}
Jul 31 22:57:19.153901 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:19.153806    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w96jg" event={"ID":"87c830a4-dd74-43c5-84d0-45cdd2dbe9a3","Type":"ContainerStarted","Data":"e8d6cf3b0345fbbc2edf111198a75df9be5f2da7c7f657b81249875752c03228"}
Jul 31 22:57:19.153901 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:19.153876    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w96jg" event={"ID":"87c830a4-dd74-43c5-84d0-45cdd2dbe9a3","Type":"ContainerStarted","Data":"5f396182168c9d0e5f037e6f4e9a952caa099f2b4f6373674e11043e8cc74cc9"}
Jul 31 22:57:19.154031 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:19.153928    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w96jg" event={"ID":"87c830a4-dd74-43c5-84d0-45cdd2dbe9a3","Type":"ContainerStarted","Data":"2edf96e252d7a92e0995ff366133b6b75bdf0c02cdb561046d9683ecc22822a0"}
Jul 31 22:57:19.156928 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:19.156910    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-shjgx" event={"ID":"eb4e0ba5-fe8b-4bf7-a86d-15db1a2532b2","Type":"ContainerStarted","Data":"b75fff75932025a8566a3cbff02821d9b9404fd5a7c21357be78442f743fb5f7"}
Jul 31 22:57:19.176574 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:19.176527    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-pbslg" podStartSLOduration=1.53626256 podStartE2EDuration="22.176512922s" totalImagesPullingTime="20.640250362s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 22:56:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:57:19.175937208 +0000 UTC m=+22.821723514" watchObservedRunningTime="2026-07-31 22:57:19.176512922 +0000 UTC m=+22.822299232"
Jul 31 22:57:19.261748 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:19.261067    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-t8qkd" podStartSLOduration=1.819289902 podStartE2EDuration="22.261045004s" totalImagesPullingTime="20.441755102s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 22:56:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:57:19.21009772 +0000 UTC m=+22.855884028" watchObservedRunningTime="2026-07-31 22:57:19.261045004 +0000 UTC m=+22.906831309"
Jul 31 22:57:19.315308 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:19.315262    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-9wt4q" podStartSLOduration=1.893845633 podStartE2EDuration="22.315244213s" totalImagesPullingTime="20.42139858s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 22:56:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:57:19.262075098 +0000 UTC m=+22.907861399" watchObservedRunningTime="2026-07-31 22:57:19.315244213 +0000 UTC m=+22.961030523"
Jul 31 22:57:19.339903 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:19.339856    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-llmj8" podStartSLOduration=2.004237883 podStartE2EDuration="22.339839602s" totalImagesPullingTime="20.335601719s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 22:56:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:57:19.315749949 +0000 UTC m=+22.961536257" watchObservedRunningTime="2026-07-31 22:57:19.339839602 +0000 UTC m=+22.985625911"
Jul 31 22:57:19.340153 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:19.340130    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-243.ec2.internal" podStartSLOduration=21.34012607 podStartE2EDuration="21.34012607s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 22:56:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:57:19.339356258 +0000 UTC m=+22.985142568" watchObservedRunningTime="2026-07-31 22:57:19.34012607 +0000 UTC m=+22.985912378"
Jul 31 22:57:19.429963 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:19.429920    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-shjgx" podStartSLOduration=2.076285479 podStartE2EDuration="22.429901144s" totalImagesPullingTime="20.353615665s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:56:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:57:19.38668307 +0000 UTC m=+23.032469384" watchObservedRunningTime="2026-07-31 22:57:19.429901144 +0000 UTC m=+23.075687453"
Jul 31 22:57:19.885454 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:19.885197    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mz5wz"
Jul 31 22:57:19.885623 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:19.885197    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-84css"
Jul 31 22:57:19.885623 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:19.885468    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mz5wz" podUID="b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945"
Jul 31 22:57:19.885623 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:19.885564    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-84css" podUID="b80644e6-82bb-4c59-86ed-b60bd24e6a05"
Jul 31 22:57:19.885623 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:19.885197    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mqwlt"
Jul 31 22:57:19.885880 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:19.885657    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mqwlt" podUID="b85a1443-4bc2-48a0-8d29-35f46cbc3d57"
Jul 31 22:57:20.161756 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:20.161732    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w96jg" event={"ID":"87c830a4-dd74-43c5-84d0-45cdd2dbe9a3","Type":"ContainerStarted","Data":"24d9ca32c04caf333d1626f5cfb2d2cc0b0a3e46b6cb146438aed31ef8835e44"}
Jul 31 22:57:20.162088 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:20.161803    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w96jg" event={"ID":"87c830a4-dd74-43c5-84d0-45cdd2dbe9a3","Type":"ContainerStarted","Data":"3c9548f7645da447f509834e78bd497f6b0342b81a9c276a42d05361f6a3059f"}
Jul 31 22:57:20.162088 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:20.161843    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w96jg" event={"ID":"87c830a4-dd74-43c5-84d0-45cdd2dbe9a3","Type":"ContainerStarted","Data":"319758c0397d3690ca01a6cba18d36d7ea65056528ad2e1aab93ffa374f3b48b"}
Jul 31 22:57:20.163153 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:20.163130    2584 generic.go:356] "Generic (PLEG): container finished" podID="59797420703085b3d07401083a2a0810" containerID="edb20e11c2d8da939137d714bfe92d33e4d468ebb4ba99edf49f21c9f8e241e0" exitCode=0
Jul 31 22:57:20.163261 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:20.163219    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-243.ec2.internal" event={"ID":"59797420703085b3d07401083a2a0810","Type":"ContainerDied","Data":"edb20e11c2d8da939137d714bfe92d33e4d468ebb4ba99edf49f21c9f8e241e0"}
Jul 31 22:57:20.164460 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:20.164425    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qchlw" event={"ID":"9a9cc748-9a60-4656-9236-34d75e5cb22a","Type":"ContainerStarted","Data":"fd6827226e7002862827539a81b16e59278a4b7e3474f355468d2884fb63be48"}
Jul 31 22:57:20.165729 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:20.165686    2584 generic.go:356] "Generic (PLEG): container finished" podID="4c7c83f2-713b-4651-b1da-f65c84f8edc9" containerID="5604a7c1bb2d6c9a8f88c422d3d5a966c621321a315ccca7c1533914e26a563e" exitCode=0
Jul 31 22:57:20.165829 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:20.165732    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2trlm" event={"ID":"4c7c83f2-713b-4651-b1da-f65c84f8edc9","Type":"ContainerDied","Data":"5604a7c1bb2d6c9a8f88c422d3d5a966c621321a315ccca7c1533914e26a563e"}
Jul 31 22:57:20.239590 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:20.239544    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-qchlw" podStartSLOduration=2.808308887 podStartE2EDuration="23.239530117s" totalImagesPullingTime="20.43122123s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 22:56:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:57:20.239124305 +0000 UTC m=+23.884910615" watchObservedRunningTime="2026-07-31 22:57:20.239530117 +0000 UTC m=+23.885316425"
Jul 31 22:57:20.469166 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:20.469139    2584 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 31 22:57:20.842576 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:20.842425    2584 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-31T22:57:20.469162805Z","UUID":"1c2401d3-b266-4e05-a34d-fef2a3b1cbe8","Handler":null,"Name":"","Endpoint":""}
Jul 31 22:57:20.843985 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:20.843955    2584 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 31 22:57:20.844106 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:20.843992    2584 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 31 22:57:21.170738 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:21.170641    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bk57" event={"ID":"3f91895a-1a8c-4ef9-8f27-0cfed935fae8","Type":"ContainerStarted","Data":"5dc4195d18869feef17ece88c948a6786782f788f69c43ecbc4d9a1deb1abda2"}
Jul 31 22:57:21.172996 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:21.172969    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-243.ec2.internal" event={"ID":"59797420703085b3d07401083a2a0810","Type":"ContainerStarted","Data":"2babeed9bf7f601461a2c0ef74f0ec94f19c239e76576cfaf96299ce9ce2d1bb"}
Jul 31 22:57:21.191382 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:21.191333    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-243.ec2.internal" podStartSLOduration=23.19131549 podStartE2EDuration="23.19131549s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 22:56:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:57:21.191154982 +0000 UTC m=+24.836941292" watchObservedRunningTime="2026-07-31 22:57:21.19131549 +0000 UTC m=+24.837101801"
Jul 31 22:57:21.885476 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:21.885394    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mqwlt"
Jul 31 22:57:21.885631 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:21.885394    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mz5wz"
Jul 31 22:57:21.885631 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:21.885499    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mqwlt" podUID="b85a1443-4bc2-48a0-8d29-35f46cbc3d57"
Jul 31 22:57:21.885631 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:21.885521    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-84css"
Jul 31 22:57:21.885631 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:21.885616    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mz5wz" podUID="b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945"
Jul 31 22:57:21.885825 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:21.885673    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-84css" podUID="b80644e6-82bb-4c59-86ed-b60bd24e6a05"
Jul 31 22:57:22.177789 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:22.177686    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bk57" event={"ID":"3f91895a-1a8c-4ef9-8f27-0cfed935fae8","Type":"ContainerStarted","Data":"fad60cd1da29836c78a7b0fe6222dd87672bcf7ca4079fe2ee3e32e2d735e903"}
Jul 31 22:57:22.181716 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:22.181672    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w96jg" event={"ID":"87c830a4-dd74-43c5-84d0-45cdd2dbe9a3","Type":"ContainerStarted","Data":"73928e18ba6d295cede515c4ec394252a193ec89d3173d4e4cfdf0ed07a9e214"}
Jul 31 22:57:22.200713 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:22.200659    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bk57" podStartSLOduration=2.475019119 podStartE2EDuration="25.200638998s" totalImagesPullingTime="22.725619879s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 22:56:57 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:57:22.200156663 +0000 UTC m=+25.845942973" watchObservedRunningTime="2026-07-31 22:57:22.200638998 +0000 UTC m=+25.846425306"
Jul 31 22:57:23.885617 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:23.885576    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-84css"
Jul 31 22:57:23.885617 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:23.885605    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mqwlt"
Jul 31 22:57:23.886107 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:23.885581    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mz5wz"
Jul 31 22:57:23.886107 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:23.885712    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-84css" podUID="b80644e6-82bb-4c59-86ed-b60bd24e6a05"
Jul 31 22:57:23.886107 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:23.885783    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mz5wz" podUID="b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945"
Jul 31 22:57:23.886107 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:23.885909    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mqwlt" podUID="b85a1443-4bc2-48a0-8d29-35f46cbc3d57"
Jul 31 22:57:23.994674 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:23.994634    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-shjgx"
Jul 31 22:57:23.995335 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:23.995313    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-shjgx"
Jul 31 22:57:24.187558 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:24.187471    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-shjgx"
Jul 31 22:57:24.188039 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:24.188022    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-shjgx"
Jul 31 22:57:25.191203 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:25.190889    2584 generic.go:356] "Generic (PLEG): container finished" podID="4c7c83f2-713b-4651-b1da-f65c84f8edc9" containerID="7cb5c32fdf1122a70ba7546a31ba7ca81d7e062e0aa098d9a2beee8fdf888b34" exitCode=0
Jul 31 22:57:25.191947 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:25.190949    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2trlm" event={"ID":"4c7c83f2-713b-4651-b1da-f65c84f8edc9","Type":"ContainerDied","Data":"7cb5c32fdf1122a70ba7546a31ba7ca81d7e062e0aa098d9a2beee8fdf888b34"}
Jul 31 22:57:25.194920 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:25.194894    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w96jg" event={"ID":"87c830a4-dd74-43c5-84d0-45cdd2dbe9a3","Type":"ContainerStarted","Data":"971b4d841b9a6149381f1646d7f85d99cd33647c13654a10d5da74291add61fb"}
Jul 31 22:57:25.243567 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:25.243517    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-w96jg" podStartSLOduration=7.599198137 podStartE2EDuration="28.243504233s" totalImagesPullingTime="20.644306096s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 22:56:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:57:25.242858838 +0000 UTC m=+28.888645143" watchObservedRunningTime="2026-07-31 22:57:25.243504233 +0000 UTC m=+28.889290541"
Jul 31 22:57:25.885197 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:25.885164    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mz5wz"
Jul 31 22:57:25.885480 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:25.885209    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-84css"
Jul 31 22:57:25.885480 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:25.885272    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mqwlt"
Jul 31 22:57:25.885480 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:25.885283    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mz5wz" podUID="b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945"
Jul 31 22:57:25.885480 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:25.885378    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mqwlt" podUID="b85a1443-4bc2-48a0-8d29-35f46cbc3d57"
Jul 31 22:57:25.885480 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:25.885450    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-84css" podUID="b80644e6-82bb-4c59-86ed-b60bd24e6a05"
Jul 31 22:57:26.197793 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:26.197714    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-w96jg"
Jul 31 22:57:26.198260 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:26.197815    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-w96jg"
Jul 31 22:57:26.198260 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:26.197862    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-w96jg"
Jul 31 22:57:26.213536 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:26.213507    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-w96jg"
Jul 31 22:57:26.214027 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:26.214011    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-w96jg"
Jul 31 22:57:26.658878 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:26.658594    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-mqwlt"]
Jul 31 22:57:26.659041 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:26.658894    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mqwlt"
Jul 31 22:57:26.659041 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:26.658994    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mqwlt" podUID="b85a1443-4bc2-48a0-8d29-35f46cbc3d57"
Jul 31 22:57:26.662129 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:26.662106    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mz5wz"]
Jul 31 22:57:26.662273 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:26.662199    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mz5wz"
Jul 31 22:57:26.662342 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:26.662296    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mz5wz" podUID="b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945"
Jul 31 22:57:26.662737 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:26.662714    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-84css"]
Jul 31 22:57:26.662820 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:26.662803    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-84css"
Jul 31 22:57:26.662900 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:26.662883    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-84css" podUID="b80644e6-82bb-4c59-86ed-b60bd24e6a05"
Jul 31 22:57:27.203029 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:27.202977    2584 generic.go:356] "Generic (PLEG): container finished" podID="4c7c83f2-713b-4651-b1da-f65c84f8edc9" containerID="3eee4eaa23027d4d7ab3d29c42ea7276d440446b2f2118133d4896051b980bde" exitCode=0
Jul 31 22:57:27.204342 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:27.204057    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2trlm" event={"ID":"4c7c83f2-713b-4651-b1da-f65c84f8edc9","Type":"ContainerDied","Data":"3eee4eaa23027d4d7ab3d29c42ea7276d440446b2f2118133d4896051b980bde"}
Jul 31 22:57:27.885259 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:27.885231    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mqwlt"
Jul 31 22:57:27.885421 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:27.885333    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mqwlt" podUID="b85a1443-4bc2-48a0-8d29-35f46cbc3d57"
Jul 31 22:57:28.207607 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:28.207522    2584 generic.go:356] "Generic (PLEG): container finished" podID="4c7c83f2-713b-4651-b1da-f65c84f8edc9" containerID="78e035db33ffc2f0a2d4d0855941493ed2b42f8e8fc2ef21552240a414febf4c" exitCode=0
Jul 31 22:57:28.208012 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:28.207609    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2trlm" event={"ID":"4c7c83f2-713b-4651-b1da-f65c84f8edc9","Type":"ContainerDied","Data":"78e035db33ffc2f0a2d4d0855941493ed2b42f8e8fc2ef21552240a414febf4c"}
Jul 31 22:57:28.885326 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:28.885292    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mz5wz"
Jul 31 22:57:28.885494 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:28.885342    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-84css"
Jul 31 22:57:28.885494 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:28.885441    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mz5wz" podUID="b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945"
Jul 31 22:57:28.885606 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:28.885579    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-84css" podUID="b80644e6-82bb-4c59-86ed-b60bd24e6a05"
Jul 31 22:57:29.539288 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:29.539257    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945-metrics-certs\") pod \"network-metrics-daemon-mz5wz\" (UID: \"b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945\") " pod="openshift-multus/network-metrics-daemon-mz5wz"
Jul 31 22:57:29.539676 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:29.539372    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 31 22:57:29.539676 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:29.539426    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945-metrics-certs podName:b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945 nodeName:}" failed. No retries permitted until 2026-07-31 22:58:01.539412527 +0000 UTC m=+65.185198813 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945-metrics-certs") pod "network-metrics-daemon-mz5wz" (UID: "b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 31 22:57:29.639886 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:29.639851    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b85a1443-4bc2-48a0-8d29-35f46cbc3d57-original-pull-secret\") pod \"global-pull-secret-syncer-mqwlt\" (UID: \"b85a1443-4bc2-48a0-8d29-35f46cbc3d57\") " pod="kube-system/global-pull-secret-syncer-mqwlt"
Jul 31 22:57:29.640073 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:29.640003    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 31 22:57:29.640132 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:29.640075    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b85a1443-4bc2-48a0-8d29-35f46cbc3d57-original-pull-secret podName:b85a1443-4bc2-48a0-8d29-35f46cbc3d57 nodeName:}" failed. No retries permitted until 2026-07-31 22:58:01.640059682 +0000 UTC m=+65.285845989 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b85a1443-4bc2-48a0-8d29-35f46cbc3d57-original-pull-secret") pod "global-pull-secret-syncer-mqwlt" (UID: "b85a1443-4bc2-48a0-8d29-35f46cbc3d57") : object "kube-system"/"original-pull-secret" not registered
Jul 31 22:57:29.740769 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:29.740739    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwbjs\" (UniqueName: \"kubernetes.io/projected/b80644e6-82bb-4c59-86ed-b60bd24e6a05-kube-api-access-bwbjs\") pod \"network-check-target-84css\" (UID: \"b80644e6-82bb-4c59-86ed-b60bd24e6a05\") " pod="openshift-network-diagnostics/network-check-target-84css"
Jul 31 22:57:29.740955 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:29.740914    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 31 22:57:29.740955 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:29.740935    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 31 22:57:29.740955 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:29.740947    2584 projected.go:196] Error preparing data for projected volume kube-api-access-bwbjs for pod openshift-network-diagnostics/network-check-target-84css: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 31 22:57:29.741110 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:29.741013    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b80644e6-82bb-4c59-86ed-b60bd24e6a05-kube-api-access-bwbjs podName:b80644e6-82bb-4c59-86ed-b60bd24e6a05 nodeName:}" failed. No retries permitted until 2026-07-31 22:58:01.740993277 +0000 UTC m=+65.386779585 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-bwbjs" (UniqueName: "kubernetes.io/projected/b80644e6-82bb-4c59-86ed-b60bd24e6a05-kube-api-access-bwbjs") pod "network-check-target-84css" (UID: "b80644e6-82bb-4c59-86ed-b60bd24e6a05") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 31 22:57:29.885414 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:29.885337    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mqwlt"
Jul 31 22:57:29.885556 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:29.885454    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mqwlt" podUID="b85a1443-4bc2-48a0-8d29-35f46cbc3d57"
Jul 31 22:57:30.885109 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:30.885080    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-84css"
Jul 31 22:57:30.885590 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:30.885204    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-84css" podUID="b80644e6-82bb-4c59-86ed-b60bd24e6a05"
Jul 31 22:57:30.885590 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:30.885265    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mz5wz"
Jul 31 22:57:30.885590 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:30.885387    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mz5wz" podUID="b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945"
Jul 31 22:57:31.070790 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:31.070757    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-shjgx_eb4e0ba5-fe8b-4bf7-a86d-15db1a2532b2/konnectivity-agent/0.log" limit=1024
Jul 31 22:57:31.213928 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:31.213848    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-shjgx_eb4e0ba5-fe8b-4bf7-a86d-15db1a2532b2/konnectivity-agent/0.log" limit=1024
Jul 31 22:57:31.345069 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:31.345023    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-shjgx_eb4e0ba5-fe8b-4bf7-a86d-15db1a2532b2/konnectivity-agent/0.log" limit=1024
Jul 31 22:57:31.649405 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:31.649316    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-243.ec2.internal" event="NodeReady"
Jul 31 22:57:31.649555 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:31.649483    2584 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 31 22:57:31.709379 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:31.709346    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-fhq7n"]
Jul 31 22:57:31.741195 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:31.741166    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-lbf5k"]
Jul 31 22:57:31.741350 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:31.741281    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-fhq7n"
Jul 31 22:57:31.744495 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:31.744470    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 31 22:57:31.744637 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:31.744552    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-7jxx7\""
Jul 31 22:57:31.744637 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:31.744580    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 31 22:57:31.755312 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:31.755290    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-fhq7n"]
Jul 31 22:57:31.755446 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:31.755331    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-lbf5k"]
Jul 31 22:57:31.755446 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:31.755424    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-lbf5k"
Jul 31 22:57:31.758198 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:31.758168    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 31 22:57:31.758301 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:31.758169    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 31 22:57:31.758301 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:31.758178    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-c9ph2\""
Jul 31 22:57:31.758830 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:31.758582    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 31 22:57:31.855935 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:31.855900    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/735a47e9-8a16-4dd3-93ec-2e3ac70100d4-tmp-dir\") pod \"dns-default-fhq7n\" (UID: \"735a47e9-8a16-4dd3-93ec-2e3ac70100d4\") " pod="openshift-dns/dns-default-fhq7n"
Jul 31 22:57:31.856113 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:31.855991    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a0e5b183-2a8a-48c8-b2fa-803bce230632-cert\") pod \"ingress-canary-lbf5k\" (UID: \"a0e5b183-2a8a-48c8-b2fa-803bce230632\") " pod="openshift-ingress-canary/ingress-canary-lbf5k"
Jul 31 22:57:31.856113 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:31.856014    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/735a47e9-8a16-4dd3-93ec-2e3ac70100d4-config-volume\") pod \"dns-default-fhq7n\" (UID: \"735a47e9-8a16-4dd3-93ec-2e3ac70100d4\") " pod="openshift-dns/dns-default-fhq7n"
Jul 31 22:57:31.856113 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:31.856041    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/735a47e9-8a16-4dd3-93ec-2e3ac70100d4-metrics-tls\") pod \"dns-default-fhq7n\" (UID: \"735a47e9-8a16-4dd3-93ec-2e3ac70100d4\") " pod="openshift-dns/dns-default-fhq7n"
Jul 31 22:57:31.856113 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:31.856089    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99wlq\" (UniqueName: \"kubernetes.io/projected/a0e5b183-2a8a-48c8-b2fa-803bce230632-kube-api-access-99wlq\") pod \"ingress-canary-lbf5k\" (UID: \"a0e5b183-2a8a-48c8-b2fa-803bce230632\") " pod="openshift-ingress-canary/ingress-canary-lbf5k"
Jul 31 22:57:31.856270 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:31.856128    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vsvq8\" (UniqueName: \"kubernetes.io/projected/735a47e9-8a16-4dd3-93ec-2e3ac70100d4-kube-api-access-vsvq8\") pod \"dns-default-fhq7n\" (UID: \"735a47e9-8a16-4dd3-93ec-2e3ac70100d4\") " pod="openshift-dns/dns-default-fhq7n"
Jul 31 22:57:31.856270 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:31.856235    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-shjgx_eb4e0ba5-fe8b-4bf7-a86d-15db1a2532b2/konnectivity-agent/0.log" limit=1024
Jul 31 22:57:31.885672 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:31.885633    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mqwlt"
Jul 31 22:57:31.888499 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:31.888475    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 31 22:57:31.957155 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:31.957071    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a0e5b183-2a8a-48c8-b2fa-803bce230632-cert\") pod \"ingress-canary-lbf5k\" (UID: \"a0e5b183-2a8a-48c8-b2fa-803bce230632\") " pod="openshift-ingress-canary/ingress-canary-lbf5k"
Jul 31 22:57:31.957155 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:31.957108    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/735a47e9-8a16-4dd3-93ec-2e3ac70100d4-config-volume\") pod \"dns-default-fhq7n\" (UID: \"735a47e9-8a16-4dd3-93ec-2e3ac70100d4\") " pod="openshift-dns/dns-default-fhq7n"
Jul 31 22:57:31.957155 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:31.957146    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/735a47e9-8a16-4dd3-93ec-2e3ac70100d4-metrics-tls\") pod \"dns-default-fhq7n\" (UID: \"735a47e9-8a16-4dd3-93ec-2e3ac70100d4\") " pod="openshift-dns/dns-default-fhq7n"
Jul 31 22:57:31.957419 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:31.957172    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99wlq\" (UniqueName: \"kubernetes.io/projected/a0e5b183-2a8a-48c8-b2fa-803bce230632-kube-api-access-99wlq\") pod \"ingress-canary-lbf5k\" (UID: \"a0e5b183-2a8a-48c8-b2fa-803bce230632\") " pod="openshift-ingress-canary/ingress-canary-lbf5k"
Jul 31 22:57:31.957419 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:31.957202    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsvq8\" (UniqueName: \"kubernetes.io/projected/735a47e9-8a16-4dd3-93ec-2e3ac70100d4-kube-api-access-vsvq8\") pod \"dns-default-fhq7n\" (UID: \"735a47e9-8a16-4dd3-93ec-2e3ac70100d4\") " pod="openshift-dns/dns-default-fhq7n"
Jul 31 22:57:31.957419 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:31.957246    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/735a47e9-8a16-4dd3-93ec-2e3ac70100d4-tmp-dir\") pod \"dns-default-fhq7n\" (UID: \"735a47e9-8a16-4dd3-93ec-2e3ac70100d4\") " pod="openshift-dns/dns-default-fhq7n"
Jul 31 22:57:31.957419 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:31.957334    2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 31 22:57:31.957419 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:31.957420    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/735a47e9-8a16-4dd3-93ec-2e3ac70100d4-metrics-tls podName:735a47e9-8a16-4dd3-93ec-2e3ac70100d4 nodeName:}" failed. No retries permitted until 2026-07-31 22:57:32.457399308 +0000 UTC m=+36.103185612 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/735a47e9-8a16-4dd3-93ec-2e3ac70100d4-metrics-tls") pod "dns-default-fhq7n" (UID: "735a47e9-8a16-4dd3-93ec-2e3ac70100d4") : secret "dns-default-metrics-tls" not found
Jul 31 22:57:31.957976 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:31.957814    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/735a47e9-8a16-4dd3-93ec-2e3ac70100d4-tmp-dir\") pod \"dns-default-fhq7n\" (UID: \"735a47e9-8a16-4dd3-93ec-2e3ac70100d4\") " pod="openshift-dns/dns-default-fhq7n"
Jul 31 22:57:31.958075 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:31.957962    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/735a47e9-8a16-4dd3-93ec-2e3ac70100d4-config-volume\") pod \"dns-default-fhq7n\" (UID: \"735a47e9-8a16-4dd3-93ec-2e3ac70100d4\") " pod="openshift-dns/dns-default-fhq7n"
Jul 31 22:57:31.961940 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:31.961915    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a0e5b183-2a8a-48c8-b2fa-803bce230632-cert\") pod \"ingress-canary-lbf5k\" (UID: \"a0e5b183-2a8a-48c8-b2fa-803bce230632\") " pod="openshift-ingress-canary/ingress-canary-lbf5k"
Jul 31 22:57:31.966025 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:31.966003    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vsvq8\" (UniqueName: \"kubernetes.io/projected/735a47e9-8a16-4dd3-93ec-2e3ac70100d4-kube-api-access-vsvq8\") pod \"dns-default-fhq7n\" (UID: \"735a47e9-8a16-4dd3-93ec-2e3ac70100d4\") " pod="openshift-dns/dns-default-fhq7n"
Jul 31 22:57:31.969332 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:31.969310    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-99wlq\" (UniqueName: \"kubernetes.io/projected/a0e5b183-2a8a-48c8-b2fa-803bce230632-kube-api-access-99wlq\") pod \"ingress-canary-lbf5k\" (UID: \"a0e5b183-2a8a-48c8-b2fa-803bce230632\") " pod="openshift-ingress-canary/ingress-canary-lbf5k"
Jul 31 22:57:31.996332 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:31.996300    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-shjgx_eb4e0ba5-fe8b-4bf7-a86d-15db1a2532b2/konnectivity-agent/0.log" limit=1024
Jul 31 22:57:32.061852 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:32.061822    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-shjgx_eb4e0ba5-fe8b-4bf7-a86d-15db1a2532b2/konnectivity-agent/0.log" limit=1024
Jul 31 22:57:32.066747 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:32.066721    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-lbf5k"
Jul 31 22:57:32.235104 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:32.235026    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-lbf5k"]
Jul 31 22:57:32.299057 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:57:32.299029    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0e5b183_2a8a_48c8_b2fa_803bce230632.slice/crio-912c6b353b051bef29e9346f1f4311246eaf66357b7ad86d7489a823632a72c2 WatchSource:0}: Error finding container 912c6b353b051bef29e9346f1f4311246eaf66357b7ad86d7489a823632a72c2: Status 404 returned error can't find the container with id 912c6b353b051bef29e9346f1f4311246eaf66357b7ad86d7489a823632a72c2
Jul 31 22:57:32.462893 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:32.462862    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/735a47e9-8a16-4dd3-93ec-2e3ac70100d4-metrics-tls\") pod \"dns-default-fhq7n\" (UID: \"735a47e9-8a16-4dd3-93ec-2e3ac70100d4\") " pod="openshift-dns/dns-default-fhq7n"
Jul 31 22:57:32.463038 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:32.463021    2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 31 22:57:32.463116 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:32.463104    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/735a47e9-8a16-4dd3-93ec-2e3ac70100d4-metrics-tls podName:735a47e9-8a16-4dd3-93ec-2e3ac70100d4 nodeName:}" failed. No retries permitted until 2026-07-31 22:57:33.46306836 +0000 UTC m=+37.108854647 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/735a47e9-8a16-4dd3-93ec-2e3ac70100d4-metrics-tls") pod "dns-default-fhq7n" (UID: "735a47e9-8a16-4dd3-93ec-2e3ac70100d4") : secret "dns-default-metrics-tls" not found
Jul 31 22:57:32.885281 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:32.885195    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mz5wz"
Jul 31 22:57:32.885281 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:32.885230    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-84css"
Jul 31 22:57:32.888213 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:32.888181    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 31 22:57:32.888997 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:32.888343    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 31 22:57:32.889503 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:32.889482    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-h8t45\""
Jul 31 22:57:32.889594 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:32.889527    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jtx5g\""
Jul 31 22:57:32.889594 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:32.889586    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 31 22:57:33.220599 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:33.220519    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-lbf5k" event={"ID":"a0e5b183-2a8a-48c8-b2fa-803bce230632","Type":"ContainerStarted","Data":"912c6b353b051bef29e9346f1f4311246eaf66357b7ad86d7489a823632a72c2"}
Jul 31 22:57:33.223353 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:33.223327    2584 generic.go:356] "Generic (PLEG): container finished" podID="4c7c83f2-713b-4651-b1da-f65c84f8edc9" containerID="dada975006c11303466a80b2d436b1e3b8d6aea8faccffc17a47b8d8265844b9" exitCode=0
Jul 31 22:57:33.223502 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:33.223381    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2trlm" event={"ID":"4c7c83f2-713b-4651-b1da-f65c84f8edc9","Type":"ContainerDied","Data":"dada975006c11303466a80b2d436b1e3b8d6aea8faccffc17a47b8d8265844b9"}
Jul 31 22:57:33.471789 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:33.471695    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/735a47e9-8a16-4dd3-93ec-2e3ac70100d4-metrics-tls\") pod \"dns-default-fhq7n\" (UID: \"735a47e9-8a16-4dd3-93ec-2e3ac70100d4\") " pod="openshift-dns/dns-default-fhq7n"
Jul 31 22:57:33.471940 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:33.471818    2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 31 22:57:33.471940 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:33.471895    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/735a47e9-8a16-4dd3-93ec-2e3ac70100d4-metrics-tls podName:735a47e9-8a16-4dd3-93ec-2e3ac70100d4 nodeName:}" failed. No retries permitted until 2026-07-31 22:57:35.471871268 +0000 UTC m=+39.117657558 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/735a47e9-8a16-4dd3-93ec-2e3ac70100d4-metrics-tls") pod "dns-default-fhq7n" (UID: "735a47e9-8a16-4dd3-93ec-2e3ac70100d4") : secret "dns-default-metrics-tls" not found
Jul 31 22:57:34.230265 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:34.230029    2584 generic.go:356] "Generic (PLEG): container finished" podID="4c7c83f2-713b-4651-b1da-f65c84f8edc9" containerID="cee4373d8bac6ed178fa68b7e0fa73dcfe5363e83e151f3739e740e5bc74a649" exitCode=0
Jul 31 22:57:34.230721 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:34.230107    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2trlm" event={"ID":"4c7c83f2-713b-4651-b1da-f65c84f8edc9","Type":"ContainerDied","Data":"cee4373d8bac6ed178fa68b7e0fa73dcfe5363e83e151f3739e740e5bc74a649"}
Jul 31 22:57:35.233910 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:35.233824    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-lbf5k" event={"ID":"a0e5b183-2a8a-48c8-b2fa-803bce230632","Type":"ContainerStarted","Data":"0bc20220ff002a52aab69cb99f8e9bcbe860f87cfe15b1b1b77812b5f1a05857"}
Jul 31 22:57:35.237092 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:35.237067    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2trlm" event={"ID":"4c7c83f2-713b-4651-b1da-f65c84f8edc9","Type":"ContainerStarted","Data":"2d70819273450e7f94000b53c839be08e0fcf7c9f2d6821cf6745a3f4b78adbc"}
Jul 31 22:57:35.251063 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:35.251015    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-lbf5k" podStartSLOduration=1.801053164 podStartE2EDuration="4.251000068s" totalImagesPullingTime="2.449946904s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:57:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:57:35.25079882 +0000 UTC m=+38.896585130" watchObservedRunningTime="2026-07-31 22:57:35.251000068 +0000 UTC m=+38.896786377"
Jul 31 22:57:35.275456 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:35.275410    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-2trlm" podStartSLOduration=7.780428533 podStartE2EDuration="38.275396067s" totalImagesPullingTime="30.494967534s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 22:56:57 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:57:35.27455025 +0000 UTC m=+38.920336559" watchObservedRunningTime="2026-07-31 22:57:35.275396067 +0000 UTC m=+38.921182375"
Jul 31 22:57:35.404638 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:35.404608    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-shjgx_eb4e0ba5-fe8b-4bf7-a86d-15db1a2532b2/konnectivity-agent/0.log" limit=1024
Jul 31 22:57:35.468334 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:35.468293    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-shjgx_eb4e0ba5-fe8b-4bf7-a86d-15db1a2532b2/konnectivity-agent/0.log" limit=1024
Jul 31 22:57:35.486745 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:35.486667    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/735a47e9-8a16-4dd3-93ec-2e3ac70100d4-metrics-tls\") pod \"dns-default-fhq7n\" (UID: \"735a47e9-8a16-4dd3-93ec-2e3ac70100d4\") " pod="openshift-dns/dns-default-fhq7n"
Jul 31 22:57:35.486825 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:35.486796    2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 31 22:57:35.486865 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:35.486846    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/735a47e9-8a16-4dd3-93ec-2e3ac70100d4-metrics-tls podName:735a47e9-8a16-4dd3-93ec-2e3ac70100d4 nodeName:}" failed. No retries permitted until 2026-07-31 22:57:39.486833144 +0000 UTC m=+43.132619432 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/735a47e9-8a16-4dd3-93ec-2e3ac70100d4-metrics-tls") pod "dns-default-fhq7n" (UID: "735a47e9-8a16-4dd3-93ec-2e3ac70100d4") : secret "dns-default-metrics-tls" not found
Jul 31 22:57:36.648785 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:36.648717    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-shjgx_eb4e0ba5-fe8b-4bf7-a86d-15db1a2532b2/konnectivity-agent/0.log" limit=1024
Jul 31 22:57:36.708917 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:36.708886    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-shjgx_eb4e0ba5-fe8b-4bf7-a86d-15db1a2532b2/konnectivity-agent/0.log" limit=1024
Jul 31 22:57:39.514891 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:39.514858    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/735a47e9-8a16-4dd3-93ec-2e3ac70100d4-metrics-tls\") pod \"dns-default-fhq7n\" (UID: \"735a47e9-8a16-4dd3-93ec-2e3ac70100d4\") " pod="openshift-dns/dns-default-fhq7n"
Jul 31 22:57:39.515258 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:39.514966    2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 31 22:57:39.515258 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:39.515016    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/735a47e9-8a16-4dd3-93ec-2e3ac70100d4-metrics-tls podName:735a47e9-8a16-4dd3-93ec-2e3ac70100d4 nodeName:}" failed. No retries permitted until 2026-07-31 22:57:47.515002438 +0000 UTC m=+51.160788725 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/735a47e9-8a16-4dd3-93ec-2e3ac70100d4-metrics-tls") pod "dns-default-fhq7n" (UID: "735a47e9-8a16-4dd3-93ec-2e3ac70100d4") : secret "dns-default-metrics-tls" not found
Jul 31 22:57:42.647328 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.647293    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-f6wq6"]
Jul 31 22:57:42.679597 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.679569    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-784f97bcf6-5jtrx"]
Jul 31 22:57:42.679755 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.679714    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-f6wq6"
Jul 31 22:57:42.685232 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.685202    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-l74sx\""
Jul 31 22:57:42.685475 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.685452    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 31 22:57:42.685566 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.685478    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 31 22:57:42.685566 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.685465    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 31 22:57:42.685566 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.685480    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 31 22:57:42.693342 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.693322    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-p58r8"]
Jul 31 22:57:42.693440 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.693431    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-5jtrx"
Jul 31 22:57:42.695929 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.695909    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 31 22:57:42.696036 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.695937    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 31 22:57:42.696036 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.695994    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 31 22:57:42.696036 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.696011    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 31 22:57:42.696149 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.696017    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-7b6g4\""
Jul 31 22:57:42.715761 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.715733    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-77696685b-k9tqs"]
Jul 31 22:57:42.715917 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.715834    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-p58r8"
Jul 31 22:57:42.718486 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.718463    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 31 22:57:42.718604 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.718497    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 31 22:57:42.718892 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.718872    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 31 22:57:42.719000 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.718893    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 31 22:57:42.719000 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.718908    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-r7gbt\""
Jul 31 22:57:42.736310 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.736284    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ed2bd0da-5ef5-45c8-bba7-ba1042dc308a-config\") pod \"service-ca-operator-784f97bcf6-5jtrx\" (UID: \"ed2bd0da-5ef5-45c8-bba7-ba1042dc308a\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-5jtrx"
Jul 31 22:57:42.736435 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.736321    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ed2bd0da-5ef5-45c8-bba7-ba1042dc308a-serving-cert\") pod \"service-ca-operator-784f97bcf6-5jtrx\" (UID: \"ed2bd0da-5ef5-45c8-bba7-ba1042dc308a\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-5jtrx"
Jul 31 22:57:42.736435 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.736346    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mndtr\" (UniqueName: \"kubernetes.io/projected/40b91dc3-a7d9-403a-8208-ca1e019553c5-kube-api-access-mndtr\") pod \"cluster-monitoring-operator-6f55c4f58d-f6wq6\" (UID: \"40b91dc3-a7d9-403a-8208-ca1e019553c5\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-f6wq6"
Jul 31 22:57:42.736435 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.736406    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwfms\" (UniqueName: \"kubernetes.io/projected/ed2bd0da-5ef5-45c8-bba7-ba1042dc308a-kube-api-access-zwfms\") pod \"service-ca-operator-784f97bcf6-5jtrx\" (UID: \"ed2bd0da-5ef5-45c8-bba7-ba1042dc308a\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-5jtrx"
Jul 31 22:57:42.736537 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.736437    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/55da6dc5-f4ad-468b-a4c8-497309dad48a-config\") pod \"kube-storage-version-migrator-operator-7878f4b88c-p58r8\" (UID: \"55da6dc5-f4ad-468b-a4c8-497309dad48a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-p58r8"
Jul 31 22:57:42.736537 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.736470    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ghzh\" (UniqueName: \"kubernetes.io/projected/55da6dc5-f4ad-468b-a4c8-497309dad48a-kube-api-access-9ghzh\") pod \"kube-storage-version-migrator-operator-7878f4b88c-p58r8\" (UID: \"55da6dc5-f4ad-468b-a4c8-497309dad48a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-p58r8"
Jul 31 22:57:42.736537 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.736530    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/55da6dc5-f4ad-468b-a4c8-497309dad48a-serving-cert\") pod \"kube-storage-version-migrator-operator-7878f4b88c-p58r8\" (UID: \"55da6dc5-f4ad-468b-a4c8-497309dad48a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-p58r8"
Jul 31 22:57:42.736624 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.736551    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/40b91dc3-a7d9-403a-8208-ca1e019553c5-telemetry-config\") pod \"cluster-monitoring-operator-6f55c4f58d-f6wq6\" (UID: \"40b91dc3-a7d9-403a-8208-ca1e019553c5\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-f6wq6"
Jul 31 22:57:42.736624 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.736566    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/40b91dc3-a7d9-403a-8208-ca1e019553c5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-f6wq6\" (UID: \"40b91dc3-a7d9-403a-8208-ca1e019553c5\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-f6wq6"
Jul 31 22:57:42.739663 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.739643    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-f6wq6"]
Jul 31 22:57:42.739772 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.739671    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-784f97bcf6-5jtrx"]
Jul 31 22:57:42.739772 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.739686    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-p58r8"]
Jul 31 22:57:42.739772 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.739768    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-77696685b-k9tqs"
Jul 31 22:57:42.739867 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.739799    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-77696685b-k9tqs"]
Jul 31 22:57:42.743227 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.743195    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 31 22:57:42.743227 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.743198    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 31 22:57:42.743391 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.743200    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 31 22:57:42.743391 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.743352    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-44mwx\""
Jul 31 22:57:42.743909 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.743895    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 31 22:57:42.749012 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.748993    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 31 22:57:42.769009 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.768981    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-xjzj5"]
Jul 31 22:57:42.795876 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.795849    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-868556846d-gx64z"]
Jul 31 22:57:42.796016 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.795965    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-xjzj5"
Jul 31 22:57:42.798389 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.798365    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 31 22:57:42.798389 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.798381    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-h4qmj\""
Jul 31 22:57:42.798575 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.798480    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 31 22:57:42.798633 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.798620    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 31 22:57:42.819677 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.819651    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-xjzj5"]
Jul 31 22:57:42.819677 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.819683    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-868556846d-gx64z"]
Jul 31 22:57:42.819904 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.819803    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-868556846d-gx64z"
Jul 31 22:57:42.822289 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.822273    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 31 22:57:42.822409 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.822311    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 31 22:57:42.822499 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.822486    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-srlnb\""
Jul 31 22:57:42.822562 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.822521    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 31 22:57:42.822752 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.822737    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 31 22:57:42.822860 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.822845    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 31 22:57:42.823087 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.823073    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 31 22:57:42.837481 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.837454    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89xbj\" (UniqueName: \"kubernetes.io/projected/715381b4-025e-436c-96d9-8ab63b3750a9-kube-api-access-89xbj\") pod \"insights-operator-77696685b-k9tqs\" (UID: \"715381b4-025e-436c-96d9-8ab63b3750a9\") " pod="openshift-insights/insights-operator-77696685b-k9tqs"
Jul 31 22:57:42.837597 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.837489    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/55da6dc5-f4ad-468b-a4c8-497309dad48a-config\") pod \"kube-storage-version-migrator-operator-7878f4b88c-p58r8\" (UID: \"55da6dc5-f4ad-468b-a4c8-497309dad48a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-p58r8"
Jul 31 22:57:42.837597 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.837510    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvbg8\" (UniqueName: \"kubernetes.io/projected/24908812-404e-4b8c-958d-335220924aff-kube-api-access-dvbg8\") pod \"router-default-868556846d-gx64z\" (UID: \"24908812-404e-4b8c-958d-335220924aff\") " pod="openshift-ingress/router-default-868556846d-gx64z"
Jul 31 22:57:42.837597 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.837528    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/40b91dc3-a7d9-403a-8208-ca1e019553c5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-f6wq6\" (UID: \"40b91dc3-a7d9-403a-8208-ca1e019553c5\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-f6wq6"
Jul 31 22:57:42.837794 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.837594    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ed2bd0da-5ef5-45c8-bba7-ba1042dc308a-serving-cert\") pod \"service-ca-operator-784f97bcf6-5jtrx\" (UID: \"ed2bd0da-5ef5-45c8-bba7-ba1042dc308a\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-5jtrx"
Jul 31 22:57:42.837794 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:42.837608    2584 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 31 22:57:42.837794 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.837619    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ed2bd0da-5ef5-45c8-bba7-ba1042dc308a-config\") pod \"service-ca-operator-784f97bcf6-5jtrx\" (UID: \"ed2bd0da-5ef5-45c8-bba7-ba1042dc308a\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-5jtrx"
Jul 31 22:57:42.837794 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.837639    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8sf8j\" (UniqueName: \"kubernetes.io/projected/7946036f-064a-4c5f-b619-1f34c3a9e16b-kube-api-access-8sf8j\") pod \"cluster-samples-operator-559cbf5745-xjzj5\" (UID: \"7946036f-064a-4c5f-b619-1f34c3a9e16b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-xjzj5"
Jul 31 22:57:42.837794 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:42.837652    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40b91dc3-a7d9-403a-8208-ca1e019553c5-cluster-monitoring-operator-tls podName:40b91dc3-a7d9-403a-8208-ca1e019553c5 nodeName:}" failed. No retries permitted until 2026-07-31 22:57:43.337636943 +0000 UTC m=+46.983423267 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/40b91dc3-a7d9-403a-8208-ca1e019553c5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-f6wq6" (UID: "40b91dc3-a7d9-403a-8208-ca1e019553c5") : secret "cluster-monitoring-operator-tls" not found
Jul 31 22:57:42.837794 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.837734    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/24908812-404e-4b8c-958d-335220924aff-service-ca-bundle\") pod \"router-default-868556846d-gx64z\" (UID: \"24908812-404e-4b8c-958d-335220924aff\") " pod="openshift-ingress/router-default-868556846d-gx64z"
Jul 31 22:57:42.837794 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.837770    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/715381b4-025e-436c-96d9-8ab63b3750a9-serving-cert\") pod \"insights-operator-77696685b-k9tqs\" (UID: \"715381b4-025e-436c-96d9-8ab63b3750a9\") " pod="openshift-insights/insights-operator-77696685b-k9tqs"
Jul 31 22:57:42.838174 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.837811    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/24908812-404e-4b8c-958d-335220924aff-default-certificate\") pod \"router-default-868556846d-gx64z\" (UID: \"24908812-404e-4b8c-958d-335220924aff\") " pod="openshift-ingress/router-default-868556846d-gx64z"
Jul 31 22:57:42.838174 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.837848    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/715381b4-025e-436c-96d9-8ab63b3750a9-tmp\") pod \"insights-operator-77696685b-k9tqs\" (UID: \"715381b4-025e-436c-96d9-8ab63b3750a9\") " pod="openshift-insights/insights-operator-77696685b-k9tqs"
Jul 31 22:57:42.838174 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.837881    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mndtr\" (UniqueName: \"kubernetes.io/projected/40b91dc3-a7d9-403a-8208-ca1e019553c5-kube-api-access-mndtr\") pod \"cluster-monitoring-operator-6f55c4f58d-f6wq6\" (UID: \"40b91dc3-a7d9-403a-8208-ca1e019553c5\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-f6wq6"
Jul 31 22:57:42.838174 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.837898    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwfms\" (UniqueName: \"kubernetes.io/projected/ed2bd0da-5ef5-45c8-bba7-ba1042dc308a-kube-api-access-zwfms\") pod \"service-ca-operator-784f97bcf6-5jtrx\" (UID: \"ed2bd0da-5ef5-45c8-bba7-ba1042dc308a\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-5jtrx"
Jul 31 22:57:42.838174 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.837916    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7946036f-064a-4c5f-b619-1f34c3a9e16b-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-xjzj5\" (UID: \"7946036f-064a-4c5f-b619-1f34c3a9e16b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-xjzj5"
Jul 31 22:57:42.838174 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.837937    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/24908812-404e-4b8c-958d-335220924aff-stats-auth\") pod \"router-default-868556846d-gx64z\" (UID: \"24908812-404e-4b8c-958d-335220924aff\") " pod="openshift-ingress/router-default-868556846d-gx64z"
Jul 31 22:57:42.838174 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.837975    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24908812-404e-4b8c-958d-335220924aff-metrics-certs\") pod \"router-default-868556846d-gx64z\" (UID: \"24908812-404e-4b8c-958d-335220924aff\") " pod="openshift-ingress/router-default-868556846d-gx64z"
Jul 31 22:57:42.838174 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.838088    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ghzh\" (UniqueName: \"kubernetes.io/projected/55da6dc5-f4ad-468b-a4c8-497309dad48a-kube-api-access-9ghzh\") pod \"kube-storage-version-migrator-operator-7878f4b88c-p58r8\" (UID: \"55da6dc5-f4ad-468b-a4c8-497309dad48a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-p58r8"
Jul 31 22:57:42.838174 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.838098    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/55da6dc5-f4ad-468b-a4c8-497309dad48a-config\") pod \"kube-storage-version-migrator-operator-7878f4b88c-p58r8\" (UID: \"55da6dc5-f4ad-468b-a4c8-497309dad48a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-p58r8"
Jul 31 22:57:42.838174 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.838119    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/40b91dc3-a7d9-403a-8208-ca1e019553c5-telemetry-config\") pod \"cluster-monitoring-operator-6f55c4f58d-f6wq6\" (UID: \"40b91dc3-a7d9-403a-8208-ca1e019553c5\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-f6wq6"
Jul 31 22:57:42.838174 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.838166    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/715381b4-025e-436c-96d9-8ab63b3750a9-snapshots\") pod \"insights-operator-77696685b-k9tqs\" (UID: \"715381b4-025e-436c-96d9-8ab63b3750a9\") " pod="openshift-insights/insights-operator-77696685b-k9tqs"
Jul 31 22:57:42.838174 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.838186    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/55da6dc5-f4ad-468b-a4c8-497309dad48a-serving-cert\") pod \"kube-storage-version-migrator-operator-7878f4b88c-p58r8\" (UID: \"55da6dc5-f4ad-468b-a4c8-497309dad48a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-p58r8"
Jul 31 22:57:42.839602 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.838209    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/715381b4-025e-436c-96d9-8ab63b3750a9-service-ca-bundle\") pod \"insights-operator-77696685b-k9tqs\" (UID: \"715381b4-025e-436c-96d9-8ab63b3750a9\") " pod="openshift-insights/insights-operator-77696685b-k9tqs"
Jul 31 22:57:42.839602 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.838246    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/715381b4-025e-436c-96d9-8ab63b3750a9-trusted-ca-bundle\") pod \"insights-operator-77696685b-k9tqs\" (UID: \"715381b4-025e-436c-96d9-8ab63b3750a9\") " pod="openshift-insights/insights-operator-77696685b-k9tqs"
Jul 31 22:57:42.839602 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.838274    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ed2bd0da-5ef5-45c8-bba7-ba1042dc308a-config\") pod \"service-ca-operator-784f97bcf6-5jtrx\" (UID: \"ed2bd0da-5ef5-45c8-bba7-ba1042dc308a\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-5jtrx"
Jul 31 22:57:42.839602 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.838835    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/40b91dc3-a7d9-403a-8208-ca1e019553c5-telemetry-config\") pod \"cluster-monitoring-operator-6f55c4f58d-f6wq6\" (UID: \"40b91dc3-a7d9-403a-8208-ca1e019553c5\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-f6wq6"
Jul 31 22:57:42.841277 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.841258    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ed2bd0da-5ef5-45c8-bba7-ba1042dc308a-serving-cert\") pod \"service-ca-operator-784f97bcf6-5jtrx\" (UID: \"ed2bd0da-5ef5-45c8-bba7-ba1042dc308a\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-5jtrx"
Jul 31 22:57:42.841369 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.841259    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/55da6dc5-f4ad-468b-a4c8-497309dad48a-serving-cert\") pod \"kube-storage-version-migrator-operator-7878f4b88c-p58r8\" (UID: \"55da6dc5-f4ad-468b-a4c8-497309dad48a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-p58r8"
Jul 31 22:57:42.847141 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.847118    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mndtr\" (UniqueName: \"kubernetes.io/projected/40b91dc3-a7d9-403a-8208-ca1e019553c5-kube-api-access-mndtr\") pod \"cluster-monitoring-operator-6f55c4f58d-f6wq6\" (UID: \"40b91dc3-a7d9-403a-8208-ca1e019553c5\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-f6wq6"
Jul 31 22:57:42.849874 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.849853    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwfms\" (UniqueName: \"kubernetes.io/projected/ed2bd0da-5ef5-45c8-bba7-ba1042dc308a-kube-api-access-zwfms\") pod \"service-ca-operator-784f97bcf6-5jtrx\" (UID: \"ed2bd0da-5ef5-45c8-bba7-ba1042dc308a\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-5jtrx"
Jul 31 22:57:42.849964 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.849906    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ghzh\" (UniqueName: \"kubernetes.io/projected/55da6dc5-f4ad-468b-a4c8-497309dad48a-kube-api-access-9ghzh\") pod \"kube-storage-version-migrator-operator-7878f4b88c-p58r8\" (UID: \"55da6dc5-f4ad-468b-a4c8-497309dad48a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-p58r8"
Jul 31 22:57:42.914417 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.914330    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-shjgx_eb4e0ba5-fe8b-4bf7-a86d-15db1a2532b2/konnectivity-agent/0.log" limit=1024
Jul 31 22:57:42.939548 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.939523    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/715381b4-025e-436c-96d9-8ab63b3750a9-snapshots\") pod \"insights-operator-77696685b-k9tqs\" (UID: \"715381b4-025e-436c-96d9-8ab63b3750a9\") " pod="openshift-insights/insights-operator-77696685b-k9tqs"
Jul 31 22:57:42.939661 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.939555    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/715381b4-025e-436c-96d9-8ab63b3750a9-service-ca-bundle\") pod \"insights-operator-77696685b-k9tqs\" (UID: \"715381b4-025e-436c-96d9-8ab63b3750a9\") " pod="openshift-insights/insights-operator-77696685b-k9tqs"
Jul 31 22:57:42.939661 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.939575    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/715381b4-025e-436c-96d9-8ab63b3750a9-trusted-ca-bundle\") pod \"insights-operator-77696685b-k9tqs\" (UID: \"715381b4-025e-436c-96d9-8ab63b3750a9\") " pod="openshift-insights/insights-operator-77696685b-k9tqs"
Jul 31 22:57:42.939661 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.939603    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89xbj\" (UniqueName: \"kubernetes.io/projected/715381b4-025e-436c-96d9-8ab63b3750a9-kube-api-access-89xbj\") pod \"insights-operator-77696685b-k9tqs\" (UID: \"715381b4-025e-436c-96d9-8ab63b3750a9\") " pod="openshift-insights/insights-operator-77696685b-k9tqs"
Jul 31 22:57:42.939661 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.939630    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvbg8\" (UniqueName: \"kubernetes.io/projected/24908812-404e-4b8c-958d-335220924aff-kube-api-access-dvbg8\") pod \"router-default-868556846d-gx64z\" (UID: \"24908812-404e-4b8c-958d-335220924aff\") " pod="openshift-ingress/router-default-868556846d-gx64z"
Jul 31 22:57:42.939874 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.939691    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8sf8j\" (UniqueName: \"kubernetes.io/projected/7946036f-064a-4c5f-b619-1f34c3a9e16b-kube-api-access-8sf8j\") pod \"cluster-samples-operator-559cbf5745-xjzj5\" (UID: \"7946036f-064a-4c5f-b619-1f34c3a9e16b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-xjzj5"
Jul 31 22:57:42.939874 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.939836    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/24908812-404e-4b8c-958d-335220924aff-service-ca-bundle\") pod \"router-default-868556846d-gx64z\" (UID: \"24908812-404e-4b8c-958d-335220924aff\") " pod="openshift-ingress/router-default-868556846d-gx64z"
Jul 31 22:57:42.939978 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.939896    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/715381b4-025e-436c-96d9-8ab63b3750a9-serving-cert\") pod \"insights-operator-77696685b-k9tqs\" (UID: \"715381b4-025e-436c-96d9-8ab63b3750a9\") " pod="openshift-insights/insights-operator-77696685b-k9tqs"
Jul 31 22:57:42.939978 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.939926    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/24908812-404e-4b8c-958d-335220924aff-default-certificate\") pod \"router-default-868556846d-gx64z\" (UID: \"24908812-404e-4b8c-958d-335220924aff\") " pod="openshift-ingress/router-default-868556846d-gx64z"
Jul 31 22:57:42.940076 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.940029    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/715381b4-025e-436c-96d9-8ab63b3750a9-tmp\") pod \"insights-operator-77696685b-k9tqs\" (UID: \"715381b4-025e-436c-96d9-8ab63b3750a9\") " pod="openshift-insights/insights-operator-77696685b-k9tqs"
Jul 31 22:57:42.940076 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:42.940064    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/24908812-404e-4b8c-958d-335220924aff-service-ca-bundle podName:24908812-404e-4b8c-958d-335220924aff nodeName:}" failed. No retries permitted until 2026-07-31 22:57:43.440041439 +0000 UTC m=+47.085827740 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/24908812-404e-4b8c-958d-335220924aff-service-ca-bundle") pod "router-default-868556846d-gx64z" (UID: "24908812-404e-4b8c-958d-335220924aff") : configmap references non-existent config key: service-ca.crt
Jul 31 22:57:42.940196 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.940109    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7946036f-064a-4c5f-b619-1f34c3a9e16b-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-xjzj5\" (UID: \"7946036f-064a-4c5f-b619-1f34c3a9e16b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-xjzj5"
Jul 31 22:57:42.940196 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.940140    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/24908812-404e-4b8c-958d-335220924aff-stats-auth\") pod \"router-default-868556846d-gx64z\" (UID: \"24908812-404e-4b8c-958d-335220924aff\") " pod="openshift-ingress/router-default-868556846d-gx64z"
Jul 31 22:57:42.940196 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.940169    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24908812-404e-4b8c-958d-335220924aff-metrics-certs\") pod \"router-default-868556846d-gx64z\" (UID: \"24908812-404e-4b8c-958d-335220924aff\") " pod="openshift-ingress/router-default-868556846d-gx64z"
Jul 31 22:57:42.940344 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:42.940310    2584 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 31 22:57:42.940344 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.940310    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/715381b4-025e-436c-96d9-8ab63b3750a9-snapshots\") pod \"insights-operator-77696685b-k9tqs\" (UID: \"715381b4-025e-436c-96d9-8ab63b3750a9\") " pod="openshift-insights/insights-operator-77696685b-k9tqs"
Jul 31 22:57:42.940427 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:42.940313    2584 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 31 22:57:42.940427 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.940314    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/715381b4-025e-436c-96d9-8ab63b3750a9-service-ca-bundle\") pod \"insights-operator-77696685b-k9tqs\" (UID: \"715381b4-025e-436c-96d9-8ab63b3750a9\") " pod="openshift-insights/insights-operator-77696685b-k9tqs"
Jul 31 22:57:42.940427 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:42.940351    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24908812-404e-4b8c-958d-335220924aff-metrics-certs podName:24908812-404e-4b8c-958d-335220924aff nodeName:}" failed. No retries permitted until 2026-07-31 22:57:43.440337907 +0000 UTC m=+47.086124197 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24908812-404e-4b8c-958d-335220924aff-metrics-certs") pod "router-default-868556846d-gx64z" (UID: "24908812-404e-4b8c-958d-335220924aff") : secret "router-metrics-certs-default" not found
Jul 31 22:57:42.940542 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:42.940428    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7946036f-064a-4c5f-b619-1f34c3a9e16b-samples-operator-tls podName:7946036f-064a-4c5f-b619-1f34c3a9e16b nodeName:}" failed. No retries permitted until 2026-07-31 22:57:43.440411757 +0000 UTC m=+47.086198057 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7946036f-064a-4c5f-b619-1f34c3a9e16b-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-xjzj5" (UID: "7946036f-064a-4c5f-b619-1f34c3a9e16b") : secret "samples-operator-tls" not found
Jul 31 22:57:42.940662 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.940645    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/715381b4-025e-436c-96d9-8ab63b3750a9-tmp\") pod \"insights-operator-77696685b-k9tqs\" (UID: \"715381b4-025e-436c-96d9-8ab63b3750a9\") " pod="openshift-insights/insights-operator-77696685b-k9tqs"
Jul 31 22:57:42.940791 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.940771    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/715381b4-025e-436c-96d9-8ab63b3750a9-trusted-ca-bundle\") pod \"insights-operator-77696685b-k9tqs\" (UID: \"715381b4-025e-436c-96d9-8ab63b3750a9\") " pod="openshift-insights/insights-operator-77696685b-k9tqs"
Jul 31 22:57:42.942302 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.942279    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/715381b4-025e-436c-96d9-8ab63b3750a9-serving-cert\") pod \"insights-operator-77696685b-k9tqs\" (UID: \"715381b4-025e-436c-96d9-8ab63b3750a9\") " pod="openshift-insights/insights-operator-77696685b-k9tqs"
Jul 31 22:57:42.942435 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.942420    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/24908812-404e-4b8c-958d-335220924aff-default-certificate\") pod \"router-default-868556846d-gx64z\" (UID: \"24908812-404e-4b8c-958d-335220924aff\") " pod="openshift-ingress/router-default-868556846d-gx64z"
Jul 31 22:57:42.942489 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.942422    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/24908812-404e-4b8c-958d-335220924aff-stats-auth\") pod \"router-default-868556846d-gx64z\" (UID: \"24908812-404e-4b8c-958d-335220924aff\") " pod="openshift-ingress/router-default-868556846d-gx64z"
Jul 31 22:57:42.949341 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.949318    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvbg8\" (UniqueName: \"kubernetes.io/projected/24908812-404e-4b8c-958d-335220924aff-kube-api-access-dvbg8\") pod \"router-default-868556846d-gx64z\" (UID: \"24908812-404e-4b8c-958d-335220924aff\") " pod="openshift-ingress/router-default-868556846d-gx64z"
Jul 31 22:57:42.949341 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.949335    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8sf8j\" (UniqueName: \"kubernetes.io/projected/7946036f-064a-4c5f-b619-1f34c3a9e16b-kube-api-access-8sf8j\") pod \"cluster-samples-operator-559cbf5745-xjzj5\" (UID: \"7946036f-064a-4c5f-b619-1f34c3a9e16b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-xjzj5"
Jul 31 22:57:42.949621 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.949604    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-89xbj\" (UniqueName: \"kubernetes.io/projected/715381b4-025e-436c-96d9-8ab63b3750a9-kube-api-access-89xbj\") pod \"insights-operator-77696685b-k9tqs\" (UID: \"715381b4-025e-436c-96d9-8ab63b3750a9\") " pod="openshift-insights/insights-operator-77696685b-k9tqs"
Jul 31 22:57:42.976770 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.976748    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-d8bd65946-zn48s"]
Jul 31 22:57:42.995185 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.995166    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-d8bd65946-zn48s"]
Jul 31 22:57:42.995273 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.995262    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d8bd65946-zn48s"
Jul 31 22:57:42.997999 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.997902    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 31 22:57:42.997999 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.997906    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-sn4bg\""
Jul 31 22:57:42.998152 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.998004    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 31 22:57:42.998152 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.998063    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 31 22:57:42.999808 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:42.999771    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-shjgx_eb4e0ba5-fe8b-4bf7-a86d-15db1a2532b2/konnectivity-agent/0.log" limit=1024
Jul 31 22:57:43.001058 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:43.001040    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-5jtrx"
Jul 31 22:57:43.003860 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:43.003843    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 31 22:57:43.024231 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:43.024214    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-p58r8"
Jul 31 22:57:43.040671 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:43.040649    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bv2x\" (UniqueName: \"kubernetes.io/projected/798610c4-70cc-4455-943f-b0d6b0dca9d9-kube-api-access-9bv2x\") pod \"image-registry-d8bd65946-zn48s\" (UID: \"798610c4-70cc-4455-943f-b0d6b0dca9d9\") " pod="openshift-image-registry/image-registry-d8bd65946-zn48s"
Jul 31 22:57:43.040826 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:43.040807    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/798610c4-70cc-4455-943f-b0d6b0dca9d9-image-registry-private-configuration\") pod \"image-registry-d8bd65946-zn48s\" (UID: \"798610c4-70cc-4455-943f-b0d6b0dca9d9\") " pod="openshift-image-registry/image-registry-d8bd65946-zn48s"
Jul 31 22:57:43.041014 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:43.040833    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/798610c4-70cc-4455-943f-b0d6b0dca9d9-ca-trust-extracted\") pod \"image-registry-d8bd65946-zn48s\" (UID: \"798610c4-70cc-4455-943f-b0d6b0dca9d9\") " pod="openshift-image-registry/image-registry-d8bd65946-zn48s"
Jul 31 22:57:43.041014 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:43.040878    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/798610c4-70cc-4455-943f-b0d6b0dca9d9-registry-tls\") pod \"image-registry-d8bd65946-zn48s\" (UID: \"798610c4-70cc-4455-943f-b0d6b0dca9d9\") " pod="openshift-image-registry/image-registry-d8bd65946-zn48s"
Jul 31 22:57:43.041014 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:43.040894    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/798610c4-70cc-4455-943f-b0d6b0dca9d9-bound-sa-token\") pod \"image-registry-d8bd65946-zn48s\" (UID: \"798610c4-70cc-4455-943f-b0d6b0dca9d9\") " pod="openshift-image-registry/image-registry-d8bd65946-zn48s"
Jul 31 22:57:43.041014 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:43.040913    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/798610c4-70cc-4455-943f-b0d6b0dca9d9-trusted-ca\") pod \"image-registry-d8bd65946-zn48s\" (UID: \"798610c4-70cc-4455-943f-b0d6b0dca9d9\") " pod="openshift-image-registry/image-registry-d8bd65946-zn48s"
Jul 31 22:57:43.041014 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:43.040985    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/798610c4-70cc-4455-943f-b0d6b0dca9d9-registry-certificates\") pod \"image-registry-d8bd65946-zn48s\" (UID: \"798610c4-70cc-4455-943f-b0d6b0dca9d9\") " pod="openshift-image-registry/image-registry-d8bd65946-zn48s"
Jul 31 22:57:43.041242 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:43.041016    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/798610c4-70cc-4455-943f-b0d6b0dca9d9-installation-pull-secrets\") pod \"image-registry-d8bd65946-zn48s\" (UID: \"798610c4-70cc-4455-943f-b0d6b0dca9d9\") " pod="openshift-image-registry/image-registry-d8bd65946-zn48s"
Jul 31 22:57:43.049874 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:43.049683    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-77696685b-k9tqs"
Jul 31 22:57:43.143236 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:43.142456    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/798610c4-70cc-4455-943f-b0d6b0dca9d9-registry-tls\") pod \"image-registry-d8bd65946-zn48s\" (UID: \"798610c4-70cc-4455-943f-b0d6b0dca9d9\") " pod="openshift-image-registry/image-registry-d8bd65946-zn48s"
Jul 31 22:57:43.143236 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:43.142499    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/798610c4-70cc-4455-943f-b0d6b0dca9d9-bound-sa-token\") pod \"image-registry-d8bd65946-zn48s\" (UID: \"798610c4-70cc-4455-943f-b0d6b0dca9d9\") " pod="openshift-image-registry/image-registry-d8bd65946-zn48s"
Jul 31 22:57:43.143236 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:43.142528    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/798610c4-70cc-4455-943f-b0d6b0dca9d9-trusted-ca\") pod \"image-registry-d8bd65946-zn48s\" (UID: \"798610c4-70cc-4455-943f-b0d6b0dca9d9\") " pod="openshift-image-registry/image-registry-d8bd65946-zn48s"
Jul 31 22:57:43.143236 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:43.142597    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/798610c4-70cc-4455-943f-b0d6b0dca9d9-registry-certificates\") pod \"image-registry-d8bd65946-zn48s\" (UID: \"798610c4-70cc-4455-943f-b0d6b0dca9d9\") " pod="openshift-image-registry/image-registry-d8bd65946-zn48s"
Jul 31 22:57:43.143236 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:43.142626    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/798610c4-70cc-4455-943f-b0d6b0dca9d9-installation-pull-secrets\") pod \"image-registry-d8bd65946-zn48s\" (UID: \"798610c4-70cc-4455-943f-b0d6b0dca9d9\") " pod="openshift-image-registry/image-registry-d8bd65946-zn48s"
Jul 31 22:57:43.143236 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:43.142670    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bv2x\" (UniqueName: \"kubernetes.io/projected/798610c4-70cc-4455-943f-b0d6b0dca9d9-kube-api-access-9bv2x\") pod \"image-registry-d8bd65946-zn48s\" (UID: \"798610c4-70cc-4455-943f-b0d6b0dca9d9\") " pod="openshift-image-registry/image-registry-d8bd65946-zn48s"
Jul 31 22:57:43.143236 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:43.142693    2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 31 22:57:43.143236 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:43.142730    2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d8bd65946-zn48s: secret "image-registry-tls" not found
Jul 31 22:57:43.143236 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:43.142792    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/798610c4-70cc-4455-943f-b0d6b0dca9d9-registry-tls podName:798610c4-70cc-4455-943f-b0d6b0dca9d9 nodeName:}" failed. No retries permitted until 2026-07-31 22:57:43.642772265 +0000 UTC m=+47.288558575 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/798610c4-70cc-4455-943f-b0d6b0dca9d9-registry-tls") pod "image-registry-d8bd65946-zn48s" (UID: "798610c4-70cc-4455-943f-b0d6b0dca9d9") : secret "image-registry-tls" not found
Jul 31 22:57:43.143236 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:43.142880    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/798610c4-70cc-4455-943f-b0d6b0dca9d9-image-registry-private-configuration\") pod \"image-registry-d8bd65946-zn48s\" (UID: \"798610c4-70cc-4455-943f-b0d6b0dca9d9\") " pod="openshift-image-registry/image-registry-d8bd65946-zn48s"
Jul 31 22:57:43.143236 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:43.142910    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/798610c4-70cc-4455-943f-b0d6b0dca9d9-ca-trust-extracted\") pod \"image-registry-d8bd65946-zn48s\" (UID: \"798610c4-70cc-4455-943f-b0d6b0dca9d9\") " pod="openshift-image-registry/image-registry-d8bd65946-zn48s"
Jul 31 22:57:43.143831 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:43.143303    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/798610c4-70cc-4455-943f-b0d6b0dca9d9-registry-certificates\") pod \"image-registry-d8bd65946-zn48s\" (UID: \"798610c4-70cc-4455-943f-b0d6b0dca9d9\") " pod="openshift-image-registry/image-registry-d8bd65946-zn48s"
Jul 31 22:57:43.143831 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:43.143502    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/798610c4-70cc-4455-943f-b0d6b0dca9d9-ca-trust-extracted\") pod \"image-registry-d8bd65946-zn48s\" (UID: \"798610c4-70cc-4455-943f-b0d6b0dca9d9\") " pod="openshift-image-registry/image-registry-d8bd65946-zn48s"
Jul 31 22:57:43.143831 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:43.143724    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/798610c4-70cc-4455-943f-b0d6b0dca9d9-trusted-ca\") pod \"image-registry-d8bd65946-zn48s\" (UID: \"798610c4-70cc-4455-943f-b0d6b0dca9d9\") " pod="openshift-image-registry/image-registry-d8bd65946-zn48s"
Jul 31 22:57:43.145830 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:43.145782    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/798610c4-70cc-4455-943f-b0d6b0dca9d9-installation-pull-secrets\") pod \"image-registry-d8bd65946-zn48s\" (UID: \"798610c4-70cc-4455-943f-b0d6b0dca9d9\") " pod="openshift-image-registry/image-registry-d8bd65946-zn48s"
Jul 31 22:57:43.145947 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:43.145931    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/798610c4-70cc-4455-943f-b0d6b0dca9d9-image-registry-private-configuration\") pod \"image-registry-d8bd65946-zn48s\" (UID: \"798610c4-70cc-4455-943f-b0d6b0dca9d9\") " pod="openshift-image-registry/image-registry-d8bd65946-zn48s"
Jul 31 22:57:43.155481 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:43.155456    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-784f97bcf6-5jtrx"]
Jul 31 22:57:43.157687 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:43.157574    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/798610c4-70cc-4455-943f-b0d6b0dca9d9-bound-sa-token\") pod \"image-registry-d8bd65946-zn48s\" (UID: \"798610c4-70cc-4455-943f-b0d6b0dca9d9\") " pod="openshift-image-registry/image-registry-d8bd65946-zn48s"
Jul 31 22:57:43.159714 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:57:43.159677    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded2bd0da_5ef5_45c8_bba7_ba1042dc308a.slice/crio-9bd735d5588a178d7b11aecb5ab25cc51b1c6c1b9751220e22d05b8aaf26dde3 WatchSource:0}: Error finding container 9bd735d5588a178d7b11aecb5ab25cc51b1c6c1b9751220e22d05b8aaf26dde3: Status 404 returned error can't find the container with id 9bd735d5588a178d7b11aecb5ab25cc51b1c6c1b9751220e22d05b8aaf26dde3
Jul 31 22:57:43.159815 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:43.159782    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bv2x\" (UniqueName: \"kubernetes.io/projected/798610c4-70cc-4455-943f-b0d6b0dca9d9-kube-api-access-9bv2x\") pod \"image-registry-d8bd65946-zn48s\" (UID: \"798610c4-70cc-4455-943f-b0d6b0dca9d9\") " pod="openshift-image-registry/image-registry-d8bd65946-zn48s"
Jul 31 22:57:43.185212 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:43.185191    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-p58r8"]
Jul 31 22:57:43.187281 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:57:43.187258    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55da6dc5_f4ad_468b_a4c8_497309dad48a.slice/crio-7a35fd867b902bf430b3046cde4267197cfcea2a5f8c5fb9b25c85dc09819534 WatchSource:0}: Error finding container 7a35fd867b902bf430b3046cde4267197cfcea2a5f8c5fb9b25c85dc09819534: Status 404 returned error can't find the container with id 7a35fd867b902bf430b3046cde4267197cfcea2a5f8c5fb9b25c85dc09819534
Jul 31 22:57:43.199165 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:43.199140    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-77696685b-k9tqs"]
Jul 31 22:57:43.201668 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:57:43.201641    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod715381b4_025e_436c_96d9_8ab63b3750a9.slice/crio-23fd5e1fb527aaea14ff1bda798c3ff000a9282399d2e1b6132a2efab4ef6283 WatchSource:0}: Error finding container 23fd5e1fb527aaea14ff1bda798c3ff000a9282399d2e1b6132a2efab4ef6283: Status 404 returned error can't find the container with id 23fd5e1fb527aaea14ff1bda798c3ff000a9282399d2e1b6132a2efab4ef6283
Jul 31 22:57:43.256886 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:43.256856    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-p58r8" event={"ID":"55da6dc5-f4ad-468b-a4c8-497309dad48a","Type":"ContainerStarted","Data":"7a35fd867b902bf430b3046cde4267197cfcea2a5f8c5fb9b25c85dc09819534"}
Jul 31 22:57:43.257851 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:43.257831    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-77696685b-k9tqs" event={"ID":"715381b4-025e-436c-96d9-8ab63b3750a9","Type":"ContainerStarted","Data":"23fd5e1fb527aaea14ff1bda798c3ff000a9282399d2e1b6132a2efab4ef6283"}
Jul 31 22:57:43.258797 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:43.258777    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-5jtrx" event={"ID":"ed2bd0da-5ef5-45c8-bba7-ba1042dc308a","Type":"ContainerStarted","Data":"9bd735d5588a178d7b11aecb5ab25cc51b1c6c1b9751220e22d05b8aaf26dde3"}
Jul 31 22:57:43.344748 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:43.344715    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/40b91dc3-a7d9-403a-8208-ca1e019553c5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-f6wq6\" (UID: \"40b91dc3-a7d9-403a-8208-ca1e019553c5\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-f6wq6"
Jul 31 22:57:43.344896 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:43.344865    2584 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 31 22:57:43.344936 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:43.344927    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40b91dc3-a7d9-403a-8208-ca1e019553c5-cluster-monitoring-operator-tls podName:40b91dc3-a7d9-403a-8208-ca1e019553c5 nodeName:}" failed. No retries permitted until 2026-07-31 22:57:44.344911771 +0000 UTC m=+47.990698057 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/40b91dc3-a7d9-403a-8208-ca1e019553c5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-f6wq6" (UID: "40b91dc3-a7d9-403a-8208-ca1e019553c5") : secret "cluster-monitoring-operator-tls" not found
Jul 31 22:57:43.445895 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:43.445816    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24908812-404e-4b8c-958d-335220924aff-metrics-certs\") pod \"router-default-868556846d-gx64z\" (UID: \"24908812-404e-4b8c-958d-335220924aff\") " pod="openshift-ingress/router-default-868556846d-gx64z"
Jul 31 22:57:43.446023 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:43.445968    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/24908812-404e-4b8c-958d-335220924aff-service-ca-bundle\") pod \"router-default-868556846d-gx64z\" (UID: \"24908812-404e-4b8c-958d-335220924aff\") " pod="openshift-ingress/router-default-868556846d-gx64z"
Jul 31 22:57:43.446023 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:43.445974    2584 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 31 22:57:43.446023 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:43.446005    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7946036f-064a-4c5f-b619-1f34c3a9e16b-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-xjzj5\" (UID: \"7946036f-064a-4c5f-b619-1f34c3a9e16b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-xjzj5"
Jul 31 22:57:43.446142 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:43.446042    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24908812-404e-4b8c-958d-335220924aff-metrics-certs podName:24908812-404e-4b8c-958d-335220924aff nodeName:}" failed. No retries permitted until 2026-07-31 22:57:44.446020967 +0000 UTC m=+48.091807304 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24908812-404e-4b8c-958d-335220924aff-metrics-certs") pod "router-default-868556846d-gx64z" (UID: "24908812-404e-4b8c-958d-335220924aff") : secret "router-metrics-certs-default" not found
Jul 31 22:57:43.446142 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:43.446100    2584 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 31 22:57:43.446142 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:43.446122    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/24908812-404e-4b8c-958d-335220924aff-service-ca-bundle podName:24908812-404e-4b8c-958d-335220924aff nodeName:}" failed. No retries permitted until 2026-07-31 22:57:44.446107322 +0000 UTC m=+48.091893610 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/24908812-404e-4b8c-958d-335220924aff-service-ca-bundle") pod "router-default-868556846d-gx64z" (UID: "24908812-404e-4b8c-958d-335220924aff") : configmap references non-existent config key: service-ca.crt
Jul 31 22:57:43.446142 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:43.446144    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7946036f-064a-4c5f-b619-1f34c3a9e16b-samples-operator-tls podName:7946036f-064a-4c5f-b619-1f34c3a9e16b nodeName:}" failed. No retries permitted until 2026-07-31 22:57:44.446136368 +0000 UTC m=+48.091922654 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7946036f-064a-4c5f-b619-1f34c3a9e16b-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-xjzj5" (UID: "7946036f-064a-4c5f-b619-1f34c3a9e16b") : secret "samples-operator-tls" not found
Jul 31 22:57:43.647828 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:43.647791    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/798610c4-70cc-4455-943f-b0d6b0dca9d9-registry-tls\") pod \"image-registry-d8bd65946-zn48s\" (UID: \"798610c4-70cc-4455-943f-b0d6b0dca9d9\") " pod="openshift-image-registry/image-registry-d8bd65946-zn48s"
Jul 31 22:57:43.648191 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:43.647938    2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 31 22:57:43.648191 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:43.647955    2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d8bd65946-zn48s: secret "image-registry-tls" not found
Jul 31 22:57:43.648191 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:43.648007    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/798610c4-70cc-4455-943f-b0d6b0dca9d9-registry-tls podName:798610c4-70cc-4455-943f-b0d6b0dca9d9 nodeName:}" failed. No retries permitted until 2026-07-31 22:57:44.647992976 +0000 UTC m=+48.293779263 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/798610c4-70cc-4455-943f-b0d6b0dca9d9-registry-tls") pod "image-registry-d8bd65946-zn48s" (UID: "798610c4-70cc-4455-943f-b0d6b0dca9d9") : secret "image-registry-tls" not found
Jul 31 22:57:44.354170 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:44.354131    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/40b91dc3-a7d9-403a-8208-ca1e019553c5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-f6wq6\" (UID: \"40b91dc3-a7d9-403a-8208-ca1e019553c5\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-f6wq6"
Jul 31 22:57:44.354428 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:44.354365    2584 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 31 22:57:44.354511 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:44.354428    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40b91dc3-a7d9-403a-8208-ca1e019553c5-cluster-monitoring-operator-tls podName:40b91dc3-a7d9-403a-8208-ca1e019553c5 nodeName:}" failed. No retries permitted until 2026-07-31 22:57:46.354410386 +0000 UTC m=+50.000196692 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/40b91dc3-a7d9-403a-8208-ca1e019553c5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-f6wq6" (UID: "40b91dc3-a7d9-403a-8208-ca1e019553c5") : secret "cluster-monitoring-operator-tls" not found
Jul 31 22:57:44.455027 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:44.454991    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7946036f-064a-4c5f-b619-1f34c3a9e16b-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-xjzj5\" (UID: \"7946036f-064a-4c5f-b619-1f34c3a9e16b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-xjzj5"
Jul 31 22:57:44.455027 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:44.455041    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24908812-404e-4b8c-958d-335220924aff-metrics-certs\") pod \"router-default-868556846d-gx64z\" (UID: \"24908812-404e-4b8c-958d-335220924aff\") " pod="openshift-ingress/router-default-868556846d-gx64z"
Jul 31 22:57:44.455278 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:44.455175    2584 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 31 22:57:44.455278 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:44.455188    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/24908812-404e-4b8c-958d-335220924aff-service-ca-bundle\") pod \"router-default-868556846d-gx64z\" (UID: \"24908812-404e-4b8c-958d-335220924aff\") " pod="openshift-ingress/router-default-868556846d-gx64z"
Jul 31 22:57:44.455278 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:44.455248    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7946036f-064a-4c5f-b619-1f34c3a9e16b-samples-operator-tls podName:7946036f-064a-4c5f-b619-1f34c3a9e16b nodeName:}" failed. No retries permitted until 2026-07-31 22:57:46.455226126 +0000 UTC m=+50.101012421 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7946036f-064a-4c5f-b619-1f34c3a9e16b-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-xjzj5" (UID: "7946036f-064a-4c5f-b619-1f34c3a9e16b") : secret "samples-operator-tls" not found
Jul 31 22:57:44.455446 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:44.455308    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/24908812-404e-4b8c-958d-335220924aff-service-ca-bundle podName:24908812-404e-4b8c-958d-335220924aff nodeName:}" failed. No retries permitted until 2026-07-31 22:57:46.455292832 +0000 UTC m=+50.101079123 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/24908812-404e-4b8c-958d-335220924aff-service-ca-bundle") pod "router-default-868556846d-gx64z" (UID: "24908812-404e-4b8c-958d-335220924aff") : configmap references non-existent config key: service-ca.crt
Jul 31 22:57:44.455446 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:44.455378    2584 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 31 22:57:44.455446 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:44.455411    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24908812-404e-4b8c-958d-335220924aff-metrics-certs podName:24908812-404e-4b8c-958d-335220924aff nodeName:}" failed. No retries permitted until 2026-07-31 22:57:46.45540083 +0000 UTC m=+50.101187141 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24908812-404e-4b8c-958d-335220924aff-metrics-certs") pod "router-default-868556846d-gx64z" (UID: "24908812-404e-4b8c-958d-335220924aff") : secret "router-metrics-certs-default" not found
Jul 31 22:57:44.656725 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:44.656626    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/798610c4-70cc-4455-943f-b0d6b0dca9d9-registry-tls\") pod \"image-registry-d8bd65946-zn48s\" (UID: \"798610c4-70cc-4455-943f-b0d6b0dca9d9\") " pod="openshift-image-registry/image-registry-d8bd65946-zn48s"
Jul 31 22:57:44.657096 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:44.656820    2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 31 22:57:44.657096 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:44.656844    2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d8bd65946-zn48s: secret "image-registry-tls" not found
Jul 31 22:57:44.657096 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:44.656925    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/798610c4-70cc-4455-943f-b0d6b0dca9d9-registry-tls podName:798610c4-70cc-4455-943f-b0d6b0dca9d9 nodeName:}" failed. No retries permitted until 2026-07-31 22:57:46.656898126 +0000 UTC m=+50.302684427 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/798610c4-70cc-4455-943f-b0d6b0dca9d9-registry-tls") pod "image-registry-d8bd65946-zn48s" (UID: "798610c4-70cc-4455-943f-b0d6b0dca9d9") : secret "image-registry-tls" not found
Jul 31 22:57:45.012061 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:45.011992    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-shjgx_eb4e0ba5-fe8b-4bf7-a86d-15db1a2532b2/konnectivity-agent/0.log" limit=1024
Jul 31 22:57:45.074434 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:45.074396    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-shjgx_eb4e0ba5-fe8b-4bf7-a86d-15db1a2532b2/konnectivity-agent/0.log" limit=1024
Jul 31 22:57:46.034933 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:46.034899    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-shjgx_eb4e0ba5-fe8b-4bf7-a86d-15db1a2532b2/konnectivity-agent/0.log" limit=1024
Jul 31 22:57:46.131145 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:46.131106    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-shjgx_eb4e0ba5-fe8b-4bf7-a86d-15db1a2532b2/konnectivity-agent/0.log" limit=1024
Jul 31 22:57:46.373077 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:46.373034    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/40b91dc3-a7d9-403a-8208-ca1e019553c5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-f6wq6\" (UID: \"40b91dc3-a7d9-403a-8208-ca1e019553c5\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-f6wq6"
Jul 31 22:57:46.373252 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:46.373223    2584 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 31 22:57:46.373317 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:46.373310    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40b91dc3-a7d9-403a-8208-ca1e019553c5-cluster-monitoring-operator-tls podName:40b91dc3-a7d9-403a-8208-ca1e019553c5 nodeName:}" failed. No retries permitted until 2026-07-31 22:57:50.373289504 +0000 UTC m=+54.019075798 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/40b91dc3-a7d9-403a-8208-ca1e019553c5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-f6wq6" (UID: "40b91dc3-a7d9-403a-8208-ca1e019553c5") : secret "cluster-monitoring-operator-tls" not found
Jul 31 22:57:46.474000 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:46.473959    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/24908812-404e-4b8c-958d-335220924aff-service-ca-bundle\") pod \"router-default-868556846d-gx64z\" (UID: \"24908812-404e-4b8c-958d-335220924aff\") " pod="openshift-ingress/router-default-868556846d-gx64z"
Jul 31 22:57:46.474150 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:46.474008    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7946036f-064a-4c5f-b619-1f34c3a9e16b-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-xjzj5\" (UID: \"7946036f-064a-4c5f-b619-1f34c3a9e16b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-xjzj5"
Jul 31 22:57:46.474150 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:46.474029    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24908812-404e-4b8c-958d-335220924aff-metrics-certs\") pod \"router-default-868556846d-gx64z\" (UID: \"24908812-404e-4b8c-958d-335220924aff\") " pod="openshift-ingress/router-default-868556846d-gx64z"
Jul 31 22:57:46.474150 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:46.474106    2584 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 31 22:57:46.474150 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:46.474113    2584 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 31 22:57:46.474150 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:46.474123    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/24908812-404e-4b8c-958d-335220924aff-service-ca-bundle podName:24908812-404e-4b8c-958d-335220924aff nodeName:}" failed. No retries permitted until 2026-07-31 22:57:50.47410562 +0000 UTC m=+54.119891906 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/24908812-404e-4b8c-958d-335220924aff-service-ca-bundle") pod "router-default-868556846d-gx64z" (UID: "24908812-404e-4b8c-958d-335220924aff") : configmap references non-existent config key: service-ca.crt
Jul 31 22:57:46.474150 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:46.474153    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24908812-404e-4b8c-958d-335220924aff-metrics-certs podName:24908812-404e-4b8c-958d-335220924aff nodeName:}" failed. No retries permitted until 2026-07-31 22:57:50.474138033 +0000 UTC m=+54.119924323 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24908812-404e-4b8c-958d-335220924aff-metrics-certs") pod "router-default-868556846d-gx64z" (UID: "24908812-404e-4b8c-958d-335220924aff") : secret "router-metrics-certs-default" not found
Jul 31 22:57:46.474378 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:46.474270    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7946036f-064a-4c5f-b619-1f34c3a9e16b-samples-operator-tls podName:7946036f-064a-4c5f-b619-1f34c3a9e16b nodeName:}" failed. No retries permitted until 2026-07-31 22:57:50.474247518 +0000 UTC m=+54.120033812 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7946036f-064a-4c5f-b619-1f34c3a9e16b-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-xjzj5" (UID: "7946036f-064a-4c5f-b619-1f34c3a9e16b") : secret "samples-operator-tls" not found
Jul 31 22:57:46.675492 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:46.675421    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/798610c4-70cc-4455-943f-b0d6b0dca9d9-registry-tls\") pod \"image-registry-d8bd65946-zn48s\" (UID: \"798610c4-70cc-4455-943f-b0d6b0dca9d9\") " pod="openshift-image-registry/image-registry-d8bd65946-zn48s"
Jul 31 22:57:46.675650 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:46.675566    2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 31 22:57:46.675650 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:46.675585    2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d8bd65946-zn48s: secret "image-registry-tls" not found
Jul 31 22:57:46.675650 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:46.675636    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/798610c4-70cc-4455-943f-b0d6b0dca9d9-registry-tls podName:798610c4-70cc-4455-943f-b0d6b0dca9d9 nodeName:}" failed. No retries permitted until 2026-07-31 22:57:50.675622568 +0000 UTC m=+54.321408858 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/798610c4-70cc-4455-943f-b0d6b0dca9d9-registry-tls") pod "image-registry-d8bd65946-zn48s" (UID: "798610c4-70cc-4455-943f-b0d6b0dca9d9") : secret "image-registry-tls" not found
Jul 31 22:57:47.271043 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:47.270942    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-77696685b-k9tqs" event={"ID":"715381b4-025e-436c-96d9-8ab63b3750a9","Type":"ContainerStarted","Data":"bedbfd81673aadaf2474d4f51b838dc9c590d4d00ba42b9a3cb0152666057eb6"}
Jul 31 22:57:47.272677 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:47.272645    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-5jtrx" event={"ID":"ed2bd0da-5ef5-45c8-bba7-ba1042dc308a","Type":"ContainerStarted","Data":"44417391135d61f5237a417145551be7fcf421eaa0b1900cc9648a6c209f917a"}
Jul 31 22:57:47.274120 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:47.274097    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-p58r8" event={"ID":"55da6dc5-f4ad-468b-a4c8-497309dad48a","Type":"ContainerStarted","Data":"02423e57f267eeb6d7f488c6ea43d413f1a153ce7ccce72ba7a674af8b17ccef"}
Jul 31 22:57:47.289298 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:47.289259    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-77696685b-k9tqs" podStartSLOduration=1.565605208 podStartE2EDuration="5.289245469s" totalImagesPullingTime="3.723640261s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:57:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:57:47.28880047 +0000 UTC m=+50.934586778" watchObservedRunningTime="2026-07-31 22:57:47.289245469 +0000 UTC m=+50.935031779"
Jul 31 22:57:47.308341 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:47.308283    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-5jtrx" podStartSLOduration=1.536438577 podStartE2EDuration="5.30826306s" totalImagesPullingTime="3.771824483s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:57:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:57:47.307744321 +0000 UTC m=+50.953530625" watchObservedRunningTime="2026-07-31 22:57:47.30826306 +0000 UTC m=+50.954049373"
Jul 31 22:57:47.330088 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:47.330034    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-p58r8" podStartSLOduration=1.592384544 podStartE2EDuration="5.330013031s" totalImagesPullingTime="3.737628487s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:57:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:57:47.329064625 +0000 UTC m=+50.974850951" watchObservedRunningTime="2026-07-31 22:57:47.330013031 +0000 UTC m=+50.975799336"
Jul 31 22:57:47.583179 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:47.583140    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/735a47e9-8a16-4dd3-93ec-2e3ac70100d4-metrics-tls\") pod \"dns-default-fhq7n\" (UID: \"735a47e9-8a16-4dd3-93ec-2e3ac70100d4\") " pod="openshift-dns/dns-default-fhq7n"
Jul 31 22:57:47.583381 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:47.583341    2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 31 22:57:47.583435 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:47.583426    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/735a47e9-8a16-4dd3-93ec-2e3ac70100d4-metrics-tls podName:735a47e9-8a16-4dd3-93ec-2e3ac70100d4 nodeName:}" failed. No retries permitted until 2026-07-31 22:58:03.583404084 +0000 UTC m=+67.229190385 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/735a47e9-8a16-4dd3-93ec-2e3ac70100d4-metrics-tls") pod "dns-default-fhq7n" (UID: "735a47e9-8a16-4dd3-93ec-2e3ac70100d4") : secret "dns-default-metrics-tls" not found
Jul 31 22:57:48.527221 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:48.527186    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-vvnwv"]
Jul 31 22:57:48.545936 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:48.545909    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-vvnwv"]
Jul 31 22:57:48.546114 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:48.546069    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vvnwv"
Jul 31 22:57:48.548539 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:48.548514    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 31 22:57:48.548641 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:48.548581    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-kmjpj\""
Jul 31 22:57:48.548686 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:48.548670    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 31 22:57:48.591808 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:48.591783    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e460911c-90d3-4ebc-baaf-3047df044bc8-crio-socket\") pod \"insights-runtime-extractor-vvnwv\" (UID: \"e460911c-90d3-4ebc-baaf-3047df044bc8\") " pod="openshift-insights/insights-runtime-extractor-vvnwv"
Jul 31 22:57:48.591973 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:48.591827    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g76zg\" (UniqueName: \"kubernetes.io/projected/e460911c-90d3-4ebc-baaf-3047df044bc8-kube-api-access-g76zg\") pod \"insights-runtime-extractor-vvnwv\" (UID: \"e460911c-90d3-4ebc-baaf-3047df044bc8\") " pod="openshift-insights/insights-runtime-extractor-vvnwv"
Jul 31 22:57:48.591973 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:48.591864    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e460911c-90d3-4ebc-baaf-3047df044bc8-data-volume\") pod \"insights-runtime-extractor-vvnwv\" (UID: \"e460911c-90d3-4ebc-baaf-3047df044bc8\") " pod="openshift-insights/insights-runtime-extractor-vvnwv"
Jul 31 22:57:48.591973 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:48.591904    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e460911c-90d3-4ebc-baaf-3047df044bc8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vvnwv\" (UID: \"e460911c-90d3-4ebc-baaf-3047df044bc8\") " pod="openshift-insights/insights-runtime-extractor-vvnwv"
Jul 31 22:57:48.591973 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:48.591951    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e460911c-90d3-4ebc-baaf-3047df044bc8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vvnwv\" (UID: \"e460911c-90d3-4ebc-baaf-3047df044bc8\") " pod="openshift-insights/insights-runtime-extractor-vvnwv"
Jul 31 22:57:48.693191 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:48.693153    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e460911c-90d3-4ebc-baaf-3047df044bc8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vvnwv\" (UID: \"e460911c-90d3-4ebc-baaf-3047df044bc8\") " pod="openshift-insights/insights-runtime-extractor-vvnwv"
Jul 31 22:57:48.693344 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:48.693276    2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 31 22:57:48.693344 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:48.693329    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e460911c-90d3-4ebc-baaf-3047df044bc8-crio-socket\") pod \"insights-runtime-extractor-vvnwv\" (UID: \"e460911c-90d3-4ebc-baaf-3047df044bc8\") " pod="openshift-insights/insights-runtime-extractor-vvnwv"
Jul 31 22:57:48.693344 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:48.693339    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e460911c-90d3-4ebc-baaf-3047df044bc8-kube-rbac-proxy-cm podName:e460911c-90d3-4ebc-baaf-3047df044bc8 nodeName:}" failed. No retries permitted until 2026-07-31 22:57:49.193324733 +0000 UTC m=+52.839111020 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e460911c-90d3-4ebc-baaf-3047df044bc8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vvnwv" (UID: "e460911c-90d3-4ebc-baaf-3047df044bc8") : configmap "kube-rbac-proxy" not found
Jul 31 22:57:48.693444 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:48.693387    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g76zg\" (UniqueName: \"kubernetes.io/projected/e460911c-90d3-4ebc-baaf-3047df044bc8-kube-api-access-g76zg\") pod \"insights-runtime-extractor-vvnwv\" (UID: \"e460911c-90d3-4ebc-baaf-3047df044bc8\") " pod="openshift-insights/insights-runtime-extractor-vvnwv"
Jul 31 22:57:48.693444 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:48.693428    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e460911c-90d3-4ebc-baaf-3047df044bc8-data-volume\") pod \"insights-runtime-extractor-vvnwv\" (UID: \"e460911c-90d3-4ebc-baaf-3047df044bc8\") " pod="openshift-insights/insights-runtime-extractor-vvnwv"
Jul 31 22:57:48.693508 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:48.693459    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e460911c-90d3-4ebc-baaf-3047df044bc8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vvnwv\" (UID: \"e460911c-90d3-4ebc-baaf-3047df044bc8\") " pod="openshift-insights/insights-runtime-extractor-vvnwv"
Jul 31 22:57:48.693540 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:48.693530    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e460911c-90d3-4ebc-baaf-3047df044bc8-crio-socket\") pod \"insights-runtime-extractor-vvnwv\" (UID: \"e460911c-90d3-4ebc-baaf-3047df044bc8\") " pod="openshift-insights/insights-runtime-extractor-vvnwv"
Jul 31 22:57:48.693581 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:48.693567    2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 31 22:57:48.693626 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:48.693616    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e460911c-90d3-4ebc-baaf-3047df044bc8-insights-runtime-extractor-tls podName:e460911c-90d3-4ebc-baaf-3047df044bc8 nodeName:}" failed. No retries permitted until 2026-07-31 22:57:49.193602709 +0000 UTC m=+52.839388996 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e460911c-90d3-4ebc-baaf-3047df044bc8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vvnwv" (UID: "e460911c-90d3-4ebc-baaf-3047df044bc8") : secret "insights-runtime-extractor-tls" not found
Jul 31 22:57:48.693862 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:48.693847    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e460911c-90d3-4ebc-baaf-3047df044bc8-data-volume\") pod \"insights-runtime-extractor-vvnwv\" (UID: \"e460911c-90d3-4ebc-baaf-3047df044bc8\") " pod="openshift-insights/insights-runtime-extractor-vvnwv"
Jul 31 22:57:48.701883 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:48.701862    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g76zg\" (UniqueName: \"kubernetes.io/projected/e460911c-90d3-4ebc-baaf-3047df044bc8-kube-api-access-g76zg\") pod \"insights-runtime-extractor-vvnwv\" (UID: \"e460911c-90d3-4ebc-baaf-3047df044bc8\") " pod="openshift-insights/insights-runtime-extractor-vvnwv"
Jul 31 22:57:49.196822 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:49.196784    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e460911c-90d3-4ebc-baaf-3047df044bc8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vvnwv\" (UID: \"e460911c-90d3-4ebc-baaf-3047df044bc8\") " pod="openshift-insights/insights-runtime-extractor-vvnwv"
Jul 31 22:57:49.197000 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:49.196841    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e460911c-90d3-4ebc-baaf-3047df044bc8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vvnwv\" (UID: \"e460911c-90d3-4ebc-baaf-3047df044bc8\") " pod="openshift-insights/insights-runtime-extractor-vvnwv"
Jul 31 22:57:49.197000 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:49.196942    2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 31 22:57:49.197000 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:49.196978    2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 31 22:57:49.197112 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:49.197015    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e460911c-90d3-4ebc-baaf-3047df044bc8-insights-runtime-extractor-tls podName:e460911c-90d3-4ebc-baaf-3047df044bc8 nodeName:}" failed. No retries permitted until 2026-07-31 22:57:50.196995535 +0000 UTC m=+53.842781822 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e460911c-90d3-4ebc-baaf-3047df044bc8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vvnwv" (UID: "e460911c-90d3-4ebc-baaf-3047df044bc8") : secret "insights-runtime-extractor-tls" not found
Jul 31 22:57:49.197112 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:49.197033    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e460911c-90d3-4ebc-baaf-3047df044bc8-kube-rbac-proxy-cm podName:e460911c-90d3-4ebc-baaf-3047df044bc8 nodeName:}" failed. No retries permitted until 2026-07-31 22:57:50.197025455 +0000 UTC m=+53.842811741 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e460911c-90d3-4ebc-baaf-3047df044bc8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vvnwv" (UID: "e460911c-90d3-4ebc-baaf-3047df044bc8") : configmap "kube-rbac-proxy" not found
Jul 31 22:57:50.085135 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:50.085110    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-llmj8_148fa8ca-4d3b-4ae1-8be5-30ea87aa412e/dns-node-resolver/0.log"
Jul 31 22:57:50.206242 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:50.206208    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e460911c-90d3-4ebc-baaf-3047df044bc8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vvnwv\" (UID: \"e460911c-90d3-4ebc-baaf-3047df044bc8\") " pod="openshift-insights/insights-runtime-extractor-vvnwv"
Jul 31 22:57:50.206430 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:50.206249    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e460911c-90d3-4ebc-baaf-3047df044bc8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vvnwv\" (UID: \"e460911c-90d3-4ebc-baaf-3047df044bc8\") " pod="openshift-insights/insights-runtime-extractor-vvnwv"
Jul 31 22:57:50.206430 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:50.206341    2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 31 22:57:50.206430 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:50.206373    2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 31 22:57:50.206430 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:50.206392    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e460911c-90d3-4ebc-baaf-3047df044bc8-kube-rbac-proxy-cm podName:e460911c-90d3-4ebc-baaf-3047df044bc8 nodeName:}" failed. No retries permitted until 2026-07-31 22:57:52.206379359 +0000 UTC m=+55.852165646 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e460911c-90d3-4ebc-baaf-3047df044bc8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vvnwv" (UID: "e460911c-90d3-4ebc-baaf-3047df044bc8") : configmap "kube-rbac-proxy" not found
Jul 31 22:57:50.206587 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:50.206444    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e460911c-90d3-4ebc-baaf-3047df044bc8-insights-runtime-extractor-tls podName:e460911c-90d3-4ebc-baaf-3047df044bc8 nodeName:}" failed. No retries permitted until 2026-07-31 22:57:52.206426088 +0000 UTC m=+55.852212396 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e460911c-90d3-4ebc-baaf-3047df044bc8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vvnwv" (UID: "e460911c-90d3-4ebc-baaf-3047df044bc8") : secret "insights-runtime-extractor-tls" not found
Jul 31 22:57:50.408252 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:50.408164    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/40b91dc3-a7d9-403a-8208-ca1e019553c5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-f6wq6\" (UID: \"40b91dc3-a7d9-403a-8208-ca1e019553c5\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-f6wq6"
Jul 31 22:57:50.408480 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:50.408306    2584 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 31 22:57:50.408480 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:50.408371    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40b91dc3-a7d9-403a-8208-ca1e019553c5-cluster-monitoring-operator-tls podName:40b91dc3-a7d9-403a-8208-ca1e019553c5 nodeName:}" failed. No retries permitted until 2026-07-31 22:57:58.408355367 +0000 UTC m=+62.054141658 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/40b91dc3-a7d9-403a-8208-ca1e019553c5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-f6wq6" (UID: "40b91dc3-a7d9-403a-8208-ca1e019553c5") : secret "cluster-monitoring-operator-tls" not found
Jul 31 22:57:50.509239 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:50.509200    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/24908812-404e-4b8c-958d-335220924aff-service-ca-bundle\") pod \"router-default-868556846d-gx64z\" (UID: \"24908812-404e-4b8c-958d-335220924aff\") " pod="openshift-ingress/router-default-868556846d-gx64z"
Jul 31 22:57:50.509424 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:50.509263    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7946036f-064a-4c5f-b619-1f34c3a9e16b-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-xjzj5\" (UID: \"7946036f-064a-4c5f-b619-1f34c3a9e16b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-xjzj5"
Jul 31 22:57:50.509424 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:50.509295    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24908812-404e-4b8c-958d-335220924aff-metrics-certs\") pod \"router-default-868556846d-gx64z\" (UID: \"24908812-404e-4b8c-958d-335220924aff\") " pod="openshift-ingress/router-default-868556846d-gx64z"
Jul 31 22:57:50.509424 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:50.509382    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/24908812-404e-4b8c-958d-335220924aff-service-ca-bundle podName:24908812-404e-4b8c-958d-335220924aff nodeName:}" failed. No retries permitted until 2026-07-31 22:57:58.509360686 +0000 UTC m=+62.155146980 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/24908812-404e-4b8c-958d-335220924aff-service-ca-bundle") pod "router-default-868556846d-gx64z" (UID: "24908812-404e-4b8c-958d-335220924aff") : configmap references non-existent config key: service-ca.crt
Jul 31 22:57:50.509424 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:50.509405    2584 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 31 22:57:50.509424 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:50.509409    2584 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 31 22:57:50.509628 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:50.509463    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24908812-404e-4b8c-958d-335220924aff-metrics-certs podName:24908812-404e-4b8c-958d-335220924aff nodeName:}" failed. No retries permitted until 2026-07-31 22:57:58.509445871 +0000 UTC m=+62.155232176 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24908812-404e-4b8c-958d-335220924aff-metrics-certs") pod "router-default-868556846d-gx64z" (UID: "24908812-404e-4b8c-958d-335220924aff") : secret "router-metrics-certs-default" not found
Jul 31 22:57:50.509628 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:50.509483    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7946036f-064a-4c5f-b619-1f34c3a9e16b-samples-operator-tls podName:7946036f-064a-4c5f-b619-1f34c3a9e16b nodeName:}" failed. No retries permitted until 2026-07-31 22:57:58.509473732 +0000 UTC m=+62.155260051 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7946036f-064a-4c5f-b619-1f34c3a9e16b-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-xjzj5" (UID: "7946036f-064a-4c5f-b619-1f34c3a9e16b") : secret "samples-operator-tls" not found
Jul 31 22:57:50.711024 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:50.710939    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/798610c4-70cc-4455-943f-b0d6b0dca9d9-registry-tls\") pod \"image-registry-d8bd65946-zn48s\" (UID: \"798610c4-70cc-4455-943f-b0d6b0dca9d9\") " pod="openshift-image-registry/image-registry-d8bd65946-zn48s"
Jul 31 22:57:50.711165 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:50.711060    2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 31 22:57:50.711165 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:50.711071    2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d8bd65946-zn48s: secret "image-registry-tls" not found
Jul 31 22:57:50.711165 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:50.711119    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/798610c4-70cc-4455-943f-b0d6b0dca9d9-registry-tls podName:798610c4-70cc-4455-943f-b0d6b0dca9d9 nodeName:}" failed. No retries permitted until 2026-07-31 22:57:58.711106512 +0000 UTC m=+62.356892799 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/798610c4-70cc-4455-943f-b0d6b0dca9d9-registry-tls") pod "image-registry-d8bd65946-zn48s" (UID: "798610c4-70cc-4455-943f-b0d6b0dca9d9") : secret "image-registry-tls" not found
Jul 31 22:57:51.285547 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:51.285521    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-t8qkd_0bb4d7d9-ff22-41eb-8883-b7c1942767fb/node-ca/0.log"
Jul 31 22:57:51.885483 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:51.885459    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-lbf5k_a0e5b183-2a8a-48c8-b2fa-803bce230632/serve-healthcheck-canary/0.log"
Jul 31 22:57:52.223130 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:52.223035    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e460911c-90d3-4ebc-baaf-3047df044bc8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vvnwv\" (UID: \"e460911c-90d3-4ebc-baaf-3047df044bc8\") " pod="openshift-insights/insights-runtime-extractor-vvnwv"
Jul 31 22:57:52.223130 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:52.223089    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e460911c-90d3-4ebc-baaf-3047df044bc8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vvnwv\" (UID: \"e460911c-90d3-4ebc-baaf-3047df044bc8\") " pod="openshift-insights/insights-runtime-extractor-vvnwv"
Jul 31 22:57:52.223314 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:52.223218    2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 31 22:57:52.223314 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:52.223232    2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 31 22:57:52.223314 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:52.223285    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e460911c-90d3-4ebc-baaf-3047df044bc8-insights-runtime-extractor-tls podName:e460911c-90d3-4ebc-baaf-3047df044bc8 nodeName:}" failed. No retries permitted until 2026-07-31 22:57:56.223268455 +0000 UTC m=+59.869054748 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e460911c-90d3-4ebc-baaf-3047df044bc8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vvnwv" (UID: "e460911c-90d3-4ebc-baaf-3047df044bc8") : secret "insights-runtime-extractor-tls" not found
Jul 31 22:57:52.223314 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:52.223301    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e460911c-90d3-4ebc-baaf-3047df044bc8-kube-rbac-proxy-cm podName:e460911c-90d3-4ebc-baaf-3047df044bc8 nodeName:}" failed. No retries permitted until 2026-07-31 22:57:56.223294154 +0000 UTC m=+59.869080440 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e460911c-90d3-4ebc-baaf-3047df044bc8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vvnwv" (UID: "e460911c-90d3-4ebc-baaf-3047df044bc8") : configmap "kube-rbac-proxy" not found
Jul 31 22:57:56.258174 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:56.258134    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e460911c-90d3-4ebc-baaf-3047df044bc8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vvnwv\" (UID: \"e460911c-90d3-4ebc-baaf-3047df044bc8\") " pod="openshift-insights/insights-runtime-extractor-vvnwv"
Jul 31 22:57:56.258595 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:56.258186    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e460911c-90d3-4ebc-baaf-3047df044bc8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vvnwv\" (UID: \"e460911c-90d3-4ebc-baaf-3047df044bc8\") " pod="openshift-insights/insights-runtime-extractor-vvnwv"
Jul 31 22:57:56.258595 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:56.258273    2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 31 22:57:56.258595 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:56.258282    2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 31 22:57:56.258595 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:56.258330    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e460911c-90d3-4ebc-baaf-3047df044bc8-kube-rbac-proxy-cm podName:e460911c-90d3-4ebc-baaf-3047df044bc8 nodeName:}" failed. No retries permitted until 2026-07-31 22:58:04.258314208 +0000 UTC m=+67.904100501 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e460911c-90d3-4ebc-baaf-3047df044bc8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vvnwv" (UID: "e460911c-90d3-4ebc-baaf-3047df044bc8") : configmap "kube-rbac-proxy" not found
Jul 31 22:57:56.258595 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:56.258344    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e460911c-90d3-4ebc-baaf-3047df044bc8-insights-runtime-extractor-tls podName:e460911c-90d3-4ebc-baaf-3047df044bc8 nodeName:}" failed. No retries permitted until 2026-07-31 22:58:04.258338712 +0000 UTC m=+67.904124998 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e460911c-90d3-4ebc-baaf-3047df044bc8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vvnwv" (UID: "e460911c-90d3-4ebc-baaf-3047df044bc8") : secret "insights-runtime-extractor-tls" not found
Jul 31 22:57:58.217677 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:58.217650    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-w96jg"
Jul 31 22:57:58.476986 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:58.476899    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/40b91dc3-a7d9-403a-8208-ca1e019553c5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-f6wq6\" (UID: \"40b91dc3-a7d9-403a-8208-ca1e019553c5\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-f6wq6"
Jul 31 22:57:58.477135 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:58.477062    2584 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 31 22:57:58.477180 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:58.477136    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40b91dc3-a7d9-403a-8208-ca1e019553c5-cluster-monitoring-operator-tls podName:40b91dc3-a7d9-403a-8208-ca1e019553c5 nodeName:}" failed. No retries permitted until 2026-07-31 22:58:14.477119756 +0000 UTC m=+78.122906067 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/40b91dc3-a7d9-403a-8208-ca1e019553c5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-f6wq6" (UID: "40b91dc3-a7d9-403a-8208-ca1e019553c5") : secret "cluster-monitoring-operator-tls" not found
Jul 31 22:57:58.578358 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:58.578321    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/24908812-404e-4b8c-958d-335220924aff-service-ca-bundle\") pod \"router-default-868556846d-gx64z\" (UID: \"24908812-404e-4b8c-958d-335220924aff\") " pod="openshift-ingress/router-default-868556846d-gx64z"
Jul 31 22:57:58.578558 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:58.578372    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7946036f-064a-4c5f-b619-1f34c3a9e16b-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-xjzj5\" (UID: \"7946036f-064a-4c5f-b619-1f34c3a9e16b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-xjzj5"
Jul 31 22:57:58.578558 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:58.578397    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24908812-404e-4b8c-958d-335220924aff-metrics-certs\") pod \"router-default-868556846d-gx64z\" (UID: \"24908812-404e-4b8c-958d-335220924aff\") " pod="openshift-ingress/router-default-868556846d-gx64z"
Jul 31 22:57:58.578558 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:58.578529    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/24908812-404e-4b8c-958d-335220924aff-service-ca-bundle podName:24908812-404e-4b8c-958d-335220924aff nodeName:}" failed. No retries permitted until 2026-07-31 22:58:14.578504812 +0000 UTC m=+78.224291098 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/24908812-404e-4b8c-958d-335220924aff-service-ca-bundle") pod "router-default-868556846d-gx64z" (UID: "24908812-404e-4b8c-958d-335220924aff") : configmap references non-existent config key: service-ca.crt
Jul 31 22:57:58.578558 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:58.578531    2584 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 31 22:57:58.578734 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:57:58.578577    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24908812-404e-4b8c-958d-335220924aff-metrics-certs podName:24908812-404e-4b8c-958d-335220924aff nodeName:}" failed. No retries permitted until 2026-07-31 22:58:14.578566614 +0000 UTC m=+78.224352921 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24908812-404e-4b8c-958d-335220924aff-metrics-certs") pod "router-default-868556846d-gx64z" (UID: "24908812-404e-4b8c-958d-335220924aff") : secret "router-metrics-certs-default" not found
Jul 31 22:57:58.580844 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:58.580811    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7946036f-064a-4c5f-b619-1f34c3a9e16b-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-xjzj5\" (UID: \"7946036f-064a-4c5f-b619-1f34c3a9e16b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-xjzj5"
Jul 31 22:57:58.707464 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:58.707433    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-h4qmj\""
Jul 31 22:57:58.714765 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:58.714743    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-xjzj5"
Jul 31 22:57:58.780611 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:58.780586    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/798610c4-70cc-4455-943f-b0d6b0dca9d9-registry-tls\") pod \"image-registry-d8bd65946-zn48s\" (UID: \"798610c4-70cc-4455-943f-b0d6b0dca9d9\") " pod="openshift-image-registry/image-registry-d8bd65946-zn48s"
Jul 31 22:57:58.782906 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:58.782882    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/798610c4-70cc-4455-943f-b0d6b0dca9d9-registry-tls\") pod \"image-registry-d8bd65946-zn48s\" (UID: \"798610c4-70cc-4455-943f-b0d6b0dca9d9\") " pod="openshift-image-registry/image-registry-d8bd65946-zn48s"
Jul 31 22:57:58.844334 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:58.844302    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-xjzj5"]
Jul 31 22:57:58.908265 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:58.908242    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-sn4bg\""
Jul 31 22:57:58.916478 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:58.916458    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d8bd65946-zn48s"
Jul 31 22:57:59.045984 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:59.045958    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-d8bd65946-zn48s"]
Jul 31 22:57:59.048391 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:57:59.048363    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod798610c4_70cc_4455_943f_b0d6b0dca9d9.slice/crio-c856bd1b56d02ff9f3be3bf3e467f0c656a4156c35421f9d7ce200b0659c7bf2 WatchSource:0}: Error finding container c856bd1b56d02ff9f3be3bf3e467f0c656a4156c35421f9d7ce200b0659c7bf2: Status 404 returned error can't find the container with id c856bd1b56d02ff9f3be3bf3e467f0c656a4156c35421f9d7ce200b0659c7bf2
Jul 31 22:57:59.306534 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:59.306506    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-d8bd65946-zn48s" event={"ID":"798610c4-70cc-4455-943f-b0d6b0dca9d9","Type":"ContainerStarted","Data":"531777a1bedd3c6a6266d9f4c788fddad6e70e73fa43a4662398dc8dde858b0d"}
Jul 31 22:57:59.306983 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:59.306566    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-d8bd65946-zn48s" event={"ID":"798610c4-70cc-4455-943f-b0d6b0dca9d9","Type":"ContainerStarted","Data":"c856bd1b56d02ff9f3be3bf3e467f0c656a4156c35421f9d7ce200b0659c7bf2"}
Jul 31 22:57:59.306983 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:59.306595    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-d8bd65946-zn48s"
Jul 31 22:57:59.307653 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:59.307631    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-xjzj5" event={"ID":"7946036f-064a-4c5f-b619-1f34c3a9e16b","Type":"ContainerStarted","Data":"b19819ba30c8dcb453eea44042150d4d403566699c9b1eb5ef1d4d9a29a56978"}
Jul 31 22:57:59.329185 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:57:59.329130    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-d8bd65946-zn48s" podStartSLOduration=17.32910996 podStartE2EDuration="17.32910996s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:57:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:57:59.328765787 +0000 UTC m=+62.974552095" watchObservedRunningTime="2026-07-31 22:57:59.32910996 +0000 UTC m=+62.974896267"
Jul 31 22:58:01.314652 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:01.314616    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-xjzj5" event={"ID":"7946036f-064a-4c5f-b619-1f34c3a9e16b","Type":"ContainerStarted","Data":"0a7739df7e3644d0c1fed952429f48d72651f8d6c74fdce8e4f73ca18b8917f3"}
Jul 31 22:58:01.315058 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:01.314670    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-xjzj5" event={"ID":"7946036f-064a-4c5f-b619-1f34c3a9e16b","Type":"ContainerStarted","Data":"f4973fc846e4999db25b3b9e26a02fea68301a6b4d363dc112a7d7b3b5094622"}
Jul 31 22:58:01.332213 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:01.332166    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-xjzj5" podStartSLOduration=17.524063873 podStartE2EDuration="19.332151874s" totalImagesPullingTime="1.808088001s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:57:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:58:01.331438142 +0000 UTC m=+64.977224452" watchObservedRunningTime="2026-07-31 22:58:01.332151874 +0000 UTC m=+64.977938182"
Jul 31 22:58:01.608618 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:01.608536    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945-metrics-certs\") pod \"network-metrics-daemon-mz5wz\" (UID: \"b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945\") " pod="openshift-multus/network-metrics-daemon-mz5wz"
Jul 31 22:58:01.611310 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:01.611292    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 31 22:58:01.619649 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:58:01.619629    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 31 22:58:01.619719 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:58:01.619687    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945-metrics-certs podName:b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945 nodeName:}" failed. No retries permitted until 2026-07-31 22:59:05.619668459 +0000 UTC m=+129.265454750 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945-metrics-certs") pod "network-metrics-daemon-mz5wz" (UID: "b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945") : secret "metrics-daemon-secret" not found
Jul 31 22:58:01.709712 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:01.709681    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b85a1443-4bc2-48a0-8d29-35f46cbc3d57-original-pull-secret\") pod \"global-pull-secret-syncer-mqwlt\" (UID: \"b85a1443-4bc2-48a0-8d29-35f46cbc3d57\") " pod="kube-system/global-pull-secret-syncer-mqwlt"
Jul 31 22:58:01.712469 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:01.712452    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 31 22:58:01.722638 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:01.722612    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b85a1443-4bc2-48a0-8d29-35f46cbc3d57-original-pull-secret\") pod \"global-pull-secret-syncer-mqwlt\" (UID: \"b85a1443-4bc2-48a0-8d29-35f46cbc3d57\") " pod="kube-system/global-pull-secret-syncer-mqwlt"
Jul 31 22:58:01.810013 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:01.809993    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwbjs\" (UniqueName: \"kubernetes.io/projected/b80644e6-82bb-4c59-86ed-b60bd24e6a05-kube-api-access-bwbjs\") pod \"network-check-target-84css\" (UID: \"b80644e6-82bb-4c59-86ed-b60bd24e6a05\") " pod="openshift-network-diagnostics/network-check-target-84css"
Jul 31 22:58:01.812559 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:01.812544    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 31 22:58:01.822951 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:01.822933    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 31 22:58:01.833106 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:01.833086    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwbjs\" (UniqueName: \"kubernetes.io/projected/b80644e6-82bb-4c59-86ed-b60bd24e6a05-kube-api-access-bwbjs\") pod \"network-check-target-84css\" (UID: \"b80644e6-82bb-4c59-86ed-b60bd24e6a05\") " pod="openshift-network-diagnostics/network-check-target-84css"
Jul 31 22:58:01.897015 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:01.896958    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mqwlt"
Jul 31 22:58:02.006493 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:02.006470    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jtx5g\""
Jul 31 22:58:02.014651 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:02.014624    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-84css"
Jul 31 22:58:02.015965 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:02.015942    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-mqwlt"]
Jul 31 22:58:02.018947 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:58:02.018929    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb85a1443_4bc2_48a0_8d29_35f46cbc3d57.slice/crio-e7f77e345d8244fb5ed3088b5434a9638f8b70d9411823ba6a7c52813810dea6 WatchSource:0}: Error finding container e7f77e345d8244fb5ed3088b5434a9638f8b70d9411823ba6a7c52813810dea6: Status 404 returned error can't find the container with id e7f77e345d8244fb5ed3088b5434a9638f8b70d9411823ba6a7c52813810dea6
Jul 31 22:58:02.138929 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:02.138813    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-84css"]
Jul 31 22:58:02.141247 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:58:02.141216    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb80644e6_82bb_4c59_86ed_b60bd24e6a05.slice/crio-d388656eb679bfe9cce6a3154abe0c7e9766bd8b8051c862d9732e0e4da92e08 WatchSource:0}: Error finding container d388656eb679bfe9cce6a3154abe0c7e9766bd8b8051c862d9732e0e4da92e08: Status 404 returned error can't find the container with id d388656eb679bfe9cce6a3154abe0c7e9766bd8b8051c862d9732e0e4da92e08
Jul 31 22:58:02.319130 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:02.319096    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-84css" event={"ID":"b80644e6-82bb-4c59-86ed-b60bd24e6a05","Type":"ContainerStarted","Data":"d388656eb679bfe9cce6a3154abe0c7e9766bd8b8051c862d9732e0e4da92e08"}
Jul 31 22:58:02.320188 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:02.320165    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-mqwlt" event={"ID":"b85a1443-4bc2-48a0-8d29-35f46cbc3d57","Type":"ContainerStarted","Data":"e7f77e345d8244fb5ed3088b5434a9638f8b70d9411823ba6a7c52813810dea6"}
Jul 31 22:58:03.625948 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:03.625890    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/735a47e9-8a16-4dd3-93ec-2e3ac70100d4-metrics-tls\") pod \"dns-default-fhq7n\" (UID: \"735a47e9-8a16-4dd3-93ec-2e3ac70100d4\") " pod="openshift-dns/dns-default-fhq7n"
Jul 31 22:58:03.629069 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:03.629036    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/735a47e9-8a16-4dd3-93ec-2e3ac70100d4-metrics-tls\") pod \"dns-default-fhq7n\" (UID: \"735a47e9-8a16-4dd3-93ec-2e3ac70100d4\") " pod="openshift-dns/dns-default-fhq7n"
Jul 31 22:58:03.854984 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:03.854953    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-7jxx7\""
Jul 31 22:58:03.863472 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:03.863253    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-fhq7n"
Jul 31 22:58:04.027513 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:04.027322    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-fhq7n"]
Jul 31 22:58:04.030807 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:58:04.030758    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod735a47e9_8a16_4dd3_93ec_2e3ac70100d4.slice/crio-749add611255ad025c0fc30500aaab3dd5385826dd3fda5b4438b44b64911a2b WatchSource:0}: Error finding container 749add611255ad025c0fc30500aaab3dd5385826dd3fda5b4438b44b64911a2b: Status 404 returned error can't find the container with id 749add611255ad025c0fc30500aaab3dd5385826dd3fda5b4438b44b64911a2b
Jul 31 22:58:04.330444 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:04.330401    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fhq7n" event={"ID":"735a47e9-8a16-4dd3-93ec-2e3ac70100d4","Type":"ContainerStarted","Data":"749add611255ad025c0fc30500aaab3dd5385826dd3fda5b4438b44b64911a2b"}
Jul 31 22:58:04.333539 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:04.332838    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e460911c-90d3-4ebc-baaf-3047df044bc8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vvnwv\" (UID: \"e460911c-90d3-4ebc-baaf-3047df044bc8\") " pod="openshift-insights/insights-runtime-extractor-vvnwv"
Jul 31 22:58:04.333539 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:04.332895    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e460911c-90d3-4ebc-baaf-3047df044bc8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vvnwv\" (UID: \"e460911c-90d3-4ebc-baaf-3047df044bc8\") " pod="openshift-insights/insights-runtime-extractor-vvnwv"
Jul 31 22:58:04.333539 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:58:04.332948    2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 31 22:58:04.333539 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:58:04.333010    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e460911c-90d3-4ebc-baaf-3047df044bc8-insights-runtime-extractor-tls podName:e460911c-90d3-4ebc-baaf-3047df044bc8 nodeName:}" failed. No retries permitted until 2026-07-31 22:58:20.332991257 +0000 UTC m=+83.978777560 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e460911c-90d3-4ebc-baaf-3047df044bc8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vvnwv" (UID: "e460911c-90d3-4ebc-baaf-3047df044bc8") : secret "insights-runtime-extractor-tls" not found
Jul 31 22:58:04.333539 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:04.333493    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e460911c-90d3-4ebc-baaf-3047df044bc8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vvnwv\" (UID: \"e460911c-90d3-4ebc-baaf-3047df044bc8\") " pod="openshift-insights/insights-runtime-extractor-vvnwv"
Jul 31 22:58:07.348005 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:07.347914    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-84css" event={"ID":"b80644e6-82bb-4c59-86ed-b60bd24e6a05","Type":"ContainerStarted","Data":"0476d8e4cb76ebee7c4e82b7cea8593d381078559710c5f689c83fafb25b4dee"}
Jul 31 22:58:07.348405 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:07.348032    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-84css"
Jul 31 22:58:07.349540 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:07.349513    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-mqwlt" event={"ID":"b85a1443-4bc2-48a0-8d29-35f46cbc3d57","Type":"ContainerStarted","Data":"d057c54d97126a1b3e7ca6b214653472168abe27394ee45a919a7c1d8df9c6a7"}
Jul 31 22:58:07.351273 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:07.351249    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fhq7n" event={"ID":"735a47e9-8a16-4dd3-93ec-2e3ac70100d4","Type":"ContainerStarted","Data":"d6965e1a3e63d01d4dcc97692d544995e94c6b0e8740556b968f5753c7594f69"}
Jul 31 22:58:07.351368 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:07.351294    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fhq7n" event={"ID":"735a47e9-8a16-4dd3-93ec-2e3ac70100d4","Type":"ContainerStarted","Data":"84b12b4bdd5a7073aa92fa3837fcaafbc653cb725132942927ffce8829f25e6c"}
Jul 31 22:58:07.351368 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:07.351331    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-fhq7n"
Jul 31 22:58:07.369645 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:07.369595    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-84css" podStartSLOduration=65.420569595 podStartE2EDuration="1m10.369580731s" totalImagesPullingTime="4.949011136s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:56:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:58:07.368188387 +0000 UTC m=+71.013974693" watchObservedRunningTime="2026-07-31 22:58:07.369580731 +0000 UTC m=+71.015367039"
Jul 31 22:58:07.396406 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:07.396347    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-mqwlt" podStartSLOduration=65.309265605 podStartE2EDuration="1m10.396330254s" totalImagesPullingTime="5.087064649s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 22:56:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:58:07.395857476 +0000 UTC m=+71.041643785" watchObservedRunningTime="2026-07-31 22:58:07.396330254 +0000 UTC m=+71.042116563"
Jul 31 22:58:07.420678 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:07.420632    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-fhq7n" podStartSLOduration=33.339269903 podStartE2EDuration="36.420616989s" totalImagesPullingTime="3.081347086s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:57:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:58:07.420516342 +0000 UTC m=+71.066302649" watchObservedRunningTime="2026-07-31 22:58:07.420616989 +0000 UTC m=+71.066403296"
Jul 31 22:58:13.525058 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.525026    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-f77f48c6d-hxkk5"]
Jul 31 22:58:13.527491 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.527472    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68fcbd4bcc-n75cn"]
Jul 31 22:58:13.527620 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.527602    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-f77f48c6d-hxkk5"
Jul 31 22:58:13.529475 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.529457    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68fcbd4bcc-n75cn"
Jul 31 22:58:13.532967 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.531604    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 31 22:58:13.532967 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.531962    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 31 22:58:13.532967 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.532340    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 31 22:58:13.532967 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.532664    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 31 22:58:13.532967 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.532810    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 31 22:58:13.533795 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.533768    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-h4bdc\""
Jul 31 22:58:13.535675 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.535636    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 31 22:58:13.536032 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.536006    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 31 22:58:13.536123 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.536109    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 31 22:58:13.536370 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.536350    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 31 22:58:13.550119 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.550083    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-d8bd65946-zn48s"]
Jul 31 22:58:13.551249 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.551223    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-f77f48c6d-hxkk5"]
Jul 31 22:58:13.555661 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.555639    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68fcbd4bcc-n75cn"]
Jul 31 22:58:13.609748 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.609695    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a1e283c7-af2a-46d7-9712-605cec1c3211-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-hxkk5\" (UID: \"a1e283c7-af2a-46d7-9712-605cec1c3211\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-hxkk5"
Jul 31 22:58:13.610046 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.610016    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/15c7a320-b9a3-4872-b08a-859f5d702070-additional-service-ca\") pod \"cluster-proxy-proxy-agent-68fcbd4bcc-n75cn\" (UID: \"15c7a320-b9a3-4872-b08a-859f5d702070\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68fcbd4bcc-n75cn"
Jul 31 22:58:13.610149 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.610081    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a1e283c7-af2a-46d7-9712-605cec1c3211-nginx-conf\") pod \"networking-console-plugin-f77f48c6d-hxkk5\" (UID: \"a1e283c7-af2a-46d7-9712-605cec1c3211\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-hxkk5"
Jul 31 22:58:13.610149 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.610126    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/15c7a320-b9a3-4872-b08a-859f5d702070-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-68fcbd4bcc-n75cn\" (UID: \"15c7a320-b9a3-4872-b08a-859f5d702070\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68fcbd4bcc-n75cn"
Jul 31 22:58:13.610226 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.610152    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/15c7a320-b9a3-4872-b08a-859f5d702070-ca\") pod \"cluster-proxy-proxy-agent-68fcbd4bcc-n75cn\" (UID: \"15c7a320-b9a3-4872-b08a-859f5d702070\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68fcbd4bcc-n75cn"
Jul 31 22:58:13.610226 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.610207    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xl62m\" (UniqueName: \"kubernetes.io/projected/15c7a320-b9a3-4872-b08a-859f5d702070-kube-api-access-xl62m\") pod \"cluster-proxy-proxy-agent-68fcbd4bcc-n75cn\" (UID: \"15c7a320-b9a3-4872-b08a-859f5d702070\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68fcbd4bcc-n75cn"
Jul 31 22:58:13.610303 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.610254    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/15c7a320-b9a3-4872-b08a-859f5d702070-hub\") pod \"cluster-proxy-proxy-agent-68fcbd4bcc-n75cn\" (UID: \"15c7a320-b9a3-4872-b08a-859f5d702070\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68fcbd4bcc-n75cn"
Jul 31 22:58:13.610337 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.610321    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/15c7a320-b9a3-4872-b08a-859f5d702070-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-68fcbd4bcc-n75cn\" (UID: \"15c7a320-b9a3-4872-b08a-859f5d702070\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68fcbd4bcc-n75cn"
Jul 31 22:58:13.621912 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.621887    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-74d85446d-tlzb9"]
Jul 31 22:58:13.624277 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.624258    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74d85446d-tlzb9"
Jul 31 22:58:13.656412 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.656390    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-74d85446d-tlzb9"]
Jul 31 22:58:13.711626 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.711599    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/15c7a320-b9a3-4872-b08a-859f5d702070-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-68fcbd4bcc-n75cn\" (UID: \"15c7a320-b9a3-4872-b08a-859f5d702070\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68fcbd4bcc-n75cn"
Jul 31 22:58:13.711626 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.711632    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/15c7a320-b9a3-4872-b08a-859f5d702070-ca\") pod \"cluster-proxy-proxy-agent-68fcbd4bcc-n75cn\" (UID: \"15c7a320-b9a3-4872-b08a-859f5d702070\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68fcbd4bcc-n75cn"
Jul 31 22:58:13.711853 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.711661    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2lg7\" (UniqueName: \"kubernetes.io/projected/61cabb50-cf7e-4874-92cb-00e7363aa33b-kube-api-access-b2lg7\") pod \"image-registry-74d85446d-tlzb9\" (UID: \"61cabb50-cf7e-4874-92cb-00e7363aa33b\") " pod="openshift-image-registry/image-registry-74d85446d-tlzb9"
Jul 31 22:58:13.711904 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.711863    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xl62m\" (UniqueName: \"kubernetes.io/projected/15c7a320-b9a3-4872-b08a-859f5d702070-kube-api-access-xl62m\") pod \"cluster-proxy-proxy-agent-68fcbd4bcc-n75cn\" (UID: \"15c7a320-b9a3-4872-b08a-859f5d702070\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68fcbd4bcc-n75cn"
Jul 31 22:58:13.711904 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.711894    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/61cabb50-cf7e-4874-92cb-00e7363aa33b-trusted-ca\") pod \"image-registry-74d85446d-tlzb9\" (UID: \"61cabb50-cf7e-4874-92cb-00e7363aa33b\") " pod="openshift-image-registry/image-registry-74d85446d-tlzb9"
Jul 31 22:58:13.712000 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.711916    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/15c7a320-b9a3-4872-b08a-859f5d702070-hub\") pod \"cluster-proxy-proxy-agent-68fcbd4bcc-n75cn\" (UID: \"15c7a320-b9a3-4872-b08a-859f5d702070\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68fcbd4bcc-n75cn"
Jul 31 22:58:13.712000 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.711937    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/61cabb50-cf7e-4874-92cb-00e7363aa33b-registry-tls\") pod \"image-registry-74d85446d-tlzb9\" (UID: \"61cabb50-cf7e-4874-92cb-00e7363aa33b\") " pod="openshift-image-registry/image-registry-74d85446d-tlzb9"
Jul 31 22:58:13.712000 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.711962    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/61cabb50-cf7e-4874-92cb-00e7363aa33b-registry-certificates\") pod \"image-registry-74d85446d-tlzb9\" (UID: \"61cabb50-cf7e-4874-92cb-00e7363aa33b\") " pod="openshift-image-registry/image-registry-74d85446d-tlzb9"
Jul 31 22:58:13.712137 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.712109    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/15c7a320-b9a3-4872-b08a-859f5d702070-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-68fcbd4bcc-n75cn\" (UID: \"15c7a320-b9a3-4872-b08a-859f5d702070\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68fcbd4bcc-n75cn"
Jul 31 22:58:13.712199 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.712161    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/61cabb50-cf7e-4874-92cb-00e7363aa33b-ca-trust-extracted\") pod \"image-registry-74d85446d-tlzb9\" (UID: \"61cabb50-cf7e-4874-92cb-00e7363aa33b\") " pod="openshift-image-registry/image-registry-74d85446d-tlzb9"
Jul 31 22:58:13.712256 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.712212    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a1e283c7-af2a-46d7-9712-605cec1c3211-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-hxkk5\" (UID: \"a1e283c7-af2a-46d7-9712-605cec1c3211\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-hxkk5"
Jul 31 22:58:13.712306 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.712258    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/61cabb50-cf7e-4874-92cb-00e7363aa33b-installation-pull-secrets\") pod \"image-registry-74d85446d-tlzb9\" (UID: \"61cabb50-cf7e-4874-92cb-00e7363aa33b\") " pod="openshift-image-registry/image-registry-74d85446d-tlzb9"
Jul 31 22:58:13.712363 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.712347    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/15c7a320-b9a3-4872-b08a-859f5d702070-additional-service-ca\") pod \"cluster-proxy-proxy-agent-68fcbd4bcc-n75cn\" (UID: \"15c7a320-b9a3-4872-b08a-859f5d702070\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68fcbd4bcc-n75cn"
Jul 31 22:58:13.712428 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.712406    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a1e283c7-af2a-46d7-9712-605cec1c3211-nginx-conf\") pod \"networking-console-plugin-f77f48c6d-hxkk5\" (UID: \"a1e283c7-af2a-46d7-9712-605cec1c3211\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-hxkk5"
Jul 31 22:58:13.712484 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.712441    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/61cabb50-cf7e-4874-92cb-00e7363aa33b-image-registry-private-configuration\") pod \"image-registry-74d85446d-tlzb9\" (UID: \"61cabb50-cf7e-4874-92cb-00e7363aa33b\") " pod="openshift-image-registry/image-registry-74d85446d-tlzb9"
Jul 31 22:58:13.712539 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.712477    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/61cabb50-cf7e-4874-92cb-00e7363aa33b-bound-sa-token\") pod \"image-registry-74d85446d-tlzb9\" (UID: \"61cabb50-cf7e-4874-92cb-00e7363aa33b\") " pod="openshift-image-registry/image-registry-74d85446d-tlzb9"
Jul 31 22:58:13.713174 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.713148    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/15c7a320-b9a3-4872-b08a-859f5d702070-additional-service-ca\") pod \"cluster-proxy-proxy-agent-68fcbd4bcc-n75cn\" (UID: \"15c7a320-b9a3-4872-b08a-859f5d702070\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68fcbd4bcc-n75cn"
Jul 31 22:58:13.713294 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.713249    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a1e283c7-af2a-46d7-9712-605cec1c3211-nginx-conf\") pod \"networking-console-plugin-f77f48c6d-hxkk5\" (UID: \"a1e283c7-af2a-46d7-9712-605cec1c3211\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-hxkk5"
Jul 31 22:58:13.714608 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.714589    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/15c7a320-b9a3-4872-b08a-859f5d702070-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-68fcbd4bcc-n75cn\" (UID: \"15c7a320-b9a3-4872-b08a-859f5d702070\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68fcbd4bcc-n75cn"
Jul 31 22:58:13.714735 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.714674    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a1e283c7-af2a-46d7-9712-605cec1c3211-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-hxkk5\" (UID: \"a1e283c7-af2a-46d7-9712-605cec1c3211\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-hxkk5"
Jul 31 22:58:13.714795 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.714777    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/15c7a320-b9a3-4872-b08a-859f5d702070-ca\") pod \"cluster-proxy-proxy-agent-68fcbd4bcc-n75cn\" (UID: \"15c7a320-b9a3-4872-b08a-859f5d702070\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68fcbd4bcc-n75cn"
Jul 31 22:58:13.714838 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.714791    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/15c7a320-b9a3-4872-b08a-859f5d702070-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-68fcbd4bcc-n75cn\" (UID: \"15c7a320-b9a3-4872-b08a-859f5d702070\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68fcbd4bcc-n75cn"
Jul 31 22:58:13.714887 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.714871    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/15c7a320-b9a3-4872-b08a-859f5d702070-hub\") pod \"cluster-proxy-proxy-agent-68fcbd4bcc-n75cn\" (UID: \"15c7a320-b9a3-4872-b08a-859f5d702070\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68fcbd4bcc-n75cn"
Jul 31 22:58:13.723216 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.723192    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xl62m\" (UniqueName: \"kubernetes.io/projected/15c7a320-b9a3-4872-b08a-859f5d702070-kube-api-access-xl62m\") pod \"cluster-proxy-proxy-agent-68fcbd4bcc-n75cn\" (UID: \"15c7a320-b9a3-4872-b08a-859f5d702070\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68fcbd4bcc-n75cn"
Jul 31 22:58:13.813465 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.813435    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/61cabb50-cf7e-4874-92cb-00e7363aa33b-registry-tls\") pod \"image-registry-74d85446d-tlzb9\" (UID: \"61cabb50-cf7e-4874-92cb-00e7363aa33b\") " pod="openshift-image-registry/image-registry-74d85446d-tlzb9"
Jul 31 22:58:13.813629 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.813473    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/61cabb50-cf7e-4874-92cb-00e7363aa33b-registry-certificates\") pod \"image-registry-74d85446d-tlzb9\" (UID: \"61cabb50-cf7e-4874-92cb-00e7363aa33b\") " pod="openshift-image-registry/image-registry-74d85446d-tlzb9"
Jul 31 22:58:13.813629 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.813513    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/61cabb50-cf7e-4874-92cb-00e7363aa33b-ca-trust-extracted\") pod \"image-registry-74d85446d-tlzb9\" (UID: \"61cabb50-cf7e-4874-92cb-00e7363aa33b\") " pod="openshift-image-registry/image-registry-74d85446d-tlzb9"
Jul 31 22:58:13.813629 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.813547    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/61cabb50-cf7e-4874-92cb-00e7363aa33b-installation-pull-secrets\") pod \"image-registry-74d85446d-tlzb9\" (UID: \"61cabb50-cf7e-4874-92cb-00e7363aa33b\") " pod="openshift-image-registry/image-registry-74d85446d-tlzb9"
Jul 31 22:58:13.813629 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.813598    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/61cabb50-cf7e-4874-92cb-00e7363aa33b-image-registry-private-configuration\") pod \"image-registry-74d85446d-tlzb9\" (UID: \"61cabb50-cf7e-4874-92cb-00e7363aa33b\") " pod="openshift-image-registry/image-registry-74d85446d-tlzb9"
Jul 31 22:58:13.813629 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.813626    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/61cabb50-cf7e-4874-92cb-00e7363aa33b-bound-sa-token\") pod \"image-registry-74d85446d-tlzb9\" (UID: \"61cabb50-cf7e-4874-92cb-00e7363aa33b\") " pod="openshift-image-registry/image-registry-74d85446d-tlzb9"
Jul 31 22:58:13.813906 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.813662    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2lg7\" (UniqueName: \"kubernetes.io/projected/61cabb50-cf7e-4874-92cb-00e7363aa33b-kube-api-access-b2lg7\") pod \"image-registry-74d85446d-tlzb9\" (UID: \"61cabb50-cf7e-4874-92cb-00e7363aa33b\") " pod="openshift-image-registry/image-registry-74d85446d-tlzb9"
Jul 31 22:58:13.813906 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.813728    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/61cabb50-cf7e-4874-92cb-00e7363aa33b-trusted-ca\") pod \"image-registry-74d85446d-tlzb9\" (UID: \"61cabb50-cf7e-4874-92cb-00e7363aa33b\") " pod="openshift-image-registry/image-registry-74d85446d-tlzb9"
Jul 31 22:58:13.814191 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.814151    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/61cabb50-cf7e-4874-92cb-00e7363aa33b-ca-trust-extracted\") pod \"image-registry-74d85446d-tlzb9\" (UID: \"61cabb50-cf7e-4874-92cb-00e7363aa33b\") " pod="openshift-image-registry/image-registry-74d85446d-tlzb9"
Jul 31 22:58:13.814567 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.814540    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/61cabb50-cf7e-4874-92cb-00e7363aa33b-registry-certificates\") pod \"image-registry-74d85446d-tlzb9\" (UID: \"61cabb50-cf7e-4874-92cb-00e7363aa33b\") " pod="openshift-image-registry/image-registry-74d85446d-tlzb9"
Jul 31 22:58:13.814664 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.814569    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/61cabb50-cf7e-4874-92cb-00e7363aa33b-trusted-ca\") pod \"image-registry-74d85446d-tlzb9\" (UID: \"61cabb50-cf7e-4874-92cb-00e7363aa33b\") " pod="openshift-image-registry/image-registry-74d85446d-tlzb9"
Jul 31 22:58:13.816168 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.816137    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/61cabb50-cf7e-4874-92cb-00e7363aa33b-registry-tls\") pod \"image-registry-74d85446d-tlzb9\" (UID: \"61cabb50-cf7e-4874-92cb-00e7363aa33b\") " pod="openshift-image-registry/image-registry-74d85446d-tlzb9"
Jul 31 22:58:13.816291 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.816188    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/61cabb50-cf7e-4874-92cb-00e7363aa33b-image-registry-private-configuration\") pod \"image-registry-74d85446d-tlzb9\" (UID: \"61cabb50-cf7e-4874-92cb-00e7363aa33b\") " pod="openshift-image-registry/image-registry-74d85446d-tlzb9"
Jul 31 22:58:13.816291 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.816187    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/61cabb50-cf7e-4874-92cb-00e7363aa33b-installation-pull-secrets\") pod \"image-registry-74d85446d-tlzb9\" (UID: \"61cabb50-cf7e-4874-92cb-00e7363aa33b\") " pod="openshift-image-registry/image-registry-74d85446d-tlzb9"
Jul 31 22:58:13.827508 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.827485    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/61cabb50-cf7e-4874-92cb-00e7363aa33b-bound-sa-token\") pod \"image-registry-74d85446d-tlzb9\" (UID: \"61cabb50-cf7e-4874-92cb-00e7363aa33b\") " pod="openshift-image-registry/image-registry-74d85446d-tlzb9"
Jul 31 22:58:13.830257 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.830238    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2lg7\" (UniqueName: \"kubernetes.io/projected/61cabb50-cf7e-4874-92cb-00e7363aa33b-kube-api-access-b2lg7\") pod \"image-registry-74d85446d-tlzb9\" (UID: \"61cabb50-cf7e-4874-92cb-00e7363aa33b\") " pod="openshift-image-registry/image-registry-74d85446d-tlzb9"
Jul 31 22:58:13.841074 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.841052    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-f77f48c6d-hxkk5"
Jul 31 22:58:13.853351 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.853328    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68fcbd4bcc-n75cn"
Jul 31 22:58:13.933171 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.933128    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74d85446d-tlzb9"
Jul 31 22:58:13.992688 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:13.992606    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-f77f48c6d-hxkk5"]
Jul 31 22:58:13.996536 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:58:13.996491    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1e283c7_af2a_46d7_9712_605cec1c3211.slice/crio-5eb4c6c33bd9537eb7b3e76f1be1c6696bc4c6313884515d3871ef396bb882d1 WatchSource:0}: Error finding container 5eb4c6c33bd9537eb7b3e76f1be1c6696bc4c6313884515d3871ef396bb882d1: Status 404 returned error can't find the container with id 5eb4c6c33bd9537eb7b3e76f1be1c6696bc4c6313884515d3871ef396bb882d1
Jul 31 22:58:14.004263 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:14.004235    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68fcbd4bcc-n75cn"]
Jul 31 22:58:14.007235 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:58:14.007216    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod15c7a320_b9a3_4872_b08a_859f5d702070.slice/crio-33fcb170d0607f2fd4834ab67a48f248f119eafa6885fec86b3c88e9cebed058 WatchSource:0}: Error finding container 33fcb170d0607f2fd4834ab67a48f248f119eafa6885fec86b3c88e9cebed058: Status 404 returned error can't find the container with id 33fcb170d0607f2fd4834ab67a48f248f119eafa6885fec86b3c88e9cebed058
Jul 31 22:58:14.072018 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:14.071991    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-74d85446d-tlzb9"]
Jul 31 22:58:14.074213 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:58:14.074180    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61cabb50_cf7e_4874_92cb_00e7363aa33b.slice/crio-ef9e933e037124aebf36874766f1441c410a7cd095ea433999730c4c3bcadbb2 WatchSource:0}: Error finding container ef9e933e037124aebf36874766f1441c410a7cd095ea433999730c4c3bcadbb2: Status 404 returned error can't find the container with id ef9e933e037124aebf36874766f1441c410a7cd095ea433999730c4c3bcadbb2
Jul 31 22:58:14.376633 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:14.376549    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-74d85446d-tlzb9" event={"ID":"61cabb50-cf7e-4874-92cb-00e7363aa33b","Type":"ContainerStarted","Data":"3e2415d1a8adb10c6206b98761e659647efb2337ecc67cc7dcf8bcd80e6ba6d8"}
Jul 31 22:58:14.376633 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:14.376597    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-74d85446d-tlzb9" event={"ID":"61cabb50-cf7e-4874-92cb-00e7363aa33b","Type":"ContainerStarted","Data":"ef9e933e037124aebf36874766f1441c410a7cd095ea433999730c4c3bcadbb2"}
Jul 31 22:58:14.376884 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:14.376670    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-74d85446d-tlzb9"
Jul 31 22:58:14.377862 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:14.377818    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68fcbd4bcc-n75cn" event={"ID":"15c7a320-b9a3-4872-b08a-859f5d702070","Type":"ContainerStarted","Data":"33fcb170d0607f2fd4834ab67a48f248f119eafa6885fec86b3c88e9cebed058"}
Jul 31 22:58:14.378897 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:14.378873    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-f77f48c6d-hxkk5" event={"ID":"a1e283c7-af2a-46d7-9712-605cec1c3211","Type":"ContainerStarted","Data":"5eb4c6c33bd9537eb7b3e76f1be1c6696bc4c6313884515d3871ef396bb882d1"}
Jul 31 22:58:14.400510 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:14.400457    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-74d85446d-tlzb9" podStartSLOduration=1.400440844 podStartE2EDuration="1.400440844s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:58:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:58:14.399006423 +0000 UTC m=+78.044792732" watchObservedRunningTime="2026-07-31 22:58:14.400440844 +0000 UTC m=+78.046227153"
Jul 31 22:58:14.520950 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:14.520575    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/40b91dc3-a7d9-403a-8208-ca1e019553c5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-f6wq6\" (UID: \"40b91dc3-a7d9-403a-8208-ca1e019553c5\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-f6wq6"
Jul 31 22:58:14.523249 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:14.523222    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/40b91dc3-a7d9-403a-8208-ca1e019553c5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-f6wq6\" (UID: \"40b91dc3-a7d9-403a-8208-ca1e019553c5\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-f6wq6"
Jul 31 22:58:14.621233 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:14.621198    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/24908812-404e-4b8c-958d-335220924aff-service-ca-bundle\") pod \"router-default-868556846d-gx64z\" (UID: \"24908812-404e-4b8c-958d-335220924aff\") " pod="openshift-ingress/router-default-868556846d-gx64z"
Jul 31 22:58:14.621673 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:14.621249    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24908812-404e-4b8c-958d-335220924aff-metrics-certs\") pod \"router-default-868556846d-gx64z\" (UID: \"24908812-404e-4b8c-958d-335220924aff\") " pod="openshift-ingress/router-default-868556846d-gx64z"
Jul 31 22:58:14.621978 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:14.621950    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/24908812-404e-4b8c-958d-335220924aff-service-ca-bundle\") pod \"router-default-868556846d-gx64z\" (UID: \"24908812-404e-4b8c-958d-335220924aff\") " pod="openshift-ingress/router-default-868556846d-gx64z"
Jul 31 22:58:14.623750 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:14.623724    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24908812-404e-4b8c-958d-335220924aff-metrics-certs\") pod \"router-default-868556846d-gx64z\" (UID: \"24908812-404e-4b8c-958d-335220924aff\") " pod="openshift-ingress/router-default-868556846d-gx64z"
Jul 31 22:58:14.632966 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:14.632908    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-srlnb\""
Jul 31 22:58:14.641071 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:14.641050    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-868556846d-gx64z"
Jul 31 22:58:14.793976 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:14.793744    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-l74sx\""
Jul 31 22:58:14.802081 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:14.802039    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-f6wq6"
Jul 31 22:58:14.809741 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:14.809664    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-868556846d-gx64z"]
Jul 31 22:58:15.200591 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:58:15.200556    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24908812_404e_4b8c_958d_335220924aff.slice/crio-857355c31c4b107f2676dfe916f5bd0d2b6fe6b26acdec172d09a9544e0df979 WatchSource:0}: Error finding container 857355c31c4b107f2676dfe916f5bd0d2b6fe6b26acdec172d09a9544e0df979: Status 404 returned error can't find the container with id 857355c31c4b107f2676dfe916f5bd0d2b6fe6b26acdec172d09a9544e0df979
Jul 31 22:58:15.380078 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:15.380009    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-f6wq6"]
Jul 31 22:58:15.385919 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:15.385726    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-868556846d-gx64z" event={"ID":"24908812-404e-4b8c-958d-335220924aff","Type":"ContainerStarted","Data":"20a31a11a20b2bf0844303b2d2d1e6bfdebd75cf7c239e48ed9b4271e16a0412"}
Jul 31 22:58:15.387583 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:15.387101    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-868556846d-gx64z" event={"ID":"24908812-404e-4b8c-958d-335220924aff","Type":"ContainerStarted","Data":"857355c31c4b107f2676dfe916f5bd0d2b6fe6b26acdec172d09a9544e0df979"}
Jul 31 22:58:15.391521 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:15.391496    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-f77f48c6d-hxkk5" event={"ID":"a1e283c7-af2a-46d7-9712-605cec1c3211","Type":"ContainerStarted","Data":"4e4a9d58bd44ca6d5d5afbc0fc71d197ea069aaf8d1adbce3d3c7d35962cd0ea"}
Jul 31 22:58:15.430625 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:15.430577    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-868556846d-gx64z" podStartSLOduration=33.430561349 podStartE2EDuration="33.430561349s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:57:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:58:15.430073623 +0000 UTC m=+79.075859946" watchObservedRunningTime="2026-07-31 22:58:15.430561349 +0000 UTC m=+79.076347695"
Jul 31 22:58:15.477420 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:15.477302    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-f77f48c6d-hxkk5" podStartSLOduration=1.214711618 podStartE2EDuration="2.47727986s" totalImagesPullingTime="1.262568242s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:58:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:58:15.474510612 +0000 UTC m=+79.120296923" watchObservedRunningTime="2026-07-31 22:58:15.47727986 +0000 UTC m=+79.123066362"
Jul 31 22:58:15.642276 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:15.642237    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-868556846d-gx64z"
Jul 31 22:58:15.645291 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:15.645267    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-868556846d-gx64z"
Jul 31 22:58:16.395541 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:16.395501    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-f6wq6" event={"ID":"40b91dc3-a7d9-403a-8208-ca1e019553c5","Type":"ContainerStarted","Data":"05bfdd675cc9d164583f22efc93f897efc41ced47b3643b7b67fb79bb30b5894"}
Jul 31 22:58:16.395950 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:16.395925    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-868556846d-gx64z"
Jul 31 22:58:16.397244 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:16.397220    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-868556846d-gx64z"
Jul 31 22:58:17.356209 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:17.356175    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-fhq7n"
Jul 31 22:58:18.410216 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:18.410187    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-f6wq6" event={"ID":"40b91dc3-a7d9-403a-8208-ca1e019553c5","Type":"ContainerStarted","Data":"e9512322dfced06b5fe45a94ac81ce0372fdb5d27be189262fd5c93e0650a449"}
Jul 31 22:58:18.411972 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:18.411808    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68fcbd4bcc-n75cn" event={"ID":"15c7a320-b9a3-4872-b08a-859f5d702070","Type":"ContainerStarted","Data":"88a52dae38e1f5c05564f2d66fadfe3e0d00f23deb824256cc52e4b6e71a4239"}
Jul 31 22:58:18.431458 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:18.431405    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-f6wq6" podStartSLOduration=33.474781795 podStartE2EDuration="36.431389929s" totalImagesPullingTime="2.956608134s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:57:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:58:18.429773535 +0000 UTC m=+82.075559845" watchObservedRunningTime="2026-07-31 22:58:18.431389929 +0000 UTC m=+82.077176237"
Jul 31 22:58:19.417206 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:19.417093    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68fcbd4bcc-n75cn" event={"ID":"15c7a320-b9a3-4872-b08a-859f5d702070","Type":"ContainerStarted","Data":"48552af8aaf77b68216349f92852d00c6d9cfbdec0573ea69d07de9f93987a16"}
Jul 31 22:58:19.417206 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:19.417143    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68fcbd4bcc-n75cn" event={"ID":"15c7a320-b9a3-4872-b08a-859f5d702070","Type":"ContainerStarted","Data":"55e10ffba31e360253b5a7a36fb4b7ed38618c618a3f4f9e8b55ec600159c318"}
Jul 31 22:58:19.438055 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:19.438008    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68fcbd4bcc-n75cn" podStartSLOduration=2.093485076 podStartE2EDuration="6.437993208s" totalImagesPullingTime="4.344508132s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:58:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:58:19.436520818 +0000 UTC m=+83.082307126" watchObservedRunningTime="2026-07-31 22:58:19.437993208 +0000 UTC m=+83.083779516"
Jul 31 22:58:20.380925 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:20.380888    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e460911c-90d3-4ebc-baaf-3047df044bc8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vvnwv\" (UID: \"e460911c-90d3-4ebc-baaf-3047df044bc8\") " pod="openshift-insights/insights-runtime-extractor-vvnwv"
Jul 31 22:58:20.383179 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:20.383156    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e460911c-90d3-4ebc-baaf-3047df044bc8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vvnwv\" (UID: \"e460911c-90d3-4ebc-baaf-3047df044bc8\") " pod="openshift-insights/insights-runtime-extractor-vvnwv"
Jul 31 22:58:20.657910 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:20.657810    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-kmjpj\""
Jul 31 22:58:20.665536 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:20.665507    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vvnwv"
Jul 31 22:58:20.795304 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:20.791681    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-vvnwv"]
Jul 31 22:58:20.798981 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:58:20.798951    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode460911c_90d3_4ebc_baaf_3047df044bc8.slice/crio-9f97f35f55165e9e622c75281f6c9205a836cb4015f27de46d49e37a3ae2c86f WatchSource:0}: Error finding container 9f97f35f55165e9e622c75281f6c9205a836cb4015f27de46d49e37a3ae2c86f: Status 404 returned error can't find the container with id 9f97f35f55165e9e622c75281f6c9205a836cb4015f27de46d49e37a3ae2c86f
Jul 31 22:58:21.425286 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:21.425261    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vvnwv" event={"ID":"e460911c-90d3-4ebc-baaf-3047df044bc8","Type":"ContainerStarted","Data":"708f3cffc78acc3a13f7081e6ac187333fe1061ab1d9cb1c58c1ce609d43aa31"}
Jul 31 22:58:21.425405 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:21.425324    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vvnwv" event={"ID":"e460911c-90d3-4ebc-baaf-3047df044bc8","Type":"ContainerStarted","Data":"8297def4895f73815aed62ddce321d55d51e7fb146870bfa3f889bd3f39d1ad2"}
Jul 31 22:58:21.425405 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:21.425351    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vvnwv" event={"ID":"e460911c-90d3-4ebc-baaf-3047df044bc8","Type":"ContainerStarted","Data":"9f97f35f55165e9e622c75281f6c9205a836cb4015f27de46d49e37a3ae2c86f"}
Jul 31 22:58:23.433522 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:23.433490    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vvnwv" event={"ID":"e460911c-90d3-4ebc-baaf-3047df044bc8","Type":"ContainerStarted","Data":"a55ab1fe0ac4c6164b48a0edfcfbb41559c5747489ea21548ad5e5957a997dcd"}
Jul 31 22:58:23.453547 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:23.453500    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-vvnwv" podStartSLOduration=33.813618653 podStartE2EDuration="35.45348576s" totalImagesPullingTime="1.639867107s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 22:57:48 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:58:23.451720474 +0000 UTC m=+87.097506776" watchObservedRunningTime="2026-07-31 22:58:23.45348576 +0000 UTC m=+87.099272094"
Jul 31 22:58:23.555568 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:23.555542    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-d8bd65946-zn48s"
Jul 31 22:58:26.580800 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:26.580767    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-fgj4b"]
Jul 31 22:58:26.585029 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:26.585014    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-fgj4b"
Jul 31 22:58:26.587884 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:26.587862    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 31 22:58:26.588868 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:26.588843    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 31 22:58:26.588974 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:26.588894    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-kgbfl\""
Jul 31 22:58:26.588974 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:26.588926    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 31 22:58:26.588974 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:26.588940    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 31 22:58:26.730018 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:26.729982    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b7b2f0b9-22d7-4e2c-a4fa-99d981acc2d2-node-exporter-textfile\") pod \"node-exporter-fgj4b\" (UID: \"b7b2f0b9-22d7-4e2c-a4fa-99d981acc2d2\") " pod="openshift-monitoring/node-exporter-fgj4b"
Jul 31 22:58:26.730018 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:26.730021    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b7b2f0b9-22d7-4e2c-a4fa-99d981acc2d2-node-exporter-wtmp\") pod \"node-exporter-fgj4b\" (UID: \"b7b2f0b9-22d7-4e2c-a4fa-99d981acc2d2\") " pod="openshift-monitoring/node-exporter-fgj4b"
Jul 31 22:58:26.730207 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:26.730045    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b7b2f0b9-22d7-4e2c-a4fa-99d981acc2d2-sys\") pod \"node-exporter-fgj4b\" (UID: \"b7b2f0b9-22d7-4e2c-a4fa-99d981acc2d2\") " pod="openshift-monitoring/node-exporter-fgj4b"
Jul 31 22:58:26.730207 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:26.730144    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tndw2\" (UniqueName: \"kubernetes.io/projected/b7b2f0b9-22d7-4e2c-a4fa-99d981acc2d2-kube-api-access-tndw2\") pod \"node-exporter-fgj4b\" (UID: \"b7b2f0b9-22d7-4e2c-a4fa-99d981acc2d2\") " pod="openshift-monitoring/node-exporter-fgj4b"
Jul 31 22:58:26.730207 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:26.730178    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b7b2f0b9-22d7-4e2c-a4fa-99d981acc2d2-root\") pod \"node-exporter-fgj4b\" (UID: \"b7b2f0b9-22d7-4e2c-a4fa-99d981acc2d2\") " pod="openshift-monitoring/node-exporter-fgj4b"
Jul 31 22:58:26.730346 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:26.730230    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b7b2f0b9-22d7-4e2c-a4fa-99d981acc2d2-node-exporter-tls\") pod \"node-exporter-fgj4b\" (UID: \"b7b2f0b9-22d7-4e2c-a4fa-99d981acc2d2\") " pod="openshift-monitoring/node-exporter-fgj4b"
Jul 31 22:58:26.730346 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:26.730251    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b7b2f0b9-22d7-4e2c-a4fa-99d981acc2d2-node-exporter-accelerators-collector-config\") pod \"node-exporter-fgj4b\" (UID: \"b7b2f0b9-22d7-4e2c-a4fa-99d981acc2d2\") " pod="openshift-monitoring/node-exporter-fgj4b"
Jul 31 22:58:26.730346 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:26.730273    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b7b2f0b9-22d7-4e2c-a4fa-99d981acc2d2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fgj4b\" (UID: \"b7b2f0b9-22d7-4e2c-a4fa-99d981acc2d2\") " pod="openshift-monitoring/node-exporter-fgj4b"
Jul 31 22:58:26.730513 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:26.730334    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b7b2f0b9-22d7-4e2c-a4fa-99d981acc2d2-metrics-client-ca\") pod \"node-exporter-fgj4b\" (UID: \"b7b2f0b9-22d7-4e2c-a4fa-99d981acc2d2\") " pod="openshift-monitoring/node-exporter-fgj4b"
Jul 31 22:58:26.831500 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:26.831385    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b7b2f0b9-22d7-4e2c-a4fa-99d981acc2d2-root\") pod \"node-exporter-fgj4b\" (UID: \"b7b2f0b9-22d7-4e2c-a4fa-99d981acc2d2\") " pod="openshift-monitoring/node-exporter-fgj4b"
Jul 31 22:58:26.831500 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:26.831467    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b7b2f0b9-22d7-4e2c-a4fa-99d981acc2d2-node-exporter-tls\") pod \"node-exporter-fgj4b\" (UID: \"b7b2f0b9-22d7-4e2c-a4fa-99d981acc2d2\") " pod="openshift-monitoring/node-exporter-fgj4b"
Jul 31 22:58:26.831741 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:26.831494    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b7b2f0b9-22d7-4e2c-a4fa-99d981acc2d2-root\") pod \"node-exporter-fgj4b\" (UID: \"b7b2f0b9-22d7-4e2c-a4fa-99d981acc2d2\") " pod="openshift-monitoring/node-exporter-fgj4b"
Jul 31 22:58:26.831741 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:26.831501    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b7b2f0b9-22d7-4e2c-a4fa-99d981acc2d2-node-exporter-accelerators-collector-config\") pod \"node-exporter-fgj4b\" (UID: \"b7b2f0b9-22d7-4e2c-a4fa-99d981acc2d2\") " pod="openshift-monitoring/node-exporter-fgj4b"
Jul 31 22:58:26.831741 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:26.831568    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b7b2f0b9-22d7-4e2c-a4fa-99d981acc2d2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fgj4b\" (UID: \"b7b2f0b9-22d7-4e2c-a4fa-99d981acc2d2\") " pod="openshift-monitoring/node-exporter-fgj4b"
Jul 31 22:58:26.831741 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:26.831605    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b7b2f0b9-22d7-4e2c-a4fa-99d981acc2d2-metrics-client-ca\") pod \"node-exporter-fgj4b\" (UID: \"b7b2f0b9-22d7-4e2c-a4fa-99d981acc2d2\") " pod="openshift-monitoring/node-exporter-fgj4b"
Jul 31 22:58:26.831741 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:26.831645    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b7b2f0b9-22d7-4e2c-a4fa-99d981acc2d2-node-exporter-textfile\") pod \"node-exporter-fgj4b\" (UID: \"b7b2f0b9-22d7-4e2c-a4fa-99d981acc2d2\") " pod="openshift-monitoring/node-exporter-fgj4b"
Jul 31 22:58:26.831741 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:26.831665    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b7b2f0b9-22d7-4e2c-a4fa-99d981acc2d2-node-exporter-wtmp\") pod \"node-exporter-fgj4b\" (UID: \"b7b2f0b9-22d7-4e2c-a4fa-99d981acc2d2\") " pod="openshift-monitoring/node-exporter-fgj4b"
Jul 31 22:58:26.831741 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:26.831691    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b7b2f0b9-22d7-4e2c-a4fa-99d981acc2d2-sys\") pod \"node-exporter-fgj4b\" (UID: \"b7b2f0b9-22d7-4e2c-a4fa-99d981acc2d2\") " pod="openshift-monitoring/node-exporter-fgj4b"
Jul 31 22:58:26.832074 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:26.831776    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tndw2\" (UniqueName: \"kubernetes.io/projected/b7b2f0b9-22d7-4e2c-a4fa-99d981acc2d2-kube-api-access-tndw2\") pod \"node-exporter-fgj4b\" (UID: \"b7b2f0b9-22d7-4e2c-a4fa-99d981acc2d2\") " pod="openshift-monitoring/node-exporter-fgj4b"
Jul 31 22:58:26.832074 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:26.831857    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b7b2f0b9-22d7-4e2c-a4fa-99d981acc2d2-node-exporter-wtmp\") pod \"node-exporter-fgj4b\" (UID: \"b7b2f0b9-22d7-4e2c-a4fa-99d981acc2d2\") " pod="openshift-monitoring/node-exporter-fgj4b"
Jul 31 22:58:26.832074 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:26.831898    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b7b2f0b9-22d7-4e2c-a4fa-99d981acc2d2-sys\") pod \"node-exporter-fgj4b\" (UID: \"b7b2f0b9-22d7-4e2c-a4fa-99d981acc2d2\") " pod="openshift-monitoring/node-exporter-fgj4b"
Jul 31 22:58:26.832466 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:26.832409    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b7b2f0b9-22d7-4e2c-a4fa-99d981acc2d2-node-exporter-accelerators-collector-config\") pod \"node-exporter-fgj4b\" (UID: \"b7b2f0b9-22d7-4e2c-a4fa-99d981acc2d2\") " pod="openshift-monitoring/node-exporter-fgj4b"
Jul 31 22:58:26.832588 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:26.832500    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b7b2f0b9-22d7-4e2c-a4fa-99d981acc2d2-node-exporter-textfile\") pod \"node-exporter-fgj4b\" (UID: \"b7b2f0b9-22d7-4e2c-a4fa-99d981acc2d2\") " pod="openshift-monitoring/node-exporter-fgj4b"
Jul 31 22:58:26.832588 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:26.832531    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b7b2f0b9-22d7-4e2c-a4fa-99d981acc2d2-metrics-client-ca\") pod \"node-exporter-fgj4b\" (UID: \"b7b2f0b9-22d7-4e2c-a4fa-99d981acc2d2\") " pod="openshift-monitoring/node-exporter-fgj4b"
Jul 31 22:58:26.834299 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:26.834275    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b7b2f0b9-22d7-4e2c-a4fa-99d981acc2d2-node-exporter-tls\") pod \"node-exporter-fgj4b\" (UID: \"b7b2f0b9-22d7-4e2c-a4fa-99d981acc2d2\") " pod="openshift-monitoring/node-exporter-fgj4b"
Jul 31 22:58:26.834427 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:26.834302    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b7b2f0b9-22d7-4e2c-a4fa-99d981acc2d2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fgj4b\" (UID: \"b7b2f0b9-22d7-4e2c-a4fa-99d981acc2d2\") " pod="openshift-monitoring/node-exporter-fgj4b"
Jul 31 22:58:26.843558 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:26.843506    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tndw2\" (UniqueName: \"kubernetes.io/projected/b7b2f0b9-22d7-4e2c-a4fa-99d981acc2d2-kube-api-access-tndw2\") pod \"node-exporter-fgj4b\" (UID: \"b7b2f0b9-22d7-4e2c-a4fa-99d981acc2d2\") " pod="openshift-monitoring/node-exporter-fgj4b"
Jul 31 22:58:26.894520 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:26.894493    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-fgj4b"
Jul 31 22:58:26.905738 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:58:26.905694    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7b2f0b9_22d7_4e2c_a4fa_99d981acc2d2.slice/crio-0a7e52540504d9bd8cdd041ed9767ff69015a6cc9861541eff504a3ee1ee6ba9 WatchSource:0}: Error finding container 0a7e52540504d9bd8cdd041ed9767ff69015a6cc9861541eff504a3ee1ee6ba9: Status 404 returned error can't find the container with id 0a7e52540504d9bd8cdd041ed9767ff69015a6cc9861541eff504a3ee1ee6ba9
Jul 31 22:58:27.449602 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:27.449565    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fgj4b" event={"ID":"b7b2f0b9-22d7-4e2c-a4fa-99d981acc2d2","Type":"ContainerStarted","Data":"0a7e52540504d9bd8cdd041ed9767ff69015a6cc9861541eff504a3ee1ee6ba9"}
Jul 31 22:58:28.458718 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:28.458669    2584 generic.go:356] "Generic (PLEG): container finished" podID="b7b2f0b9-22d7-4e2c-a4fa-99d981acc2d2" containerID="127856ee5dfeef6f3b120df1f8d4c31d962e93ec5818c81b4ae3ea9498d5da1e" exitCode=0
Jul 31 22:58:28.459073 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:28.458740    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fgj4b" event={"ID":"b7b2f0b9-22d7-4e2c-a4fa-99d981acc2d2","Type":"ContainerDied","Data":"127856ee5dfeef6f3b120df1f8d4c31d962e93ec5818c81b4ae3ea9498d5da1e"}
Jul 31 22:58:29.464595 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:29.464566    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fgj4b" event={"ID":"b7b2f0b9-22d7-4e2c-a4fa-99d981acc2d2","Type":"ContainerStarted","Data":"b9d2822afd694f4dfc7da51eb4f9acc04a3ecc2b0b7feb07b076bb268c94837d"}
Jul 31 22:58:29.464986 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:29.464614    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fgj4b" event={"ID":"b7b2f0b9-22d7-4e2c-a4fa-99d981acc2d2","Type":"ContainerStarted","Data":"50c390f95f3094795224620beb205546262859e84dd41dfec8ddfce3bca8da2f"}
Jul 31 22:58:29.493394 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:29.493352    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-fgj4b" podStartSLOduration=2.416043146 podStartE2EDuration="3.493337775s" totalImagesPullingTime="1.012279299s" totalInitContainerRuntime="65.01533ms" isStatefulPod=true podCreationTimestamp="2026-07-31 22:58:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:58:29.492262527 +0000 UTC m=+93.138048836" watchObservedRunningTime="2026-07-31 22:58:29.493337775 +0000 UTC m=+93.139124083"
Jul 31 22:58:35.395549 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:35.395517    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-74d85446d-tlzb9"
Jul 31 22:58:38.356736 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:38.356691    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-84css"
Jul 31 22:58:38.571268 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:38.571223    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-d8bd65946-zn48s" podUID="798610c4-70cc-4455-943f-b0d6b0dca9d9" containerName="registry" containerID="cri-o://531777a1bedd3c6a6266d9f4c788fddad6e70e73fa43a4662398dc8dde858b0d" gracePeriod=30
Jul 31 22:58:38.811120 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:38.811095    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d8bd65946-zn48s"
Jul 31 22:58:38.933223 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:38.933191    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/798610c4-70cc-4455-943f-b0d6b0dca9d9-trusted-ca\") pod \"798610c4-70cc-4455-943f-b0d6b0dca9d9\" (UID: \"798610c4-70cc-4455-943f-b0d6b0dca9d9\") "
Jul 31 22:58:38.933398 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:38.933247    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/798610c4-70cc-4455-943f-b0d6b0dca9d9-installation-pull-secrets\") pod \"798610c4-70cc-4455-943f-b0d6b0dca9d9\" (UID: \"798610c4-70cc-4455-943f-b0d6b0dca9d9\") "
Jul 31 22:58:38.933398 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:38.933279    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/798610c4-70cc-4455-943f-b0d6b0dca9d9-image-registry-private-configuration\") pod \"798610c4-70cc-4455-943f-b0d6b0dca9d9\" (UID: \"798610c4-70cc-4455-943f-b0d6b0dca9d9\") "
Jul 31 22:58:38.933398 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:38.933323    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9bv2x\" (UniqueName: \"kubernetes.io/projected/798610c4-70cc-4455-943f-b0d6b0dca9d9-kube-api-access-9bv2x\") pod \"798610c4-70cc-4455-943f-b0d6b0dca9d9\" (UID: \"798610c4-70cc-4455-943f-b0d6b0dca9d9\") "
Jul 31 22:58:38.933398 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:38.933346    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/798610c4-70cc-4455-943f-b0d6b0dca9d9-bound-sa-token\") pod \"798610c4-70cc-4455-943f-b0d6b0dca9d9\" (UID: \"798610c4-70cc-4455-943f-b0d6b0dca9d9\") "
Jul 31 22:58:38.933398 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:38.933375    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/798610c4-70cc-4455-943f-b0d6b0dca9d9-registry-certificates\") pod \"798610c4-70cc-4455-943f-b0d6b0dca9d9\" (UID: \"798610c4-70cc-4455-943f-b0d6b0dca9d9\") "
Jul 31 22:58:38.933653 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:38.933408    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/798610c4-70cc-4455-943f-b0d6b0dca9d9-ca-trust-extracted\") pod \"798610c4-70cc-4455-943f-b0d6b0dca9d9\" (UID: \"798610c4-70cc-4455-943f-b0d6b0dca9d9\") "
Jul 31 22:58:38.933653 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:38.933483    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/798610c4-70cc-4455-943f-b0d6b0dca9d9-registry-tls\") pod \"798610c4-70cc-4455-943f-b0d6b0dca9d9\" (UID: \"798610c4-70cc-4455-943f-b0d6b0dca9d9\") "
Jul 31 22:58:38.933820 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:38.933644    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/798610c4-70cc-4455-943f-b0d6b0dca9d9-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "798610c4-70cc-4455-943f-b0d6b0dca9d9" (UID: "798610c4-70cc-4455-943f-b0d6b0dca9d9"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 22:58:38.934097 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:38.934061    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/798610c4-70cc-4455-943f-b0d6b0dca9d9-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "798610c4-70cc-4455-943f-b0d6b0dca9d9" (UID: "798610c4-70cc-4455-943f-b0d6b0dca9d9"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 22:58:38.935905 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:38.935876    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/798610c4-70cc-4455-943f-b0d6b0dca9d9-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "798610c4-70cc-4455-943f-b0d6b0dca9d9" (UID: "798610c4-70cc-4455-943f-b0d6b0dca9d9"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 22:58:38.936023 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:38.936004    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/798610c4-70cc-4455-943f-b0d6b0dca9d9-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "798610c4-70cc-4455-943f-b0d6b0dca9d9" (UID: "798610c4-70cc-4455-943f-b0d6b0dca9d9"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 22:58:38.936138 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:38.936123    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/798610c4-70cc-4455-943f-b0d6b0dca9d9-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "798610c4-70cc-4455-943f-b0d6b0dca9d9" (UID: "798610c4-70cc-4455-943f-b0d6b0dca9d9"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 22:58:38.936213 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:38.936189    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/798610c4-70cc-4455-943f-b0d6b0dca9d9-kube-api-access-9bv2x" (OuterVolumeSpecName: "kube-api-access-9bv2x") pod "798610c4-70cc-4455-943f-b0d6b0dca9d9" (UID: "798610c4-70cc-4455-943f-b0d6b0dca9d9"). InnerVolumeSpecName "kube-api-access-9bv2x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 22:58:38.936291 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:38.936268    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/798610c4-70cc-4455-943f-b0d6b0dca9d9-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "798610c4-70cc-4455-943f-b0d6b0dca9d9" (UID: "798610c4-70cc-4455-943f-b0d6b0dca9d9"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 22:58:38.944236 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:38.944204    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/798610c4-70cc-4455-943f-b0d6b0dca9d9-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "798610c4-70cc-4455-943f-b0d6b0dca9d9" (UID: "798610c4-70cc-4455-943f-b0d6b0dca9d9"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 22:58:39.034887 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:39.034848    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9bv2x\" (UniqueName: \"kubernetes.io/projected/798610c4-70cc-4455-943f-b0d6b0dca9d9-kube-api-access-9bv2x\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 22:58:39.034887 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:39.034880    2584 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/798610c4-70cc-4455-943f-b0d6b0dca9d9-bound-sa-token\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 22:58:39.034887 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:39.034890    2584 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/798610c4-70cc-4455-943f-b0d6b0dca9d9-registry-certificates\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 22:58:39.034887 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:39.034899    2584 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/798610c4-70cc-4455-943f-b0d6b0dca9d9-ca-trust-extracted\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 22:58:39.035148 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:39.034908    2584 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/798610c4-70cc-4455-943f-b0d6b0dca9d9-registry-tls\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 22:58:39.035148 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:39.034916    2584 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/798610c4-70cc-4455-943f-b0d6b0dca9d9-trusted-ca\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 22:58:39.035148 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:39.034925    2584 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/798610c4-70cc-4455-943f-b0d6b0dca9d9-installation-pull-secrets\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 22:58:39.035148 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:39.034936    2584 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/798610c4-70cc-4455-943f-b0d6b0dca9d9-image-registry-private-configuration\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 22:58:39.501585 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:39.501554    2584 generic.go:356] "Generic (PLEG): container finished" podID="798610c4-70cc-4455-943f-b0d6b0dca9d9" containerID="531777a1bedd3c6a6266d9f4c788fddad6e70e73fa43a4662398dc8dde858b0d" exitCode=0
Jul 31 22:58:39.502031 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:39.501598    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d8bd65946-zn48s"
Jul 31 22:58:39.502031 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:39.501635    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-d8bd65946-zn48s" event={"ID":"798610c4-70cc-4455-943f-b0d6b0dca9d9","Type":"ContainerDied","Data":"531777a1bedd3c6a6266d9f4c788fddad6e70e73fa43a4662398dc8dde858b0d"}
Jul 31 22:58:39.502031 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:39.501679    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-d8bd65946-zn48s" event={"ID":"798610c4-70cc-4455-943f-b0d6b0dca9d9","Type":"ContainerDied","Data":"c856bd1b56d02ff9f3be3bf3e467f0c656a4156c35421f9d7ce200b0659c7bf2"}
Jul 31 22:58:39.502031 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:39.501733    2584 scope.go:117] "RemoveContainer" containerID="531777a1bedd3c6a6266d9f4c788fddad6e70e73fa43a4662398dc8dde858b0d"
Jul 31 22:58:39.510155 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:39.510139    2584 scope.go:117] "RemoveContainer" containerID="531777a1bedd3c6a6266d9f4c788fddad6e70e73fa43a4662398dc8dde858b0d"
Jul 31 22:58:39.510415 ip-10-0-128-243 kubenswrapper[2584]: E0731 22:58:39.510394    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"531777a1bedd3c6a6266d9f4c788fddad6e70e73fa43a4662398dc8dde858b0d\": container with ID starting with 531777a1bedd3c6a6266d9f4c788fddad6e70e73fa43a4662398dc8dde858b0d not found: ID does not exist" containerID="531777a1bedd3c6a6266d9f4c788fddad6e70e73fa43a4662398dc8dde858b0d"
Jul 31 22:58:39.510491 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:39.510428    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"531777a1bedd3c6a6266d9f4c788fddad6e70e73fa43a4662398dc8dde858b0d"} err="failed to get container status \"531777a1bedd3c6a6266d9f4c788fddad6e70e73fa43a4662398dc8dde858b0d\": rpc error: code = NotFound desc = could not find container \"531777a1bedd3c6a6266d9f4c788fddad6e70e73fa43a4662398dc8dde858b0d\": container with ID starting with 531777a1bedd3c6a6266d9f4c788fddad6e70e73fa43a4662398dc8dde858b0d not found: ID does not exist"
Jul 31 22:58:39.523559 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:39.523535    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-d8bd65946-zn48s"]
Jul 31 22:58:39.532413 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:39.532391    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-d8bd65946-zn48s"]
Jul 31 22:58:40.890123 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:40.890090    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="798610c4-70cc-4455-943f-b0d6b0dca9d9" path="/var/lib/kubelet/pods/798610c4-70cc-4455-943f-b0d6b0dca9d9/volumes"
Jul 31 22:58:58.565468 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:58.565429    2584 generic.go:356] "Generic (PLEG): container finished" podID="715381b4-025e-436c-96d9-8ab63b3750a9" containerID="bedbfd81673aadaf2474d4f51b838dc9c590d4d00ba42b9a3cb0152666057eb6" exitCode=0
Jul 31 22:58:58.566016 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:58.565511    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-77696685b-k9tqs" event={"ID":"715381b4-025e-436c-96d9-8ab63b3750a9","Type":"ContainerDied","Data":"bedbfd81673aadaf2474d4f51b838dc9c590d4d00ba42b9a3cb0152666057eb6"}
Jul 31 22:58:58.566016 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:58.565983    2584 scope.go:117] "RemoveContainer" containerID="bedbfd81673aadaf2474d4f51b838dc9c590d4d00ba42b9a3cb0152666057eb6"
Jul 31 22:58:59.570749 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:58:59.570714    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-77696685b-k9tqs" event={"ID":"715381b4-025e-436c-96d9-8ab63b3750a9","Type":"ContainerStarted","Data":"ff32f227a2ced94655ebb06cb03ef9acd487b8572970a60b7e7dd2ea16c059f4"}
Jul 31 22:59:05.664996 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:59:05.664952    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945-metrics-certs\") pod \"network-metrics-daemon-mz5wz\" (UID: \"b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945\") " pod="openshift-multus/network-metrics-daemon-mz5wz"
Jul 31 22:59:05.667290 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:59:05.667264    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945-metrics-certs\") pod \"network-metrics-daemon-mz5wz\" (UID: \"b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945\") " pod="openshift-multus/network-metrics-daemon-mz5wz"
Jul 31 22:59:05.900640 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:59:05.900613    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-h8t45\""
Jul 31 22:59:05.907822 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:59:05.907805    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mz5wz"
Jul 31 22:59:06.040943 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:59:06.040909    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mz5wz"]
Jul 31 22:59:06.043801 ip-10-0-128-243 kubenswrapper[2584]: W0731 22:59:06.043775    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb57a1e5f_f0c7_4ac1_b7d0_7456b22c2945.slice/crio-d04e8631a1c01c1b695eacf07f4a3279ac100e4b861f3bccd505c567478c04c6 WatchSource:0}: Error finding container d04e8631a1c01c1b695eacf07f4a3279ac100e4b861f3bccd505c567478c04c6: Status 404 returned error can't find the container with id d04e8631a1c01c1b695eacf07f4a3279ac100e4b861f3bccd505c567478c04c6
Jul 31 22:59:06.596358 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:59:06.596319    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mz5wz" event={"ID":"b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945","Type":"ContainerStarted","Data":"d04e8631a1c01c1b695eacf07f4a3279ac100e4b861f3bccd505c567478c04c6"}
Jul 31 22:59:07.601522 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:59:07.601483    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mz5wz" event={"ID":"b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945","Type":"ContainerStarted","Data":"ddbbd2e4b22683d13c0e5886a3961fe37edd68272ed9e879be69b5f39e328d2a"}
Jul 31 22:59:07.602141 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:59:07.601581    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mz5wz" event={"ID":"b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945","Type":"ContainerStarted","Data":"8e66f0bb301cc4ac40f0960d7036d1ace620f29db249a5072c54fe8557319fee"}
Jul 31 22:59:07.619115 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:59:07.619068    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-mz5wz" podStartSLOduration=129.757670854 podStartE2EDuration="2m10.619052657s" totalImagesPullingTime="861.381803ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 22:56:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 22:59:07.617164042 +0000 UTC m=+131.262950352" watchObservedRunningTime="2026-07-31 22:59:07.619052657 +0000 UTC m=+131.264838965"
Jul 31 22:59:13.623081 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:59:13.623041    2584 generic.go:356] "Generic (PLEG): container finished" podID="ed2bd0da-5ef5-45c8-bba7-ba1042dc308a" containerID="44417391135d61f5237a417145551be7fcf421eaa0b1900cc9648a6c209f917a" exitCode=0
Jul 31 22:59:13.623471 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:59:13.623087    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-5jtrx" event={"ID":"ed2bd0da-5ef5-45c8-bba7-ba1042dc308a","Type":"ContainerDied","Data":"44417391135d61f5237a417145551be7fcf421eaa0b1900cc9648a6c209f917a"}
Jul 31 22:59:13.623471 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:59:13.623436    2584 scope.go:117] "RemoveContainer" containerID="44417391135d61f5237a417145551be7fcf421eaa0b1900cc9648a6c209f917a"
Jul 31 22:59:14.628637 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:59:14.628601    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-5jtrx" event={"ID":"ed2bd0da-5ef5-45c8-bba7-ba1042dc308a","Type":"ContainerStarted","Data":"84f454df9b7765d084f2777a7b012ddd50d8769d44750f9cf3f326ebb8cf90e6"}
Jul 31 22:59:17.642146 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:59:17.642114    2584 generic.go:356] "Generic (PLEG): container finished" podID="55da6dc5-f4ad-468b-a4c8-497309dad48a" containerID="02423e57f267eeb6d7f488c6ea43d413f1a153ce7ccce72ba7a674af8b17ccef" exitCode=0
Jul 31 22:59:17.642553 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:59:17.642160    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-p58r8" event={"ID":"55da6dc5-f4ad-468b-a4c8-497309dad48a","Type":"ContainerDied","Data":"02423e57f267eeb6d7f488c6ea43d413f1a153ce7ccce72ba7a674af8b17ccef"}
Jul 31 22:59:17.642553 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:59:17.642437    2584 scope.go:117] "RemoveContainer" containerID="02423e57f267eeb6d7f488c6ea43d413f1a153ce7ccce72ba7a674af8b17ccef"
Jul 31 22:59:18.647242 ip-10-0-128-243 kubenswrapper[2584]: I0731 22:59:18.647207    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-p58r8" event={"ID":"55da6dc5-f4ad-468b-a4c8-497309dad48a","Type":"ContainerStarted","Data":"407eb403a80cffd6fe617ec5c0cc304bfbe304c555fa658ab47e09352cad5982"}
Jul 31 23:01:56.775090 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:01:56.775064    2584 kubelet.go:1648] "Image garbage collection succeeded"
Jul 31 23:03:06.317874 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:06.317796    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-8567dd98f6-dqqc2"]
Jul 31 23:03:06.318309 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:06.318052    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="798610c4-70cc-4455-943f-b0d6b0dca9d9" containerName="registry"
Jul 31 23:03:06.318309 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:06.318062    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="798610c4-70cc-4455-943f-b0d6b0dca9d9" containerName="registry"
Jul 31 23:03:06.318309 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:06.318114    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="798610c4-70cc-4455-943f-b0d6b0dca9d9" containerName="registry"
Jul 31 23:03:06.320920 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:06.320901    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-8567dd98f6-dqqc2"
Jul 31 23:03:06.324969 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:06.324941    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 31 23:03:06.325089 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:06.324972    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 31 23:03:06.325089 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:06.325023    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-zbcqc\""
Jul 31 23:03:06.325339 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:06.325261    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 31 23:03:06.325339 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:06.325292    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 31 23:03:06.347031 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:06.346999    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-8567dd98f6-dqqc2"]
Jul 31 23:03:06.488167 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:06.488129    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c31127e6-7ea2-4b22-8bb0-a8ba1fed312b-manifests\") pod \"opendatahub-operator-controller-manager-8567dd98f6-dqqc2\" (UID: \"c31127e6-7ea2-4b22-8bb0-a8ba1fed312b\") " pod="opendatahub/opendatahub-operator-controller-manager-8567dd98f6-dqqc2"
Jul 31 23:03:06.488352 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:06.488184    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c31127e6-7ea2-4b22-8bb0-a8ba1fed312b-webhook-cert\") pod \"opendatahub-operator-controller-manager-8567dd98f6-dqqc2\" (UID: \"c31127e6-7ea2-4b22-8bb0-a8ba1fed312b\") " pod="opendatahub/opendatahub-operator-controller-manager-8567dd98f6-dqqc2"
Jul 31 23:03:06.488352 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:06.488263    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c31127e6-7ea2-4b22-8bb0-a8ba1fed312b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8567dd98f6-dqqc2\" (UID: \"c31127e6-7ea2-4b22-8bb0-a8ba1fed312b\") " pod="opendatahub/opendatahub-operator-controller-manager-8567dd98f6-dqqc2"
Jul 31 23:03:06.488352 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:06.488293    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6rf4\" (UniqueName: \"kubernetes.io/projected/c31127e6-7ea2-4b22-8bb0-a8ba1fed312b-kube-api-access-d6rf4\") pod \"opendatahub-operator-controller-manager-8567dd98f6-dqqc2\" (UID: \"c31127e6-7ea2-4b22-8bb0-a8ba1fed312b\") " pod="opendatahub/opendatahub-operator-controller-manager-8567dd98f6-dqqc2"
Jul 31 23:03:06.488352 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:06.488316    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c31127e6-7ea2-4b22-8bb0-a8ba1fed312b-tmp\") pod \"opendatahub-operator-controller-manager-8567dd98f6-dqqc2\" (UID: \"c31127e6-7ea2-4b22-8bb0-a8ba1fed312b\") " pod="opendatahub/opendatahub-operator-controller-manager-8567dd98f6-dqqc2"
Jul 31 23:03:06.588843 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:06.588756    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c31127e6-7ea2-4b22-8bb0-a8ba1fed312b-tmp\") pod \"opendatahub-operator-controller-manager-8567dd98f6-dqqc2\" (UID: \"c31127e6-7ea2-4b22-8bb0-a8ba1fed312b\") " pod="opendatahub/opendatahub-operator-controller-manager-8567dd98f6-dqqc2"
Jul 31 23:03:06.588843 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:06.588813    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c31127e6-7ea2-4b22-8bb0-a8ba1fed312b-manifests\") pod \"opendatahub-operator-controller-manager-8567dd98f6-dqqc2\" (UID: \"c31127e6-7ea2-4b22-8bb0-a8ba1fed312b\") " pod="opendatahub/opendatahub-operator-controller-manager-8567dd98f6-dqqc2"
Jul 31 23:03:06.588843 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:06.588840    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c31127e6-7ea2-4b22-8bb0-a8ba1fed312b-webhook-cert\") pod \"opendatahub-operator-controller-manager-8567dd98f6-dqqc2\" (UID: \"c31127e6-7ea2-4b22-8bb0-a8ba1fed312b\") " pod="opendatahub/opendatahub-operator-controller-manager-8567dd98f6-dqqc2"
Jul 31 23:03:06.589046 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:06.588877    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c31127e6-7ea2-4b22-8bb0-a8ba1fed312b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8567dd98f6-dqqc2\" (UID: \"c31127e6-7ea2-4b22-8bb0-a8ba1fed312b\") " pod="opendatahub/opendatahub-operator-controller-manager-8567dd98f6-dqqc2"
Jul 31 23:03:06.589046 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:06.588894    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6rf4\" (UniqueName: \"kubernetes.io/projected/c31127e6-7ea2-4b22-8bb0-a8ba1fed312b-kube-api-access-d6rf4\") pod \"opendatahub-operator-controller-manager-8567dd98f6-dqqc2\" (UID: \"c31127e6-7ea2-4b22-8bb0-a8ba1fed312b\") " pod="opendatahub/opendatahub-operator-controller-manager-8567dd98f6-dqqc2"
Jul 31 23:03:06.589343 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:06.589319    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c31127e6-7ea2-4b22-8bb0-a8ba1fed312b-manifests\") pod \"opendatahub-operator-controller-manager-8567dd98f6-dqqc2\" (UID: \"c31127e6-7ea2-4b22-8bb0-a8ba1fed312b\") " pod="opendatahub/opendatahub-operator-controller-manager-8567dd98f6-dqqc2"
Jul 31 23:03:06.591206 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:06.591181    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c31127e6-7ea2-4b22-8bb0-a8ba1fed312b-tmp\") pod \"opendatahub-operator-controller-manager-8567dd98f6-dqqc2\" (UID: \"c31127e6-7ea2-4b22-8bb0-a8ba1fed312b\") " pod="opendatahub/opendatahub-operator-controller-manager-8567dd98f6-dqqc2"
Jul 31 23:03:06.591329 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:06.591310    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c31127e6-7ea2-4b22-8bb0-a8ba1fed312b-webhook-cert\") pod \"opendatahub-operator-controller-manager-8567dd98f6-dqqc2\" (UID: \"c31127e6-7ea2-4b22-8bb0-a8ba1fed312b\") " pod="opendatahub/opendatahub-operator-controller-manager-8567dd98f6-dqqc2"
Jul 31 23:03:06.591537 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:06.591515    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c31127e6-7ea2-4b22-8bb0-a8ba1fed312b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8567dd98f6-dqqc2\" (UID: \"c31127e6-7ea2-4b22-8bb0-a8ba1fed312b\") " pod="opendatahub/opendatahub-operator-controller-manager-8567dd98f6-dqqc2"
Jul 31 23:03:06.598227 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:06.598203    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6rf4\" (UniqueName: \"kubernetes.io/projected/c31127e6-7ea2-4b22-8bb0-a8ba1fed312b-kube-api-access-d6rf4\") pod \"opendatahub-operator-controller-manager-8567dd98f6-dqqc2\" (UID: \"c31127e6-7ea2-4b22-8bb0-a8ba1fed312b\") " pod="opendatahub/opendatahub-operator-controller-manager-8567dd98f6-dqqc2"
Jul 31 23:03:06.630924 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:06.630896    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-8567dd98f6-dqqc2"
Jul 31 23:03:06.758996 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:06.758964    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-8567dd98f6-dqqc2"]
Jul 31 23:03:06.762200 ip-10-0-128-243 kubenswrapper[2584]: W0731 23:03:06.762173    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc31127e6_7ea2_4b22_8bb0_a8ba1fed312b.slice/crio-62788221a02c13932cdeac261847c8add2ad79ca38f05d0035c8bfefe95c8735 WatchSource:0}: Error finding container 62788221a02c13932cdeac261847c8add2ad79ca38f05d0035c8bfefe95c8735: Status 404 returned error can't find the container with id 62788221a02c13932cdeac261847c8add2ad79ca38f05d0035c8bfefe95c8735
Jul 31 23:03:06.764030 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:06.764012    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 31 23:03:07.448110 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:07.448023    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8567dd98f6-dqqc2" event={"ID":"c31127e6-7ea2-4b22-8bb0-a8ba1fed312b","Type":"ContainerStarted","Data":"62788221a02c13932cdeac261847c8add2ad79ca38f05d0035c8bfefe95c8735"}
Jul 31 23:03:10.462360 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:10.462326    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8567dd98f6-dqqc2" event={"ID":"c31127e6-7ea2-4b22-8bb0-a8ba1fed312b","Type":"ContainerStarted","Data":"4882c17dd4c31bacaf18614808f02ad54ccfb97e184d9ff23205abec167b074b"}
Jul 31 23:03:11.467631 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:11.467590    2584 generic.go:356] "Generic (PLEG): container finished" podID="c31127e6-7ea2-4b22-8bb0-a8ba1fed312b" containerID="4882c17dd4c31bacaf18614808f02ad54ccfb97e184d9ff23205abec167b074b" exitCode=0
Jul 31 23:03:11.468066 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:11.467674    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8567dd98f6-dqqc2" event={"ID":"c31127e6-7ea2-4b22-8bb0-a8ba1fed312b","Type":"ContainerDied","Data":"4882c17dd4c31bacaf18614808f02ad54ccfb97e184d9ff23205abec167b074b"}
Jul 31 23:03:12.472911 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:12.472875    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8567dd98f6-dqqc2" event={"ID":"c31127e6-7ea2-4b22-8bb0-a8ba1fed312b","Type":"ContainerStarted","Data":"01d212f5694facb102dc87927a9a4caef4f59bafe54c319fe16978b1a70f30fe"}
Jul 31 23:03:12.473289 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:12.473009    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-8567dd98f6-dqqc2"
Jul 31 23:03:12.495648 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:12.495562    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-8567dd98f6-dqqc2" podStartSLOduration=2.187690096 podStartE2EDuration="6.495542329s" totalImagesPullingTime="3.274234654s" totalInitContainerRuntime="1.033617579s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:03:06 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:03:12.493182083 +0000 UTC m=+376.138968393" watchObservedRunningTime="2026-07-31 23:03:12.495542329 +0000 UTC m=+376.141328639"
Jul 31 23:03:23.478974 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:23.478939    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-8567dd98f6-dqqc2"
Jul 31 23:03:26.161847 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:26.161809    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-85c4c7ff5b-mwq54"]
Jul 31 23:03:26.167187 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:26.167165    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-85c4c7ff5b-mwq54"
Jul 31 23:03:26.188883 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:26.188862    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 31 23:03:26.188972 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:26.188870    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-vjkqq\""
Jul 31 23:03:26.196597 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:26.196576    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 31 23:03:26.199664 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:26.199641    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-85c4c7ff5b-mwq54"]
Jul 31 23:03:26.229038 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:26.229015    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/378ff678-9bd2-42d8-88c8-ae81e219d8d0-tmp\") pod \"kube-auth-proxy-85c4c7ff5b-mwq54\" (UID: \"378ff678-9bd2-42d8-88c8-ae81e219d8d0\") " pod="openshift-ingress/kube-auth-proxy-85c4c7ff5b-mwq54"
Jul 31 23:03:26.229169 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:26.229045    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndkjl\" (UniqueName: \"kubernetes.io/projected/378ff678-9bd2-42d8-88c8-ae81e219d8d0-kube-api-access-ndkjl\") pod \"kube-auth-proxy-85c4c7ff5b-mwq54\" (UID: \"378ff678-9bd2-42d8-88c8-ae81e219d8d0\") " pod="openshift-ingress/kube-auth-proxy-85c4c7ff5b-mwq54"
Jul 31 23:03:26.229169 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:26.229083    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/378ff678-9bd2-42d8-88c8-ae81e219d8d0-tls-certs\") pod \"kube-auth-proxy-85c4c7ff5b-mwq54\" (UID: \"378ff678-9bd2-42d8-88c8-ae81e219d8d0\") " pod="openshift-ingress/kube-auth-proxy-85c4c7ff5b-mwq54"
Jul 31 23:03:26.329685 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:26.329657    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/378ff678-9bd2-42d8-88c8-ae81e219d8d0-tmp\") pod \"kube-auth-proxy-85c4c7ff5b-mwq54\" (UID: \"378ff678-9bd2-42d8-88c8-ae81e219d8d0\") " pod="openshift-ingress/kube-auth-proxy-85c4c7ff5b-mwq54"
Jul 31 23:03:26.329836 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:26.329687    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndkjl\" (UniqueName: \"kubernetes.io/projected/378ff678-9bd2-42d8-88c8-ae81e219d8d0-kube-api-access-ndkjl\") pod \"kube-auth-proxy-85c4c7ff5b-mwq54\" (UID: \"378ff678-9bd2-42d8-88c8-ae81e219d8d0\") " pod="openshift-ingress/kube-auth-proxy-85c4c7ff5b-mwq54"
Jul 31 23:03:26.329836 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:26.329739    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/378ff678-9bd2-42d8-88c8-ae81e219d8d0-tls-certs\") pod \"kube-auth-proxy-85c4c7ff5b-mwq54\" (UID: \"378ff678-9bd2-42d8-88c8-ae81e219d8d0\") " pod="openshift-ingress/kube-auth-proxy-85c4c7ff5b-mwq54"
Jul 31 23:03:26.329907 ip-10-0-128-243 kubenswrapper[2584]: E0731 23:03:26.329849    2584 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 31 23:03:26.329939 ip-10-0-128-243 kubenswrapper[2584]: E0731 23:03:26.329911    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/378ff678-9bd2-42d8-88c8-ae81e219d8d0-tls-certs podName:378ff678-9bd2-42d8-88c8-ae81e219d8d0 nodeName:}" failed. No retries permitted until 2026-07-31 23:03:26.829894735 +0000 UTC m=+390.475681021 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/378ff678-9bd2-42d8-88c8-ae81e219d8d0-tls-certs") pod "kube-auth-proxy-85c4c7ff5b-mwq54" (UID: "378ff678-9bd2-42d8-88c8-ae81e219d8d0") : secret "kube-auth-proxy-tls" not found
Jul 31 23:03:26.331740 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:26.331716    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/378ff678-9bd2-42d8-88c8-ae81e219d8d0-tmp\") pod \"kube-auth-proxy-85c4c7ff5b-mwq54\" (UID: \"378ff678-9bd2-42d8-88c8-ae81e219d8d0\") " pod="openshift-ingress/kube-auth-proxy-85c4c7ff5b-mwq54"
Jul 31 23:03:26.349683 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:26.349661    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndkjl\" (UniqueName: \"kubernetes.io/projected/378ff678-9bd2-42d8-88c8-ae81e219d8d0-kube-api-access-ndkjl\") pod \"kube-auth-proxy-85c4c7ff5b-mwq54\" (UID: \"378ff678-9bd2-42d8-88c8-ae81e219d8d0\") " pod="openshift-ingress/kube-auth-proxy-85c4c7ff5b-mwq54"
Jul 31 23:03:26.832793 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:26.832757    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/378ff678-9bd2-42d8-88c8-ae81e219d8d0-tls-certs\") pod \"kube-auth-proxy-85c4c7ff5b-mwq54\" (UID: \"378ff678-9bd2-42d8-88c8-ae81e219d8d0\") " pod="openshift-ingress/kube-auth-proxy-85c4c7ff5b-mwq54"
Jul 31 23:03:26.835092 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:26.835068    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/378ff678-9bd2-42d8-88c8-ae81e219d8d0-tls-certs\") pod \"kube-auth-proxy-85c4c7ff5b-mwq54\" (UID: \"378ff678-9bd2-42d8-88c8-ae81e219d8d0\") " pod="openshift-ingress/kube-auth-proxy-85c4c7ff5b-mwq54"
Jul 31 23:03:27.076488 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:27.076459    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-85c4c7ff5b-mwq54"
Jul 31 23:03:27.204233 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:27.204209    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-85c4c7ff5b-mwq54"]
Jul 31 23:03:27.206583 ip-10-0-128-243 kubenswrapper[2584]: W0731 23:03:27.206551    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod378ff678_9bd2_42d8_88c8_ae81e219d8d0.slice/crio-a9278677661b2344e883d3779001e9d7c53846420b27af99abddb3ab77ab5e92 WatchSource:0}: Error finding container a9278677661b2344e883d3779001e9d7c53846420b27af99abddb3ab77ab5e92: Status 404 returned error can't find the container with id a9278677661b2344e883d3779001e9d7c53846420b27af99abddb3ab77ab5e92
Jul 31 23:03:27.494182 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:27.494100    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-dr5kw"]
Jul 31 23:03:27.497052 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:27.497036    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-dr5kw"
Jul 31 23:03:27.499362 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:27.499341    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 31 23:03:27.499458 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:27.499437    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-tq675\""
Jul 31 23:03:27.507714 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:27.507681    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-dr5kw"]
Jul 31 23:03:27.531036 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:27.530997    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-85c4c7ff5b-mwq54" event={"ID":"378ff678-9bd2-42d8-88c8-ae81e219d8d0","Type":"ContainerStarted","Data":"a9278677661b2344e883d3779001e9d7c53846420b27af99abddb3ab77ab5e92"}
Jul 31 23:03:27.537968 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:27.537941    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8dbdab97-abc5-48d8-94ea-1f808b9d772f-tls-certs\") pod \"model-serving-api-54d89bd5fc-dr5kw\" (UID: \"8dbdab97-abc5-48d8-94ea-1f808b9d772f\") " pod="opendatahub/model-serving-api-54d89bd5fc-dr5kw"
Jul 31 23:03:27.538084 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:27.537984    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8x5p7\" (UniqueName: \"kubernetes.io/projected/8dbdab97-abc5-48d8-94ea-1f808b9d772f-kube-api-access-8x5p7\") pod \"model-serving-api-54d89bd5fc-dr5kw\" (UID: \"8dbdab97-abc5-48d8-94ea-1f808b9d772f\") " pod="opendatahub/model-serving-api-54d89bd5fc-dr5kw"
Jul 31 23:03:27.638562 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:27.638529    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8dbdab97-abc5-48d8-94ea-1f808b9d772f-tls-certs\") pod \"model-serving-api-54d89bd5fc-dr5kw\" (UID: \"8dbdab97-abc5-48d8-94ea-1f808b9d772f\") " pod="opendatahub/model-serving-api-54d89bd5fc-dr5kw"
Jul 31 23:03:27.638752 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:27.638575    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8x5p7\" (UniqueName: \"kubernetes.io/projected/8dbdab97-abc5-48d8-94ea-1f808b9d772f-kube-api-access-8x5p7\") pod \"model-serving-api-54d89bd5fc-dr5kw\" (UID: \"8dbdab97-abc5-48d8-94ea-1f808b9d772f\") " pod="opendatahub/model-serving-api-54d89bd5fc-dr5kw"
Jul 31 23:03:27.638752 ip-10-0-128-243 kubenswrapper[2584]: E0731 23:03:27.638679    2584 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 31 23:03:27.638883 ip-10-0-128-243 kubenswrapper[2584]: E0731 23:03:27.638776    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dbdab97-abc5-48d8-94ea-1f808b9d772f-tls-certs podName:8dbdab97-abc5-48d8-94ea-1f808b9d772f nodeName:}" failed. No retries permitted until 2026-07-31 23:03:28.138755707 +0000 UTC m=+391.784542009 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/8dbdab97-abc5-48d8-94ea-1f808b9d772f-tls-certs") pod "model-serving-api-54d89bd5fc-dr5kw" (UID: "8dbdab97-abc5-48d8-94ea-1f808b9d772f") : secret "model-serving-api-tls" not found
Jul 31 23:03:27.648319 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:27.648292    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8x5p7\" (UniqueName: \"kubernetes.io/projected/8dbdab97-abc5-48d8-94ea-1f808b9d772f-kube-api-access-8x5p7\") pod \"model-serving-api-54d89bd5fc-dr5kw\" (UID: \"8dbdab97-abc5-48d8-94ea-1f808b9d772f\") " pod="opendatahub/model-serving-api-54d89bd5fc-dr5kw"
Jul 31 23:03:28.141342 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:28.141303    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8dbdab97-abc5-48d8-94ea-1f808b9d772f-tls-certs\") pod \"model-serving-api-54d89bd5fc-dr5kw\" (UID: \"8dbdab97-abc5-48d8-94ea-1f808b9d772f\") " pod="opendatahub/model-serving-api-54d89bd5fc-dr5kw"
Jul 31 23:03:28.141542 ip-10-0-128-243 kubenswrapper[2584]: E0731 23:03:28.141463    2584 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 31 23:03:28.141614 ip-10-0-128-243 kubenswrapper[2584]: E0731 23:03:28.141541    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dbdab97-abc5-48d8-94ea-1f808b9d772f-tls-certs podName:8dbdab97-abc5-48d8-94ea-1f808b9d772f nodeName:}" failed. No retries permitted until 2026-07-31 23:03:29.141520467 +0000 UTC m=+392.787306757 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/8dbdab97-abc5-48d8-94ea-1f808b9d772f-tls-certs") pod "model-serving-api-54d89bd5fc-dr5kw" (UID: "8dbdab97-abc5-48d8-94ea-1f808b9d772f") : secret "model-serving-api-tls" not found
Jul 31 23:03:29.147618 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:29.147574    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8dbdab97-abc5-48d8-94ea-1f808b9d772f-tls-certs\") pod \"model-serving-api-54d89bd5fc-dr5kw\" (UID: \"8dbdab97-abc5-48d8-94ea-1f808b9d772f\") " pod="opendatahub/model-serving-api-54d89bd5fc-dr5kw"
Jul 31 23:03:29.149985 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:29.149952    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8dbdab97-abc5-48d8-94ea-1f808b9d772f-tls-certs\") pod \"model-serving-api-54d89bd5fc-dr5kw\" (UID: \"8dbdab97-abc5-48d8-94ea-1f808b9d772f\") " pod="opendatahub/model-serving-api-54d89bd5fc-dr5kw"
Jul 31 23:03:29.308512 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:29.308474    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-dr5kw"
Jul 31 23:03:30.115103 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:30.115072    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-dr5kw"]
Jul 31 23:03:30.117919 ip-10-0-128-243 kubenswrapper[2584]: W0731 23:03:30.117890    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8dbdab97_abc5_48d8_94ea_1f808b9d772f.slice/crio-979ff19e68ea781218fcf5cc315a59194d32893e0981d42489a5628ed8698363 WatchSource:0}: Error finding container 979ff19e68ea781218fcf5cc315a59194d32893e0981d42489a5628ed8698363: Status 404 returned error can't find the container with id 979ff19e68ea781218fcf5cc315a59194d32893e0981d42489a5628ed8698363
Jul 31 23:03:30.545422 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:30.545387    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-85c4c7ff5b-mwq54" event={"ID":"378ff678-9bd2-42d8-88c8-ae81e219d8d0","Type":"ContainerStarted","Data":"9deabbbe26e9a66ee3795a35756bc9d96380bd22c326852aad3b81764502e6c9"}
Jul 31 23:03:30.546590 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:30.546566    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-dr5kw" event={"ID":"8dbdab97-abc5-48d8-94ea-1f808b9d772f","Type":"ContainerStarted","Data":"979ff19e68ea781218fcf5cc315a59194d32893e0981d42489a5628ed8698363"}
Jul 31 23:03:30.562270 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:30.562225    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-85c4c7ff5b-mwq54" podStartSLOduration=1.726759374 podStartE2EDuration="4.562210986s" totalImagesPullingTime="2.835451612s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:03:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:03:30.560982859 +0000 UTC m=+394.206769182" watchObservedRunningTime="2026-07-31 23:03:30.562210986 +0000 UTC m=+394.207997295"
Jul 31 23:03:32.556775 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:32.556726    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-dr5kw" event={"ID":"8dbdab97-abc5-48d8-94ea-1f808b9d772f","Type":"ContainerStarted","Data":"65631e4cc00c07c885660005debc0be4850cc21d25b2f7d0ab5f69aab9e0ce91"}
Jul 31 23:03:32.557264 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:32.556869    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-dr5kw"
Jul 31 23:03:32.577050 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:32.576967    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-dr5kw" podStartSLOduration=3.746447616 podStartE2EDuration="5.576950953s" totalImagesPullingTime="1.830503337s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:03:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:03:32.576565201 +0000 UTC m=+396.222351509" watchObservedRunningTime="2026-07-31 23:03:32.576950953 +0000 UTC m=+396.222737263"
Jul 31 23:03:35.425343 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:35.425309    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-hwhbc"]
Jul 31 23:03:35.428485 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:35.428467    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hwhbc"
Jul 31 23:03:35.430942 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:35.430917    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 31 23:03:35.431205 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:35.431189    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 31 23:03:35.431378 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:35.431359    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 31 23:03:35.431489 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:35.431473    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-pzrd6\""
Jul 31 23:03:35.431652 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:35.431632    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 31 23:03:35.448646 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:35.448626    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-hwhbc"]
Jul 31 23:03:35.488582 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:35.488562    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f77d34f-4a8c-4282-9923-35909e938fa6-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-hwhbc\" (UID: \"6f77d34f-4a8c-4282-9923-35909e938fa6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hwhbc"
Jul 31 23:03:35.488754 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:35.488591    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pt56j\" (UniqueName: \"kubernetes.io/projected/6f77d34f-4a8c-4282-9923-35909e938fa6-kube-api-access-pt56j\") pod \"istiod-openshift-gateway-5677ccb49b-hwhbc\" (UID: \"6f77d34f-4a8c-4282-9923-35909e938fa6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hwhbc"
Jul 31 23:03:35.488754 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:35.488657    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f77d34f-4a8c-4282-9923-35909e938fa6-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-hwhbc\" (UID: \"6f77d34f-4a8c-4282-9923-35909e938fa6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hwhbc"
Jul 31 23:03:35.488754 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:35.488680    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6f77d34f-4a8c-4282-9923-35909e938fa6-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-hwhbc\" (UID: \"6f77d34f-4a8c-4282-9923-35909e938fa6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hwhbc"
Jul 31 23:03:35.488754 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:35.488746    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6f77d34f-4a8c-4282-9923-35909e938fa6-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-hwhbc\" (UID: \"6f77d34f-4a8c-4282-9923-35909e938fa6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hwhbc"
Jul 31 23:03:35.488902 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:35.488783    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6f77d34f-4a8c-4282-9923-35909e938fa6-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-hwhbc\" (UID: \"6f77d34f-4a8c-4282-9923-35909e938fa6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hwhbc"
Jul 31 23:03:35.488902 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:35.488804    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6f77d34f-4a8c-4282-9923-35909e938fa6-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-hwhbc\" (UID: \"6f77d34f-4a8c-4282-9923-35909e938fa6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hwhbc"
Jul 31 23:03:35.589482 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:35.589457    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f77d34f-4a8c-4282-9923-35909e938fa6-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-hwhbc\" (UID: \"6f77d34f-4a8c-4282-9923-35909e938fa6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hwhbc"
Jul 31 23:03:35.589635 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:35.589494    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6f77d34f-4a8c-4282-9923-35909e938fa6-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-hwhbc\" (UID: \"6f77d34f-4a8c-4282-9923-35909e938fa6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hwhbc"
Jul 31 23:03:35.589733 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:35.589677    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6f77d34f-4a8c-4282-9923-35909e938fa6-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-hwhbc\" (UID: \"6f77d34f-4a8c-4282-9923-35909e938fa6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hwhbc"
Jul 31 23:03:35.589800 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:35.589769    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6f77d34f-4a8c-4282-9923-35909e938fa6-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-hwhbc\" (UID: \"6f77d34f-4a8c-4282-9923-35909e938fa6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hwhbc"
Jul 31 23:03:35.589853 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:35.589813    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6f77d34f-4a8c-4282-9923-35909e938fa6-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-hwhbc\" (UID: \"6f77d34f-4a8c-4282-9923-35909e938fa6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hwhbc"
Jul 31 23:03:35.589904 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:35.589867    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f77d34f-4a8c-4282-9923-35909e938fa6-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-hwhbc\" (UID: \"6f77d34f-4a8c-4282-9923-35909e938fa6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hwhbc"
Jul 31 23:03:35.589904 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:35.589894    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pt56j\" (UniqueName: \"kubernetes.io/projected/6f77d34f-4a8c-4282-9923-35909e938fa6-kube-api-access-pt56j\") pod \"istiod-openshift-gateway-5677ccb49b-hwhbc\" (UID: \"6f77d34f-4a8c-4282-9923-35909e938fa6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hwhbc"
Jul 31 23:03:35.590509 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:35.590487    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6f77d34f-4a8c-4282-9923-35909e938fa6-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-hwhbc\" (UID: \"6f77d34f-4a8c-4282-9923-35909e938fa6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hwhbc"
Jul 31 23:03:35.592032 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:35.592011    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6f77d34f-4a8c-4282-9923-35909e938fa6-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-hwhbc\" (UID: \"6f77d34f-4a8c-4282-9923-35909e938fa6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hwhbc"
Jul 31 23:03:35.592250 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:35.592061    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6f77d34f-4a8c-4282-9923-35909e938fa6-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-hwhbc\" (UID: \"6f77d34f-4a8c-4282-9923-35909e938fa6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hwhbc"
Jul 31 23:03:35.592303 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:35.592252    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f77d34f-4a8c-4282-9923-35909e938fa6-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-hwhbc\" (UID: \"6f77d34f-4a8c-4282-9923-35909e938fa6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hwhbc"
Jul 31 23:03:35.592343 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:35.592331    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6f77d34f-4a8c-4282-9923-35909e938fa6-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-hwhbc\" (UID: \"6f77d34f-4a8c-4282-9923-35909e938fa6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hwhbc"
Jul 31 23:03:35.600318 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:35.600297    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f77d34f-4a8c-4282-9923-35909e938fa6-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-hwhbc\" (UID: \"6f77d34f-4a8c-4282-9923-35909e938fa6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hwhbc"
Jul 31 23:03:35.606907 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:35.606886    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pt56j\" (UniqueName: \"kubernetes.io/projected/6f77d34f-4a8c-4282-9923-35909e938fa6-kube-api-access-pt56j\") pod \"istiod-openshift-gateway-5677ccb49b-hwhbc\" (UID: \"6f77d34f-4a8c-4282-9923-35909e938fa6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hwhbc"
Jul 31 23:03:35.737447 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:35.737371    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hwhbc"
Jul 31 23:03:35.873500 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:35.873459    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-hwhbc"]
Jul 31 23:03:35.879419 ip-10-0-128-243 kubenswrapper[2584]: W0731 23:03:35.879391    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f77d34f_4a8c_4282_9923_35909e938fa6.slice/crio-19cda1463bdf6e1c77dee66912565542c02777eaff8cabac1462989559865ccb WatchSource:0}: Error finding container 19cda1463bdf6e1c77dee66912565542c02777eaff8cabac1462989559865ccb: Status 404 returned error can't find the container with id 19cda1463bdf6e1c77dee66912565542c02777eaff8cabac1462989559865ccb
Jul 31 23:03:36.575897 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:36.575853    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hwhbc" event={"ID":"6f77d34f-4a8c-4282-9923-35909e938fa6","Type":"ContainerStarted","Data":"19cda1463bdf6e1c77dee66912565542c02777eaff8cabac1462989559865ccb"}
Jul 31 23:03:38.032594 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:38.032551    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-gsrc4"]
Jul 31 23:03:38.043020 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:38.042997    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gsrc4"
Jul 31 23:03:38.045962 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:38.045931    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 31 23:03:38.046489 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:38.046470    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-6r8x5\""
Jul 31 23:03:38.059326 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:38.059299    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-gsrc4"]
Jul 31 23:03:38.110073 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:38.110046    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0f609f8c-bab9-4664-9986-ecfa0943a982-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-gsrc4\" (UID: \"0f609f8c-bab9-4664-9986-ecfa0943a982\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gsrc4"
Jul 31 23:03:38.110168 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:38.110075    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgwqf\" (UniqueName: \"kubernetes.io/projected/0f609f8c-bab9-4664-9986-ecfa0943a982-kube-api-access-mgwqf\") pod \"llmisvc-controller-manager-777c6bd7d5-gsrc4\" (UID: \"0f609f8c-bab9-4664-9986-ecfa0943a982\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gsrc4"
Jul 31 23:03:38.132469 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:38.132416    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 31 23:03:38.132541 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:38.132506    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 31 23:03:38.211473 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:38.211447    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0f609f8c-bab9-4664-9986-ecfa0943a982-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-gsrc4\" (UID: \"0f609f8c-bab9-4664-9986-ecfa0943a982\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gsrc4"
Jul 31 23:03:38.211594 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:38.211480    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgwqf\" (UniqueName: \"kubernetes.io/projected/0f609f8c-bab9-4664-9986-ecfa0943a982-kube-api-access-mgwqf\") pod \"llmisvc-controller-manager-777c6bd7d5-gsrc4\" (UID: \"0f609f8c-bab9-4664-9986-ecfa0943a982\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gsrc4"
Jul 31 23:03:38.211636 ip-10-0-128-243 kubenswrapper[2584]: E0731 23:03:38.211603    2584 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 31 23:03:38.211684 ip-10-0-128-243 kubenswrapper[2584]: E0731 23:03:38.211672    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f609f8c-bab9-4664-9986-ecfa0943a982-cert podName:0f609f8c-bab9-4664-9986-ecfa0943a982 nodeName:}" failed. No retries permitted until 2026-07-31 23:03:38.711653321 +0000 UTC m=+402.357439613 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/0f609f8c-bab9-4664-9986-ecfa0943a982-cert") pod "llmisvc-controller-manager-777c6bd7d5-gsrc4" (UID: "0f609f8c-bab9-4664-9986-ecfa0943a982") : secret "llmisvc-webhook-server-cert" not found
Jul 31 23:03:38.227288 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:38.227265    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgwqf\" (UniqueName: \"kubernetes.io/projected/0f609f8c-bab9-4664-9986-ecfa0943a982-kube-api-access-mgwqf\") pod \"llmisvc-controller-manager-777c6bd7d5-gsrc4\" (UID: \"0f609f8c-bab9-4664-9986-ecfa0943a982\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gsrc4"
Jul 31 23:03:38.586770 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:38.586735    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hwhbc" event={"ID":"6f77d34f-4a8c-4282-9923-35909e938fa6","Type":"ContainerStarted","Data":"0b9d9c4350eea022b9406c2838bf7872637bd43ec3bbdafec6dbc8d00c810b68"}
Jul 31 23:03:38.587053 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:38.586881    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hwhbc"
Jul 31 23:03:38.621063 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:38.621005    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hwhbc" podStartSLOduration=1.370243939 podStartE2EDuration="3.620983855s" totalImagesPullingTime="2.250739916s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:03:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:03:38.619041155 +0000 UTC m=+402.264827466" watchObservedRunningTime="2026-07-31 23:03:38.620983855 +0000 UTC m=+402.266770170"
Jul 31 23:03:38.716318 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:38.716275    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0f609f8c-bab9-4664-9986-ecfa0943a982-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-gsrc4\" (UID: \"0f609f8c-bab9-4664-9986-ecfa0943a982\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gsrc4"
Jul 31 23:03:38.718745 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:38.718691    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0f609f8c-bab9-4664-9986-ecfa0943a982-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-gsrc4\" (UID: \"0f609f8c-bab9-4664-9986-ecfa0943a982\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gsrc4"
Jul 31 23:03:38.954003 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:38.953918    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gsrc4"
Jul 31 23:03:39.080715 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:39.080667    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-gsrc4"]
Jul 31 23:03:39.084228 ip-10-0-128-243 kubenswrapper[2584]: W0731 23:03:39.084204    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod0f609f8c_bab9_4664_9986_ecfa0943a982.slice/crio-744f10a2127ad9ec30e29d5a74593b72fe340741a2e8d86cbfebd5e1ac2045fa WatchSource:0}: Error finding container 744f10a2127ad9ec30e29d5a74593b72fe340741a2e8d86cbfebd5e1ac2045fa: Status 404 returned error can't find the container with id 744f10a2127ad9ec30e29d5a74593b72fe340741a2e8d86cbfebd5e1ac2045fa
Jul 31 23:03:39.591641 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:39.591603    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gsrc4" event={"ID":"0f609f8c-bab9-4664-9986-ecfa0943a982","Type":"ContainerStarted","Data":"744f10a2127ad9ec30e29d5a74593b72fe340741a2e8d86cbfebd5e1ac2045fa"}
Jul 31 23:03:39.593329 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:39.593308    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hwhbc"
Jul 31 23:03:42.605540 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:42.605503    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gsrc4" event={"ID":"0f609f8c-bab9-4664-9986-ecfa0943a982","Type":"ContainerStarted","Data":"39a581fdb1995b06a43f77f66898ec9a040976d3b107a2d31d7e1665e1bbf463"}
Jul 31 23:03:42.606016 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:42.605598    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gsrc4"
Jul 31 23:03:42.621131 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:42.621082    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gsrc4" podStartSLOduration=1.2792334300000001 podStartE2EDuration="4.62106589s" totalImagesPullingTime="3.34183246s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:03:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:03:42.620670343 +0000 UTC m=+406.266456651" watchObservedRunningTime="2026-07-31 23:03:42.62106589 +0000 UTC m=+406.266852199"
Jul 31 23:03:43.565426 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:03:43.565400    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-dr5kw"
Jul 31 23:04:13.612978 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:04:13.612949    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gsrc4"
Jul 31 23:04:43.298546 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:04:43.298510    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-f4r7v"]
Jul 31 23:04:43.301765 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:04:43.301748    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-5985564776-f4r7v"
Jul 31 23:04:43.305278 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:04:43.305256    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 31 23:04:43.305549 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:04:43.305532    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 31 23:04:43.305936 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:04:43.305918    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 31 23:04:43.313867 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:04:43.313830    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-5n79f\""
Jul 31 23:04:43.334505 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:04:43.334481    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-f4r7v"]
Jul 31 23:04:43.354424 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:04:43.354398    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lf6t\" (UniqueName: \"kubernetes.io/projected/17cc18ad-0998-4f95-841a-88ec1e5f6fbf-kube-api-access-5lf6t\") pod \"dns-operator-controller-manager-5985564776-f4r7v\" (UID: \"17cc18ad-0998-4f95-841a-88ec1e5f6fbf\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-f4r7v"
Jul 31 23:04:43.455285 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:04:43.455250    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lf6t\" (UniqueName: \"kubernetes.io/projected/17cc18ad-0998-4f95-841a-88ec1e5f6fbf-kube-api-access-5lf6t\") pod \"dns-operator-controller-manager-5985564776-f4r7v\" (UID: \"17cc18ad-0998-4f95-841a-88ec1e5f6fbf\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-f4r7v"
Jul 31 23:04:43.490286 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:04:43.490255    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lf6t\" (UniqueName: \"kubernetes.io/projected/17cc18ad-0998-4f95-841a-88ec1e5f6fbf-kube-api-access-5lf6t\") pod \"dns-operator-controller-manager-5985564776-f4r7v\" (UID: \"17cc18ad-0998-4f95-841a-88ec1e5f6fbf\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-f4r7v"
Jul 31 23:04:43.611268 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:04:43.611204    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-5985564776-f4r7v"
Jul 31 23:04:43.739308 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:04:43.739279    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-f4r7v"]
Jul 31 23:04:43.741864 ip-10-0-128-243 kubenswrapper[2584]: W0731 23:04:43.741833    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17cc18ad_0998_4f95_841a_88ec1e5f6fbf.slice/crio-ff234881e19927a5d4a6695b9dc5ef1cc947521a081d34539894edbe6326686f WatchSource:0}: Error finding container ff234881e19927a5d4a6695b9dc5ef1cc947521a081d34539894edbe6326686f: Status 404 returned error can't find the container with id ff234881e19927a5d4a6695b9dc5ef1cc947521a081d34539894edbe6326686f
Jul 31 23:04:43.856977 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:04:43.856943    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-5985564776-f4r7v" event={"ID":"17cc18ad-0998-4f95-841a-88ec1e5f6fbf","Type":"ContainerStarted","Data":"ff234881e19927a5d4a6695b9dc5ef1cc947521a081d34539894edbe6326686f"}
Jul 31 23:04:45.958234 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:04:45.958199    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-trjxn"]
Jul 31 23:04:45.961691 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:04:45.961664    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-68d88948d8-trjxn"
Jul 31 23:04:45.964437 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:04:45.964414    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-4nlfq\""
Jul 31 23:04:45.975355 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:04:45.975325    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-trjxn"]
Jul 31 23:04:46.078834 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:04:46.078792    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2lmjt\" (UniqueName: \"kubernetes.io/projected/2346a376-c5a4-4062-a78f-13e2fead3c61-kube-api-access-2lmjt\") pod \"authorino-operator-68d88948d8-trjxn\" (UID: \"2346a376-c5a4-4062-a78f-13e2fead3c61\") " pod="kuadrant-system/authorino-operator-68d88948d8-trjxn"
Jul 31 23:04:46.179472 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:04:46.179451    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2lmjt\" (UniqueName: \"kubernetes.io/projected/2346a376-c5a4-4062-a78f-13e2fead3c61-kube-api-access-2lmjt\") pod \"authorino-operator-68d88948d8-trjxn\" (UID: \"2346a376-c5a4-4062-a78f-13e2fead3c61\") " pod="kuadrant-system/authorino-operator-68d88948d8-trjxn"
Jul 31 23:04:46.190723 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:04:46.190685    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2lmjt\" (UniqueName: \"kubernetes.io/projected/2346a376-c5a4-4062-a78f-13e2fead3c61-kube-api-access-2lmjt\") pod \"authorino-operator-68d88948d8-trjxn\" (UID: \"2346a376-c5a4-4062-a78f-13e2fead3c61\") " pod="kuadrant-system/authorino-operator-68d88948d8-trjxn"
Jul 31 23:04:46.274915 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:04:46.274881    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-68d88948d8-trjxn"
Jul 31 23:04:46.403805 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:04:46.403782    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-trjxn"]
Jul 31 23:04:46.405858 ip-10-0-128-243 kubenswrapper[2584]: W0731 23:04:46.405833    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2346a376_c5a4_4062_a78f_13e2fead3c61.slice/crio-d6526aff9298513a1879fc8c8d35a85f0efc8a4b25a93f86f0232d93e8c08e41 WatchSource:0}: Error finding container d6526aff9298513a1879fc8c8d35a85f0efc8a4b25a93f86f0232d93e8c08e41: Status 404 returned error can't find the container with id d6526aff9298513a1879fc8c8d35a85f0efc8a4b25a93f86f0232d93e8c08e41
Jul 31 23:04:46.875662 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:04:46.875613    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-5985564776-f4r7v" event={"ID":"17cc18ad-0998-4f95-841a-88ec1e5f6fbf","Type":"ContainerStarted","Data":"542ecc624ca336b8f2541e4f6d95c1ec6dd857d8d6e0437c91fa84d17afaf248"}
Jul 31 23:04:46.875878 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:04:46.875746    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-5985564776-f4r7v"
Jul 31 23:04:46.876982 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:04:46.876956    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-68d88948d8-trjxn" event={"ID":"2346a376-c5a4-4062-a78f-13e2fead3c61","Type":"ContainerStarted","Data":"d6526aff9298513a1879fc8c8d35a85f0efc8a4b25a93f86f0232d93e8c08e41"}
Jul 31 23:04:46.893325 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:04:46.893284    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-5985564776-f4r7v" podStartSLOduration=1.475001319 podStartE2EDuration="3.893269985s" totalImagesPullingTime="2.418268666s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:04:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:04:46.89246874 +0000 UTC m=+470.538255043" watchObservedRunningTime="2026-07-31 23:04:46.893269985 +0000 UTC m=+470.539056295"
Jul 31 23:04:48.893993 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:04:48.893960    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-68d88948d8-trjxn" event={"ID":"2346a376-c5a4-4062-a78f-13e2fead3c61","Type":"ContainerStarted","Data":"80231f89a3fe362baf2748549da66ec9541ff8f076376a8f7c4f7115e5215626"}
Jul 31 23:04:48.894350 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:04:48.894014    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-68d88948d8-trjxn"
Jul 31 23:04:48.911810 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:04:48.911756    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-68d88948d8-trjxn" podStartSLOduration=2.312682088 podStartE2EDuration="3.911738012s" totalImagesPullingTime="1.599055924s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:04:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:04:48.911580112 +0000 UTC m=+472.557366424" watchObservedRunningTime="2026-07-31 23:04:48.911738012 +0000 UTC m=+472.557524322"
Jul 31 23:04:57.884249 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:04:57.884220    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-5985564776-f4r7v"
Jul 31 23:04:59.899222 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:04:59.899194    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-68d88948d8-trjxn"
Jul 31 23:05:07.249365 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:07.249325    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-fhk5g"]
Jul 31 23:05:07.255322 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:07.255298    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-fhk5g"
Jul 31 23:05:07.257938 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:07.257916    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-mcgdw\""
Jul 31 23:05:07.265695 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:07.265663    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-fhk5g"]
Jul 31 23:05:07.365734 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:07.365663    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xd2nr\" (UniqueName: \"kubernetes.io/projected/342dc193-c375-448c-99d4-e11acc8660bb-kube-api-access-xd2nr\") pod \"kuadrant-operator-controller-manager-96db98c47-fhk5g\" (UID: \"342dc193-c375-448c-99d4-e11acc8660bb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-fhk5g"
Jul 31 23:05:07.365925 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:07.365746    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/342dc193-c375-448c-99d4-e11acc8660bb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-96db98c47-fhk5g\" (UID: \"342dc193-c375-448c-99d4-e11acc8660bb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-fhk5g"
Jul 31 23:05:07.466468 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:07.466433    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xd2nr\" (UniqueName: \"kubernetes.io/projected/342dc193-c375-448c-99d4-e11acc8660bb-kube-api-access-xd2nr\") pod \"kuadrant-operator-controller-manager-96db98c47-fhk5g\" (UID: \"342dc193-c375-448c-99d4-e11acc8660bb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-fhk5g"
Jul 31 23:05:07.466637 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:07.466478    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/342dc193-c375-448c-99d4-e11acc8660bb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-96db98c47-fhk5g\" (UID: \"342dc193-c375-448c-99d4-e11acc8660bb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-fhk5g"
Jul 31 23:05:07.466840 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:07.466824    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/342dc193-c375-448c-99d4-e11acc8660bb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-96db98c47-fhk5g\" (UID: \"342dc193-c375-448c-99d4-e11acc8660bb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-fhk5g"
Jul 31 23:05:07.476797 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:07.476767    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xd2nr\" (UniqueName: \"kubernetes.io/projected/342dc193-c375-448c-99d4-e11acc8660bb-kube-api-access-xd2nr\") pod \"kuadrant-operator-controller-manager-96db98c47-fhk5g\" (UID: \"342dc193-c375-448c-99d4-e11acc8660bb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-fhk5g"
Jul 31 23:05:07.566423 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:07.566389    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-fhk5g"
Jul 31 23:05:07.714684 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:07.714648    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-fhk5g"]
Jul 31 23:05:07.718447 ip-10-0-128-243 kubenswrapper[2584]: W0731 23:05:07.718421    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod342dc193_c375_448c_99d4_e11acc8660bb.slice/crio-33695a26c0a3b077938a0aa21299f4851ed2da9bf9936d8ecab655f561f50c39 WatchSource:0}: Error finding container 33695a26c0a3b077938a0aa21299f4851ed2da9bf9936d8ecab655f561f50c39: Status 404 returned error can't find the container with id 33695a26c0a3b077938a0aa21299f4851ed2da9bf9936d8ecab655f561f50c39
Jul 31 23:05:07.974270 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:07.974183    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-fhk5g" event={"ID":"342dc193-c375-448c-99d4-e11acc8660bb","Type":"ContainerStarted","Data":"33695a26c0a3b077938a0aa21299f4851ed2da9bf9936d8ecab655f561f50c39"}
Jul 31 23:05:07.997323 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:07.997284    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-fhk5g"]
Jul 31 23:05:08.008604 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:08.008568    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-fhk5g"]
Jul 31 23:05:08.031852 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:08.031819    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-cq894"]
Jul 31 23:05:08.040463 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:08.040438    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-cq894"
Jul 31 23:05:08.050884 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:08.050855    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-cq894"]
Jul 31 23:05:08.172383 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:08.172348    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/51cc79a4-c3f9-4943-921b-08c19ef933bf-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-96db98c47-cq894\" (UID: \"51cc79a4-c3f9-4943-921b-08c19ef933bf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-cq894"
Jul 31 23:05:08.172546 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:08.172409    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlnnn\" (UniqueName: \"kubernetes.io/projected/51cc79a4-c3f9-4943-921b-08c19ef933bf-kube-api-access-jlnnn\") pod \"kuadrant-operator-controller-manager-96db98c47-cq894\" (UID: \"51cc79a4-c3f9-4943-921b-08c19ef933bf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-cq894"
Jul 31 23:05:08.273045 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:08.272967    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlnnn\" (UniqueName: \"kubernetes.io/projected/51cc79a4-c3f9-4943-921b-08c19ef933bf-kube-api-access-jlnnn\") pod \"kuadrant-operator-controller-manager-96db98c47-cq894\" (UID: \"51cc79a4-c3f9-4943-921b-08c19ef933bf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-cq894"
Jul 31 23:05:08.273045 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:08.273036    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/51cc79a4-c3f9-4943-921b-08c19ef933bf-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-96db98c47-cq894\" (UID: \"51cc79a4-c3f9-4943-921b-08c19ef933bf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-cq894"
Jul 31 23:05:08.273438 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:08.273364    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/51cc79a4-c3f9-4943-921b-08c19ef933bf-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-96db98c47-cq894\" (UID: \"51cc79a4-c3f9-4943-921b-08c19ef933bf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-cq894"
Jul 31 23:05:08.283307 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:08.283277    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlnnn\" (UniqueName: \"kubernetes.io/projected/51cc79a4-c3f9-4943-921b-08c19ef933bf-kube-api-access-jlnnn\") pod \"kuadrant-operator-controller-manager-96db98c47-cq894\" (UID: \"51cc79a4-c3f9-4943-921b-08c19ef933bf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-cq894"
Jul 31 23:05:08.354291 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:08.354255    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-cq894"
Jul 31 23:05:08.509497 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:08.509465    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-cq894"]
Jul 31 23:05:08.511660 ip-10-0-128-243 kubenswrapper[2584]: W0731 23:05:08.511625    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51cc79a4_c3f9_4943_921b_08c19ef933bf.slice/crio-d8b10ec356c1ece22e8ea6fd515ff8ef4925d66f32c7c5160069035dd4865341 WatchSource:0}: Error finding container d8b10ec356c1ece22e8ea6fd515ff8ef4925d66f32c7c5160069035dd4865341: Status 404 returned error can't find the container with id d8b10ec356c1ece22e8ea6fd515ff8ef4925d66f32c7c5160069035dd4865341
Jul 31 23:05:08.981230 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:08.981195    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-cq894" event={"ID":"51cc79a4-c3f9-4943-921b-08c19ef933bf","Type":"ContainerStarted","Data":"d8b10ec356c1ece22e8ea6fd515ff8ef4925d66f32c7c5160069035dd4865341"}
Jul 31 23:05:11.997579 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:11.997477    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-fhk5g" podUID="342dc193-c375-448c-99d4-e11acc8660bb" containerName="manager" containerID="cri-o://e73e5a8ce04a0fb78db47ed7239912107b63fb95c016ac70b2921193520a8d96" gracePeriod=2
Jul 31 23:05:11.999258 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:11.999236    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-cq894" event={"ID":"51cc79a4-c3f9-4943-921b-08c19ef933bf","Type":"ContainerStarted","Data":"39dc9a26dbf5526ada8db8ae88a26b378cefaa490af48c9605176d5c63c23099"}
Jul 31 23:05:11.999377 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:11.999358    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-cq894"
Jul 31 23:05:12.025597 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:12.025549    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-cq894" podStartSLOduration=0.854055857 podStartE2EDuration="4.025530992s" totalImagesPullingTime="3.171475135s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:05:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:05:12.022903568 +0000 UTC m=+495.668689878" watchObservedRunningTime="2026-07-31 23:05:12.025530992 +0000 UTC m=+495.671317301"
Jul 31 23:05:12.246577 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:12.246549    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-fhk5g"
Jul 31 23:05:12.252158 ip-10-0-128-243 kubenswrapper[2584]: E0731 23:05:12.249323    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-96db98c47-fhk5g\" is forbidden: User \"system:node:ip-10-0-128-243.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-243.ec2.internal' and this object" podUID="342dc193-c375-448c-99d4-e11acc8660bb" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-fhk5g"
Jul 31 23:05:12.414302 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:12.414267    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xd2nr\" (UniqueName: \"kubernetes.io/projected/342dc193-c375-448c-99d4-e11acc8660bb-kube-api-access-xd2nr\") pod \"342dc193-c375-448c-99d4-e11acc8660bb\" (UID: \"342dc193-c375-448c-99d4-e11acc8660bb\") "
Jul 31 23:05:12.414469 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:12.414311    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/342dc193-c375-448c-99d4-e11acc8660bb-extensions-socket-volume\") pod \"342dc193-c375-448c-99d4-e11acc8660bb\" (UID: \"342dc193-c375-448c-99d4-e11acc8660bb\") "
Jul 31 23:05:12.414621 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:12.414598    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/342dc193-c375-448c-99d4-e11acc8660bb-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "342dc193-c375-448c-99d4-e11acc8660bb" (UID: "342dc193-c375-448c-99d4-e11acc8660bb"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 23:05:12.416424 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:12.416395    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/342dc193-c375-448c-99d4-e11acc8660bb-kube-api-access-xd2nr" (OuterVolumeSpecName: "kube-api-access-xd2nr") pod "342dc193-c375-448c-99d4-e11acc8660bb" (UID: "342dc193-c375-448c-99d4-e11acc8660bb"). InnerVolumeSpecName "kube-api-access-xd2nr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 23:05:12.515159 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:12.515064    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xd2nr\" (UniqueName: \"kubernetes.io/projected/342dc193-c375-448c-99d4-e11acc8660bb-kube-api-access-xd2nr\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:05:12.515159 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:12.515107    2584 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/342dc193-c375-448c-99d4-e11acc8660bb-extensions-socket-volume\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:05:12.890761 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:12.890727    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="342dc193-c375-448c-99d4-e11acc8660bb" path="/var/lib/kubelet/pods/342dc193-c375-448c-99d4-e11acc8660bb/volumes"
Jul 31 23:05:13.005487 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:13.005452    2584 generic.go:356] "Generic (PLEG): container finished" podID="342dc193-c375-448c-99d4-e11acc8660bb" containerID="e73e5a8ce04a0fb78db47ed7239912107b63fb95c016ac70b2921193520a8d96" exitCode=2
Jul 31 23:05:13.005943 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:13.005494    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-fhk5g"
Jul 31 23:05:13.005943 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:13.005564    2584 scope.go:117] "RemoveContainer" containerID="e73e5a8ce04a0fb78db47ed7239912107b63fb95c016ac70b2921193520a8d96"
Jul 31 23:05:13.010505 ip-10-0-128-243 kubenswrapper[2584]: E0731 23:05:13.010478    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-96db98c47-fhk5g\" is forbidden: User \"system:node:ip-10-0-128-243.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-243.ec2.internal' and this object" podUID="342dc193-c375-448c-99d4-e11acc8660bb" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-fhk5g"
Jul 31 23:05:13.013660 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:13.013645    2584 scope.go:117] "RemoveContainer" containerID="e73e5a8ce04a0fb78db47ed7239912107b63fb95c016ac70b2921193520a8d96"
Jul 31 23:05:13.013958 ip-10-0-128-243 kubenswrapper[2584]: E0731 23:05:13.013939    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e73e5a8ce04a0fb78db47ed7239912107b63fb95c016ac70b2921193520a8d96\": container with ID starting with e73e5a8ce04a0fb78db47ed7239912107b63fb95c016ac70b2921193520a8d96 not found: ID does not exist" containerID="e73e5a8ce04a0fb78db47ed7239912107b63fb95c016ac70b2921193520a8d96"
Jul 31 23:05:13.014018 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:13.013972    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e73e5a8ce04a0fb78db47ed7239912107b63fb95c016ac70b2921193520a8d96"} err="failed to get container status \"e73e5a8ce04a0fb78db47ed7239912107b63fb95c016ac70b2921193520a8d96\": rpc error: code = NotFound desc = could not find container \"e73e5a8ce04a0fb78db47ed7239912107b63fb95c016ac70b2921193520a8d96\": container with ID starting with e73e5a8ce04a0fb78db47ed7239912107b63fb95c016ac70b2921193520a8d96 not found: ID does not exist"
Jul 31 23:05:23.008406 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:23.008375    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-cq894"
Jul 31 23:05:35.503604 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:35.503536    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-cq894"]
Jul 31 23:05:35.504056 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:35.503779    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-cq894" podUID="51cc79a4-c3f9-4943-921b-08c19ef933bf" containerName="manager" containerID="cri-o://39dc9a26dbf5526ada8db8ae88a26b378cefaa490af48c9605176d5c63c23099" gracePeriod=10
Jul 31 23:05:35.753784 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:35.753723    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-cq894"
Jul 31 23:05:35.809177 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:35.809146    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jlnnn\" (UniqueName: \"kubernetes.io/projected/51cc79a4-c3f9-4943-921b-08c19ef933bf-kube-api-access-jlnnn\") pod \"51cc79a4-c3f9-4943-921b-08c19ef933bf\" (UID: \"51cc79a4-c3f9-4943-921b-08c19ef933bf\") "
Jul 31 23:05:35.809319 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:35.809189    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/51cc79a4-c3f9-4943-921b-08c19ef933bf-extensions-socket-volume\") pod \"51cc79a4-c3f9-4943-921b-08c19ef933bf\" (UID: \"51cc79a4-c3f9-4943-921b-08c19ef933bf\") "
Jul 31 23:05:35.809663 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:35.809639    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51cc79a4-c3f9-4943-921b-08c19ef933bf-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "51cc79a4-c3f9-4943-921b-08c19ef933bf" (UID: "51cc79a4-c3f9-4943-921b-08c19ef933bf"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 23:05:35.811142 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:35.811117    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51cc79a4-c3f9-4943-921b-08c19ef933bf-kube-api-access-jlnnn" (OuterVolumeSpecName: "kube-api-access-jlnnn") pod "51cc79a4-c3f9-4943-921b-08c19ef933bf" (UID: "51cc79a4-c3f9-4943-921b-08c19ef933bf"). InnerVolumeSpecName "kube-api-access-jlnnn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 23:05:35.910183 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:35.910143    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jlnnn\" (UniqueName: \"kubernetes.io/projected/51cc79a4-c3f9-4943-921b-08c19ef933bf-kube-api-access-jlnnn\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:05:35.910183 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:35.910178    2584 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/51cc79a4-c3f9-4943-921b-08c19ef933bf-extensions-socket-volume\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:05:36.109086 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:36.109056    2584 generic.go:356] "Generic (PLEG): container finished" podID="51cc79a4-c3f9-4943-921b-08c19ef933bf" containerID="39dc9a26dbf5526ada8db8ae88a26b378cefaa490af48c9605176d5c63c23099" exitCode=0
Jul 31 23:05:36.109243 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:36.109102    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-cq894"
Jul 31 23:05:36.109243 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:36.109125    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-cq894" event={"ID":"51cc79a4-c3f9-4943-921b-08c19ef933bf","Type":"ContainerDied","Data":"39dc9a26dbf5526ada8db8ae88a26b378cefaa490af48c9605176d5c63c23099"}
Jul 31 23:05:36.109243 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:36.109170    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-cq894" event={"ID":"51cc79a4-c3f9-4943-921b-08c19ef933bf","Type":"ContainerDied","Data":"d8b10ec356c1ece22e8ea6fd515ff8ef4925d66f32c7c5160069035dd4865341"}
Jul 31 23:05:36.109243 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:36.109191    2584 scope.go:117] "RemoveContainer" containerID="39dc9a26dbf5526ada8db8ae88a26b378cefaa490af48c9605176d5c63c23099"
Jul 31 23:05:36.117466 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:36.117436    2584 scope.go:117] "RemoveContainer" containerID="39dc9a26dbf5526ada8db8ae88a26b378cefaa490af48c9605176d5c63c23099"
Jul 31 23:05:36.117759 ip-10-0-128-243 kubenswrapper[2584]: E0731 23:05:36.117737    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39dc9a26dbf5526ada8db8ae88a26b378cefaa490af48c9605176d5c63c23099\": container with ID starting with 39dc9a26dbf5526ada8db8ae88a26b378cefaa490af48c9605176d5c63c23099 not found: ID does not exist" containerID="39dc9a26dbf5526ada8db8ae88a26b378cefaa490af48c9605176d5c63c23099"
Jul 31 23:05:36.117820 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:36.117766    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39dc9a26dbf5526ada8db8ae88a26b378cefaa490af48c9605176d5c63c23099"} err="failed to get container status \"39dc9a26dbf5526ada8db8ae88a26b378cefaa490af48c9605176d5c63c23099\": rpc error: code = NotFound desc = could not find container \"39dc9a26dbf5526ada8db8ae88a26b378cefaa490af48c9605176d5c63c23099\": container with ID starting with 39dc9a26dbf5526ada8db8ae88a26b378cefaa490af48c9605176d5c63c23099 not found: ID does not exist"
Jul 31 23:05:36.132252 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:36.132226    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-cq894"]
Jul 31 23:05:36.143162 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:36.143139    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-cq894"]
Jul 31 23:05:36.890420 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:36.890387    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51cc79a4-c3f9-4943-921b-08c19ef933bf" path="/var/lib/kubelet/pods/51cc79a4-c3f9-4943-921b-08c19ef933bf/volumes"
Jul 31 23:05:52.881230 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:52.881156    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-zktdg"]
Jul 31 23:05:52.881638 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:52.881464    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="342dc193-c375-448c-99d4-e11acc8660bb" containerName="manager"
Jul 31 23:05:52.881638 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:52.881475    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="342dc193-c375-448c-99d4-e11acc8660bb" containerName="manager"
Jul 31 23:05:52.881638 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:52.881482    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51cc79a4-c3f9-4943-921b-08c19ef933bf" containerName="manager"
Jul 31 23:05:52.881638 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:52.881487    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="51cc79a4-c3f9-4943-921b-08c19ef933bf" containerName="manager"
Jul 31 23:05:52.881638 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:52.881543    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="51cc79a4-c3f9-4943-921b-08c19ef933bf" containerName="manager"
Jul 31 23:05:52.881638 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:52.881552    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="342dc193-c375-448c-99d4-e11acc8660bb" containerName="manager"
Jul 31 23:05:52.883837 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:52.883821    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-zktdg"
Jul 31 23:05:52.886662 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:52.886643    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-9qxhs\""
Jul 31 23:05:52.886793 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:52.886669    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 31 23:05:52.893442 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:52.893415    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-zktdg"]
Jul 31 23:05:52.945262 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:52.945231    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/875aa77a-50ed-4928-b9a8-4ce032b50f4e-config-file\") pod \"limitador-limitador-859c4d96d6-zktdg\" (UID: \"875aa77a-50ed-4928-b9a8-4ce032b50f4e\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-zktdg"
Jul 31 23:05:52.945404 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:52.945273    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rc85n\" (UniqueName: \"kubernetes.io/projected/875aa77a-50ed-4928-b9a8-4ce032b50f4e-kube-api-access-rc85n\") pod \"limitador-limitador-859c4d96d6-zktdg\" (UID: \"875aa77a-50ed-4928-b9a8-4ce032b50f4e\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-zktdg"
Jul 31 23:05:52.978481 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:52.978446    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-zktdg"]
Jul 31 23:05:53.046641 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:53.046611    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/875aa77a-50ed-4928-b9a8-4ce032b50f4e-config-file\") pod \"limitador-limitador-859c4d96d6-zktdg\" (UID: \"875aa77a-50ed-4928-b9a8-4ce032b50f4e\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-zktdg"
Jul 31 23:05:53.046815 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:53.046648    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rc85n\" (UniqueName: \"kubernetes.io/projected/875aa77a-50ed-4928-b9a8-4ce032b50f4e-kube-api-access-rc85n\") pod \"limitador-limitador-859c4d96d6-zktdg\" (UID: \"875aa77a-50ed-4928-b9a8-4ce032b50f4e\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-zktdg"
Jul 31 23:05:53.047324 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:53.047306    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/875aa77a-50ed-4928-b9a8-4ce032b50f4e-config-file\") pod \"limitador-limitador-859c4d96d6-zktdg\" (UID: \"875aa77a-50ed-4928-b9a8-4ce032b50f4e\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-zktdg"
Jul 31 23:05:53.054567 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:53.054540    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rc85n\" (UniqueName: \"kubernetes.io/projected/875aa77a-50ed-4928-b9a8-4ce032b50f4e-kube-api-access-rc85n\") pod \"limitador-limitador-859c4d96d6-zktdg\" (UID: \"875aa77a-50ed-4928-b9a8-4ce032b50f4e\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-zktdg"
Jul 31 23:05:53.195848 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:53.195774    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-zktdg"
Jul 31 23:05:53.336537 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:53.336488    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-zktdg"]
Jul 31 23:05:53.341114 ip-10-0-128-243 kubenswrapper[2584]: W0731 23:05:53.341087    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod875aa77a_50ed_4928_b9a8_4ce032b50f4e.slice/crio-e5bf1a20bd6d200c9a3c2bd5079989a70324c3caed3b68cb98c0fe5513789053 WatchSource:0}: Error finding container e5bf1a20bd6d200c9a3c2bd5079989a70324c3caed3b68cb98c0fe5513789053: Status 404 returned error can't find the container with id e5bf1a20bd6d200c9a3c2bd5079989a70324c3caed3b68cb98c0fe5513789053
Jul 31 23:05:54.185081 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:54.185038    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-zktdg" event={"ID":"875aa77a-50ed-4928-b9a8-4ce032b50f4e","Type":"ContainerStarted","Data":"e5bf1a20bd6d200c9a3c2bd5079989a70324c3caed3b68cb98c0fe5513789053"}
Jul 31 23:05:55.191260 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:55.191221    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-zktdg" event={"ID":"875aa77a-50ed-4928-b9a8-4ce032b50f4e","Type":"ContainerStarted","Data":"58fbe98d89b68a3874c0226db132d43746aafe74111dbcf58b305dd45292e8e7"}
Jul 31 23:05:55.191659 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:55.191364    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-859c4d96d6-zktdg"
Jul 31 23:05:55.208154 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:05:55.208109    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-859c4d96d6-zktdg" podStartSLOduration=1.840563429 podStartE2EDuration="3.208096291s" totalImagesPullingTime="1.367532862s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:05:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:05:55.20697841 +0000 UTC m=+538.852764720" watchObservedRunningTime="2026-07-31 23:05:55.208096291 +0000 UTC m=+538.853882600"
Jul 31 23:06:06.197930 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:06.197895    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-859c4d96d6-zktdg"
Jul 31 23:06:08.097073 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:08.097036    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-zktdg"]
Jul 31 23:06:08.097450 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:08.097257    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-859c4d96d6-zktdg" podUID="875aa77a-50ed-4928-b9a8-4ce032b50f4e" containerName="limitador" containerID="cri-o://58fbe98d89b68a3874c0226db132d43746aafe74111dbcf58b305dd45292e8e7" gracePeriod=30
Jul 31 23:06:08.639151 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:08.639125    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-zktdg"
Jul 31 23:06:08.787801 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:08.787692    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rc85n\" (UniqueName: \"kubernetes.io/projected/875aa77a-50ed-4928-b9a8-4ce032b50f4e-kube-api-access-rc85n\") pod \"875aa77a-50ed-4928-b9a8-4ce032b50f4e\" (UID: \"875aa77a-50ed-4928-b9a8-4ce032b50f4e\") "
Jul 31 23:06:08.787801 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:08.787779    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/875aa77a-50ed-4928-b9a8-4ce032b50f4e-config-file\") pod \"875aa77a-50ed-4928-b9a8-4ce032b50f4e\" (UID: \"875aa77a-50ed-4928-b9a8-4ce032b50f4e\") "
Jul 31 23:06:08.788212 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:08.788184    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/875aa77a-50ed-4928-b9a8-4ce032b50f4e-config-file" (OuterVolumeSpecName: "config-file") pod "875aa77a-50ed-4928-b9a8-4ce032b50f4e" (UID: "875aa77a-50ed-4928-b9a8-4ce032b50f4e"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:06:08.789917 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:08.789893    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/875aa77a-50ed-4928-b9a8-4ce032b50f4e-kube-api-access-rc85n" (OuterVolumeSpecName: "kube-api-access-rc85n") pod "875aa77a-50ed-4928-b9a8-4ce032b50f4e" (UID: "875aa77a-50ed-4928-b9a8-4ce032b50f4e"). InnerVolumeSpecName "kube-api-access-rc85n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 23:06:08.888813 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:08.888781    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rc85n\" (UniqueName: \"kubernetes.io/projected/875aa77a-50ed-4928-b9a8-4ce032b50f4e-kube-api-access-rc85n\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:06:08.888813 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:08.888805    2584 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/875aa77a-50ed-4928-b9a8-4ce032b50f4e-config-file\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:06:09.252212 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:09.252179    2584 generic.go:356] "Generic (PLEG): container finished" podID="875aa77a-50ed-4928-b9a8-4ce032b50f4e" containerID="58fbe98d89b68a3874c0226db132d43746aafe74111dbcf58b305dd45292e8e7" exitCode=0
Jul 31 23:06:09.252569 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:09.252217    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-zktdg" event={"ID":"875aa77a-50ed-4928-b9a8-4ce032b50f4e","Type":"ContainerDied","Data":"58fbe98d89b68a3874c0226db132d43746aafe74111dbcf58b305dd45292e8e7"}
Jul 31 23:06:09.252569 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:09.252224    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-zktdg"
Jul 31 23:06:09.252569 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:09.252256    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-zktdg" event={"ID":"875aa77a-50ed-4928-b9a8-4ce032b50f4e","Type":"ContainerDied","Data":"e5bf1a20bd6d200c9a3c2bd5079989a70324c3caed3b68cb98c0fe5513789053"}
Jul 31 23:06:09.252569 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:09.252278    2584 scope.go:117] "RemoveContainer" containerID="58fbe98d89b68a3874c0226db132d43746aafe74111dbcf58b305dd45292e8e7"
Jul 31 23:06:09.260359 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:09.260343    2584 scope.go:117] "RemoveContainer" containerID="58fbe98d89b68a3874c0226db132d43746aafe74111dbcf58b305dd45292e8e7"
Jul 31 23:06:09.260600 ip-10-0-128-243 kubenswrapper[2584]: E0731 23:06:09.260578    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58fbe98d89b68a3874c0226db132d43746aafe74111dbcf58b305dd45292e8e7\": container with ID starting with 58fbe98d89b68a3874c0226db132d43746aafe74111dbcf58b305dd45292e8e7 not found: ID does not exist" containerID="58fbe98d89b68a3874c0226db132d43746aafe74111dbcf58b305dd45292e8e7"
Jul 31 23:06:09.260656 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:09.260608    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58fbe98d89b68a3874c0226db132d43746aafe74111dbcf58b305dd45292e8e7"} err="failed to get container status \"58fbe98d89b68a3874c0226db132d43746aafe74111dbcf58b305dd45292e8e7\": rpc error: code = NotFound desc = could not find container \"58fbe98d89b68a3874c0226db132d43746aafe74111dbcf58b305dd45292e8e7\": container with ID starting with 58fbe98d89b68a3874c0226db132d43746aafe74111dbcf58b305dd45292e8e7 not found: ID does not exist"
Jul 31 23:06:09.269276 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:09.269253    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-zktdg"]
Jul 31 23:06:09.274526 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:09.274507    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-zktdg"]
Jul 31 23:06:10.891033 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:10.891001    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="875aa77a-50ed-4928-b9a8-4ce032b50f4e" path="/var/lib/kubelet/pods/875aa77a-50ed-4928-b9a8-4ce032b50f4e/volumes"
Jul 31 23:06:13.362897 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:13.362861    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-xjm54"]
Jul 31 23:06:13.363327 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:13.363311    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="875aa77a-50ed-4928-b9a8-4ce032b50f4e" containerName="limitador"
Jul 31 23:06:13.363377 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:13.363331    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="875aa77a-50ed-4928-b9a8-4ce032b50f4e" containerName="limitador"
Jul 31 23:06:13.363427 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:13.363414    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="875aa77a-50ed-4928-b9a8-4ce032b50f4e" containerName="limitador"
Jul 31 23:06:13.368112 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:13.368093    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-xjm54"
Jul 31 23:06:13.371066 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:13.371046    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 31 23:06:13.371185 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:13.371068    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 31 23:06:13.371683 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:13.371659    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-lmrw8\""
Jul 31 23:06:13.371805 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:13.371689    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 31 23:06:13.372401 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:13.372376    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-xjm54"]
Jul 31 23:06:13.528799 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:13.528765    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmmpw\" (UniqueName: \"kubernetes.io/projected/282883c3-8777-4c49-9ca0-81581f0d33d5-kube-api-access-wmmpw\") pod \"postgres-69fb4594cf-xjm54\" (UID: \"282883c3-8777-4c49-9ca0-81581f0d33d5\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-xjm54"
Jul 31 23:06:13.528961 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:13.528890    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/282883c3-8777-4c49-9ca0-81581f0d33d5-data\") pod \"postgres-69fb4594cf-xjm54\" (UID: \"282883c3-8777-4c49-9ca0-81581f0d33d5\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-xjm54"
Jul 31 23:06:13.629636 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:13.629562    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmmpw\" (UniqueName: \"kubernetes.io/projected/282883c3-8777-4c49-9ca0-81581f0d33d5-kube-api-access-wmmpw\") pod \"postgres-69fb4594cf-xjm54\" (UID: \"282883c3-8777-4c49-9ca0-81581f0d33d5\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-xjm54"
Jul 31 23:06:13.629823 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:13.629745    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/282883c3-8777-4c49-9ca0-81581f0d33d5-data\") pod \"postgres-69fb4594cf-xjm54\" (UID: \"282883c3-8777-4c49-9ca0-81581f0d33d5\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-xjm54"
Jul 31 23:06:13.630075 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:13.630058    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/282883c3-8777-4c49-9ca0-81581f0d33d5-data\") pod \"postgres-69fb4594cf-xjm54\" (UID: \"282883c3-8777-4c49-9ca0-81581f0d33d5\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-xjm54"
Jul 31 23:06:13.649077 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:13.649053    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmmpw\" (UniqueName: \"kubernetes.io/projected/282883c3-8777-4c49-9ca0-81581f0d33d5-kube-api-access-wmmpw\") pod \"postgres-69fb4594cf-xjm54\" (UID: \"282883c3-8777-4c49-9ca0-81581f0d33d5\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-xjm54"
Jul 31 23:06:13.679861 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:13.679839    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-xjm54"
Jul 31 23:06:13.805364 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:13.805214    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-xjm54"]
Jul 31 23:06:13.808007 ip-10-0-128-243 kubenswrapper[2584]: W0731 23:06:13.807979    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod282883c3_8777_4c49_9ca0_81581f0d33d5.slice/crio-f47af8072b574c91d8c2c97d9a20b529c9f06ef4bfd5df77058728fe93161414 WatchSource:0}: Error finding container f47af8072b574c91d8c2c97d9a20b529c9f06ef4bfd5df77058728fe93161414: Status 404 returned error can't find the container with id f47af8072b574c91d8c2c97d9a20b529c9f06ef4bfd5df77058728fe93161414
Jul 31 23:06:14.277509 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:14.277475    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-xjm54" event={"ID":"282883c3-8777-4c49-9ca0-81581f0d33d5","Type":"ContainerStarted","Data":"f47af8072b574c91d8c2c97d9a20b529c9f06ef4bfd5df77058728fe93161414"}
Jul 31 23:06:19.301649 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:19.301609    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-xjm54" event={"ID":"282883c3-8777-4c49-9ca0-81581f0d33d5","Type":"ContainerStarted","Data":"232968aa8e6dffeff59e5474b342cbde467657f8a6dce21be75ae1f8636bfffe"}
Jul 31 23:06:19.302013 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:19.301741    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-xjm54"
Jul 31 23:06:19.318393 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:19.318348    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-xjm54" podStartSLOduration=1.665976573 podStartE2EDuration="6.318335072s" totalImagesPullingTime="4.652358499s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:06:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:06:19.317686368 +0000 UTC m=+562.963472655" watchObservedRunningTime="2026-07-31 23:06:19.318335072 +0000 UTC m=+562.964121381"
Jul 31 23:06:25.334040 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:25.334018    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-xjm54"
Jul 31 23:06:32.280035 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:32.279999    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-694fdffc9b-tj5r9"]
Jul 31 23:06:32.297516 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:32.297489    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-694fdffc9b-tj5r9"
Jul 31 23:06:32.302445 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:32.302406    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 31 23:06:32.302588 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:32.302519    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 31 23:06:32.303374 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:32.303351    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-rfzlf\""
Jul 31 23:06:32.305029 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:32.304653    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-694fdffc9b-tj5r9"]
Jul 31 23:06:32.369761 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:32.369727    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e14742e8-0a27-4e24-970d-45522eb1504f-webhook-cert\") pod \"maas-controller-694fdffc9b-tj5r9\" (UID: \"e14742e8-0a27-4e24-970d-45522eb1504f\") " pod="opendatahub/maas-controller-694fdffc9b-tj5r9"
Jul 31 23:06:32.369929 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:32.369769    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jh768\" (UniqueName: \"kubernetes.io/projected/e14742e8-0a27-4e24-970d-45522eb1504f-kube-api-access-jh768\") pod \"maas-controller-694fdffc9b-tj5r9\" (UID: \"e14742e8-0a27-4e24-970d-45522eb1504f\") " pod="opendatahub/maas-controller-694fdffc9b-tj5r9"
Jul 31 23:06:32.451187 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:32.451151    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-74f5bd9766-jmq85"]
Jul 31 23:06:32.453421 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:32.453405    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74f5bd9766-jmq85"
Jul 31 23:06:32.462199 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:32.462175    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-74f5bd9766-jmq85"]
Jul 31 23:06:32.470816 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:32.470795    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4rxk\" (UniqueName: \"kubernetes.io/projected/405ea669-5dae-46ad-b64f-59a9364f222b-kube-api-access-c4rxk\") pod \"maas-controller-74f5bd9766-jmq85\" (UID: \"405ea669-5dae-46ad-b64f-59a9364f222b\") " pod="opendatahub/maas-controller-74f5bd9766-jmq85"
Jul 31 23:06:32.470946 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:32.470861    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e14742e8-0a27-4e24-970d-45522eb1504f-webhook-cert\") pod \"maas-controller-694fdffc9b-tj5r9\" (UID: \"e14742e8-0a27-4e24-970d-45522eb1504f\") " pod="opendatahub/maas-controller-694fdffc9b-tj5r9"
Jul 31 23:06:32.470946 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:32.470902    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jh768\" (UniqueName: \"kubernetes.io/projected/e14742e8-0a27-4e24-970d-45522eb1504f-kube-api-access-jh768\") pod \"maas-controller-694fdffc9b-tj5r9\" (UID: \"e14742e8-0a27-4e24-970d-45522eb1504f\") " pod="opendatahub/maas-controller-694fdffc9b-tj5r9"
Jul 31 23:06:32.471054 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:32.470960    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/405ea669-5dae-46ad-b64f-59a9364f222b-webhook-cert\") pod \"maas-controller-74f5bd9766-jmq85\" (UID: \"405ea669-5dae-46ad-b64f-59a9364f222b\") " pod="opendatahub/maas-controller-74f5bd9766-jmq85"
Jul 31 23:06:32.473190 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:32.473164    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e14742e8-0a27-4e24-970d-45522eb1504f-webhook-cert\") pod \"maas-controller-694fdffc9b-tj5r9\" (UID: \"e14742e8-0a27-4e24-970d-45522eb1504f\") " pod="opendatahub/maas-controller-694fdffc9b-tj5r9"
Jul 31 23:06:32.486483 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:32.486454    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jh768\" (UniqueName: \"kubernetes.io/projected/e14742e8-0a27-4e24-970d-45522eb1504f-kube-api-access-jh768\") pod \"maas-controller-694fdffc9b-tj5r9\" (UID: \"e14742e8-0a27-4e24-970d-45522eb1504f\") " pod="opendatahub/maas-controller-694fdffc9b-tj5r9"
Jul 31 23:06:32.572081 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:32.572051    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/405ea669-5dae-46ad-b64f-59a9364f222b-webhook-cert\") pod \"maas-controller-74f5bd9766-jmq85\" (UID: \"405ea669-5dae-46ad-b64f-59a9364f222b\") " pod="opendatahub/maas-controller-74f5bd9766-jmq85"
Jul 31 23:06:32.572219 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:32.572101    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4rxk\" (UniqueName: \"kubernetes.io/projected/405ea669-5dae-46ad-b64f-59a9364f222b-kube-api-access-c4rxk\") pod \"maas-controller-74f5bd9766-jmq85\" (UID: \"405ea669-5dae-46ad-b64f-59a9364f222b\") " pod="opendatahub/maas-controller-74f5bd9766-jmq85"
Jul 31 23:06:32.574405 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:32.574382    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/405ea669-5dae-46ad-b64f-59a9364f222b-webhook-cert\") pod \"maas-controller-74f5bd9766-jmq85\" (UID: \"405ea669-5dae-46ad-b64f-59a9364f222b\") " pod="opendatahub/maas-controller-74f5bd9766-jmq85"
Jul 31 23:06:32.580297 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:32.580275    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4rxk\" (UniqueName: \"kubernetes.io/projected/405ea669-5dae-46ad-b64f-59a9364f222b-kube-api-access-c4rxk\") pod \"maas-controller-74f5bd9766-jmq85\" (UID: \"405ea669-5dae-46ad-b64f-59a9364f222b\") " pod="opendatahub/maas-controller-74f5bd9766-jmq85"
Jul 31 23:06:32.609117 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:32.609096    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-694fdffc9b-tj5r9"
Jul 31 23:06:32.732777 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:32.732744    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-694fdffc9b-tj5r9"]
Jul 31 23:06:32.734781 ip-10-0-128-243 kubenswrapper[2584]: W0731 23:06:32.734754    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode14742e8_0a27_4e24_970d_45522eb1504f.slice/crio-49f80249c360419f2536a831a52f274937e7c367c80573a603ecd7169f61129d WatchSource:0}: Error finding container 49f80249c360419f2536a831a52f274937e7c367c80573a603ecd7169f61129d: Status 404 returned error can't find the container with id 49f80249c360419f2536a831a52f274937e7c367c80573a603ecd7169f61129d
Jul 31 23:06:32.764575 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:32.764545    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74f5bd9766-jmq85"
Jul 31 23:06:32.892591 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:32.892567    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-74f5bd9766-jmq85"]
Jul 31 23:06:32.893507 ip-10-0-128-243 kubenswrapper[2584]: W0731 23:06:32.893482    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod405ea669_5dae_46ad_b64f_59a9364f222b.slice/crio-5e455ac32334405083d495b28fa345d195512a1688410903a331cf3f23aca345 WatchSource:0}: Error finding container 5e455ac32334405083d495b28fa345d195512a1688410903a331cf3f23aca345: Status 404 returned error can't find the container with id 5e455ac32334405083d495b28fa345d195512a1688410903a331cf3f23aca345
Jul 31 23:06:33.364484 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:33.364444    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74f5bd9766-jmq85" event={"ID":"405ea669-5dae-46ad-b64f-59a9364f222b","Type":"ContainerStarted","Data":"5e455ac32334405083d495b28fa345d195512a1688410903a331cf3f23aca345"}
Jul 31 23:06:33.366398 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:33.366369    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-694fdffc9b-tj5r9" event={"ID":"e14742e8-0a27-4e24-970d-45522eb1504f","Type":"ContainerStarted","Data":"49f80249c360419f2536a831a52f274937e7c367c80573a603ecd7169f61129d"}
Jul 31 23:06:34.373877 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:34.373836    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74f5bd9766-jmq85" event={"ID":"405ea669-5dae-46ad-b64f-59a9364f222b","Type":"ContainerStarted","Data":"dd739d8053c878b7c2477ea0dad1310f15b47bd4730fbaba69bcf91826fc9063"}
Jul 31 23:06:34.374332 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:34.374001    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-74f5bd9766-jmq85"
Jul 31 23:06:34.375423 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:34.375386    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-694fdffc9b-tj5r9" event={"ID":"e14742e8-0a27-4e24-970d-45522eb1504f","Type":"ContainerStarted","Data":"95348da1b9594e3ecbb8d5ac439622c3f18e4998eb3fc4d0a20343289b00541d"}
Jul 31 23:06:34.375557 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:34.375519    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-694fdffc9b-tj5r9"
Jul 31 23:06:34.395934 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:34.395893    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-74f5bd9766-jmq85" podStartSLOduration=1.084569646 podStartE2EDuration="2.395878767s" totalImagesPullingTime="1.311309121s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:06:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:06:34.39493762 +0000 UTC m=+578.040723930" watchObservedRunningTime="2026-07-31 23:06:34.395878767 +0000 UTC m=+578.041665078"
Jul 31 23:06:34.414719 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:34.414657    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-694fdffc9b-tj5r9" podStartSLOduration=0.955972574 podStartE2EDuration="2.41464175s" totalImagesPullingTime="1.458669176s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:06:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:06:34.412617598 +0000 UTC m=+578.058403908" watchObservedRunningTime="2026-07-31 23:06:34.41464175 +0000 UTC m=+578.060428072"
Jul 31 23:06:45.382206 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:45.382176    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-694fdffc9b-tj5r9"
Jul 31 23:06:45.382667 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:45.382340    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-74f5bd9766-jmq85"
Jul 31 23:06:45.440309 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:45.440279    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-694fdffc9b-tj5r9"]
Jul 31 23:06:45.440485 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:45.440465    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-694fdffc9b-tj5r9" podUID="e14742e8-0a27-4e24-970d-45522eb1504f" containerName="manager" containerID="cri-o://95348da1b9594e3ecbb8d5ac439622c3f18e4998eb3fc4d0a20343289b00541d" gracePeriod=30
Jul 31 23:06:45.688004 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:45.687979    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-694fdffc9b-tj5r9"
Jul 31 23:06:45.793364 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:45.793327    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e14742e8-0a27-4e24-970d-45522eb1504f-webhook-cert\") pod \"e14742e8-0a27-4e24-970d-45522eb1504f\" (UID: \"e14742e8-0a27-4e24-970d-45522eb1504f\") "
Jul 31 23:06:45.793558 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:45.793380    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jh768\" (UniqueName: \"kubernetes.io/projected/e14742e8-0a27-4e24-970d-45522eb1504f-kube-api-access-jh768\") pod \"e14742e8-0a27-4e24-970d-45522eb1504f\" (UID: \"e14742e8-0a27-4e24-970d-45522eb1504f\") "
Jul 31 23:06:45.795539 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:45.795508    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e14742e8-0a27-4e24-970d-45522eb1504f-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "e14742e8-0a27-4e24-970d-45522eb1504f" (UID: "e14742e8-0a27-4e24-970d-45522eb1504f"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 23:06:45.795648 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:45.795549    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e14742e8-0a27-4e24-970d-45522eb1504f-kube-api-access-jh768" (OuterVolumeSpecName: "kube-api-access-jh768") pod "e14742e8-0a27-4e24-970d-45522eb1504f" (UID: "e14742e8-0a27-4e24-970d-45522eb1504f"). InnerVolumeSpecName "kube-api-access-jh768". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 23:06:45.894392 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:45.894347    2584 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e14742e8-0a27-4e24-970d-45522eb1504f-webhook-cert\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:06:45.894392 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:45.894385    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jh768\" (UniqueName: \"kubernetes.io/projected/e14742e8-0a27-4e24-970d-45522eb1504f-kube-api-access-jh768\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:06:46.427970 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:46.427933    2584 generic.go:356] "Generic (PLEG): container finished" podID="e14742e8-0a27-4e24-970d-45522eb1504f" containerID="95348da1b9594e3ecbb8d5ac439622c3f18e4998eb3fc4d0a20343289b00541d" exitCode=0
Jul 31 23:06:46.428377 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:46.427979    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-694fdffc9b-tj5r9"
Jul 31 23:06:46.428377 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:46.428017    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-694fdffc9b-tj5r9" event={"ID":"e14742e8-0a27-4e24-970d-45522eb1504f","Type":"ContainerDied","Data":"95348da1b9594e3ecbb8d5ac439622c3f18e4998eb3fc4d0a20343289b00541d"}
Jul 31 23:06:46.428377 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:46.428069    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-694fdffc9b-tj5r9" event={"ID":"e14742e8-0a27-4e24-970d-45522eb1504f","Type":"ContainerDied","Data":"49f80249c360419f2536a831a52f274937e7c367c80573a603ecd7169f61129d"}
Jul 31 23:06:46.428377 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:46.428090    2584 scope.go:117] "RemoveContainer" containerID="95348da1b9594e3ecbb8d5ac439622c3f18e4998eb3fc4d0a20343289b00541d"
Jul 31 23:06:46.437025 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:46.437005    2584 scope.go:117] "RemoveContainer" containerID="95348da1b9594e3ecbb8d5ac439622c3f18e4998eb3fc4d0a20343289b00541d"
Jul 31 23:06:46.437285 ip-10-0-128-243 kubenswrapper[2584]: E0731 23:06:46.437250    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95348da1b9594e3ecbb8d5ac439622c3f18e4998eb3fc4d0a20343289b00541d\": container with ID starting with 95348da1b9594e3ecbb8d5ac439622c3f18e4998eb3fc4d0a20343289b00541d not found: ID does not exist" containerID="95348da1b9594e3ecbb8d5ac439622c3f18e4998eb3fc4d0a20343289b00541d"
Jul 31 23:06:46.437322 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:46.437294    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95348da1b9594e3ecbb8d5ac439622c3f18e4998eb3fc4d0a20343289b00541d"} err="failed to get container status \"95348da1b9594e3ecbb8d5ac439622c3f18e4998eb3fc4d0a20343289b00541d\": rpc error: code = NotFound desc = could not find container \"95348da1b9594e3ecbb8d5ac439622c3f18e4998eb3fc4d0a20343289b00541d\": container with ID starting with 95348da1b9594e3ecbb8d5ac439622c3f18e4998eb3fc4d0a20343289b00541d not found: ID does not exist"
Jul 31 23:06:46.448771 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:46.448747    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-694fdffc9b-tj5r9"]
Jul 31 23:06:46.454331 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:46.454311    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-694fdffc9b-tj5r9"]
Jul 31 23:06:46.890196 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:06:46.890163    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e14742e8-0a27-4e24-970d-45522eb1504f" path="/var/lib/kubelet/pods/e14742e8-0a27-4e24-970d-45522eb1504f/volumes"
Jul 31 23:07:05.428079 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:05.428047    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7c89d6c77-2msn9"]
Jul 31 23:07:05.428439 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:05.428378    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e14742e8-0a27-4e24-970d-45522eb1504f" containerName="manager"
Jul 31 23:07:05.428439 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:05.428390    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="e14742e8-0a27-4e24-970d-45522eb1504f" containerName="manager"
Jul 31 23:07:05.428508 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:05.428458    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="e14742e8-0a27-4e24-970d-45522eb1504f" containerName="manager"
Jul 31 23:07:05.432957 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:05.432939    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7c89d6c77-2msn9"
Jul 31 23:07:05.435425 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:05.435402    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-sc5t2\""
Jul 31 23:07:05.436192 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:05.436176    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 31 23:07:05.442154 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:05.442128    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7c89d6c77-2msn9"]
Jul 31 23:07:05.555518 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:05.555463    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/353f23a4-5683-4e87-b657-fd0a7a0e73fc-maas-api-tls\") pod \"maas-api-7c89d6c77-2msn9\" (UID: \"353f23a4-5683-4e87-b657-fd0a7a0e73fc\") " pod="odh-ai-gateway-infra/maas-api-7c89d6c77-2msn9"
Jul 31 23:07:05.555518 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:05.555530    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/353f23a4-5683-4e87-b657-fd0a7a0e73fc-openshift-service-ca\") pod \"maas-api-7c89d6c77-2msn9\" (UID: \"353f23a4-5683-4e87-b657-fd0a7a0e73fc\") " pod="odh-ai-gateway-infra/maas-api-7c89d6c77-2msn9"
Jul 31 23:07:05.555770 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:05.555652    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pw7n\" (UniqueName: \"kubernetes.io/projected/353f23a4-5683-4e87-b657-fd0a7a0e73fc-kube-api-access-4pw7n\") pod \"maas-api-7c89d6c77-2msn9\" (UID: \"353f23a4-5683-4e87-b657-fd0a7a0e73fc\") " pod="odh-ai-gateway-infra/maas-api-7c89d6c77-2msn9"
Jul 31 23:07:05.656442 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:05.656406    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/353f23a4-5683-4e87-b657-fd0a7a0e73fc-openshift-service-ca\") pod \"maas-api-7c89d6c77-2msn9\" (UID: \"353f23a4-5683-4e87-b657-fd0a7a0e73fc\") " pod="odh-ai-gateway-infra/maas-api-7c89d6c77-2msn9"
Jul 31 23:07:05.656614 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:05.656476    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pw7n\" (UniqueName: \"kubernetes.io/projected/353f23a4-5683-4e87-b657-fd0a7a0e73fc-kube-api-access-4pw7n\") pod \"maas-api-7c89d6c77-2msn9\" (UID: \"353f23a4-5683-4e87-b657-fd0a7a0e73fc\") " pod="odh-ai-gateway-infra/maas-api-7c89d6c77-2msn9"
Jul 31 23:07:05.656614 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:05.656502    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/353f23a4-5683-4e87-b657-fd0a7a0e73fc-maas-api-tls\") pod \"maas-api-7c89d6c77-2msn9\" (UID: \"353f23a4-5683-4e87-b657-fd0a7a0e73fc\") " pod="odh-ai-gateway-infra/maas-api-7c89d6c77-2msn9"
Jul 31 23:07:05.657119 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:05.657088    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/353f23a4-5683-4e87-b657-fd0a7a0e73fc-openshift-service-ca\") pod \"maas-api-7c89d6c77-2msn9\" (UID: \"353f23a4-5683-4e87-b657-fd0a7a0e73fc\") " pod="odh-ai-gateway-infra/maas-api-7c89d6c77-2msn9"
Jul 31 23:07:05.658950 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:05.658930    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/353f23a4-5683-4e87-b657-fd0a7a0e73fc-maas-api-tls\") pod \"maas-api-7c89d6c77-2msn9\" (UID: \"353f23a4-5683-4e87-b657-fd0a7a0e73fc\") " pod="odh-ai-gateway-infra/maas-api-7c89d6c77-2msn9"
Jul 31 23:07:05.664943 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:05.664916    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pw7n\" (UniqueName: \"kubernetes.io/projected/353f23a4-5683-4e87-b657-fd0a7a0e73fc-kube-api-access-4pw7n\") pod \"maas-api-7c89d6c77-2msn9\" (UID: \"353f23a4-5683-4e87-b657-fd0a7a0e73fc\") " pod="odh-ai-gateway-infra/maas-api-7c89d6c77-2msn9"
Jul 31 23:07:05.743425 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:05.743339    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7c89d6c77-2msn9"
Jul 31 23:07:05.882782 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:05.882757    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7c89d6c77-2msn9"]
Jul 31 23:07:05.885413 ip-10-0-128-243 kubenswrapper[2584]: W0731 23:07:05.885379    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod353f23a4_5683_4e87_b657_fd0a7a0e73fc.slice/crio-955d9cd6af65f05667a4ec6a84aa2acd1b388c00fc6af655a5abddc0f0de5c14 WatchSource:0}: Error finding container 955d9cd6af65f05667a4ec6a84aa2acd1b388c00fc6af655a5abddc0f0de5c14: Status 404 returned error can't find the container with id 955d9cd6af65f05667a4ec6a84aa2acd1b388c00fc6af655a5abddc0f0de5c14
Jul 31 23:07:05.942613 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:05.942575    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-29phv"]
Jul 31 23:07:05.947436 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:05.947414    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-29phv"
Jul 31 23:07:05.950057 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:05.950040    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-mhmmr\""
Jul 31 23:07:05.951323 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:05.951307    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 31 23:07:05.959021 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:05.958978    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-29phv"]
Jul 31 23:07:06.059679 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:06.059651    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkzf4\" (UniqueName: \"kubernetes.io/projected/f3023383-4445-4c1e-86a8-446b2e1307fa-kube-api-access-lkzf4\") pod \"payload-processing-5cc6cd444b-29phv\" (UID: \"f3023383-4445-4c1e-86a8-446b2e1307fa\") " pod="openshift-ingress/payload-processing-5cc6cd444b-29phv"
Jul 31 23:07:06.059839 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:06.059720    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f3023383-4445-4c1e-86a8-446b2e1307fa-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-29phv\" (UID: \"f3023383-4445-4c1e-86a8-446b2e1307fa\") " pod="openshift-ingress/payload-processing-5cc6cd444b-29phv"
Jul 31 23:07:06.160982 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:06.160943    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkzf4\" (UniqueName: \"kubernetes.io/projected/f3023383-4445-4c1e-86a8-446b2e1307fa-kube-api-access-lkzf4\") pod \"payload-processing-5cc6cd444b-29phv\" (UID: \"f3023383-4445-4c1e-86a8-446b2e1307fa\") " pod="openshift-ingress/payload-processing-5cc6cd444b-29phv"
Jul 31 23:07:06.161178 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:06.161041    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f3023383-4445-4c1e-86a8-446b2e1307fa-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-29phv\" (UID: \"f3023383-4445-4c1e-86a8-446b2e1307fa\") " pod="openshift-ingress/payload-processing-5cc6cd444b-29phv"
Jul 31 23:07:06.161838 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:06.161822    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f3023383-4445-4c1e-86a8-446b2e1307fa-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-29phv\" (UID: \"f3023383-4445-4c1e-86a8-446b2e1307fa\") " pod="openshift-ingress/payload-processing-5cc6cd444b-29phv"
Jul 31 23:07:06.168970 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:06.168950    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkzf4\" (UniqueName: \"kubernetes.io/projected/f3023383-4445-4c1e-86a8-446b2e1307fa-kube-api-access-lkzf4\") pod \"payload-processing-5cc6cd444b-29phv\" (UID: \"f3023383-4445-4c1e-86a8-446b2e1307fa\") " pod="openshift-ingress/payload-processing-5cc6cd444b-29phv"
Jul 31 23:07:06.257774 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:06.257743    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-29phv"
Jul 31 23:07:06.386950 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:06.386924    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-29phv"]
Jul 31 23:07:06.390144 ip-10-0-128-243 kubenswrapper[2584]: W0731 23:07:06.390115    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3023383_4445_4c1e_86a8_446b2e1307fa.slice/crio-b618c8ef74e00d80d7b9500d3f017ce0ac9d97f2b0dbf4a2dae012642720773d WatchSource:0}: Error finding container b618c8ef74e00d80d7b9500d3f017ce0ac9d97f2b0dbf4a2dae012642720773d: Status 404 returned error can't find the container with id b618c8ef74e00d80d7b9500d3f017ce0ac9d97f2b0dbf4a2dae012642720773d
Jul 31 23:07:06.517114 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:06.517077    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7c89d6c77-2msn9" event={"ID":"353f23a4-5683-4e87-b657-fd0a7a0e73fc","Type":"ContainerStarted","Data":"955d9cd6af65f05667a4ec6a84aa2acd1b388c00fc6af655a5abddc0f0de5c14"}
Jul 31 23:07:06.518394 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:06.518366    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-29phv" event={"ID":"f3023383-4445-4c1e-86a8-446b2e1307fa","Type":"ContainerStarted","Data":"b618c8ef74e00d80d7b9500d3f017ce0ac9d97f2b0dbf4a2dae012642720773d"}
Jul 31 23:07:07.526144 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:07.526106    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7c89d6c77-2msn9" event={"ID":"353f23a4-5683-4e87-b657-fd0a7a0e73fc","Type":"ContainerStarted","Data":"a4695cf69e20efed450e574bcbc13e750ceba1a2d41786d8d5fff872c2e54aa8"}
Jul 31 23:07:07.526623 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:07.526499    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7c89d6c77-2msn9"
Jul 31 23:07:07.550003 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:07.549949    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7c89d6c77-2msn9" podStartSLOduration=1.041192653 podStartE2EDuration="2.549931129s" totalImagesPullingTime="1.508738476s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:07:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:07:07.549108695 +0000 UTC m=+611.194895005" watchObservedRunningTime="2026-07-31 23:07:07.549931129 +0000 UTC m=+611.195717441"
Jul 31 23:07:09.538978 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:09.538938    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-29phv" event={"ID":"f3023383-4445-4c1e-86a8-446b2e1307fa","Type":"ContainerStarted","Data":"113c39353ec5cb6768e15581eb40d0be4e432a60d16014d9e21fe75aa2d103f6"}
Jul 31 23:07:09.539394 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:09.539091    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5cc6cd444b-29phv"
Jul 31 23:07:09.555387 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:09.555325    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5cc6cd444b-29phv" podStartSLOduration=2.233192195 podStartE2EDuration="4.555304732s" totalImagesPullingTime="2.322112537s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:07:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:07:09.553749936 +0000 UTC m=+613.199536245" watchObservedRunningTime="2026-07-31 23:07:09.555304732 +0000 UTC m=+613.201091042"
Jul 31 23:07:13.538688 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:13.538655    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7c89d6c77-2msn9"
Jul 31 23:07:18.901012 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:18.900977    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bdwhx"]
Jul 31 23:07:18.905973 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:18.905951    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bdwhx"
Jul 31 23:07:18.908826 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:18.908808    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 31 23:07:18.909974 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:18.909949    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-xm9gz\""
Jul 31 23:07:18.910086 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:18.910014    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 31 23:07:18.910086 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:18.910017    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 31 23:07:18.913674 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:18.913519    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bdwhx"]
Jul 31 23:07:18.975575 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:18.975546    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aaaf6395-4024-4492-b68d-77b34efb2204-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bdwhx\" (UID: \"aaaf6395-4024-4492-b68d-77b34efb2204\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bdwhx"
Jul 31 23:07:18.975774 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:18.975580    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aaaf6395-4024-4492-b68d-77b34efb2204-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bdwhx\" (UID: \"aaaf6395-4024-4492-b68d-77b34efb2204\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bdwhx"
Jul 31 23:07:18.975774 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:18.975608    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aaaf6395-4024-4492-b68d-77b34efb2204-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bdwhx\" (UID: \"aaaf6395-4024-4492-b68d-77b34efb2204\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bdwhx"
Jul 31 23:07:18.975774 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:18.975655    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aaaf6395-4024-4492-b68d-77b34efb2204-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bdwhx\" (UID: \"aaaf6395-4024-4492-b68d-77b34efb2204\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bdwhx"
Jul 31 23:07:18.975774 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:18.975694    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9vd2\" (UniqueName: \"kubernetes.io/projected/aaaf6395-4024-4492-b68d-77b34efb2204-kube-api-access-z9vd2\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bdwhx\" (UID: \"aaaf6395-4024-4492-b68d-77b34efb2204\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bdwhx"
Jul 31 23:07:18.975967 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:18.975806    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aaaf6395-4024-4492-b68d-77b34efb2204-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bdwhx\" (UID: \"aaaf6395-4024-4492-b68d-77b34efb2204\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bdwhx"
Jul 31 23:07:19.076658 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:19.076621    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aaaf6395-4024-4492-b68d-77b34efb2204-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bdwhx\" (UID: \"aaaf6395-4024-4492-b68d-77b34efb2204\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bdwhx"
Jul 31 23:07:19.076841 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:19.076764    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aaaf6395-4024-4492-b68d-77b34efb2204-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bdwhx\" (UID: \"aaaf6395-4024-4492-b68d-77b34efb2204\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bdwhx"
Jul 31 23:07:19.076841 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:19.076785    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aaaf6395-4024-4492-b68d-77b34efb2204-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bdwhx\" (UID: \"aaaf6395-4024-4492-b68d-77b34efb2204\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bdwhx"
Jul 31 23:07:19.076841 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:19.076811    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aaaf6395-4024-4492-b68d-77b34efb2204-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bdwhx\" (UID: \"aaaf6395-4024-4492-b68d-77b34efb2204\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bdwhx"
Jul 31 23:07:19.076841 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:19.076825    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aaaf6395-4024-4492-b68d-77b34efb2204-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bdwhx\" (UID: \"aaaf6395-4024-4492-b68d-77b34efb2204\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bdwhx"
Jul 31 23:07:19.077008 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:19.076989    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9vd2\" (UniqueName: \"kubernetes.io/projected/aaaf6395-4024-4492-b68d-77b34efb2204-kube-api-access-z9vd2\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bdwhx\" (UID: \"aaaf6395-4024-4492-b68d-77b34efb2204\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bdwhx"
Jul 31 23:07:19.077165 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:19.077145    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aaaf6395-4024-4492-b68d-77b34efb2204-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bdwhx\" (UID: \"aaaf6395-4024-4492-b68d-77b34efb2204\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bdwhx"
Jul 31 23:07:19.077253 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:19.077229    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aaaf6395-4024-4492-b68d-77b34efb2204-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bdwhx\" (UID: \"aaaf6395-4024-4492-b68d-77b34efb2204\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bdwhx"
Jul 31 23:07:19.077306 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:19.077237    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aaaf6395-4024-4492-b68d-77b34efb2204-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bdwhx\" (UID: \"aaaf6395-4024-4492-b68d-77b34efb2204\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bdwhx"
Jul 31 23:07:19.079012 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:19.078990    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aaaf6395-4024-4492-b68d-77b34efb2204-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bdwhx\" (UID: \"aaaf6395-4024-4492-b68d-77b34efb2204\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bdwhx"
Jul 31 23:07:19.079175 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:19.079159    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aaaf6395-4024-4492-b68d-77b34efb2204-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bdwhx\" (UID: \"aaaf6395-4024-4492-b68d-77b34efb2204\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bdwhx"
Jul 31 23:07:19.085125 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:19.085104    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9vd2\" (UniqueName: \"kubernetes.io/projected/aaaf6395-4024-4492-b68d-77b34efb2204-kube-api-access-z9vd2\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bdwhx\" (UID: \"aaaf6395-4024-4492-b68d-77b34efb2204\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bdwhx"
Jul 31 23:07:19.217745 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:19.217624    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bdwhx"
Jul 31 23:07:19.363627 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:19.363594    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bdwhx"]
Jul 31 23:07:19.365450 ip-10-0-128-243 kubenswrapper[2584]: W0731 23:07:19.365423    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaaaf6395_4024_4492_b68d_77b34efb2204.slice/crio-605da8e092071045a2ce9401c5eed8cc21e993711a2b200a12af6632e634874d WatchSource:0}: Error finding container 605da8e092071045a2ce9401c5eed8cc21e993711a2b200a12af6632e634874d: Status 404 returned error can't find the container with id 605da8e092071045a2ce9401c5eed8cc21e993711a2b200a12af6632e634874d
Jul 31 23:07:19.581592 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:19.581559    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bdwhx" event={"ID":"aaaf6395-4024-4492-b68d-77b34efb2204","Type":"ContainerStarted","Data":"605da8e092071045a2ce9401c5eed8cc21e993711a2b200a12af6632e634874d"}
Jul 31 23:07:20.545583 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:20.545553    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5cc6cd444b-29phv"
Jul 31 23:07:20.592678 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:20.592645    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bdwhx" event={"ID":"aaaf6395-4024-4492-b68d-77b34efb2204","Type":"ContainerStarted","Data":"e1265396cd32f01d052743d1890c1f1ab4000eb2a7a9d3356757e7d006fc9010"}
Jul 31 23:07:20.612633 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:20.612588    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bdwhx" podStartSLOduration=1.539200939 podStartE2EDuration="2.612575183s" totalImagesPullingTime="1.073374244s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:07:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:07:20.610468353 +0000 UTC m=+624.256254663" watchObservedRunningTime="2026-07-31 23:07:20.612575183 +0000 UTC m=+624.258361492"
Jul 31 23:07:28.488482 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:28.488447    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-srtqk"]
Jul 31 23:07:28.491942 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:28.491920    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-srtqk"
Jul 31 23:07:28.494266 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:28.494243    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 31 23:07:28.501448 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:28.501422    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-srtqk"]
Jul 31 23:07:28.546164 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:28.546130    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/26a9bdf8-5d25-4d59-a1a4-14ee72ca24a4-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-srtqk\" (UID: \"26a9bdf8-5d25-4d59-a1a4-14ee72ca24a4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-srtqk"
Jul 31 23:07:28.546361 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:28.546184    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/26a9bdf8-5d25-4d59-a1a4-14ee72ca24a4-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-srtqk\" (UID: \"26a9bdf8-5d25-4d59-a1a4-14ee72ca24a4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-srtqk"
Jul 31 23:07:28.546361 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:28.546212    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbp88\" (UniqueName: \"kubernetes.io/projected/26a9bdf8-5d25-4d59-a1a4-14ee72ca24a4-kube-api-access-gbp88\") pod \"e2e-trlp-test-simulated-kserve-598466548d-srtqk\" (UID: \"26a9bdf8-5d25-4d59-a1a4-14ee72ca24a4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-srtqk"
Jul 31 23:07:28.546361 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:28.546242    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/26a9bdf8-5d25-4d59-a1a4-14ee72ca24a4-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-srtqk\" (UID: \"26a9bdf8-5d25-4d59-a1a4-14ee72ca24a4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-srtqk"
Jul 31 23:07:28.546520 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:28.546353    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/26a9bdf8-5d25-4d59-a1a4-14ee72ca24a4-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-srtqk\" (UID: \"26a9bdf8-5d25-4d59-a1a4-14ee72ca24a4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-srtqk"
Jul 31 23:07:28.546520 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:28.546445    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/26a9bdf8-5d25-4d59-a1a4-14ee72ca24a4-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-srtqk\" (UID: \"26a9bdf8-5d25-4d59-a1a4-14ee72ca24a4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-srtqk"
Jul 31 23:07:28.647719 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:28.647661    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/26a9bdf8-5d25-4d59-a1a4-14ee72ca24a4-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-srtqk\" (UID: \"26a9bdf8-5d25-4d59-a1a4-14ee72ca24a4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-srtqk"
Jul 31 23:07:28.647719 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:28.647723    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/26a9bdf8-5d25-4d59-a1a4-14ee72ca24a4-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-srtqk\" (UID: \"26a9bdf8-5d25-4d59-a1a4-14ee72ca24a4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-srtqk"
Jul 31 23:07:28.647959 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:28.647760    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/26a9bdf8-5d25-4d59-a1a4-14ee72ca24a4-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-srtqk\" (UID: \"26a9bdf8-5d25-4d59-a1a4-14ee72ca24a4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-srtqk"
Jul 31 23:07:28.647959 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:28.647783    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/26a9bdf8-5d25-4d59-a1a4-14ee72ca24a4-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-srtqk\" (UID: \"26a9bdf8-5d25-4d59-a1a4-14ee72ca24a4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-srtqk"
Jul 31 23:07:28.647959 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:28.647798    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbp88\" (UniqueName: \"kubernetes.io/projected/26a9bdf8-5d25-4d59-a1a4-14ee72ca24a4-kube-api-access-gbp88\") pod \"e2e-trlp-test-simulated-kserve-598466548d-srtqk\" (UID: \"26a9bdf8-5d25-4d59-a1a4-14ee72ca24a4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-srtqk"
Jul 31 23:07:28.647959 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:28.647819    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/26a9bdf8-5d25-4d59-a1a4-14ee72ca24a4-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-srtqk\" (UID: \"26a9bdf8-5d25-4d59-a1a4-14ee72ca24a4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-srtqk"
Jul 31 23:07:28.648220 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:28.648195    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/26a9bdf8-5d25-4d59-a1a4-14ee72ca24a4-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-srtqk\" (UID: \"26a9bdf8-5d25-4d59-a1a4-14ee72ca24a4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-srtqk"
Jul 31 23:07:28.648281 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:28.648227    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/26a9bdf8-5d25-4d59-a1a4-14ee72ca24a4-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-srtqk\" (UID: \"26a9bdf8-5d25-4d59-a1a4-14ee72ca24a4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-srtqk"
Jul 31 23:07:28.648281 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:28.648262    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/26a9bdf8-5d25-4d59-a1a4-14ee72ca24a4-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-srtqk\" (UID: \"26a9bdf8-5d25-4d59-a1a4-14ee72ca24a4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-srtqk"
Jul 31 23:07:28.650347 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:28.650280    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/26a9bdf8-5d25-4d59-a1a4-14ee72ca24a4-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-srtqk\" (UID: \"26a9bdf8-5d25-4d59-a1a4-14ee72ca24a4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-srtqk"
Jul 31 23:07:28.650476 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:28.650440    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/26a9bdf8-5d25-4d59-a1a4-14ee72ca24a4-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-srtqk\" (UID: \"26a9bdf8-5d25-4d59-a1a4-14ee72ca24a4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-srtqk"
Jul 31 23:07:28.656376 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:28.656356    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbp88\" (UniqueName: \"kubernetes.io/projected/26a9bdf8-5d25-4d59-a1a4-14ee72ca24a4-kube-api-access-gbp88\") pod \"e2e-trlp-test-simulated-kserve-598466548d-srtqk\" (UID: \"26a9bdf8-5d25-4d59-a1a4-14ee72ca24a4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-srtqk"
Jul 31 23:07:28.802954 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:28.802870    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-srtqk"
Jul 31 23:07:28.945230 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:28.945201    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-srtqk"]
Jul 31 23:07:28.948204 ip-10-0-128-243 kubenswrapper[2584]: W0731 23:07:28.948170    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26a9bdf8_5d25_4d59_a1a4_14ee72ca24a4.slice/crio-e471b172ad2cf43bbef9eb4c527bcc8cf8d0a64486e214c269f95f582e2ed1df WatchSource:0}: Error finding container e471b172ad2cf43bbef9eb4c527bcc8cf8d0a64486e214c269f95f582e2ed1df: Status 404 returned error can't find the container with id e471b172ad2cf43bbef9eb4c527bcc8cf8d0a64486e214c269f95f582e2ed1df
Jul 31 23:07:29.218496 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:29.218467    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bdwhx"
Jul 31 23:07:29.218656 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:29.218532    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bdwhx"
Jul 31 23:07:29.231743 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:29.231687    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bdwhx"
Jul 31 23:07:29.637971 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:29.637937    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-srtqk" event={"ID":"26a9bdf8-5d25-4d59-a1a4-14ee72ca24a4","Type":"ContainerStarted","Data":"ccb87527e0456e470113299da9c4c0dc34bb5c01d74e6f6ecab109934ca224a6"}
Jul 31 23:07:29.638343 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:29.638000    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-srtqk" event={"ID":"26a9bdf8-5d25-4d59-a1a4-14ee72ca24a4","Type":"ContainerStarted","Data":"e471b172ad2cf43bbef9eb4c527bcc8cf8d0a64486e214c269f95f582e2ed1df"}
Jul 31 23:07:29.649562 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:29.649533    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bdwhx"
Jul 31 23:07:29.656281 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:29.656237    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-srtqk" podStartSLOduration=1.440899252 podStartE2EDuration="1.656222859s" totalImagesPullingTime="215.323607ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:07:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:07:29.655233503 +0000 UTC m=+633.301019815" watchObservedRunningTime="2026-07-31 23:07:29.656222859 +0000 UTC m=+633.302009202"
Jul 31 23:07:38.803540 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:38.803498    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-srtqk"
Jul 31 23:07:38.804153 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:38.803615    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-srtqk"
Jul 31 23:07:38.815888 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:38.815861    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-srtqk"
Jul 31 23:07:39.698714 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:07:39.698677    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-srtqk"
Jul 31 23:09:57.677436 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:09:57.677402    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-74f5bd9766-jmq85"]
Jul 31 23:09:57.677884 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:09:57.677597    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-74f5bd9766-jmq85" podUID="405ea669-5dae-46ad-b64f-59a9364f222b" containerName="manager" containerID="cri-o://dd739d8053c878b7c2477ea0dad1310f15b47bd4730fbaba69bcf91826fc9063" gracePeriod=30
Jul 31 23:09:57.924845 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:09:57.924822    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74f5bd9766-jmq85"
Jul 31 23:09:58.034250 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:09:58.034158    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c4rxk\" (UniqueName: \"kubernetes.io/projected/405ea669-5dae-46ad-b64f-59a9364f222b-kube-api-access-c4rxk\") pod \"405ea669-5dae-46ad-b64f-59a9364f222b\" (UID: \"405ea669-5dae-46ad-b64f-59a9364f222b\") "
Jul 31 23:09:58.034250 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:09:58.034209    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/405ea669-5dae-46ad-b64f-59a9364f222b-webhook-cert\") pod \"405ea669-5dae-46ad-b64f-59a9364f222b\" (UID: \"405ea669-5dae-46ad-b64f-59a9364f222b\") "
Jul 31 23:09:58.036258 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:09:58.036228    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/405ea669-5dae-46ad-b64f-59a9364f222b-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "405ea669-5dae-46ad-b64f-59a9364f222b" (UID: "405ea669-5dae-46ad-b64f-59a9364f222b"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 23:09:58.036392 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:09:58.036282    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/405ea669-5dae-46ad-b64f-59a9364f222b-kube-api-access-c4rxk" (OuterVolumeSpecName: "kube-api-access-c4rxk") pod "405ea669-5dae-46ad-b64f-59a9364f222b" (UID: "405ea669-5dae-46ad-b64f-59a9364f222b"). InnerVolumeSpecName "kube-api-access-c4rxk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 23:09:58.134937 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:09:58.134906    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c4rxk\" (UniqueName: \"kubernetes.io/projected/405ea669-5dae-46ad-b64f-59a9364f222b-kube-api-access-c4rxk\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:09:58.134937 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:09:58.134930    2584 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/405ea669-5dae-46ad-b64f-59a9364f222b-webhook-cert\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:09:58.299064 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:09:58.298976    2584 generic.go:356] "Generic (PLEG): container finished" podID="405ea669-5dae-46ad-b64f-59a9364f222b" containerID="dd739d8053c878b7c2477ea0dad1310f15b47bd4730fbaba69bcf91826fc9063" exitCode=0
Jul 31 23:09:58.299064 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:09:58.299015    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74f5bd9766-jmq85"
Jul 31 23:09:58.299225 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:09:58.299064    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74f5bd9766-jmq85" event={"ID":"405ea669-5dae-46ad-b64f-59a9364f222b","Type":"ContainerDied","Data":"dd739d8053c878b7c2477ea0dad1310f15b47bd4730fbaba69bcf91826fc9063"}
Jul 31 23:09:58.299225 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:09:58.299117    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74f5bd9766-jmq85" event={"ID":"405ea669-5dae-46ad-b64f-59a9364f222b","Type":"ContainerDied","Data":"5e455ac32334405083d495b28fa345d195512a1688410903a331cf3f23aca345"}
Jul 31 23:09:58.299225 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:09:58.299156    2584 scope.go:117] "RemoveContainer" containerID="dd739d8053c878b7c2477ea0dad1310f15b47bd4730fbaba69bcf91826fc9063"
Jul 31 23:09:58.308678 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:09:58.308662    2584 scope.go:117] "RemoveContainer" containerID="dd739d8053c878b7c2477ea0dad1310f15b47bd4730fbaba69bcf91826fc9063"
Jul 31 23:09:58.308956 ip-10-0-128-243 kubenswrapper[2584]: E0731 23:09:58.308936    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd739d8053c878b7c2477ea0dad1310f15b47bd4730fbaba69bcf91826fc9063\": container with ID starting with dd739d8053c878b7c2477ea0dad1310f15b47bd4730fbaba69bcf91826fc9063 not found: ID does not exist" containerID="dd739d8053c878b7c2477ea0dad1310f15b47bd4730fbaba69bcf91826fc9063"
Jul 31 23:09:58.309024 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:09:58.308966    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd739d8053c878b7c2477ea0dad1310f15b47bd4730fbaba69bcf91826fc9063"} err="failed to get container status \"dd739d8053c878b7c2477ea0dad1310f15b47bd4730fbaba69bcf91826fc9063\": rpc error: code = NotFound desc = could not find container \"dd739d8053c878b7c2477ea0dad1310f15b47bd4730fbaba69bcf91826fc9063\": container with ID starting with dd739d8053c878b7c2477ea0dad1310f15b47bd4730fbaba69bcf91826fc9063 not found: ID does not exist"
Jul 31 23:09:58.322817 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:09:58.322784    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-74f5bd9766-jmq85"]
Jul 31 23:09:58.327787 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:09:58.327764    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-74f5bd9766-jmq85"]
Jul 31 23:09:58.890647 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:09:58.890613    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="405ea669-5dae-46ad-b64f-59a9364f222b" path="/var/lib/kubelet/pods/405ea669-5dae-46ad-b64f-59a9364f222b/volumes"
Jul 31 23:10:59.152800 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:10:59.152767    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-74f5bd9766-cvt6n"]
Jul 31 23:10:59.153216 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:10:59.153190    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="405ea669-5dae-46ad-b64f-59a9364f222b" containerName="manager"
Jul 31 23:10:59.153216 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:10:59.153202    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="405ea669-5dae-46ad-b64f-59a9364f222b" containerName="manager"
Jul 31 23:10:59.153302 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:10:59.153262    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="405ea669-5dae-46ad-b64f-59a9364f222b" containerName="manager"
Jul 31 23:10:59.156586 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:10:59.156566    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74f5bd9766-cvt6n"
Jul 31 23:10:59.160550 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:10:59.160519    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-rfzlf\""
Jul 31 23:10:59.160670 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:10:59.160616    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 31 23:10:59.160743 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:10:59.160670    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 31 23:10:59.166266 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:10:59.166242    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-74f5bd9766-cvt6n"]
Jul 31 23:10:59.275332 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:10:59.275293    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpg22\" (UniqueName: \"kubernetes.io/projected/9356cb7a-f3df-4418-b755-2260b3d83cad-kube-api-access-mpg22\") pod \"maas-controller-74f5bd9766-cvt6n\" (UID: \"9356cb7a-f3df-4418-b755-2260b3d83cad\") " pod="opendatahub/maas-controller-74f5bd9766-cvt6n"
Jul 31 23:10:59.275332 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:10:59.275341    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9356cb7a-f3df-4418-b755-2260b3d83cad-webhook-cert\") pod \"maas-controller-74f5bd9766-cvt6n\" (UID: \"9356cb7a-f3df-4418-b755-2260b3d83cad\") " pod="opendatahub/maas-controller-74f5bd9766-cvt6n"
Jul 31 23:10:59.376210 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:10:59.376171    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9356cb7a-f3df-4418-b755-2260b3d83cad-webhook-cert\") pod \"maas-controller-74f5bd9766-cvt6n\" (UID: \"9356cb7a-f3df-4418-b755-2260b3d83cad\") " pod="opendatahub/maas-controller-74f5bd9766-cvt6n"
Jul 31 23:10:59.376395 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:10:59.376303    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpg22\" (UniqueName: \"kubernetes.io/projected/9356cb7a-f3df-4418-b755-2260b3d83cad-kube-api-access-mpg22\") pod \"maas-controller-74f5bd9766-cvt6n\" (UID: \"9356cb7a-f3df-4418-b755-2260b3d83cad\") " pod="opendatahub/maas-controller-74f5bd9766-cvt6n"
Jul 31 23:10:59.378777 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:10:59.378746    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9356cb7a-f3df-4418-b755-2260b3d83cad-webhook-cert\") pod \"maas-controller-74f5bd9766-cvt6n\" (UID: \"9356cb7a-f3df-4418-b755-2260b3d83cad\") " pod="opendatahub/maas-controller-74f5bd9766-cvt6n"
Jul 31 23:10:59.385253 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:10:59.385221    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpg22\" (UniqueName: \"kubernetes.io/projected/9356cb7a-f3df-4418-b755-2260b3d83cad-kube-api-access-mpg22\") pod \"maas-controller-74f5bd9766-cvt6n\" (UID: \"9356cb7a-f3df-4418-b755-2260b3d83cad\") " pod="opendatahub/maas-controller-74f5bd9766-cvt6n"
Jul 31 23:10:59.467912 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:10:59.467816    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74f5bd9766-cvt6n"
Jul 31 23:10:59.599623 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:10:59.599598    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-74f5bd9766-cvt6n"]
Jul 31 23:10:59.601271 ip-10-0-128-243 kubenswrapper[2584]: W0731 23:10:59.601243    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9356cb7a_f3df_4418_b755_2260b3d83cad.slice/crio-acaeb3b3300ef400f6270b1881b8bd117a28bd0f8146a621d370f0d75e97d0ba WatchSource:0}: Error finding container acaeb3b3300ef400f6270b1881b8bd117a28bd0f8146a621d370f0d75e97d0ba: Status 404 returned error can't find the container with id acaeb3b3300ef400f6270b1881b8bd117a28bd0f8146a621d370f0d75e97d0ba
Jul 31 23:10:59.602552 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:10:59.602532    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 31 23:11:00.584795 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:11:00.584761    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74f5bd9766-cvt6n" event={"ID":"9356cb7a-f3df-4418-b755-2260b3d83cad","Type":"ContainerStarted","Data":"6d0bf4fdb98013423cb854254b8a2f486d13794bca908d5b5342284e2ea0d808"}
Jul 31 23:11:00.585188 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:11:00.584815    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74f5bd9766-cvt6n" event={"ID":"9356cb7a-f3df-4418-b755-2260b3d83cad","Type":"ContainerStarted","Data":"acaeb3b3300ef400f6270b1881b8bd117a28bd0f8146a621d370f0d75e97d0ba"}
Jul 31 23:11:00.585188 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:11:00.584884    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-74f5bd9766-cvt6n"
Jul 31 23:11:00.604429 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:11:00.604380    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-74f5bd9766-cvt6n" podStartSLOduration=1.169443289 podStartE2EDuration="1.604364516s" totalImagesPullingTime="434.921227ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:10:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:11:00.602567173 +0000 UTC m=+844.248353495" watchObservedRunningTime="2026-07-31 23:11:00.604364516 +0000 UTC m=+844.250150825"
Jul 31 23:11:11.591872 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:11:11.591838    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-74f5bd9766-cvt6n"
Jul 31 23:11:16.230120 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:11:16.230089    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-74f5bd9766-cvt6n"]
Jul 31 23:11:16.230568 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:11:16.230253    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-74f5bd9766-cvt6n" podUID="9356cb7a-f3df-4418-b755-2260b3d83cad" containerName="manager" containerID="cri-o://6d0bf4fdb98013423cb854254b8a2f486d13794bca908d5b5342284e2ea0d808" gracePeriod=30
Jul 31 23:11:16.483538 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:11:16.483481    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74f5bd9766-cvt6n"
Jul 31 23:11:16.624099 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:11:16.624067    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9356cb7a-f3df-4418-b755-2260b3d83cad-webhook-cert\") pod \"9356cb7a-f3df-4418-b755-2260b3d83cad\" (UID: \"9356cb7a-f3df-4418-b755-2260b3d83cad\") "
Jul 31 23:11:16.624295 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:11:16.624207    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mpg22\" (UniqueName: \"kubernetes.io/projected/9356cb7a-f3df-4418-b755-2260b3d83cad-kube-api-access-mpg22\") pod \"9356cb7a-f3df-4418-b755-2260b3d83cad\" (UID: \"9356cb7a-f3df-4418-b755-2260b3d83cad\") "
Jul 31 23:11:16.626189 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:11:16.626158    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9356cb7a-f3df-4418-b755-2260b3d83cad-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "9356cb7a-f3df-4418-b755-2260b3d83cad" (UID: "9356cb7a-f3df-4418-b755-2260b3d83cad"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 23:11:16.626189 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:11:16.626168    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9356cb7a-f3df-4418-b755-2260b3d83cad-kube-api-access-mpg22" (OuterVolumeSpecName: "kube-api-access-mpg22") pod "9356cb7a-f3df-4418-b755-2260b3d83cad" (UID: "9356cb7a-f3df-4418-b755-2260b3d83cad"). InnerVolumeSpecName "kube-api-access-mpg22". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 23:11:16.657136 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:11:16.657108    2584 generic.go:356] "Generic (PLEG): container finished" podID="9356cb7a-f3df-4418-b755-2260b3d83cad" containerID="6d0bf4fdb98013423cb854254b8a2f486d13794bca908d5b5342284e2ea0d808" exitCode=0
Jul 31 23:11:16.657242 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:11:16.657146    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74f5bd9766-cvt6n" event={"ID":"9356cb7a-f3df-4418-b755-2260b3d83cad","Type":"ContainerDied","Data":"6d0bf4fdb98013423cb854254b8a2f486d13794bca908d5b5342284e2ea0d808"}
Jul 31 23:11:16.657242 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:11:16.657152    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74f5bd9766-cvt6n"
Jul 31 23:11:16.657242 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:11:16.657187    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74f5bd9766-cvt6n" event={"ID":"9356cb7a-f3df-4418-b755-2260b3d83cad","Type":"ContainerDied","Data":"acaeb3b3300ef400f6270b1881b8bd117a28bd0f8146a621d370f0d75e97d0ba"}
Jul 31 23:11:16.657242 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:11:16.657209    2584 scope.go:117] "RemoveContainer" containerID="6d0bf4fdb98013423cb854254b8a2f486d13794bca908d5b5342284e2ea0d808"
Jul 31 23:11:16.666162 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:11:16.666146    2584 scope.go:117] "RemoveContainer" containerID="6d0bf4fdb98013423cb854254b8a2f486d13794bca908d5b5342284e2ea0d808"
Jul 31 23:11:16.666419 ip-10-0-128-243 kubenswrapper[2584]: E0731 23:11:16.666398    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d0bf4fdb98013423cb854254b8a2f486d13794bca908d5b5342284e2ea0d808\": container with ID starting with 6d0bf4fdb98013423cb854254b8a2f486d13794bca908d5b5342284e2ea0d808 not found: ID does not exist" containerID="6d0bf4fdb98013423cb854254b8a2f486d13794bca908d5b5342284e2ea0d808"
Jul 31 23:11:16.666482 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:11:16.666431    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d0bf4fdb98013423cb854254b8a2f486d13794bca908d5b5342284e2ea0d808"} err="failed to get container status \"6d0bf4fdb98013423cb854254b8a2f486d13794bca908d5b5342284e2ea0d808\": rpc error: code = NotFound desc = could not find container \"6d0bf4fdb98013423cb854254b8a2f486d13794bca908d5b5342284e2ea0d808\": container with ID starting with 6d0bf4fdb98013423cb854254b8a2f486d13794bca908d5b5342284e2ea0d808 not found: ID does not exist"
Jul 31 23:11:16.679445 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:11:16.679411    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-74f5bd9766-cvt6n"]
Jul 31 23:11:16.681454 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:11:16.681432    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-74f5bd9766-cvt6n"]
Jul 31 23:11:16.725236 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:11:16.725212    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mpg22\" (UniqueName: \"kubernetes.io/projected/9356cb7a-f3df-4418-b755-2260b3d83cad-kube-api-access-mpg22\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:11:16.725236 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:11:16.725233    2584 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9356cb7a-f3df-4418-b755-2260b3d83cad-webhook-cert\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:11:16.891108 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:11:16.891076    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9356cb7a-f3df-4418-b755-2260b3d83cad" path="/var/lib/kubelet/pods/9356cb7a-f3df-4418-b755-2260b3d83cad/volumes"
Jul 31 23:11:17.439213 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:11:17.439173    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-74f5bd9766-t8s7d"]
Jul 31 23:11:17.439745 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:11:17.439726    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9356cb7a-f3df-4418-b755-2260b3d83cad" containerName="manager"
Jul 31 23:11:17.439834 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:11:17.439748    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="9356cb7a-f3df-4418-b755-2260b3d83cad" containerName="manager"
Jul 31 23:11:17.439888 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:11:17.439852    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="9356cb7a-f3df-4418-b755-2260b3d83cad" containerName="manager"
Jul 31 23:11:17.444434 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:11:17.444411    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74f5bd9766-t8s7d"
Jul 31 23:11:17.448012 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:11:17.447987    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 31 23:11:17.448136 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:11:17.448035    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 31 23:11:17.448136 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:11:17.448041    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-rfzlf\""
Jul 31 23:11:17.452430 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:11:17.452402    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-74f5bd9766-t8s7d"]
Jul 31 23:11:17.533394 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:11:17.533363    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjfgt\" (UniqueName: \"kubernetes.io/projected/6e07685d-6508-4ba5-be33-cd0b86504595-kube-api-access-wjfgt\") pod \"maas-controller-74f5bd9766-t8s7d\" (UID: \"6e07685d-6508-4ba5-be33-cd0b86504595\") " pod="opendatahub/maas-controller-74f5bd9766-t8s7d"
Jul 31 23:11:17.533577 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:11:17.533410    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6e07685d-6508-4ba5-be33-cd0b86504595-webhook-cert\") pod \"maas-controller-74f5bd9766-t8s7d\" (UID: \"6e07685d-6508-4ba5-be33-cd0b86504595\") " pod="opendatahub/maas-controller-74f5bd9766-t8s7d"
Jul 31 23:11:17.634673 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:11:17.634635    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjfgt\" (UniqueName: \"kubernetes.io/projected/6e07685d-6508-4ba5-be33-cd0b86504595-kube-api-access-wjfgt\") pod \"maas-controller-74f5bd9766-t8s7d\" (UID: \"6e07685d-6508-4ba5-be33-cd0b86504595\") " pod="opendatahub/maas-controller-74f5bd9766-t8s7d"
Jul 31 23:11:17.634862 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:11:17.634721    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6e07685d-6508-4ba5-be33-cd0b86504595-webhook-cert\") pod \"maas-controller-74f5bd9766-t8s7d\" (UID: \"6e07685d-6508-4ba5-be33-cd0b86504595\") " pod="opendatahub/maas-controller-74f5bd9766-t8s7d"
Jul 31 23:11:17.637078 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:11:17.637059    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6e07685d-6508-4ba5-be33-cd0b86504595-webhook-cert\") pod \"maas-controller-74f5bd9766-t8s7d\" (UID: \"6e07685d-6508-4ba5-be33-cd0b86504595\") " pod="opendatahub/maas-controller-74f5bd9766-t8s7d"
Jul 31 23:11:17.643508 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:11:17.643476    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjfgt\" (UniqueName: \"kubernetes.io/projected/6e07685d-6508-4ba5-be33-cd0b86504595-kube-api-access-wjfgt\") pod \"maas-controller-74f5bd9766-t8s7d\" (UID: \"6e07685d-6508-4ba5-be33-cd0b86504595\") " pod="opendatahub/maas-controller-74f5bd9766-t8s7d"
Jul 31 23:11:17.756290 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:11:17.756211    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74f5bd9766-t8s7d"
Jul 31 23:11:17.899038 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:11:17.899014    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-74f5bd9766-t8s7d"]
Jul 31 23:11:17.901879 ip-10-0-128-243 kubenswrapper[2584]: W0731 23:11:17.901845    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e07685d_6508_4ba5_be33_cd0b86504595.slice/crio-cc77ee6658effdba75f436b72a4bcc65a1036dc7494a35c18e215389dcfd2977 WatchSource:0}: Error finding container cc77ee6658effdba75f436b72a4bcc65a1036dc7494a35c18e215389dcfd2977: Status 404 returned error can't find the container with id cc77ee6658effdba75f436b72a4bcc65a1036dc7494a35c18e215389dcfd2977
Jul 31 23:11:18.668517 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:11:18.668486    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74f5bd9766-t8s7d" event={"ID":"6e07685d-6508-4ba5-be33-cd0b86504595","Type":"ContainerStarted","Data":"554d9026e597a0c85cd42a70910316e6d3f7c3f29383ce899379b866aa251d33"}
Jul 31 23:11:18.668899 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:11:18.668535    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74f5bd9766-t8s7d" event={"ID":"6e07685d-6508-4ba5-be33-cd0b86504595","Type":"ContainerStarted","Data":"cc77ee6658effdba75f436b72a4bcc65a1036dc7494a35c18e215389dcfd2977"}
Jul 31 23:11:18.668899 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:11:18.668595    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-74f5bd9766-t8s7d"
Jul 31 23:11:18.688614 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:11:18.688564    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-74f5bd9766-t8s7d" podStartSLOduration=1.326413087 podStartE2EDuration="1.68855042s" totalImagesPullingTime="362.137333ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:11:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:11:18.687202793 +0000 UTC m=+862.332989102" watchObservedRunningTime="2026-07-31 23:11:18.68855042 +0000 UTC m=+862.334336746"
Jul 31 23:11:29.676881 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:11:29.676850    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-74f5bd9766-t8s7d"
Jul 31 23:15:00.151489 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:15:00.151407    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29758995-bnxj2"]
Jul 31 23:15:00.154845 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:15:00.154827    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29758995-bnxj2"
Jul 31 23:15:00.157402 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:15:00.157381    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qxwgn\""
Jul 31 23:15:00.164298 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:15:00.164276    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29758995-bnxj2"
Jul 31 23:15:00.170898 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:15:00.170872    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29758995-bnxj2"]
Jul 31 23:15:00.504294 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:15:00.504267    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29758995-bnxj2"]
Jul 31 23:15:00.506023 ip-10-0-128-243 kubenswrapper[2584]: W0731 23:15:00.505995    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbf8b16a_00b6_40c4_bdb4_5b214960658b.slice/crio-26b590a1f6fc8f23a582a782bbc56708585f998ced886a9fc2dfae96e6be565f WatchSource:0}: Error finding container 26b590a1f6fc8f23a582a782bbc56708585f998ced886a9fc2dfae96e6be565f: Status 404 returned error can't find the container with id 26b590a1f6fc8f23a582a782bbc56708585f998ced886a9fc2dfae96e6be565f
Jul 31 23:15:00.667663 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:15:00.667618    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29758995-bnxj2" event={"ID":"dbf8b16a-00b6-40c4-bdb4-5b214960658b","Type":"ContainerStarted","Data":"26b590a1f6fc8f23a582a782bbc56708585f998ced886a9fc2dfae96e6be565f"}
Jul 31 23:15:03.684936 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:15:03.684900    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29758995-bnxj2" event={"ID":"dbf8b16a-00b6-40c4-bdb4-5b214960658b","Type":"ContainerStarted","Data":"cc9ea5e6cf91f67826e14c3078549d78b47c3d837a725024aea0c16be8b21d07"}
Jul 31 23:15:03.702081 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:15:03.702033    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29758995-bnxj2" podStartSLOduration=1.3744197630000001 podStartE2EDuration="3.702017345s" totalImagesPullingTime="2.327597582s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:15:03.700523867 +0000 UTC m=+1087.346310176" watchObservedRunningTime="2026-07-31 23:15:03.702017345 +0000 UTC m=+1087.347803708"
Jul 31 23:15:04.690527 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:15:04.690493    2584 generic.go:356] "Generic (PLEG): container finished" podID="dbf8b16a-00b6-40c4-bdb4-5b214960658b" containerID="cc9ea5e6cf91f67826e14c3078549d78b47c3d837a725024aea0c16be8b21d07" exitCode=0
Jul 31 23:15:04.690930 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:15:04.690587    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29758995-bnxj2" event={"ID":"dbf8b16a-00b6-40c4-bdb4-5b214960658b","Type":"ContainerDied","Data":"cc9ea5e6cf91f67826e14c3078549d78b47c3d837a725024aea0c16be8b21d07"}
Jul 31 23:15:05.832961 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:15:05.832938    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29758995-bnxj2"
Jul 31 23:15:06.701622 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:15:06.701593    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29758995-bnxj2"
Jul 31 23:15:06.701622 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:15:06.701609    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29758995-bnxj2" event={"ID":"dbf8b16a-00b6-40c4-bdb4-5b214960658b","Type":"ContainerDied","Data":"26b590a1f6fc8f23a582a782bbc56708585f998ced886a9fc2dfae96e6be565f"}
Jul 31 23:15:06.701858 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:15:06.701658    2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="26b590a1f6fc8f23a582a782bbc56708585f998ced886a9fc2dfae96e6be565f"
Jul 31 23:21:58.295789 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:21:58.295755    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-a038c70d-kserve-d455b8995-jbmlt"]
Jul 31 23:21:58.296230 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:21:58.296109    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dbf8b16a-00b6-40c4-bdb4-5b214960658b" containerName="cleanup"
Jul 31 23:21:58.296230 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:21:58.296119    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="dbf8b16a-00b6-40c4-bdb4-5b214960658b" containerName="cleanup"
Jul 31 23:21:58.296230 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:21:58.296180    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="dbf8b16a-00b6-40c4-bdb4-5b214960658b" containerName="cleanup"
Jul 31 23:21:58.298962 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:21:58.298946    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-a038c70d-kserve-d455b8995-jbmlt"
Jul 31 23:21:58.301357 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:21:58.301334    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-a038c70d-kserve-self-signed-certs\""
Jul 31 23:21:58.315036 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:21:58.315011    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-a038c70d-kserve-d455b8995-jbmlt"]
Jul 31 23:21:58.396521 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:21:58.396486    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84b17b4a-b8e0-419f-a0e2-5ca7b0aef717-tmp-dir\") pod \"e2e-same-id-a-a038c70d-kserve-d455b8995-jbmlt\" (UID: \"84b17b4a-b8e0-419f-a0e2-5ca7b0aef717\") " pod="llm/e2e-same-id-a-a038c70d-kserve-d455b8995-jbmlt"
Jul 31 23:21:58.396521 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:21:58.396520    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84b17b4a-b8e0-419f-a0e2-5ca7b0aef717-model-cache\") pod \"e2e-same-id-a-a038c70d-kserve-d455b8995-jbmlt\" (UID: \"84b17b4a-b8e0-419f-a0e2-5ca7b0aef717\") " pod="llm/e2e-same-id-a-a038c70d-kserve-d455b8995-jbmlt"
Jul 31 23:21:58.396816 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:21:58.396552    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84b17b4a-b8e0-419f-a0e2-5ca7b0aef717-tls-certs\") pod \"e2e-same-id-a-a038c70d-kserve-d455b8995-jbmlt\" (UID: \"84b17b4a-b8e0-419f-a0e2-5ca7b0aef717\") " pod="llm/e2e-same-id-a-a038c70d-kserve-d455b8995-jbmlt"
Jul 31 23:21:58.396816 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:21:58.396671    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84b17b4a-b8e0-419f-a0e2-5ca7b0aef717-home\") pod \"e2e-same-id-a-a038c70d-kserve-d455b8995-jbmlt\" (UID: \"84b17b4a-b8e0-419f-a0e2-5ca7b0aef717\") " pod="llm/e2e-same-id-a-a038c70d-kserve-d455b8995-jbmlt"
Jul 31 23:21:58.396816 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:21:58.396737    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqg4n\" (UniqueName: \"kubernetes.io/projected/84b17b4a-b8e0-419f-a0e2-5ca7b0aef717-kube-api-access-fqg4n\") pod \"e2e-same-id-a-a038c70d-kserve-d455b8995-jbmlt\" (UID: \"84b17b4a-b8e0-419f-a0e2-5ca7b0aef717\") " pod="llm/e2e-same-id-a-a038c70d-kserve-d455b8995-jbmlt"
Jul 31 23:21:58.396816 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:21:58.396773    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84b17b4a-b8e0-419f-a0e2-5ca7b0aef717-dshm\") pod \"e2e-same-id-a-a038c70d-kserve-d455b8995-jbmlt\" (UID: \"84b17b4a-b8e0-419f-a0e2-5ca7b0aef717\") " pod="llm/e2e-same-id-a-a038c70d-kserve-d455b8995-jbmlt"
Jul 31 23:21:58.497341 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:21:58.497311    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84b17b4a-b8e0-419f-a0e2-5ca7b0aef717-home\") pod \"e2e-same-id-a-a038c70d-kserve-d455b8995-jbmlt\" (UID: \"84b17b4a-b8e0-419f-a0e2-5ca7b0aef717\") " pod="llm/e2e-same-id-a-a038c70d-kserve-d455b8995-jbmlt"
Jul 31 23:21:58.497341 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:21:58.497344    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqg4n\" (UniqueName: \"kubernetes.io/projected/84b17b4a-b8e0-419f-a0e2-5ca7b0aef717-kube-api-access-fqg4n\") pod \"e2e-same-id-a-a038c70d-kserve-d455b8995-jbmlt\" (UID: \"84b17b4a-b8e0-419f-a0e2-5ca7b0aef717\") " pod="llm/e2e-same-id-a-a038c70d-kserve-d455b8995-jbmlt"
Jul 31 23:21:58.497559 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:21:58.497367    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84b17b4a-b8e0-419f-a0e2-5ca7b0aef717-dshm\") pod \"e2e-same-id-a-a038c70d-kserve-d455b8995-jbmlt\" (UID: \"84b17b4a-b8e0-419f-a0e2-5ca7b0aef717\") " pod="llm/e2e-same-id-a-a038c70d-kserve-d455b8995-jbmlt"
Jul 31 23:21:58.497559 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:21:58.497512    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84b17b4a-b8e0-419f-a0e2-5ca7b0aef717-tmp-dir\") pod \"e2e-same-id-a-a038c70d-kserve-d455b8995-jbmlt\" (UID: \"84b17b4a-b8e0-419f-a0e2-5ca7b0aef717\") " pod="llm/e2e-same-id-a-a038c70d-kserve-d455b8995-jbmlt"
Jul 31 23:21:58.497669 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:21:58.497557    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84b17b4a-b8e0-419f-a0e2-5ca7b0aef717-model-cache\") pod \"e2e-same-id-a-a038c70d-kserve-d455b8995-jbmlt\" (UID: \"84b17b4a-b8e0-419f-a0e2-5ca7b0aef717\") " pod="llm/e2e-same-id-a-a038c70d-kserve-d455b8995-jbmlt"
Jul 31 23:21:58.497669 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:21:58.497645    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84b17b4a-b8e0-419f-a0e2-5ca7b0aef717-tls-certs\") pod \"e2e-same-id-a-a038c70d-kserve-d455b8995-jbmlt\" (UID: \"84b17b4a-b8e0-419f-a0e2-5ca7b0aef717\") " pod="llm/e2e-same-id-a-a038c70d-kserve-d455b8995-jbmlt"
Jul 31 23:21:58.497818 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:21:58.497792    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84b17b4a-b8e0-419f-a0e2-5ca7b0aef717-home\") pod \"e2e-same-id-a-a038c70d-kserve-d455b8995-jbmlt\" (UID: \"84b17b4a-b8e0-419f-a0e2-5ca7b0aef717\") " pod="llm/e2e-same-id-a-a038c70d-kserve-d455b8995-jbmlt"
Jul 31 23:21:58.497916 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:21:58.497894    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84b17b4a-b8e0-419f-a0e2-5ca7b0aef717-tmp-dir\") pod \"e2e-same-id-a-a038c70d-kserve-d455b8995-jbmlt\" (UID: \"84b17b4a-b8e0-419f-a0e2-5ca7b0aef717\") " pod="llm/e2e-same-id-a-a038c70d-kserve-d455b8995-jbmlt"
Jul 31 23:21:58.497967 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:21:58.497931    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84b17b4a-b8e0-419f-a0e2-5ca7b0aef717-model-cache\") pod \"e2e-same-id-a-a038c70d-kserve-d455b8995-jbmlt\" (UID: \"84b17b4a-b8e0-419f-a0e2-5ca7b0aef717\") " pod="llm/e2e-same-id-a-a038c70d-kserve-d455b8995-jbmlt"
Jul 31 23:21:58.499569 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:21:58.499549    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84b17b4a-b8e0-419f-a0e2-5ca7b0aef717-dshm\") pod \"e2e-same-id-a-a038c70d-kserve-d455b8995-jbmlt\" (UID: \"84b17b4a-b8e0-419f-a0e2-5ca7b0aef717\") " pod="llm/e2e-same-id-a-a038c70d-kserve-d455b8995-jbmlt"
Jul 31 23:21:58.500067 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:21:58.500048    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84b17b4a-b8e0-419f-a0e2-5ca7b0aef717-tls-certs\") pod \"e2e-same-id-a-a038c70d-kserve-d455b8995-jbmlt\" (UID: \"84b17b4a-b8e0-419f-a0e2-5ca7b0aef717\") " pod="llm/e2e-same-id-a-a038c70d-kserve-d455b8995-jbmlt"
Jul 31 23:21:58.504492 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:21:58.504469    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqg4n\" (UniqueName: \"kubernetes.io/projected/84b17b4a-b8e0-419f-a0e2-5ca7b0aef717-kube-api-access-fqg4n\") pod \"e2e-same-id-a-a038c70d-kserve-d455b8995-jbmlt\" (UID: \"84b17b4a-b8e0-419f-a0e2-5ca7b0aef717\") " pod="llm/e2e-same-id-a-a038c70d-kserve-d455b8995-jbmlt"
Jul 31 23:21:58.608051 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:21:58.608020    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-a038c70d-kserve-d455b8995-jbmlt"
Jul 31 23:21:58.742977 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:21:58.742952    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-a038c70d-kserve-d455b8995-jbmlt"]
Jul 31 23:21:58.745064 ip-10-0-128-243 kubenswrapper[2584]: W0731 23:21:58.745030    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod84b17b4a_b8e0_419f_a0e2_5ca7b0aef717.slice/crio-417ba7822248cba05fc0d84f8cdfd07c83914624b4a3283286341063dcd519d9 WatchSource:0}: Error finding container 417ba7822248cba05fc0d84f8cdfd07c83914624b4a3283286341063dcd519d9: Status 404 returned error can't find the container with id 417ba7822248cba05fc0d84f8cdfd07c83914624b4a3283286341063dcd519d9
Jul 31 23:21:59.563648 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:21:59.563613    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-a038c70d-kserve-d455b8995-jbmlt" event={"ID":"84b17b4a-b8e0-419f-a0e2-5ca7b0aef717","Type":"ContainerStarted","Data":"25c0b190e023a7518f69e349c944042c7a4d8c1c394adb0b23032d2a4a02560a"}
Jul 31 23:21:59.564064 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:21:59.563656    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-a038c70d-kserve-d455b8995-jbmlt" event={"ID":"84b17b4a-b8e0-419f-a0e2-5ca7b0aef717","Type":"ContainerStarted","Data":"417ba7822248cba05fc0d84f8cdfd07c83914624b4a3283286341063dcd519d9"}
Jul 31 23:21:59.580484 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:21:59.580428    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-a038c70d-kserve-d455b8995-jbmlt" podStartSLOduration=1.580411563 podStartE2EDuration="1.580411563s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:21:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:21:59.578911155 +0000 UTC m=+1503.224697465" watchObservedRunningTime="2026-07-31 23:21:59.580411563 +0000 UTC m=+1503.226197872"
Jul 31 23:22:08.608374 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:22:08.608344    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-a038c70d-kserve-d455b8995-jbmlt"
Jul 31 23:22:08.608802 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:22:08.608400    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-a038c70d-kserve-d455b8995-jbmlt"
Jul 31 23:22:08.620766 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:22:08.620747    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-a038c70d-kserve-d455b8995-jbmlt"
Jul 31 23:22:09.621990 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:22:09.621955    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-a038c70d-kserve-d455b8995-jbmlt"
Jul 31 23:22:35.631757 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:22:35.631639    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-a038c70d-kserve-d455b8995-jbmlt"]
Jul 31 23:22:50.651571 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:22:50.651522    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-a038c70d-kserve-d455b8995-jbmlt" podUID="84b17b4a-b8e0-419f-a0e2-5ca7b0aef717" containerName="main" containerID="cri-o://25c0b190e023a7518f69e349c944042c7a4d8c1c394adb0b23032d2a4a02560a" gracePeriod=45
Jul 31 23:22:50.800748 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:22:50.800688    2584 generic.go:356] "Generic (PLEG): container finished" podID="84b17b4a-b8e0-419f-a0e2-5ca7b0aef717" containerID="25c0b190e023a7518f69e349c944042c7a4d8c1c394adb0b23032d2a4a02560a" exitCode=0
Jul 31 23:22:50.800903 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:22:50.800762    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-a038c70d-kserve-d455b8995-jbmlt" event={"ID":"84b17b4a-b8e0-419f-a0e2-5ca7b0aef717","Type":"ContainerDied","Data":"25c0b190e023a7518f69e349c944042c7a4d8c1c394adb0b23032d2a4a02560a"}
Jul 31 23:22:50.897151 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:22:50.897122    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-a038c70d-kserve-d455b8995-jbmlt"
Jul 31 23:22:51.057354 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:22:51.057328    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84b17b4a-b8e0-419f-a0e2-5ca7b0aef717-model-cache\") pod \"84b17b4a-b8e0-419f-a0e2-5ca7b0aef717\" (UID: \"84b17b4a-b8e0-419f-a0e2-5ca7b0aef717\") "
Jul 31 23:22:51.057526 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:22:51.057384    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84b17b4a-b8e0-419f-a0e2-5ca7b0aef717-dshm\") pod \"84b17b4a-b8e0-419f-a0e2-5ca7b0aef717\" (UID: \"84b17b4a-b8e0-419f-a0e2-5ca7b0aef717\") "
Jul 31 23:22:51.057526 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:22:51.057447    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84b17b4a-b8e0-419f-a0e2-5ca7b0aef717-tmp-dir\") pod \"84b17b4a-b8e0-419f-a0e2-5ca7b0aef717\" (UID: \"84b17b4a-b8e0-419f-a0e2-5ca7b0aef717\") "
Jul 31 23:22:51.057526 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:22:51.057462    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84b17b4a-b8e0-419f-a0e2-5ca7b0aef717-tls-certs\") pod \"84b17b4a-b8e0-419f-a0e2-5ca7b0aef717\" (UID: \"84b17b4a-b8e0-419f-a0e2-5ca7b0aef717\") "
Jul 31 23:22:51.057526 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:22:51.057491    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84b17b4a-b8e0-419f-a0e2-5ca7b0aef717-home\") pod \"84b17b4a-b8e0-419f-a0e2-5ca7b0aef717\" (UID: \"84b17b4a-b8e0-419f-a0e2-5ca7b0aef717\") "
Jul 31 23:22:51.057526 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:22:51.057521    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fqg4n\" (UniqueName: \"kubernetes.io/projected/84b17b4a-b8e0-419f-a0e2-5ca7b0aef717-kube-api-access-fqg4n\") pod \"84b17b4a-b8e0-419f-a0e2-5ca7b0aef717\" (UID: \"84b17b4a-b8e0-419f-a0e2-5ca7b0aef717\") "
Jul 31 23:22:51.057828 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:22:51.057623    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84b17b4a-b8e0-419f-a0e2-5ca7b0aef717-model-cache" (OuterVolumeSpecName: "model-cache") pod "84b17b4a-b8e0-419f-a0e2-5ca7b0aef717" (UID: "84b17b4a-b8e0-419f-a0e2-5ca7b0aef717"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 23:22:51.057828 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:22:51.057749    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84b17b4a-b8e0-419f-a0e2-5ca7b0aef717-home" (OuterVolumeSpecName: "home") pod "84b17b4a-b8e0-419f-a0e2-5ca7b0aef717" (UID: "84b17b4a-b8e0-419f-a0e2-5ca7b0aef717"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 23:22:51.057828 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:22:51.057765    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84b17b4a-b8e0-419f-a0e2-5ca7b0aef717-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "84b17b4a-b8e0-419f-a0e2-5ca7b0aef717" (UID: "84b17b4a-b8e0-419f-a0e2-5ca7b0aef717"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 23:22:51.057988 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:22:51.057871    2584 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84b17b4a-b8e0-419f-a0e2-5ca7b0aef717-tmp-dir\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:22:51.057988 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:22:51.057889    2584 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84b17b4a-b8e0-419f-a0e2-5ca7b0aef717-home\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:22:51.057988 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:22:51.057902    2584 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84b17b4a-b8e0-419f-a0e2-5ca7b0aef717-model-cache\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:22:51.059757 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:22:51.059732    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84b17b4a-b8e0-419f-a0e2-5ca7b0aef717-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "84b17b4a-b8e0-419f-a0e2-5ca7b0aef717" (UID: "84b17b4a-b8e0-419f-a0e2-5ca7b0aef717"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 23:22:51.059983 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:22:51.059959    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84b17b4a-b8e0-419f-a0e2-5ca7b0aef717-dshm" (OuterVolumeSpecName: "dshm") pod "84b17b4a-b8e0-419f-a0e2-5ca7b0aef717" (UID: "84b17b4a-b8e0-419f-a0e2-5ca7b0aef717"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 23:22:51.060069 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:22:51.060047    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84b17b4a-b8e0-419f-a0e2-5ca7b0aef717-kube-api-access-fqg4n" (OuterVolumeSpecName: "kube-api-access-fqg4n") pod "84b17b4a-b8e0-419f-a0e2-5ca7b0aef717" (UID: "84b17b4a-b8e0-419f-a0e2-5ca7b0aef717"). InnerVolumeSpecName "kube-api-access-fqg4n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 23:22:51.159189 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:22:51.159148    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fqg4n\" (UniqueName: \"kubernetes.io/projected/84b17b4a-b8e0-419f-a0e2-5ca7b0aef717-kube-api-access-fqg4n\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:22:51.159189 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:22:51.159191    2584 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84b17b4a-b8e0-419f-a0e2-5ca7b0aef717-dshm\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:22:51.159425 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:22:51.159206    2584 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84b17b4a-b8e0-419f-a0e2-5ca7b0aef717-tls-certs\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:22:51.806618 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:22:51.806574    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-a038c70d-kserve-d455b8995-jbmlt" event={"ID":"84b17b4a-b8e0-419f-a0e2-5ca7b0aef717","Type":"ContainerDied","Data":"417ba7822248cba05fc0d84f8cdfd07c83914624b4a3283286341063dcd519d9"}
Jul 31 23:22:51.806618 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:22:51.806604    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-a038c70d-kserve-d455b8995-jbmlt"
Jul 31 23:22:51.807091 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:22:51.806640    2584 scope.go:117] "RemoveContainer" containerID="25c0b190e023a7518f69e349c944042c7a4d8c1c394adb0b23032d2a4a02560a"
Jul 31 23:22:51.826844 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:22:51.826821    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-a038c70d-kserve-d455b8995-jbmlt"]
Jul 31 23:22:51.830619 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:22:51.830598    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-a038c70d-kserve-d455b8995-jbmlt"]
Jul 31 23:22:52.891387 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:22:52.891353    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84b17b4a-b8e0-419f-a0e2-5ca7b0aef717" path="/var/lib/kubelet/pods/84b17b4a-b8e0-419f-a0e2-5ca7b0aef717/volumes"
Jul 31 23:26:35.513278 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:26:35.512636    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fec77b77-78dff4696b-vp5gc"]
Jul 31 23:26:35.513278 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:26:35.513178    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84b17b4a-b8e0-419f-a0e2-5ca7b0aef717" containerName="main"
Jul 31 23:26:35.513278 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:26:35.513199    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="84b17b4a-b8e0-419f-a0e2-5ca7b0aef717" containerName="main"
Jul 31 23:26:35.513278 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:26:35.513279    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="84b17b4a-b8e0-419f-a0e2-5ca7b0aef717" containerName="main"
Jul 31 23:26:35.516076 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:26:35.516053    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fec77b77-78dff4696b-vp5gc"
Jul 31 23:26:35.518219 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:26:35.518195    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-fec77b77-serving-cert\""
Jul 31 23:26:35.526762 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:26:35.526734    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fec77b77-78dff4696b-vp5gc"]
Jul 31 23:26:35.609369 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:26:35.609334    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2fc2a5bf-fb7f-4d1b-8422-c28af24e71ff-maas-api-tls\") pod \"maas-api-e2e-ait-fec77b77-78dff4696b-vp5gc\" (UID: \"2fc2a5bf-fb7f-4d1b-8422-c28af24e71ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fec77b77-78dff4696b-vp5gc"
Jul 31 23:26:35.609567 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:26:35.609374    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2fc2a5bf-fb7f-4d1b-8422-c28af24e71ff-openshift-service-ca\") pod \"maas-api-e2e-ait-fec77b77-78dff4696b-vp5gc\" (UID: \"2fc2a5bf-fb7f-4d1b-8422-c28af24e71ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fec77b77-78dff4696b-vp5gc"
Jul 31 23:26:35.609641 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:26:35.609570    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kmqn\" (UniqueName: \"kubernetes.io/projected/2fc2a5bf-fb7f-4d1b-8422-c28af24e71ff-kube-api-access-7kmqn\") pod \"maas-api-e2e-ait-fec77b77-78dff4696b-vp5gc\" (UID: \"2fc2a5bf-fb7f-4d1b-8422-c28af24e71ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fec77b77-78dff4696b-vp5gc"
Jul 31 23:26:35.711085 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:26:35.711052    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kmqn\" (UniqueName: \"kubernetes.io/projected/2fc2a5bf-fb7f-4d1b-8422-c28af24e71ff-kube-api-access-7kmqn\") pod \"maas-api-e2e-ait-fec77b77-78dff4696b-vp5gc\" (UID: \"2fc2a5bf-fb7f-4d1b-8422-c28af24e71ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fec77b77-78dff4696b-vp5gc"
Jul 31 23:26:35.711085 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:26:35.711091    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2fc2a5bf-fb7f-4d1b-8422-c28af24e71ff-maas-api-tls\") pod \"maas-api-e2e-ait-fec77b77-78dff4696b-vp5gc\" (UID: \"2fc2a5bf-fb7f-4d1b-8422-c28af24e71ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fec77b77-78dff4696b-vp5gc"
Jul 31 23:26:35.711302 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:26:35.711113    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2fc2a5bf-fb7f-4d1b-8422-c28af24e71ff-openshift-service-ca\") pod \"maas-api-e2e-ait-fec77b77-78dff4696b-vp5gc\" (UID: \"2fc2a5bf-fb7f-4d1b-8422-c28af24e71ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fec77b77-78dff4696b-vp5gc"
Jul 31 23:26:35.711746 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:26:35.711689    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2fc2a5bf-fb7f-4d1b-8422-c28af24e71ff-openshift-service-ca\") pod \"maas-api-e2e-ait-fec77b77-78dff4696b-vp5gc\" (UID: \"2fc2a5bf-fb7f-4d1b-8422-c28af24e71ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fec77b77-78dff4696b-vp5gc"
Jul 31 23:26:35.713855 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:26:35.713832    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2fc2a5bf-fb7f-4d1b-8422-c28af24e71ff-maas-api-tls\") pod \"maas-api-e2e-ait-fec77b77-78dff4696b-vp5gc\" (UID: \"2fc2a5bf-fb7f-4d1b-8422-c28af24e71ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fec77b77-78dff4696b-vp5gc"
Jul 31 23:26:35.724481 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:26:35.724456    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kmqn\" (UniqueName: \"kubernetes.io/projected/2fc2a5bf-fb7f-4d1b-8422-c28af24e71ff-kube-api-access-7kmqn\") pod \"maas-api-e2e-ait-fec77b77-78dff4696b-vp5gc\" (UID: \"2fc2a5bf-fb7f-4d1b-8422-c28af24e71ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fec77b77-78dff4696b-vp5gc"
Jul 31 23:26:35.827582 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:26:35.827551    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fec77b77-78dff4696b-vp5gc"
Jul 31 23:26:35.962725 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:26:35.962685    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fec77b77-78dff4696b-vp5gc"]
Jul 31 23:26:35.964286 ip-10-0-128-243 kubenswrapper[2584]: W0731 23:26:35.964258    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fc2a5bf_fb7f_4d1b_8422_c28af24e71ff.slice/crio-2bdf1ee46b1a9242e163c01155b93944b7556a663b3e1bbff3ef69c6cbce7e9d WatchSource:0}: Error finding container 2bdf1ee46b1a9242e163c01155b93944b7556a663b3e1bbff3ef69c6cbce7e9d: Status 404 returned error can't find the container with id 2bdf1ee46b1a9242e163c01155b93944b7556a663b3e1bbff3ef69c6cbce7e9d
Jul 31 23:26:35.965718 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:26:35.965673    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 31 23:26:36.286120 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:26:36.286025    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-fec77b77-9864f5b4-jtq9w"]
Jul 31 23:26:36.290474 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:26:36.290451    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-fec77b77-9864f5b4-jtq9w"
Jul 31 23:26:36.296525 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:26:36.296499    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-fec77b77\""
Jul 31 23:26:36.296645 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:26:36.296626    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-fec77b77-dockercfg-zs976\""
Jul 31 23:26:36.302863 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:26:36.302838    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-fec77b77-9864f5b4-jtq9w"]
Jul 31 23:26:36.335897 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:26:36.335858    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-fec77b77-5cf88f98cb-4zfn6"]
Jul 31 23:26:36.339510 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:26:36.339484    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-fec77b77-5cf88f98cb-4zfn6"
Jul 31 23:26:36.349343 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:26:36.349319    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-fec77b77-5cf88f98cb-4zfn6"]
Jul 31 23:26:36.418486 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:26:36.418465    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bdca4cce-1c0b-4711-9a87-ba7f4dadf6ba-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-fec77b77-5cf88f98cb-4zfn6\" (UID: \"bdca4cce-1c0b-4711-9a87-ba7f4dadf6ba\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-fec77b77-5cf88f98cb-4zfn6"
Jul 31 23:26:36.418554 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:26:36.418501    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtszk\" (UniqueName: \"kubernetes.io/projected/bdca4cce-1c0b-4711-9a87-ba7f4dadf6ba-kube-api-access-wtszk\") pod \"payload-pre-processing-e2e-ait-fec77b77-5cf88f98cb-4zfn6\" (UID: \"bdca4cce-1c0b-4711-9a87-ba7f4dadf6ba\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-fec77b77-5cf88f98cb-4zfn6"
Jul 31 23:26:36.418596 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:26:36.418576    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kg25t\" (UniqueName: \"kubernetes.io/projected/3683604e-9696-4c44-aad1-f72b971cac56-kube-api-access-kg25t\") pod \"payload-processing-e2e-ait-fec77b77-9864f5b4-jtq9w\" (UID: \"3683604e-9696-4c44-aad1-f72b971cac56\") " pod="openshift-ingress/payload-processing-e2e-ait-fec77b77-9864f5b4-jtq9w"
Jul 31 23:26:36.418652 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:26:36.418599    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3683604e-9696-4c44-aad1-f72b971cac56-plugins-config-volume\") pod \"payload-processing-e2e-ait-fec77b77-9864f5b4-jtq9w\" (UID: \"3683604e-9696-4c44-aad1-f72b971cac56\") " pod="openshift-ingress/payload-processing-e2e-ait-fec77b77-9864f5b4-jtq9w"
Jul 31 23:26:36.519556 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:26:36.519523    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bdca4cce-1c0b-4711-9a87-ba7f4dadf6ba-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-fec77b77-5cf88f98cb-4zfn6\" (UID: \"bdca4cce-1c0b-4711-9a87-ba7f4dadf6ba\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-fec77b77-5cf88f98cb-4zfn6"
Jul 31 23:26:36.519979 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:26:36.519572    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtszk\" (UniqueName: \"kubernetes.io/projected/bdca4cce-1c0b-4711-9a87-ba7f4dadf6ba-kube-api-access-wtszk\") pod \"payload-pre-processing-e2e-ait-fec77b77-5cf88f98cb-4zfn6\" (UID: \"bdca4cce-1c0b-4711-9a87-ba7f4dadf6ba\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-fec77b77-5cf88f98cb-4zfn6"
Jul 31 23:26:36.519979 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:26:36.519631    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kg25t\" (UniqueName: \"kubernetes.io/projected/3683604e-9696-4c44-aad1-f72b971cac56-kube-api-access-kg25t\") pod \"payload-processing-e2e-ait-fec77b77-9864f5b4-jtq9w\" (UID: \"3683604e-9696-4c44-aad1-f72b971cac56\") " pod="openshift-ingress/payload-processing-e2e-ait-fec77b77-9864f5b4-jtq9w"
Jul 31 23:26:36.519979 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:26:36.519649    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3683604e-9696-4c44-aad1-f72b971cac56-plugins-config-volume\") pod \"payload-processing-e2e-ait-fec77b77-9864f5b4-jtq9w\" (UID: \"3683604e-9696-4c44-aad1-f72b971cac56\") " pod="openshift-ingress/payload-processing-e2e-ait-fec77b77-9864f5b4-jtq9w"
Jul 31 23:26:36.520337 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:26:36.520318    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bdca4cce-1c0b-4711-9a87-ba7f4dadf6ba-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-fec77b77-5cf88f98cb-4zfn6\" (UID: \"bdca4cce-1c0b-4711-9a87-ba7f4dadf6ba\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-fec77b77-5cf88f98cb-4zfn6"
Jul 31 23:26:36.520376 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:26:36.520339    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3683604e-9696-4c44-aad1-f72b971cac56-plugins-config-volume\") pod \"payload-processing-e2e-ait-fec77b77-9864f5b4-jtq9w\" (UID: \"3683604e-9696-4c44-aad1-f72b971cac56\") " pod="openshift-ingress/payload-processing-e2e-ait-fec77b77-9864f5b4-jtq9w"
Jul 31 23:26:36.527110 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:26:36.527085    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kg25t\" (UniqueName: \"kubernetes.io/projected/3683604e-9696-4c44-aad1-f72b971cac56-kube-api-access-kg25t\") pod \"payload-processing-e2e-ait-fec77b77-9864f5b4-jtq9w\" (UID: \"3683604e-9696-4c44-aad1-f72b971cac56\") " pod="openshift-ingress/payload-processing-e2e-ait-fec77b77-9864f5b4-jtq9w"
Jul 31 23:26:36.527272 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:26:36.527254    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtszk\" (UniqueName: \"kubernetes.io/projected/bdca4cce-1c0b-4711-9a87-ba7f4dadf6ba-kube-api-access-wtszk\") pod \"payload-pre-processing-e2e-ait-fec77b77-5cf88f98cb-4zfn6\" (UID: \"bdca4cce-1c0b-4711-9a87-ba7f4dadf6ba\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-fec77b77-5cf88f98cb-4zfn6"
Jul 31 23:26:36.601528 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:26:36.601492    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-fec77b77-9864f5b4-jtq9w"
Jul 31 23:26:36.650908 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:26:36.650737    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-fec77b77-5cf88f98cb-4zfn6"
Jul 31 23:26:36.742220 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:26:36.742047    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-fec77b77-9864f5b4-jtq9w"]
Jul 31 23:26:36.744554 ip-10-0-128-243 kubenswrapper[2584]: W0731 23:26:36.744516    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3683604e_9696_4c44_aad1_f72b971cac56.slice/crio-8b08b39553dd25bf55b2d13ff71530afeff8845f8d78670fadb1df1ddac19a6f WatchSource:0}: Error finding container 8b08b39553dd25bf55b2d13ff71530afeff8845f8d78670fadb1df1ddac19a6f: Status 404 returned error can't find the container with id 8b08b39553dd25bf55b2d13ff71530afeff8845f8d78670fadb1df1ddac19a6f
Jul 31 23:26:36.800261 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:26:36.800238    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-fec77b77-5cf88f98cb-4zfn6"]
Jul 31 23:26:36.801974 ip-10-0-128-243 kubenswrapper[2584]: W0731 23:26:36.801946    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdca4cce_1c0b_4711_9a87_ba7f4dadf6ba.slice/crio-69466f5e6ee01a37f07af4d35af29dab21e4f171b9f5e95d866f2b731cfac8e5 WatchSource:0}: Error finding container 69466f5e6ee01a37f07af4d35af29dab21e4f171b9f5e95d866f2b731cfac8e5: Status 404 returned error can't find the container with id 69466f5e6ee01a37f07af4d35af29dab21e4f171b9f5e95d866f2b731cfac8e5
Jul 31 23:26:36.840479 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:26:36.840442    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fec77b77-78dff4696b-vp5gc" event={"ID":"2fc2a5bf-fb7f-4d1b-8422-c28af24e71ff","Type":"ContainerStarted","Data":"ec98a002e0b25a96705bf3380f64e778c6b7b0180e986120384f750a5dbf9de2"}
Jul 31 23:26:36.840589 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:26:36.840547    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fec77b77-78dff4696b-vp5gc" event={"ID":"2fc2a5bf-fb7f-4d1b-8422-c28af24e71ff","Type":"ContainerStarted","Data":"2bdf1ee46b1a9242e163c01155b93944b7556a663b3e1bbff3ef69c6cbce7e9d"}
Jul 31 23:26:36.841958 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:26:36.841591    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fec77b77-78dff4696b-vp5gc"
Jul 31 23:26:36.843749 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:26:36.843724    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-fec77b77-9864f5b4-jtq9w" event={"ID":"3683604e-9696-4c44-aad1-f72b971cac56","Type":"ContainerStarted","Data":"11417adf64fd667ecde15641db0e85ae213cc88102b4f4b859a805530ea023e3"}
Jul 31 23:26:36.843867 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:26:36.843767    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-fec77b77-9864f5b4-jtq9w" event={"ID":"3683604e-9696-4c44-aad1-f72b971cac56","Type":"ContainerStarted","Data":"8b08b39553dd25bf55b2d13ff71530afeff8845f8d78670fadb1df1ddac19a6f"}
Jul 31 23:26:36.843867 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:26:36.843814    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-fec77b77-9864f5b4-jtq9w"
Jul 31 23:26:36.845356 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:26:36.845333    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-fec77b77-5cf88f98cb-4zfn6" event={"ID":"bdca4cce-1c0b-4711-9a87-ba7f4dadf6ba","Type":"ContainerStarted","Data":"69466f5e6ee01a37f07af4d35af29dab21e4f171b9f5e95d866f2b731cfac8e5"}
Jul 31 23:26:36.859487 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:26:36.859432    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fec77b77-78dff4696b-vp5gc" podStartSLOduration=1.406620234 podStartE2EDuration="1.859411875s" totalImagesPullingTime="452.791641ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:26:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:26:36.856519057 +0000 UTC m=+1780.502305367" watchObservedRunningTime="2026-07-31 23:26:36.859411875 +0000 UTC m=+1780.505198184"
Jul 31 23:26:36.872934 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:26:36.872887    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-fec77b77-9864f5b4-jtq9w" podStartSLOduration=0.872869862 podStartE2EDuration="872.869862ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:26:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:26:36.871214461 +0000 UTC m=+1780.517000793" watchObservedRunningTime="2026-07-31 23:26:36.872869862 +0000 UTC m=+1780.518656170"
Jul 31 23:26:37.851733 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:26:37.851674    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-fec77b77-5cf88f98cb-4zfn6" event={"ID":"bdca4cce-1c0b-4711-9a87-ba7f4dadf6ba","Type":"ContainerStarted","Data":"9216dfd4781800ff46bd9fbb38d814788d80491f36fd08cda6ab79474f327748"}
Jul 31 23:26:37.852178 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:26:37.851847    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-fec77b77-5cf88f98cb-4zfn6"
Jul 31 23:26:37.866578 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:26:37.866530    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-fec77b77-5cf88f98cb-4zfn6" podStartSLOduration=1.866513415 podStartE2EDuration="1.866513415s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:26:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:26:37.865867462 +0000 UTC m=+1781.511653771" watchObservedRunningTime="2026-07-31 23:26:37.866513415 +0000 UTC m=+1781.512299726"
Jul 31 23:26:43.864741 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:26:43.864689    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fec77b77-78dff4696b-vp5gc"
Jul 31 23:26:47.853470 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:26:47.853440    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-fec77b77-9864f5b4-jtq9w"
Jul 31 23:26:48.861811 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:26:48.861782    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-fec77b77-5cf88f98cb-4zfn6"
Jul 31 23:27:16.160486 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:16.160450    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fec77b77-78dff4696b-vp5gc"]
Jul 31 23:27:16.161079 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:16.160725    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fec77b77-78dff4696b-vp5gc" podUID="2fc2a5bf-fb7f-4d1b-8422-c28af24e71ff" containerName="maas-api" containerID="cri-o://ec98a002e0b25a96705bf3380f64e778c6b7b0180e986120384f750a5dbf9de2" gracePeriod=30
Jul 31 23:27:16.208002 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:16.207959    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-fec77b77-9864f5b4-jtq9w"]
Jul 31 23:27:16.209233 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:16.209179    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-fec77b77-9864f5b4-jtq9w" podUID="3683604e-9696-4c44-aad1-f72b971cac56" containerName="payload-processing" containerID="cri-o://11417adf64fd667ecde15641db0e85ae213cc88102b4f4b859a805530ea023e3" gracePeriod=30
Jul 31 23:27:16.213251 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:16.212862    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-fec77b77-5cf88f98cb-4zfn6"]
Jul 31 23:27:16.213251 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:16.213066    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-fec77b77-5cf88f98cb-4zfn6" podUID="bdca4cce-1c0b-4711-9a87-ba7f4dadf6ba" containerName="payload-pre-processing" containerID="cri-o://9216dfd4781800ff46bd9fbb38d814788d80491f36fd08cda6ab79474f327748" gracePeriod=30
Jul 31 23:27:16.524445 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:16.524419    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-fec77b77-9864f5b4-jtq9w"
Jul 31 23:27:16.527169 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:16.527146    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-fec77b77-5cf88f98cb-4zfn6"
Jul 31 23:27:16.532964 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:16.532938    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fec77b77-78dff4696b-vp5gc"
Jul 31 23:27:16.589562 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:16.589531    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3683604e-9696-4c44-aad1-f72b971cac56-plugins-config-volume\") pod \"3683604e-9696-4c44-aad1-f72b971cac56\" (UID: \"3683604e-9696-4c44-aad1-f72b971cac56\") "
Jul 31 23:27:16.589767 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:16.589616    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kg25t\" (UniqueName: \"kubernetes.io/projected/3683604e-9696-4c44-aad1-f72b971cac56-kube-api-access-kg25t\") pod \"3683604e-9696-4c44-aad1-f72b971cac56\" (UID: \"3683604e-9696-4c44-aad1-f72b971cac56\") "
Jul 31 23:27:16.590012 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:16.589988    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3683604e-9696-4c44-aad1-f72b971cac56-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3683604e-9696-4c44-aad1-f72b971cac56" (UID: "3683604e-9696-4c44-aad1-f72b971cac56"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:27:16.591853 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:16.591824    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3683604e-9696-4c44-aad1-f72b971cac56-kube-api-access-kg25t" (OuterVolumeSpecName: "kube-api-access-kg25t") pod "3683604e-9696-4c44-aad1-f72b971cac56" (UID: "3683604e-9696-4c44-aad1-f72b971cac56"). InnerVolumeSpecName "kube-api-access-kg25t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 23:27:16.690341 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:16.690260    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2fc2a5bf-fb7f-4d1b-8422-c28af24e71ff-openshift-service-ca\") pod \"2fc2a5bf-fb7f-4d1b-8422-c28af24e71ff\" (UID: \"2fc2a5bf-fb7f-4d1b-8422-c28af24e71ff\") "
Jul 31 23:27:16.690341 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:16.690299    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wtszk\" (UniqueName: \"kubernetes.io/projected/bdca4cce-1c0b-4711-9a87-ba7f4dadf6ba-kube-api-access-wtszk\") pod \"bdca4cce-1c0b-4711-9a87-ba7f4dadf6ba\" (UID: \"bdca4cce-1c0b-4711-9a87-ba7f4dadf6ba\") "
Jul 31 23:27:16.690557 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:16.690350    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2fc2a5bf-fb7f-4d1b-8422-c28af24e71ff-maas-api-tls\") pod \"2fc2a5bf-fb7f-4d1b-8422-c28af24e71ff\" (UID: \"2fc2a5bf-fb7f-4d1b-8422-c28af24e71ff\") "
Jul 31 23:27:16.690557 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:16.690366    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7kmqn\" (UniqueName: \"kubernetes.io/projected/2fc2a5bf-fb7f-4d1b-8422-c28af24e71ff-kube-api-access-7kmqn\") pod \"2fc2a5bf-fb7f-4d1b-8422-c28af24e71ff\" (UID: \"2fc2a5bf-fb7f-4d1b-8422-c28af24e71ff\") "
Jul 31 23:27:16.690557 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:16.690438    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bdca4cce-1c0b-4711-9a87-ba7f4dadf6ba-plugins-config-volume\") pod \"bdca4cce-1c0b-4711-9a87-ba7f4dadf6ba\" (UID: \"bdca4cce-1c0b-4711-9a87-ba7f4dadf6ba\") "
Jul 31 23:27:16.690736 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:16.690640    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2fc2a5bf-fb7f-4d1b-8422-c28af24e71ff-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2fc2a5bf-fb7f-4d1b-8422-c28af24e71ff" (UID: "2fc2a5bf-fb7f-4d1b-8422-c28af24e71ff"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:27:16.690736 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:16.690669    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kg25t\" (UniqueName: \"kubernetes.io/projected/3683604e-9696-4c44-aad1-f72b971cac56-kube-api-access-kg25t\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:27:16.690736 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:16.690690    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3683604e-9696-4c44-aad1-f72b971cac56-plugins-config-volume\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:27:16.691020 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:16.690993    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bdca4cce-1c0b-4711-9a87-ba7f4dadf6ba-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bdca4cce-1c0b-4711-9a87-ba7f4dadf6ba" (UID: "bdca4cce-1c0b-4711-9a87-ba7f4dadf6ba"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:27:16.692588 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:16.692560    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bdca4cce-1c0b-4711-9a87-ba7f4dadf6ba-kube-api-access-wtszk" (OuterVolumeSpecName: "kube-api-access-wtszk") pod "bdca4cce-1c0b-4711-9a87-ba7f4dadf6ba" (UID: "bdca4cce-1c0b-4711-9a87-ba7f4dadf6ba"). InnerVolumeSpecName "kube-api-access-wtszk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 23:27:16.692712 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:16.692676    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2fc2a5bf-fb7f-4d1b-8422-c28af24e71ff-kube-api-access-7kmqn" (OuterVolumeSpecName: "kube-api-access-7kmqn") pod "2fc2a5bf-fb7f-4d1b-8422-c28af24e71ff" (UID: "2fc2a5bf-fb7f-4d1b-8422-c28af24e71ff"). InnerVolumeSpecName "kube-api-access-7kmqn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 23:27:16.692774 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:16.692743    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2fc2a5bf-fb7f-4d1b-8422-c28af24e71ff-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2fc2a5bf-fb7f-4d1b-8422-c28af24e71ff" (UID: "2fc2a5bf-fb7f-4d1b-8422-c28af24e71ff"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 23:27:16.791711 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:16.791661    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2fc2a5bf-fb7f-4d1b-8422-c28af24e71ff-maas-api-tls\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:27:16.791711 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:16.791694    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7kmqn\" (UniqueName: \"kubernetes.io/projected/2fc2a5bf-fb7f-4d1b-8422-c28af24e71ff-kube-api-access-7kmqn\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:27:16.791910 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:16.791730    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bdca4cce-1c0b-4711-9a87-ba7f4dadf6ba-plugins-config-volume\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:27:16.791910 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:16.791740    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2fc2a5bf-fb7f-4d1b-8422-c28af24e71ff-openshift-service-ca\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:27:16.791910 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:16.791749    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wtszk\" (UniqueName: \"kubernetes.io/projected/bdca4cce-1c0b-4711-9a87-ba7f4dadf6ba-kube-api-access-wtszk\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:27:17.039518 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:17.039430    2584 generic.go:356] "Generic (PLEG): container finished" podID="bdca4cce-1c0b-4711-9a87-ba7f4dadf6ba" containerID="9216dfd4781800ff46bd9fbb38d814788d80491f36fd08cda6ab79474f327748" exitCode=0
Jul 31 23:27:17.039518 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:17.039479    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-fec77b77-5cf88f98cb-4zfn6"
Jul 31 23:27:17.039845 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:17.039529    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-fec77b77-5cf88f98cb-4zfn6" event={"ID":"bdca4cce-1c0b-4711-9a87-ba7f4dadf6ba","Type":"ContainerDied","Data":"9216dfd4781800ff46bd9fbb38d814788d80491f36fd08cda6ab79474f327748"}
Jul 31 23:27:17.039845 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:17.039591    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-fec77b77-5cf88f98cb-4zfn6" event={"ID":"bdca4cce-1c0b-4711-9a87-ba7f4dadf6ba","Type":"ContainerDied","Data":"69466f5e6ee01a37f07af4d35af29dab21e4f171b9f5e95d866f2b731cfac8e5"}
Jul 31 23:27:17.039845 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:17.039622    2584 scope.go:117] "RemoveContainer" containerID="9216dfd4781800ff46bd9fbb38d814788d80491f36fd08cda6ab79474f327748"
Jul 31 23:27:17.041035 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:17.041011    2584 generic.go:356] "Generic (PLEG): container finished" podID="2fc2a5bf-fb7f-4d1b-8422-c28af24e71ff" containerID="ec98a002e0b25a96705bf3380f64e778c6b7b0180e986120384f750a5dbf9de2" exitCode=0
Jul 31 23:27:17.041148 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:17.041056    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fec77b77-78dff4696b-vp5gc"
Jul 31 23:27:17.041201 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:17.041050    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fec77b77-78dff4696b-vp5gc" event={"ID":"2fc2a5bf-fb7f-4d1b-8422-c28af24e71ff","Type":"ContainerDied","Data":"ec98a002e0b25a96705bf3380f64e778c6b7b0180e986120384f750a5dbf9de2"}
Jul 31 23:27:17.041252 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:17.041207    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fec77b77-78dff4696b-vp5gc" event={"ID":"2fc2a5bf-fb7f-4d1b-8422-c28af24e71ff","Type":"ContainerDied","Data":"2bdf1ee46b1a9242e163c01155b93944b7556a663b3e1bbff3ef69c6cbce7e9d"}
Jul 31 23:27:17.042542 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:17.042519    2584 generic.go:356] "Generic (PLEG): container finished" podID="3683604e-9696-4c44-aad1-f72b971cac56" containerID="11417adf64fd667ecde15641db0e85ae213cc88102b4f4b859a805530ea023e3" exitCode=0
Jul 31 23:27:17.042661 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:17.042556    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-fec77b77-9864f5b4-jtq9w"
Jul 31 23:27:17.042661 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:17.042586    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-fec77b77-9864f5b4-jtq9w" event={"ID":"3683604e-9696-4c44-aad1-f72b971cac56","Type":"ContainerDied","Data":"11417adf64fd667ecde15641db0e85ae213cc88102b4f4b859a805530ea023e3"}
Jul 31 23:27:17.042661 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:17.042628    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-fec77b77-9864f5b4-jtq9w" event={"ID":"3683604e-9696-4c44-aad1-f72b971cac56","Type":"ContainerDied","Data":"8b08b39553dd25bf55b2d13ff71530afeff8845f8d78670fadb1df1ddac19a6f"}
Jul 31 23:27:17.049138 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:17.049121    2584 scope.go:117] "RemoveContainer" containerID="9216dfd4781800ff46bd9fbb38d814788d80491f36fd08cda6ab79474f327748"
Jul 31 23:27:17.049393 ip-10-0-128-243 kubenswrapper[2584]: E0731 23:27:17.049374    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9216dfd4781800ff46bd9fbb38d814788d80491f36fd08cda6ab79474f327748\": container with ID starting with 9216dfd4781800ff46bd9fbb38d814788d80491f36fd08cda6ab79474f327748 not found: ID does not exist" containerID="9216dfd4781800ff46bd9fbb38d814788d80491f36fd08cda6ab79474f327748"
Jul 31 23:27:17.049456 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:17.049400    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9216dfd4781800ff46bd9fbb38d814788d80491f36fd08cda6ab79474f327748"} err="failed to get container status \"9216dfd4781800ff46bd9fbb38d814788d80491f36fd08cda6ab79474f327748\": rpc error: code = NotFound desc = could not find container \"9216dfd4781800ff46bd9fbb38d814788d80491f36fd08cda6ab79474f327748\": container with ID starting with 9216dfd4781800ff46bd9fbb38d814788d80491f36fd08cda6ab79474f327748 not found: ID does not exist"
Jul 31 23:27:17.049456 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:17.049418    2584 scope.go:117] "RemoveContainer" containerID="ec98a002e0b25a96705bf3380f64e778c6b7b0180e986120384f750a5dbf9de2"
Jul 31 23:27:17.057202 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:17.057186    2584 scope.go:117] "RemoveContainer" containerID="ec98a002e0b25a96705bf3380f64e778c6b7b0180e986120384f750a5dbf9de2"
Jul 31 23:27:17.057450 ip-10-0-128-243 kubenswrapper[2584]: E0731 23:27:17.057429    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec98a002e0b25a96705bf3380f64e778c6b7b0180e986120384f750a5dbf9de2\": container with ID starting with ec98a002e0b25a96705bf3380f64e778c6b7b0180e986120384f750a5dbf9de2 not found: ID does not exist" containerID="ec98a002e0b25a96705bf3380f64e778c6b7b0180e986120384f750a5dbf9de2"
Jul 31 23:27:17.057496 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:17.057457    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec98a002e0b25a96705bf3380f64e778c6b7b0180e986120384f750a5dbf9de2"} err="failed to get container status \"ec98a002e0b25a96705bf3380f64e778c6b7b0180e986120384f750a5dbf9de2\": rpc error: code = NotFound desc = could not find container \"ec98a002e0b25a96705bf3380f64e778c6b7b0180e986120384f750a5dbf9de2\": container with ID starting with ec98a002e0b25a96705bf3380f64e778c6b7b0180e986120384f750a5dbf9de2 not found: ID does not exist"
Jul 31 23:27:17.057496 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:17.057478    2584 scope.go:117] "RemoveContainer" containerID="11417adf64fd667ecde15641db0e85ae213cc88102b4f4b859a805530ea023e3"
Jul 31 23:27:17.064121 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:17.064100    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fec77b77-78dff4696b-vp5gc"]
Jul 31 23:27:17.065449 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:17.065434    2584 scope.go:117] "RemoveContainer" containerID="11417adf64fd667ecde15641db0e85ae213cc88102b4f4b859a805530ea023e3"
Jul 31 23:27:17.065920 ip-10-0-128-243 kubenswrapper[2584]: E0731 23:27:17.065893    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11417adf64fd667ecde15641db0e85ae213cc88102b4f4b859a805530ea023e3\": container with ID starting with 11417adf64fd667ecde15641db0e85ae213cc88102b4f4b859a805530ea023e3 not found: ID does not exist" containerID="11417adf64fd667ecde15641db0e85ae213cc88102b4f4b859a805530ea023e3"
Jul 31 23:27:17.066048 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:17.065931    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11417adf64fd667ecde15641db0e85ae213cc88102b4f4b859a805530ea023e3"} err="failed to get container status \"11417adf64fd667ecde15641db0e85ae213cc88102b4f4b859a805530ea023e3\": rpc error: code = NotFound desc = could not find container \"11417adf64fd667ecde15641db0e85ae213cc88102b4f4b859a805530ea023e3\": container with ID starting with 11417adf64fd667ecde15641db0e85ae213cc88102b4f4b859a805530ea023e3 not found: ID does not exist"
Jul 31 23:27:17.067682 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:17.067664    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fec77b77-78dff4696b-vp5gc"]
Jul 31 23:27:17.077220 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:17.077200    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-fec77b77-9864f5b4-jtq9w"]
Jul 31 23:27:17.081592 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:17.081571    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-fec77b77-9864f5b4-jtq9w"]
Jul 31 23:27:17.090408 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:17.090389    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-fec77b77-5cf88f98cb-4zfn6"]
Jul 31 23:27:17.093763 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:17.093745    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-fec77b77-5cf88f98cb-4zfn6"]
Jul 31 23:27:18.891258 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:18.891224    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2fc2a5bf-fb7f-4d1b-8422-c28af24e71ff" path="/var/lib/kubelet/pods/2fc2a5bf-fb7f-4d1b-8422-c28af24e71ff/volumes"
Jul 31 23:27:18.891608 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:18.891577    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3683604e-9696-4c44-aad1-f72b971cac56" path="/var/lib/kubelet/pods/3683604e-9696-4c44-aad1-f72b971cac56/volumes"
Jul 31 23:27:18.891923 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:18.891909    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bdca4cce-1c0b-4711-9a87-ba7f4dadf6ba" path="/var/lib/kubelet/pods/bdca4cce-1c0b-4711-9a87-ba7f4dadf6ba/volumes"
Jul 31 23:27:19.909516 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:19.909480    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-13204f82-5d75bbc95-gh75s"]
Jul 31 23:27:19.910039 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:19.909880    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bdca4cce-1c0b-4711-9a87-ba7f4dadf6ba" containerName="payload-pre-processing"
Jul 31 23:27:19.910039 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:19.909898    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="bdca4cce-1c0b-4711-9a87-ba7f4dadf6ba" containerName="payload-pre-processing"
Jul 31 23:27:19.910039 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:19.909914    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2fc2a5bf-fb7f-4d1b-8422-c28af24e71ff" containerName="maas-api"
Jul 31 23:27:19.910039 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:19.909924    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="2fc2a5bf-fb7f-4d1b-8422-c28af24e71ff" containerName="maas-api"
Jul 31 23:27:19.910039 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:19.909958    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3683604e-9696-4c44-aad1-f72b971cac56" containerName="payload-processing"
Jul 31 23:27:19.910039 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:19.909966    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="3683604e-9696-4c44-aad1-f72b971cac56" containerName="payload-processing"
Jul 31 23:27:19.910347 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:19.910056    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="bdca4cce-1c0b-4711-9a87-ba7f4dadf6ba" containerName="payload-pre-processing"
Jul 31 23:27:19.910347 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:19.910071    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="2fc2a5bf-fb7f-4d1b-8422-c28af24e71ff" containerName="maas-api"
Jul 31 23:27:19.910347 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:19.910079    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="3683604e-9696-4c44-aad1-f72b971cac56" containerName="payload-processing"
Jul 31 23:27:19.914784 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:19.914760    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-13204f82-5d75bbc95-gh75s"
Jul 31 23:27:19.917206 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:19.917183    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-13204f82-serving-cert\""
Jul 31 23:27:19.921943 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:19.921916    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-13204f82-5d75bbc95-gh75s"]
Jul 31 23:27:20.027072 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:20.027039    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ebd2d52-7bbc-496e-a539-ab9eacf42eb1-maas-api-tls\") pod \"maas-api-e2e-migrate-13204f82-5d75bbc95-gh75s\" (UID: \"9ebd2d52-7bbc-496e-a539-ab9eacf42eb1\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-13204f82-5d75bbc95-gh75s"
Jul 31 23:27:20.027072 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:20.027083    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ebd2d52-7bbc-496e-a539-ab9eacf42eb1-openshift-service-ca\") pod \"maas-api-e2e-migrate-13204f82-5d75bbc95-gh75s\" (UID: \"9ebd2d52-7bbc-496e-a539-ab9eacf42eb1\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-13204f82-5d75bbc95-gh75s"
Jul 31 23:27:20.027295 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:20.027145    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cstl4\" (UniqueName: \"kubernetes.io/projected/9ebd2d52-7bbc-496e-a539-ab9eacf42eb1-kube-api-access-cstl4\") pod \"maas-api-e2e-migrate-13204f82-5d75bbc95-gh75s\" (UID: \"9ebd2d52-7bbc-496e-a539-ab9eacf42eb1\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-13204f82-5d75bbc95-gh75s"
Jul 31 23:27:20.128434 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:20.128394    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ebd2d52-7bbc-496e-a539-ab9eacf42eb1-maas-api-tls\") pod \"maas-api-e2e-migrate-13204f82-5d75bbc95-gh75s\" (UID: \"9ebd2d52-7bbc-496e-a539-ab9eacf42eb1\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-13204f82-5d75bbc95-gh75s"
Jul 31 23:27:20.128434 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:20.128448    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ebd2d52-7bbc-496e-a539-ab9eacf42eb1-openshift-service-ca\") pod \"maas-api-e2e-migrate-13204f82-5d75bbc95-gh75s\" (UID: \"9ebd2d52-7bbc-496e-a539-ab9eacf42eb1\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-13204f82-5d75bbc95-gh75s"
Jul 31 23:27:20.128749 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:20.128472    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cstl4\" (UniqueName: \"kubernetes.io/projected/9ebd2d52-7bbc-496e-a539-ab9eacf42eb1-kube-api-access-cstl4\") pod \"maas-api-e2e-migrate-13204f82-5d75bbc95-gh75s\" (UID: \"9ebd2d52-7bbc-496e-a539-ab9eacf42eb1\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-13204f82-5d75bbc95-gh75s"
Jul 31 23:27:20.129147 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:20.129124    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ebd2d52-7bbc-496e-a539-ab9eacf42eb1-openshift-service-ca\") pod \"maas-api-e2e-migrate-13204f82-5d75bbc95-gh75s\" (UID: \"9ebd2d52-7bbc-496e-a539-ab9eacf42eb1\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-13204f82-5d75bbc95-gh75s"
Jul 31 23:27:20.130997 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:20.130975    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ebd2d52-7bbc-496e-a539-ab9eacf42eb1-maas-api-tls\") pod \"maas-api-e2e-migrate-13204f82-5d75bbc95-gh75s\" (UID: \"9ebd2d52-7bbc-496e-a539-ab9eacf42eb1\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-13204f82-5d75bbc95-gh75s"
Jul 31 23:27:20.136865 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:20.136842    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cstl4\" (UniqueName: \"kubernetes.io/projected/9ebd2d52-7bbc-496e-a539-ab9eacf42eb1-kube-api-access-cstl4\") pod \"maas-api-e2e-migrate-13204f82-5d75bbc95-gh75s\" (UID: \"9ebd2d52-7bbc-496e-a539-ab9eacf42eb1\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-13204f82-5d75bbc95-gh75s"
Jul 31 23:27:20.228355 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:20.228260    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-13204f82-5d75bbc95-gh75s"
Jul 31 23:27:20.374219 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:20.374191    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-13204f82-5d75bbc95-gh75s"]
Jul 31 23:27:20.376498 ip-10-0-128-243 kubenswrapper[2584]: W0731 23:27:20.376460    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ebd2d52_7bbc_496e_a539_ab9eacf42eb1.slice/crio-6f196b222596245bfa225d0e0470bee1f9f7b108abf420a4502031f6393a7383 WatchSource:0}: Error finding container 6f196b222596245bfa225d0e0470bee1f9f7b108abf420a4502031f6393a7383: Status 404 returned error can't find the container with id 6f196b222596245bfa225d0e0470bee1f9f7b108abf420a4502031f6393a7383
Jul 31 23:27:20.533428 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:20.533350    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-13204f82-bbd9b8b76-ftpn5"]
Jul 31 23:27:20.538050 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:20.538026    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-13204f82-bbd9b8b76-ftpn5"
Jul 31 23:27:20.540783 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:20.540763    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-13204f82-dockercfg-kqfhx\""
Jul 31 23:27:20.540921 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:20.540816    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-13204f82\""
Jul 31 23:27:20.546403 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:20.546357    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-13204f82-bbd9b8b76-ftpn5"]
Jul 31 23:27:20.573864 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:20.573832    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-13204f82-577d7795d5-t7qtd"]
Jul 31 23:27:20.577088 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:20.577072    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-13204f82-577d7795d5-t7qtd"
Jul 31 23:27:20.586958 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:20.586932    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-13204f82-577d7795d5-t7qtd"]
Jul 31 23:27:20.632322 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:20.632270    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c8a785c-4de3-40f1-be52-9ba1294f3df1-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-13204f82-bbd9b8b76-ftpn5\" (UID: \"9c8a785c-4de3-40f1-be52-9ba1294f3df1\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-13204f82-bbd9b8b76-ftpn5"
Jul 31 23:27:20.632322 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:20.632319    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wcpgs\" (UniqueName: \"kubernetes.io/projected/9c8a785c-4de3-40f1-be52-9ba1294f3df1-kube-api-access-wcpgs\") pod \"payload-pre-processing-e2e-migrate-13204f82-bbd9b8b76-ftpn5\" (UID: \"9c8a785c-4de3-40f1-be52-9ba1294f3df1\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-13204f82-bbd9b8b76-ftpn5"
Jul 31 23:27:20.733786 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:20.733746    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ac3bbf5-596c-40d0-a5bd-a8a9f81d507f-plugins-config-volume\") pod \"payload-processing-e2e-migrate-13204f82-577d7795d5-t7qtd\" (UID: \"7ac3bbf5-596c-40d0-a5bd-a8a9f81d507f\") " pod="openshift-ingress/payload-processing-e2e-migrate-13204f82-577d7795d5-t7qtd"
Jul 31 23:27:20.733936 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:20.733789    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c8a785c-4de3-40f1-be52-9ba1294f3df1-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-13204f82-bbd9b8b76-ftpn5\" (UID: \"9c8a785c-4de3-40f1-be52-9ba1294f3df1\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-13204f82-bbd9b8b76-ftpn5"
Jul 31 23:27:20.733936 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:20.733817    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcpgs\" (UniqueName: \"kubernetes.io/projected/9c8a785c-4de3-40f1-be52-9ba1294f3df1-kube-api-access-wcpgs\") pod \"payload-pre-processing-e2e-migrate-13204f82-bbd9b8b76-ftpn5\" (UID: \"9c8a785c-4de3-40f1-be52-9ba1294f3df1\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-13204f82-bbd9b8b76-ftpn5"
Jul 31 23:27:20.733936 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:20.733901    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p69jh\" (UniqueName: \"kubernetes.io/projected/7ac3bbf5-596c-40d0-a5bd-a8a9f81d507f-kube-api-access-p69jh\") pod \"payload-processing-e2e-migrate-13204f82-577d7795d5-t7qtd\" (UID: \"7ac3bbf5-596c-40d0-a5bd-a8a9f81d507f\") " pod="openshift-ingress/payload-processing-e2e-migrate-13204f82-577d7795d5-t7qtd"
Jul 31 23:27:20.734763 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:20.734733    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c8a785c-4de3-40f1-be52-9ba1294f3df1-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-13204f82-bbd9b8b76-ftpn5\" (UID: \"9c8a785c-4de3-40f1-be52-9ba1294f3df1\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-13204f82-bbd9b8b76-ftpn5"
Jul 31 23:27:20.741955 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:20.741934    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wcpgs\" (UniqueName: \"kubernetes.io/projected/9c8a785c-4de3-40f1-be52-9ba1294f3df1-kube-api-access-wcpgs\") pod \"payload-pre-processing-e2e-migrate-13204f82-bbd9b8b76-ftpn5\" (UID: \"9c8a785c-4de3-40f1-be52-9ba1294f3df1\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-13204f82-bbd9b8b76-ftpn5"
Jul 31 23:27:20.834408 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:20.834365    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ac3bbf5-596c-40d0-a5bd-a8a9f81d507f-plugins-config-volume\") pod \"payload-processing-e2e-migrate-13204f82-577d7795d5-t7qtd\" (UID: \"7ac3bbf5-596c-40d0-a5bd-a8a9f81d507f\") " pod="openshift-ingress/payload-processing-e2e-migrate-13204f82-577d7795d5-t7qtd"
Jul 31 23:27:20.834408 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:20.834410    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p69jh\" (UniqueName: \"kubernetes.io/projected/7ac3bbf5-596c-40d0-a5bd-a8a9f81d507f-kube-api-access-p69jh\") pod \"payload-processing-e2e-migrate-13204f82-577d7795d5-t7qtd\" (UID: \"7ac3bbf5-596c-40d0-a5bd-a8a9f81d507f\") " pod="openshift-ingress/payload-processing-e2e-migrate-13204f82-577d7795d5-t7qtd"
Jul 31 23:27:20.835095 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:20.835073    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ac3bbf5-596c-40d0-a5bd-a8a9f81d507f-plugins-config-volume\") pod \"payload-processing-e2e-migrate-13204f82-577d7795d5-t7qtd\" (UID: \"7ac3bbf5-596c-40d0-a5bd-a8a9f81d507f\") " pod="openshift-ingress/payload-processing-e2e-migrate-13204f82-577d7795d5-t7qtd"
Jul 31 23:27:20.843933 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:20.843906    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p69jh\" (UniqueName: \"kubernetes.io/projected/7ac3bbf5-596c-40d0-a5bd-a8a9f81d507f-kube-api-access-p69jh\") pod \"payload-processing-e2e-migrate-13204f82-577d7795d5-t7qtd\" (UID: \"7ac3bbf5-596c-40d0-a5bd-a8a9f81d507f\") " pod="openshift-ingress/payload-processing-e2e-migrate-13204f82-577d7795d5-t7qtd"
Jul 31 23:27:20.850717 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:20.850682    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-13204f82-bbd9b8b76-ftpn5"
Jul 31 23:27:20.888388 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:20.888359    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-13204f82-577d7795d5-t7qtd"
Jul 31 23:27:20.999288 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:20.999254    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-13204f82-bbd9b8b76-ftpn5"]
Jul 31 23:27:21.002688 ip-10-0-128-243 kubenswrapper[2584]: W0731 23:27:21.002640    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c8a785c_4de3_40f1_be52_9ba1294f3df1.slice/crio-8ad53a154891f1e3a06f012ce90f62208ca31cac4119c824da476db381a39776 WatchSource:0}: Error finding container 8ad53a154891f1e3a06f012ce90f62208ca31cac4119c824da476db381a39776: Status 404 returned error can't find the container with id 8ad53a154891f1e3a06f012ce90f62208ca31cac4119c824da476db381a39776
Jul 31 23:27:21.058889 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:21.058856    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-13204f82-577d7795d5-t7qtd"]
Jul 31 23:27:21.063977 ip-10-0-128-243 kubenswrapper[2584]: W0731 23:27:21.063902    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ac3bbf5_596c_40d0_a5bd_a8a9f81d507f.slice/crio-ef0bb812f045aa89fd4e4c96d1f7ae3356231930c52973b61d29db901ab974c5 WatchSource:0}: Error finding container ef0bb812f045aa89fd4e4c96d1f7ae3356231930c52973b61d29db901ab974c5: Status 404 returned error can't find the container with id ef0bb812f045aa89fd4e4c96d1f7ae3356231930c52973b61d29db901ab974c5
Jul 31 23:27:21.070833 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:21.070212    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-13204f82-5d75bbc95-gh75s" event={"ID":"9ebd2d52-7bbc-496e-a539-ab9eacf42eb1","Type":"ContainerStarted","Data":"33a7a96d04cbdd6b22f2cf9c3201607c37c9c16698fee494587d43a62b715690"}
Jul 31 23:27:21.070833 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:21.070274    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-13204f82-5d75bbc95-gh75s" event={"ID":"9ebd2d52-7bbc-496e-a539-ab9eacf42eb1","Type":"ContainerStarted","Data":"6f196b222596245bfa225d0e0470bee1f9f7b108abf420a4502031f6393a7383"}
Jul 31 23:27:21.070833 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:21.070330    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-13204f82-5d75bbc95-gh75s"
Jul 31 23:27:21.073243 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:21.073216    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-13204f82-bbd9b8b76-ftpn5" event={"ID":"9c8a785c-4de3-40f1-be52-9ba1294f3df1","Type":"ContainerStarted","Data":"8ad53a154891f1e3a06f012ce90f62208ca31cac4119c824da476db381a39776"}
Jul 31 23:27:21.086270 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:21.086197    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-13204f82-5d75bbc95-gh75s" podStartSLOduration=1.761826095 podStartE2EDuration="2.086180598s" totalImagesPullingTime="324.354503ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:27:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:27:21.08479749 +0000 UTC m=+1824.730583799" watchObservedRunningTime="2026-07-31 23:27:21.086180598 +0000 UTC m=+1824.731966908"
Jul 31 23:27:22.080516 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:22.080481    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-13204f82-577d7795d5-t7qtd" event={"ID":"7ac3bbf5-596c-40d0-a5bd-a8a9f81d507f","Type":"ContainerStarted","Data":"1d6ccede17b31e7006f5e598210b410d5c3a7a6f8e76c5c4e30bbed95de13f70"}
Jul 31 23:27:22.081089 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:22.080564    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-13204f82-577d7795d5-t7qtd" event={"ID":"7ac3bbf5-596c-40d0-a5bd-a8a9f81d507f","Type":"ContainerStarted","Data":"ef0bb812f045aa89fd4e4c96d1f7ae3356231930c52973b61d29db901ab974c5"}
Jul 31 23:27:22.081089 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:22.080677    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-13204f82-577d7795d5-t7qtd"
Jul 31 23:27:22.082093 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:22.082068    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-13204f82-bbd9b8b76-ftpn5" event={"ID":"9c8a785c-4de3-40f1-be52-9ba1294f3df1","Type":"ContainerStarted","Data":"6ba9e79ccc17d149833e95f020908b19a25337800c623d4c1364ca16904a8810"}
Jul 31 23:27:22.082182 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:22.082128    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-13204f82-bbd9b8b76-ftpn5"
Jul 31 23:27:22.096337 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:22.096289    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-13204f82-577d7795d5-t7qtd" podStartSLOduration=2.096274399 podStartE2EDuration="2.096274399s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:27:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:27:22.095289937 +0000 UTC m=+1825.741076248" watchObservedRunningTime="2026-07-31 23:27:22.096274399 +0000 UTC m=+1825.742060708"
Jul 31 23:27:22.109972 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:22.109928    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-13204f82-bbd9b8b76-ftpn5" podStartSLOduration=2.109913935 podStartE2EDuration="2.109913935s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:27:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:27:22.109016231 +0000 UTC m=+1825.754802542" watchObservedRunningTime="2026-07-31 23:27:22.109913935 +0000 UTC m=+1825.755700244"
Jul 31 23:27:27.088166 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:27.088140    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-13204f82-5d75bbc95-gh75s"
Jul 31 23:27:33.089429 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:33.089399    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-13204f82-577d7795d5-t7qtd"
Jul 31 23:27:33.089842 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:33.089472    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-13204f82-bbd9b8b76-ftpn5"
Jul 31 23:27:50.393030 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:50.392979    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-13204f82-5d75bbc95-gh75s"]
Jul 31 23:27:50.394361 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:50.393226    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-13204f82-5d75bbc95-gh75s" podUID="9ebd2d52-7bbc-496e-a539-ab9eacf42eb1" containerName="maas-api" containerID="cri-o://33a7a96d04cbdd6b22f2cf9c3201607c37c9c16698fee494587d43a62b715690" gracePeriod=30
Jul 31 23:27:50.444499 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:50.444463    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-13204f82-577d7795d5-t7qtd"]
Jul 31 23:27:50.444761 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:50.444735    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-13204f82-577d7795d5-t7qtd" podUID="7ac3bbf5-596c-40d0-a5bd-a8a9f81d507f" containerName="payload-processing" containerID="cri-o://1d6ccede17b31e7006f5e598210b410d5c3a7a6f8e76c5c4e30bbed95de13f70" gracePeriod=30
Jul 31 23:27:50.449125 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:50.449098    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-13204f82-bbd9b8b76-ftpn5"]
Jul 31 23:27:50.449409 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:50.449361    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-13204f82-bbd9b8b76-ftpn5" podUID="9c8a785c-4de3-40f1-be52-9ba1294f3df1" containerName="payload-pre-processing" containerID="cri-o://6ba9e79ccc17d149833e95f020908b19a25337800c623d4c1364ca16904a8810" gracePeriod=30
Jul 31 23:27:50.727294 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:50.727264    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-13204f82-5d75bbc95-gh75s"
Jul 31 23:27:50.743051 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:50.743028    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-13204f82-577d7795d5-t7qtd"
Jul 31 23:27:50.747666 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:50.747648    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-13204f82-bbd9b8b76-ftpn5"
Jul 31 23:27:50.806408 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:50.806337    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ebd2d52-7bbc-496e-a539-ab9eacf42eb1-openshift-service-ca\") pod \"9ebd2d52-7bbc-496e-a539-ab9eacf42eb1\" (UID: \"9ebd2d52-7bbc-496e-a539-ab9eacf42eb1\") "
Jul 31 23:27:50.806605 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:50.806425    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cstl4\" (UniqueName: \"kubernetes.io/projected/9ebd2d52-7bbc-496e-a539-ab9eacf42eb1-kube-api-access-cstl4\") pod \"9ebd2d52-7bbc-496e-a539-ab9eacf42eb1\" (UID: \"9ebd2d52-7bbc-496e-a539-ab9eacf42eb1\") "
Jul 31 23:27:50.806605 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:50.806478    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ac3bbf5-596c-40d0-a5bd-a8a9f81d507f-plugins-config-volume\") pod \"7ac3bbf5-596c-40d0-a5bd-a8a9f81d507f\" (UID: \"7ac3bbf5-596c-40d0-a5bd-a8a9f81d507f\") "
Jul 31 23:27:50.806605 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:50.806520    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wcpgs\" (UniqueName: \"kubernetes.io/projected/9c8a785c-4de3-40f1-be52-9ba1294f3df1-kube-api-access-wcpgs\") pod \"9c8a785c-4de3-40f1-be52-9ba1294f3df1\" (UID: \"9c8a785c-4de3-40f1-be52-9ba1294f3df1\") "
Jul 31 23:27:50.806605 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:50.806544    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p69jh\" (UniqueName: \"kubernetes.io/projected/7ac3bbf5-596c-40d0-a5bd-a8a9f81d507f-kube-api-access-p69jh\") pod \"7ac3bbf5-596c-40d0-a5bd-a8a9f81d507f\" (UID: \"7ac3bbf5-596c-40d0-a5bd-a8a9f81d507f\") "
Jul 31 23:27:50.806962 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:50.806620    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ebd2d52-7bbc-496e-a539-ab9eacf42eb1-maas-api-tls\") pod \"9ebd2d52-7bbc-496e-a539-ab9eacf42eb1\" (UID: \"9ebd2d52-7bbc-496e-a539-ab9eacf42eb1\") "
Jul 31 23:27:50.806962 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:50.806664    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c8a785c-4de3-40f1-be52-9ba1294f3df1-plugins-config-volume\") pod \"9c8a785c-4de3-40f1-be52-9ba1294f3df1\" (UID: \"9c8a785c-4de3-40f1-be52-9ba1294f3df1\") "
Jul 31 23:27:50.806962 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:50.806802    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ebd2d52-7bbc-496e-a539-ab9eacf42eb1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9ebd2d52-7bbc-496e-a539-ab9eacf42eb1" (UID: "9ebd2d52-7bbc-496e-a539-ab9eacf42eb1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:27:50.807118 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:50.806988    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ebd2d52-7bbc-496e-a539-ab9eacf42eb1-openshift-service-ca\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:27:50.807215 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:50.807186    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ac3bbf5-596c-40d0-a5bd-a8a9f81d507f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7ac3bbf5-596c-40d0-a5bd-a8a9f81d507f" (UID: "7ac3bbf5-596c-40d0-a5bd-a8a9f81d507f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:27:50.807271 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:50.807215    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c8a785c-4de3-40f1-be52-9ba1294f3df1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9c8a785c-4de3-40f1-be52-9ba1294f3df1" (UID: "9c8a785c-4de3-40f1-be52-9ba1294f3df1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:27:50.809149 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:50.809112    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c8a785c-4de3-40f1-be52-9ba1294f3df1-kube-api-access-wcpgs" (OuterVolumeSpecName: "kube-api-access-wcpgs") pod "9c8a785c-4de3-40f1-be52-9ba1294f3df1" (UID: "9c8a785c-4de3-40f1-be52-9ba1294f3df1"). InnerVolumeSpecName "kube-api-access-wcpgs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 23:27:50.809286 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:50.809171    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ebd2d52-7bbc-496e-a539-ab9eacf42eb1-kube-api-access-cstl4" (OuterVolumeSpecName: "kube-api-access-cstl4") pod "9ebd2d52-7bbc-496e-a539-ab9eacf42eb1" (UID: "9ebd2d52-7bbc-496e-a539-ab9eacf42eb1"). InnerVolumeSpecName "kube-api-access-cstl4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 23:27:50.809347 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:50.809319    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ac3bbf5-596c-40d0-a5bd-a8a9f81d507f-kube-api-access-p69jh" (OuterVolumeSpecName: "kube-api-access-p69jh") pod "7ac3bbf5-596c-40d0-a5bd-a8a9f81d507f" (UID: "7ac3bbf5-596c-40d0-a5bd-a8a9f81d507f"). InnerVolumeSpecName "kube-api-access-p69jh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 23:27:50.809435 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:50.809419    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ebd2d52-7bbc-496e-a539-ab9eacf42eb1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9ebd2d52-7bbc-496e-a539-ab9eacf42eb1" (UID: "9ebd2d52-7bbc-496e-a539-ab9eacf42eb1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 23:27:50.908405 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:50.908381    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cstl4\" (UniqueName: \"kubernetes.io/projected/9ebd2d52-7bbc-496e-a539-ab9eacf42eb1-kube-api-access-cstl4\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:27:50.908405 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:50.908408    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ac3bbf5-596c-40d0-a5bd-a8a9f81d507f-plugins-config-volume\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:27:50.908557 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:50.908421    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wcpgs\" (UniqueName: \"kubernetes.io/projected/9c8a785c-4de3-40f1-be52-9ba1294f3df1-kube-api-access-wcpgs\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:27:50.908557 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:50.908432    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p69jh\" (UniqueName: \"kubernetes.io/projected/7ac3bbf5-596c-40d0-a5bd-a8a9f81d507f-kube-api-access-p69jh\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:27:50.908557 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:50.908444    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ebd2d52-7bbc-496e-a539-ab9eacf42eb1-maas-api-tls\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:27:50.908557 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:50.908456    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c8a785c-4de3-40f1-be52-9ba1294f3df1-plugins-config-volume\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:27:51.230918 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:51.230883    2584 generic.go:356] "Generic (PLEG): container finished" podID="7ac3bbf5-596c-40d0-a5bd-a8a9f81d507f" containerID="1d6ccede17b31e7006f5e598210b410d5c3a7a6f8e76c5c4e30bbed95de13f70" exitCode=0
Jul 31 23:27:51.231113 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:51.230931    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-13204f82-577d7795d5-t7qtd"
Jul 31 23:27:51.231113 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:51.230967    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-13204f82-577d7795d5-t7qtd" event={"ID":"7ac3bbf5-596c-40d0-a5bd-a8a9f81d507f","Type":"ContainerDied","Data":"1d6ccede17b31e7006f5e598210b410d5c3a7a6f8e76c5c4e30bbed95de13f70"}
Jul 31 23:27:51.231113 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:51.231022    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-13204f82-577d7795d5-t7qtd" event={"ID":"7ac3bbf5-596c-40d0-a5bd-a8a9f81d507f","Type":"ContainerDied","Data":"ef0bb812f045aa89fd4e4c96d1f7ae3356231930c52973b61d29db901ab974c5"}
Jul 31 23:27:51.231113 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:51.231048    2584 scope.go:117] "RemoveContainer" containerID="1d6ccede17b31e7006f5e598210b410d5c3a7a6f8e76c5c4e30bbed95de13f70"
Jul 31 23:27:51.232636 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:51.232471    2584 generic.go:356] "Generic (PLEG): container finished" podID="9c8a785c-4de3-40f1-be52-9ba1294f3df1" containerID="6ba9e79ccc17d149833e95f020908b19a25337800c623d4c1364ca16904a8810" exitCode=0
Jul 31 23:27:51.232636 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:51.232520    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-13204f82-bbd9b8b76-ftpn5"
Jul 31 23:27:51.232636 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:51.232577    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-13204f82-bbd9b8b76-ftpn5" event={"ID":"9c8a785c-4de3-40f1-be52-9ba1294f3df1","Type":"ContainerDied","Data":"6ba9e79ccc17d149833e95f020908b19a25337800c623d4c1364ca16904a8810"}
Jul 31 23:27:51.232636 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:51.232626    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-13204f82-bbd9b8b76-ftpn5" event={"ID":"9c8a785c-4de3-40f1-be52-9ba1294f3df1","Type":"ContainerDied","Data":"8ad53a154891f1e3a06f012ce90f62208ca31cac4119c824da476db381a39776"}
Jul 31 23:27:51.235039 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:51.235016    2584 generic.go:356] "Generic (PLEG): container finished" podID="9ebd2d52-7bbc-496e-a539-ab9eacf42eb1" containerID="33a7a96d04cbdd6b22f2cf9c3201607c37c9c16698fee494587d43a62b715690" exitCode=0
Jul 31 23:27:51.235167 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:51.235082    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-13204f82-5d75bbc95-gh75s"
Jul 31 23:27:51.235371 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:51.235085    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-13204f82-5d75bbc95-gh75s" event={"ID":"9ebd2d52-7bbc-496e-a539-ab9eacf42eb1","Type":"ContainerDied","Data":"33a7a96d04cbdd6b22f2cf9c3201607c37c9c16698fee494587d43a62b715690"}
Jul 31 23:27:51.235794 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:51.235772    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-13204f82-5d75bbc95-gh75s" event={"ID":"9ebd2d52-7bbc-496e-a539-ab9eacf42eb1","Type":"ContainerDied","Data":"6f196b222596245bfa225d0e0470bee1f9f7b108abf420a4502031f6393a7383"}
Jul 31 23:27:51.240478 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:51.240458    2584 scope.go:117] "RemoveContainer" containerID="1d6ccede17b31e7006f5e598210b410d5c3a7a6f8e76c5c4e30bbed95de13f70"
Jul 31 23:27:51.240735 ip-10-0-128-243 kubenswrapper[2584]: E0731 23:27:51.240715    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d6ccede17b31e7006f5e598210b410d5c3a7a6f8e76c5c4e30bbed95de13f70\": container with ID starting with 1d6ccede17b31e7006f5e598210b410d5c3a7a6f8e76c5c4e30bbed95de13f70 not found: ID does not exist" containerID="1d6ccede17b31e7006f5e598210b410d5c3a7a6f8e76c5c4e30bbed95de13f70"
Jul 31 23:27:51.240787 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:51.240744    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d6ccede17b31e7006f5e598210b410d5c3a7a6f8e76c5c4e30bbed95de13f70"} err="failed to get container status \"1d6ccede17b31e7006f5e598210b410d5c3a7a6f8e76c5c4e30bbed95de13f70\": rpc error: code = NotFound desc = could not find container \"1d6ccede17b31e7006f5e598210b410d5c3a7a6f8e76c5c4e30bbed95de13f70\": container with ID starting with 1d6ccede17b31e7006f5e598210b410d5c3a7a6f8e76c5c4e30bbed95de13f70 not found: ID does not exist"
Jul 31 23:27:51.240787 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:51.240759    2584 scope.go:117] "RemoveContainer" containerID="6ba9e79ccc17d149833e95f020908b19a25337800c623d4c1364ca16904a8810"
Jul 31 23:27:51.248052 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:51.248035    2584 scope.go:117] "RemoveContainer" containerID="6ba9e79ccc17d149833e95f020908b19a25337800c623d4c1364ca16904a8810"
Jul 31 23:27:51.248295 ip-10-0-128-243 kubenswrapper[2584]: E0731 23:27:51.248265    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ba9e79ccc17d149833e95f020908b19a25337800c623d4c1364ca16904a8810\": container with ID starting with 6ba9e79ccc17d149833e95f020908b19a25337800c623d4c1364ca16904a8810 not found: ID does not exist" containerID="6ba9e79ccc17d149833e95f020908b19a25337800c623d4c1364ca16904a8810"
Jul 31 23:27:51.248377 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:51.248303    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ba9e79ccc17d149833e95f020908b19a25337800c623d4c1364ca16904a8810"} err="failed to get container status \"6ba9e79ccc17d149833e95f020908b19a25337800c623d4c1364ca16904a8810\": rpc error: code = NotFound desc = could not find container \"6ba9e79ccc17d149833e95f020908b19a25337800c623d4c1364ca16904a8810\": container with ID starting with 6ba9e79ccc17d149833e95f020908b19a25337800c623d4c1364ca16904a8810 not found: ID does not exist"
Jul 31 23:27:51.248377 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:51.248325    2584 scope.go:117] "RemoveContainer" containerID="33a7a96d04cbdd6b22f2cf9c3201607c37c9c16698fee494587d43a62b715690"
Jul 31 23:27:51.250477 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:51.250460    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-13204f82-bbd9b8b76-ftpn5"]
Jul 31 23:27:51.253974 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:51.253955    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-13204f82-bbd9b8b76-ftpn5"]
Jul 31 23:27:51.256230 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:51.256214    2584 scope.go:117] "RemoveContainer" containerID="33a7a96d04cbdd6b22f2cf9c3201607c37c9c16698fee494587d43a62b715690"
Jul 31 23:27:51.256478 ip-10-0-128-243 kubenswrapper[2584]: E0731 23:27:51.256462    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33a7a96d04cbdd6b22f2cf9c3201607c37c9c16698fee494587d43a62b715690\": container with ID starting with 33a7a96d04cbdd6b22f2cf9c3201607c37c9c16698fee494587d43a62b715690 not found: ID does not exist" containerID="33a7a96d04cbdd6b22f2cf9c3201607c37c9c16698fee494587d43a62b715690"
Jul 31 23:27:51.256527 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:51.256483    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33a7a96d04cbdd6b22f2cf9c3201607c37c9c16698fee494587d43a62b715690"} err="failed to get container status \"33a7a96d04cbdd6b22f2cf9c3201607c37c9c16698fee494587d43a62b715690\": rpc error: code = NotFound desc = could not find container \"33a7a96d04cbdd6b22f2cf9c3201607c37c9c16698fee494587d43a62b715690\": container with ID starting with 33a7a96d04cbdd6b22f2cf9c3201607c37c9c16698fee494587d43a62b715690 not found: ID does not exist"
Jul 31 23:27:51.263286 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:51.263267    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-13204f82-577d7795d5-t7qtd"]
Jul 31 23:27:51.266592 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:51.266572    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-13204f82-577d7795d5-t7qtd"]
Jul 31 23:27:51.278879 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:51.278861    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-13204f82-5d75bbc95-gh75s"]
Jul 31 23:27:51.282358 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:51.282339    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-13204f82-5d75bbc95-gh75s"]
Jul 31 23:27:52.896732 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:52.896679    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ac3bbf5-596c-40d0-a5bd-a8a9f81d507f" path="/var/lib/kubelet/pods/7ac3bbf5-596c-40d0-a5bd-a8a9f81d507f/volumes"
Jul 31 23:27:52.897180 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:52.897050    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c8a785c-4de3-40f1-be52-9ba1294f3df1" path="/var/lib/kubelet/pods/9c8a785c-4de3-40f1-be52-9ba1294f3df1/volumes"
Jul 31 23:27:52.897385 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:52.897372    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ebd2d52-7bbc-496e-a539-ab9eacf42eb1" path="/var/lib/kubelet/pods/9ebd2d52-7bbc-496e-a539-ab9eacf42eb1/volumes"
Jul 31 23:27:54.989623 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:54.989586    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-86538af5-758959bf6b-gcxjr"]
Jul 31 23:27:54.990169 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:54.990149    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ac3bbf5-596c-40d0-a5bd-a8a9f81d507f" containerName="payload-processing"
Jul 31 23:27:54.990217 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:54.990175    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ac3bbf5-596c-40d0-a5bd-a8a9f81d507f" containerName="payload-processing"
Jul 31 23:27:54.990217 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:54.990188    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ebd2d52-7bbc-496e-a539-ab9eacf42eb1" containerName="maas-api"
Jul 31 23:27:54.990217 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:54.990196    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ebd2d52-7bbc-496e-a539-ab9eacf42eb1" containerName="maas-api"
Jul 31 23:27:54.990217 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:54.990212    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c8a785c-4de3-40f1-be52-9ba1294f3df1" containerName="payload-pre-processing"
Jul 31 23:27:54.990400 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:54.990222    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c8a785c-4de3-40f1-be52-9ba1294f3df1" containerName="payload-pre-processing"
Jul 31 23:27:54.990400 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:54.990308    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ebd2d52-7bbc-496e-a539-ab9eacf42eb1" containerName="maas-api"
Jul 31 23:27:54.990400 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:54.990324    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c8a785c-4de3-40f1-be52-9ba1294f3df1" containerName="payload-pre-processing"
Jul 31 23:27:54.990400 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:54.990335    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ac3bbf5-596c-40d0-a5bd-a8a9f81d507f" containerName="payload-processing"
Jul 31 23:27:54.994838 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:54.994819    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-86538af5-758959bf6b-gcxjr"
Jul 31 23:27:54.997069 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:54.997044    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-86538af5-serving-cert\""
Jul 31 23:27:55.002032 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:55.002006    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-86538af5-758959bf6b-gcxjr"]
Jul 31 23:27:55.147396 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:55.147364    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6ccca49-0e98-406b-b45e-ac8c3289879c-maas-api-tls\") pod \"maas-api-e2e-ait-86538af5-758959bf6b-gcxjr\" (UID: \"d6ccca49-0e98-406b-b45e-ac8c3289879c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-86538af5-758959bf6b-gcxjr"
Jul 31 23:27:55.147548 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:55.147417    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6ccca49-0e98-406b-b45e-ac8c3289879c-openshift-service-ca\") pod \"maas-api-e2e-ait-86538af5-758959bf6b-gcxjr\" (UID: \"d6ccca49-0e98-406b-b45e-ac8c3289879c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-86538af5-758959bf6b-gcxjr"
Jul 31 23:27:55.147548 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:55.147488    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfbzq\" (UniqueName: \"kubernetes.io/projected/d6ccca49-0e98-406b-b45e-ac8c3289879c-kube-api-access-wfbzq\") pod \"maas-api-e2e-ait-86538af5-758959bf6b-gcxjr\" (UID: \"d6ccca49-0e98-406b-b45e-ac8c3289879c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-86538af5-758959bf6b-gcxjr"
Jul 31 23:27:55.248940 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:55.248865    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6ccca49-0e98-406b-b45e-ac8c3289879c-maas-api-tls\") pod \"maas-api-e2e-ait-86538af5-758959bf6b-gcxjr\" (UID: \"d6ccca49-0e98-406b-b45e-ac8c3289879c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-86538af5-758959bf6b-gcxjr"
Jul 31 23:27:55.248940 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:55.248918    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6ccca49-0e98-406b-b45e-ac8c3289879c-openshift-service-ca\") pod \"maas-api-e2e-ait-86538af5-758959bf6b-gcxjr\" (UID: \"d6ccca49-0e98-406b-b45e-ac8c3289879c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-86538af5-758959bf6b-gcxjr"
Jul 31 23:27:55.249115 ip-10-0-128-243 kubenswrapper[2584]: E0731 23:27:55.249014    2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-86538af5-serving-cert: secret "maas-api-e2e-ait-86538af5-serving-cert" not found
Jul 31 23:27:55.249115 ip-10-0-128-243 kubenswrapper[2584]: E0731 23:27:55.249072    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6ccca49-0e98-406b-b45e-ac8c3289879c-maas-api-tls podName:d6ccca49-0e98-406b-b45e-ac8c3289879c nodeName:}" failed. No retries permitted until 2026-07-31 23:27:55.749056463 +0000 UTC m=+1859.394842750 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d6ccca49-0e98-406b-b45e-ac8c3289879c-maas-api-tls") pod "maas-api-e2e-ait-86538af5-758959bf6b-gcxjr" (UID: "d6ccca49-0e98-406b-b45e-ac8c3289879c") : secret "maas-api-e2e-ait-86538af5-serving-cert" not found
Jul 31 23:27:55.249115 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:55.249091    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfbzq\" (UniqueName: \"kubernetes.io/projected/d6ccca49-0e98-406b-b45e-ac8c3289879c-kube-api-access-wfbzq\") pod \"maas-api-e2e-ait-86538af5-758959bf6b-gcxjr\" (UID: \"d6ccca49-0e98-406b-b45e-ac8c3289879c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-86538af5-758959bf6b-gcxjr"
Jul 31 23:27:55.249486 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:55.249471    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6ccca49-0e98-406b-b45e-ac8c3289879c-openshift-service-ca\") pod \"maas-api-e2e-ait-86538af5-758959bf6b-gcxjr\" (UID: \"d6ccca49-0e98-406b-b45e-ac8c3289879c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-86538af5-758959bf6b-gcxjr"
Jul 31 23:27:55.257971 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:55.257949    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfbzq\" (UniqueName: \"kubernetes.io/projected/d6ccca49-0e98-406b-b45e-ac8c3289879c-kube-api-access-wfbzq\") pod \"maas-api-e2e-ait-86538af5-758959bf6b-gcxjr\" (UID: \"d6ccca49-0e98-406b-b45e-ac8c3289879c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-86538af5-758959bf6b-gcxjr"
Jul 31 23:27:55.672364 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:55.672332    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-86538af5-7b9b4bdbf6-wm7xn"]
Jul 31 23:27:55.675907 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:55.675888    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-86538af5-7b9b4bdbf6-wm7xn"
Jul 31 23:27:55.678254 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:55.678232    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-86538af5\""
Jul 31 23:27:55.678455 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:55.678437    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-86538af5-dockercfg-bg984\""
Jul 31 23:27:55.685425 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:55.685401    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-86538af5-7b9b4bdbf6-wm7xn"]
Jul 31 23:27:55.711882 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:55.711857    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-86538af5-7f9476b98d-vfrfs"]
Jul 31 23:27:55.715223 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:55.715199    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-86538af5-7f9476b98d-vfrfs"
Jul 31 23:27:55.724566 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:55.724541    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-86538af5-7f9476b98d-vfrfs"]
Jul 31 23:27:55.753108 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:55.753080    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0653682e-d1ad-4023-a8cd-93c0d41c2a7f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-86538af5-7b9b4bdbf6-wm7xn\" (UID: \"0653682e-d1ad-4023-a8cd-93c0d41c2a7f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-86538af5-7b9b4bdbf6-wm7xn"
Jul 31 23:27:55.753216 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:55.753135    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6ccca49-0e98-406b-b45e-ac8c3289879c-maas-api-tls\") pod \"maas-api-e2e-ait-86538af5-758959bf6b-gcxjr\" (UID: \"d6ccca49-0e98-406b-b45e-ac8c3289879c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-86538af5-758959bf6b-gcxjr"
Jul 31 23:27:55.753216 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:55.753196    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hz8vw\" (UniqueName: \"kubernetes.io/projected/0653682e-d1ad-4023-a8cd-93c0d41c2a7f-kube-api-access-hz8vw\") pod \"payload-pre-processing-e2e-ait-86538af5-7b9b4bdbf6-wm7xn\" (UID: \"0653682e-d1ad-4023-a8cd-93c0d41c2a7f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-86538af5-7b9b4bdbf6-wm7xn"
Jul 31 23:27:55.755485 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:55.755456    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6ccca49-0e98-406b-b45e-ac8c3289879c-maas-api-tls\") pod \"maas-api-e2e-ait-86538af5-758959bf6b-gcxjr\" (UID: \"d6ccca49-0e98-406b-b45e-ac8c3289879c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-86538af5-758959bf6b-gcxjr"
Jul 31 23:27:55.854002 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:55.853968    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0653682e-d1ad-4023-a8cd-93c0d41c2a7f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-86538af5-7b9b4bdbf6-wm7xn\" (UID: \"0653682e-d1ad-4023-a8cd-93c0d41c2a7f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-86538af5-7b9b4bdbf6-wm7xn"
Jul 31 23:27:55.854002 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:55.854008    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pn5b8\" (UniqueName: \"kubernetes.io/projected/04fba67b-282e-476d-8cfa-b732b893b610-kube-api-access-pn5b8\") pod \"payload-processing-e2e-ait-86538af5-7f9476b98d-vfrfs\" (UID: \"04fba67b-282e-476d-8cfa-b732b893b610\") " pod="openshift-ingress/payload-processing-e2e-ait-86538af5-7f9476b98d-vfrfs"
Jul 31 23:27:55.854252 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:55.854061    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hz8vw\" (UniqueName: \"kubernetes.io/projected/0653682e-d1ad-4023-a8cd-93c0d41c2a7f-kube-api-access-hz8vw\") pod \"payload-pre-processing-e2e-ait-86538af5-7b9b4bdbf6-wm7xn\" (UID: \"0653682e-d1ad-4023-a8cd-93c0d41c2a7f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-86538af5-7b9b4bdbf6-wm7xn"
Jul 31 23:27:55.854252 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:55.854080    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/04fba67b-282e-476d-8cfa-b732b893b610-plugins-config-volume\") pod \"payload-processing-e2e-ait-86538af5-7f9476b98d-vfrfs\" (UID: \"04fba67b-282e-476d-8cfa-b732b893b610\") " pod="openshift-ingress/payload-processing-e2e-ait-86538af5-7f9476b98d-vfrfs"
Jul 31 23:27:55.854756 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:55.854733    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0653682e-d1ad-4023-a8cd-93c0d41c2a7f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-86538af5-7b9b4bdbf6-wm7xn\" (UID: \"0653682e-d1ad-4023-a8cd-93c0d41c2a7f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-86538af5-7b9b4bdbf6-wm7xn"
Jul 31 23:27:55.861521 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:55.861500    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hz8vw\" (UniqueName: \"kubernetes.io/projected/0653682e-d1ad-4023-a8cd-93c0d41c2a7f-kube-api-access-hz8vw\") pod \"payload-pre-processing-e2e-ait-86538af5-7b9b4bdbf6-wm7xn\" (UID: \"0653682e-d1ad-4023-a8cd-93c0d41c2a7f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-86538af5-7b9b4bdbf6-wm7xn"
Jul 31 23:27:55.907033 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:55.907000    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-86538af5-758959bf6b-gcxjr"
Jul 31 23:27:55.955582 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:55.955089    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/04fba67b-282e-476d-8cfa-b732b893b610-plugins-config-volume\") pod \"payload-processing-e2e-ait-86538af5-7f9476b98d-vfrfs\" (UID: \"04fba67b-282e-476d-8cfa-b732b893b610\") " pod="openshift-ingress/payload-processing-e2e-ait-86538af5-7f9476b98d-vfrfs"
Jul 31 23:27:55.955582 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:55.955200    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pn5b8\" (UniqueName: \"kubernetes.io/projected/04fba67b-282e-476d-8cfa-b732b893b610-kube-api-access-pn5b8\") pod \"payload-processing-e2e-ait-86538af5-7f9476b98d-vfrfs\" (UID: \"04fba67b-282e-476d-8cfa-b732b893b610\") " pod="openshift-ingress/payload-processing-e2e-ait-86538af5-7f9476b98d-vfrfs"
Jul 31 23:27:55.956270 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:55.956242    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/04fba67b-282e-476d-8cfa-b732b893b610-plugins-config-volume\") pod \"payload-processing-e2e-ait-86538af5-7f9476b98d-vfrfs\" (UID: \"04fba67b-282e-476d-8cfa-b732b893b610\") " pod="openshift-ingress/payload-processing-e2e-ait-86538af5-7f9476b98d-vfrfs"
Jul 31 23:27:55.963225 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:55.963195    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pn5b8\" (UniqueName: \"kubernetes.io/projected/04fba67b-282e-476d-8cfa-b732b893b610-kube-api-access-pn5b8\") pod \"payload-processing-e2e-ait-86538af5-7f9476b98d-vfrfs\" (UID: \"04fba67b-282e-476d-8cfa-b732b893b610\") " pod="openshift-ingress/payload-processing-e2e-ait-86538af5-7f9476b98d-vfrfs"
Jul 31 23:27:55.987949 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:55.987918    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-86538af5-7b9b4bdbf6-wm7xn"
Jul 31 23:27:56.026102 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:56.025923    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-86538af5-7f9476b98d-vfrfs"
Jul 31 23:27:56.056171 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:56.056082    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-86538af5-758959bf6b-gcxjr"]
Jul 31 23:27:56.057396 ip-10-0-128-243 kubenswrapper[2584]: W0731 23:27:56.057364    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6ccca49_0e98_406b_b45e_ac8c3289879c.slice/crio-3f5d90b7d9369b2b1f1bd15ffd092ae86b4513af8f36defa48f54cb2ef4f7f5b WatchSource:0}: Error finding container 3f5d90b7d9369b2b1f1bd15ffd092ae86b4513af8f36defa48f54cb2ef4f7f5b: Status 404 returned error can't find the container with id 3f5d90b7d9369b2b1f1bd15ffd092ae86b4513af8f36defa48f54cb2ef4f7f5b
Jul 31 23:27:56.146354 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:56.146317    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-86538af5-7b9b4bdbf6-wm7xn"]
Jul 31 23:27:56.150047 ip-10-0-128-243 kubenswrapper[2584]: W0731 23:27:56.150011    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0653682e_d1ad_4023_a8cd_93c0d41c2a7f.slice/crio-879abc8d52dd5739ec2c144833fb581f06803fd536b8dd13c6aa176df562d161 WatchSource:0}: Error finding container 879abc8d52dd5739ec2c144833fb581f06803fd536b8dd13c6aa176df562d161: Status 404 returned error can't find the container with id 879abc8d52dd5739ec2c144833fb581f06803fd536b8dd13c6aa176df562d161
Jul 31 23:27:56.182156 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:56.182116    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-86538af5-7f9476b98d-vfrfs"]
Jul 31 23:27:56.185142 ip-10-0-128-243 kubenswrapper[2584]: W0731 23:27:56.185112    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04fba67b_282e_476d_8cfa_b732b893b610.slice/crio-1ee13dfd8dda98783ed27dcf89a6b456f9342b2f976b1cc5e98ebe1abe994e74 WatchSource:0}: Error finding container 1ee13dfd8dda98783ed27dcf89a6b456f9342b2f976b1cc5e98ebe1abe994e74: Status 404 returned error can't find the container with id 1ee13dfd8dda98783ed27dcf89a6b456f9342b2f976b1cc5e98ebe1abe994e74
Jul 31 23:27:56.262148 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:56.262117    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-86538af5-758959bf6b-gcxjr" event={"ID":"d6ccca49-0e98-406b-b45e-ac8c3289879c","Type":"ContainerStarted","Data":"3f5d90b7d9369b2b1f1bd15ffd092ae86b4513af8f36defa48f54cb2ef4f7f5b"}
Jul 31 23:27:56.264048 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:56.264014    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-86538af5-7b9b4bdbf6-wm7xn" event={"ID":"0653682e-d1ad-4023-a8cd-93c0d41c2a7f","Type":"ContainerStarted","Data":"0cedd3e1ebcb464cfbe91be611301b75a9e34d4f191cdd65cd0adf8b5703a7c4"}
Jul 31 23:27:56.264172 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:56.264073    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-86538af5-7b9b4bdbf6-wm7xn" event={"ID":"0653682e-d1ad-4023-a8cd-93c0d41c2a7f","Type":"ContainerStarted","Data":"879abc8d52dd5739ec2c144833fb581f06803fd536b8dd13c6aa176df562d161"}
Jul 31 23:27:56.264172 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:56.264102    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-86538af5-7b9b4bdbf6-wm7xn"
Jul 31 23:27:56.265679 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:56.265659    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-86538af5-7f9476b98d-vfrfs" event={"ID":"04fba67b-282e-476d-8cfa-b732b893b610","Type":"ContainerStarted","Data":"9be4aa8dc4855f2db61b1957c134dd35a1f58246ecb7ba36a23d0d65d26fa2f3"}
Jul 31 23:27:56.265812 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:56.265720    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-86538af5-7f9476b98d-vfrfs" event={"ID":"04fba67b-282e-476d-8cfa-b732b893b610","Type":"ContainerStarted","Data":"1ee13dfd8dda98783ed27dcf89a6b456f9342b2f976b1cc5e98ebe1abe994e74"}
Jul 31 23:27:56.265812 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:56.265774    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-86538af5-7f9476b98d-vfrfs"
Jul 31 23:27:56.279822 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:56.279774    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-86538af5-7b9b4bdbf6-wm7xn" podStartSLOduration=1.279759638 podStartE2EDuration="1.279759638s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:27:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:27:56.277640374 +0000 UTC m=+1859.923426709" watchObservedRunningTime="2026-07-31 23:27:56.279759638 +0000 UTC m=+1859.925545947"
Jul 31 23:27:56.295599 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:56.295538    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-86538af5-7f9476b98d-vfrfs" podStartSLOduration=1.295515542 podStartE2EDuration="1.295515542s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:27:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:27:56.293088524 +0000 UTC m=+1859.938874834" watchObservedRunningTime="2026-07-31 23:27:56.295515542 +0000 UTC m=+1859.941301847"
Jul 31 23:27:57.272397 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:57.272312    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-86538af5-758959bf6b-gcxjr" event={"ID":"d6ccca49-0e98-406b-b45e-ac8c3289879c","Type":"ContainerStarted","Data":"2e931bf243277480c630651b558d8558c2ba9d7615b1e473accb9090c5020b98"}
Jul 31 23:27:57.272866 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:57.272617    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-86538af5-758959bf6b-gcxjr"
Jul 31 23:27:57.288245 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:27:57.288192    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-86538af5-758959bf6b-gcxjr" podStartSLOduration=2.845600585 podStartE2EDuration="3.288179126s" totalImagesPullingTime="442.578541ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:27:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:27:57.287367601 +0000 UTC m=+1860.933153915" watchObservedRunningTime="2026-07-31 23:27:57.288179126 +0000 UTC m=+1860.933965446"
Jul 31 23:28:03.283132 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:03.283100    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-86538af5-758959bf6b-gcxjr"
Jul 31 23:28:07.274688 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:07.274657    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-86538af5-7f9476b98d-vfrfs"
Jul 31 23:28:07.275105 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:07.274774    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-86538af5-7b9b4bdbf6-wm7xn"
Jul 31 23:28:25.463639 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:25.463539    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-86538af5-758959bf6b-gcxjr"]
Jul 31 23:28:25.464404 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:25.464348    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-86538af5-758959bf6b-gcxjr" podUID="d6ccca49-0e98-406b-b45e-ac8c3289879c" containerName="maas-api" containerID="cri-o://2e931bf243277480c630651b558d8558c2ba9d7615b1e473accb9090c5020b98" gracePeriod=30
Jul 31 23:28:25.507905 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:25.507874    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-86538af5-7f9476b98d-vfrfs"]
Jul 31 23:28:25.508170 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:25.508125    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-86538af5-7f9476b98d-vfrfs" podUID="04fba67b-282e-476d-8cfa-b732b893b610" containerName="payload-processing" containerID="cri-o://9be4aa8dc4855f2db61b1957c134dd35a1f58246ecb7ba36a23d0d65d26fa2f3" gracePeriod=30
Jul 31 23:28:25.514891 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:25.514864    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-86538af5-7b9b4bdbf6-wm7xn"]
Jul 31 23:28:25.515127 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:25.515087    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-86538af5-7b9b4bdbf6-wm7xn" podUID="0653682e-d1ad-4023-a8cd-93c0d41c2a7f" containerName="payload-pre-processing" containerID="cri-o://0cedd3e1ebcb464cfbe91be611301b75a9e34d4f191cdd65cd0adf8b5703a7c4" gracePeriod=30
Jul 31 23:28:25.783207 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:25.783179    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-86538af5-758959bf6b-gcxjr"
Jul 31 23:28:25.800716 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:25.800677    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-86538af5-7b9b4bdbf6-wm7xn"
Jul 31 23:28:25.811611 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:25.811579    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6ccca49-0e98-406b-b45e-ac8c3289879c-maas-api-tls\") pod \"d6ccca49-0e98-406b-b45e-ac8c3289879c\" (UID: \"d6ccca49-0e98-406b-b45e-ac8c3289879c\") "
Jul 31 23:28:25.811808 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:25.811673    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wfbzq\" (UniqueName: \"kubernetes.io/projected/d6ccca49-0e98-406b-b45e-ac8c3289879c-kube-api-access-wfbzq\") pod \"d6ccca49-0e98-406b-b45e-ac8c3289879c\" (UID: \"d6ccca49-0e98-406b-b45e-ac8c3289879c\") "
Jul 31 23:28:25.811808 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:25.811761    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6ccca49-0e98-406b-b45e-ac8c3289879c-openshift-service-ca\") pod \"d6ccca49-0e98-406b-b45e-ac8c3289879c\" (UID: \"d6ccca49-0e98-406b-b45e-ac8c3289879c\") "
Jul 31 23:28:25.812124 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:25.812096    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6ccca49-0e98-406b-b45e-ac8c3289879c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d6ccca49-0e98-406b-b45e-ac8c3289879c" (UID: "d6ccca49-0e98-406b-b45e-ac8c3289879c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:28:25.813834 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:25.813801    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d6ccca49-0e98-406b-b45e-ac8c3289879c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d6ccca49-0e98-406b-b45e-ac8c3289879c" (UID: "d6ccca49-0e98-406b-b45e-ac8c3289879c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 23:28:25.814223 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:25.814201    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6ccca49-0e98-406b-b45e-ac8c3289879c-kube-api-access-wfbzq" (OuterVolumeSpecName: "kube-api-access-wfbzq") pod "d6ccca49-0e98-406b-b45e-ac8c3289879c" (UID: "d6ccca49-0e98-406b-b45e-ac8c3289879c"). InnerVolumeSpecName "kube-api-access-wfbzq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 23:28:25.828997 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:25.828967    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-86538af5-7f9476b98d-vfrfs"
Jul 31 23:28:25.913026 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:25.912989    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pn5b8\" (UniqueName: \"kubernetes.io/projected/04fba67b-282e-476d-8cfa-b732b893b610-kube-api-access-pn5b8\") pod \"04fba67b-282e-476d-8cfa-b732b893b610\" (UID: \"04fba67b-282e-476d-8cfa-b732b893b610\") "
Jul 31 23:28:25.913217 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:25.913056    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hz8vw\" (UniqueName: \"kubernetes.io/projected/0653682e-d1ad-4023-a8cd-93c0d41c2a7f-kube-api-access-hz8vw\") pod \"0653682e-d1ad-4023-a8cd-93c0d41c2a7f\" (UID: \"0653682e-d1ad-4023-a8cd-93c0d41c2a7f\") "
Jul 31 23:28:25.913217 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:25.913139    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0653682e-d1ad-4023-a8cd-93c0d41c2a7f-plugins-config-volume\") pod \"0653682e-d1ad-4023-a8cd-93c0d41c2a7f\" (UID: \"0653682e-d1ad-4023-a8cd-93c0d41c2a7f\") "
Jul 31 23:28:25.913217 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:25.913171    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/04fba67b-282e-476d-8cfa-b732b893b610-plugins-config-volume\") pod \"04fba67b-282e-476d-8cfa-b732b893b610\" (UID: \"04fba67b-282e-476d-8cfa-b732b893b610\") "
Jul 31 23:28:25.913473 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:25.913457    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6ccca49-0e98-406b-b45e-ac8c3289879c-maas-api-tls\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:28:25.913539 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:25.913479    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wfbzq\" (UniqueName: \"kubernetes.io/projected/d6ccca49-0e98-406b-b45e-ac8c3289879c-kube-api-access-wfbzq\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:28:25.913539 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:25.913495    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6ccca49-0e98-406b-b45e-ac8c3289879c-openshift-service-ca\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:28:25.913539 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:25.913504    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0653682e-d1ad-4023-a8cd-93c0d41c2a7f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0653682e-d1ad-4023-a8cd-93c0d41c2a7f" (UID: "0653682e-d1ad-4023-a8cd-93c0d41c2a7f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:28:25.913737 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:25.913593    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04fba67b-282e-476d-8cfa-b732b893b610-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "04fba67b-282e-476d-8cfa-b732b893b610" (UID: "04fba67b-282e-476d-8cfa-b732b893b610"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:28:25.915195 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:25.915174    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04fba67b-282e-476d-8cfa-b732b893b610-kube-api-access-pn5b8" (OuterVolumeSpecName: "kube-api-access-pn5b8") pod "04fba67b-282e-476d-8cfa-b732b893b610" (UID: "04fba67b-282e-476d-8cfa-b732b893b610"). InnerVolumeSpecName "kube-api-access-pn5b8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 23:28:25.915248 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:25.915209    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0653682e-d1ad-4023-a8cd-93c0d41c2a7f-kube-api-access-hz8vw" (OuterVolumeSpecName: "kube-api-access-hz8vw") pod "0653682e-d1ad-4023-a8cd-93c0d41c2a7f" (UID: "0653682e-d1ad-4023-a8cd-93c0d41c2a7f"). InnerVolumeSpecName "kube-api-access-hz8vw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 23:28:26.014690 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:26.014584    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0653682e-d1ad-4023-a8cd-93c0d41c2a7f-plugins-config-volume\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:28:26.014690 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:26.014631    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/04fba67b-282e-476d-8cfa-b732b893b610-plugins-config-volume\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:28:26.014690 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:26.014641    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pn5b8\" (UniqueName: \"kubernetes.io/projected/04fba67b-282e-476d-8cfa-b732b893b610-kube-api-access-pn5b8\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:28:26.014690 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:26.014650    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hz8vw\" (UniqueName: \"kubernetes.io/projected/0653682e-d1ad-4023-a8cd-93c0d41c2a7f-kube-api-access-hz8vw\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:28:26.411602 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:26.411569    2584 generic.go:356] "Generic (PLEG): container finished" podID="0653682e-d1ad-4023-a8cd-93c0d41c2a7f" containerID="0cedd3e1ebcb464cfbe91be611301b75a9e34d4f191cdd65cd0adf8b5703a7c4" exitCode=0
Jul 31 23:28:26.411814 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:26.411609    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-86538af5-7b9b4bdbf6-wm7xn"
Jul 31 23:28:26.411814 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:26.411649    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-86538af5-7b9b4bdbf6-wm7xn" event={"ID":"0653682e-d1ad-4023-a8cd-93c0d41c2a7f","Type":"ContainerDied","Data":"0cedd3e1ebcb464cfbe91be611301b75a9e34d4f191cdd65cd0adf8b5703a7c4"}
Jul 31 23:28:26.411814 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:26.411718    2584 scope.go:117] "RemoveContainer" containerID="0cedd3e1ebcb464cfbe91be611301b75a9e34d4f191cdd65cd0adf8b5703a7c4"
Jul 31 23:28:26.411992 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:26.411719    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-86538af5-7b9b4bdbf6-wm7xn" event={"ID":"0653682e-d1ad-4023-a8cd-93c0d41c2a7f","Type":"ContainerDied","Data":"879abc8d52dd5739ec2c144833fb581f06803fd536b8dd13c6aa176df562d161"}
Jul 31 23:28:26.412944 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:26.412919    2584 generic.go:356] "Generic (PLEG): container finished" podID="04fba67b-282e-476d-8cfa-b732b893b610" containerID="9be4aa8dc4855f2db61b1957c134dd35a1f58246ecb7ba36a23d0d65d26fa2f3" exitCode=0
Jul 31 23:28:26.413041 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:26.412955    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-86538af5-7f9476b98d-vfrfs"
Jul 31 23:28:26.413041 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:26.412984    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-86538af5-7f9476b98d-vfrfs" event={"ID":"04fba67b-282e-476d-8cfa-b732b893b610","Type":"ContainerDied","Data":"9be4aa8dc4855f2db61b1957c134dd35a1f58246ecb7ba36a23d0d65d26fa2f3"}
Jul 31 23:28:26.413041 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:26.413035    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-86538af5-7f9476b98d-vfrfs" event={"ID":"04fba67b-282e-476d-8cfa-b732b893b610","Type":"ContainerDied","Data":"1ee13dfd8dda98783ed27dcf89a6b456f9342b2f976b1cc5e98ebe1abe994e74"}
Jul 31 23:28:26.420156 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:26.420128    2584 generic.go:356] "Generic (PLEG): container finished" podID="d6ccca49-0e98-406b-b45e-ac8c3289879c" containerID="2e931bf243277480c630651b558d8558c2ba9d7615b1e473accb9090c5020b98" exitCode=0
Jul 31 23:28:26.420439 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:26.420417    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-86538af5-758959bf6b-gcxjr" event={"ID":"d6ccca49-0e98-406b-b45e-ac8c3289879c","Type":"ContainerDied","Data":"2e931bf243277480c630651b558d8558c2ba9d7615b1e473accb9090c5020b98"}
Jul 31 23:28:26.420758 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:26.420737    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-86538af5-758959bf6b-gcxjr" event={"ID":"d6ccca49-0e98-406b-b45e-ac8c3289879c","Type":"ContainerDied","Data":"3f5d90b7d9369b2b1f1bd15ffd092ae86b4513af8f36defa48f54cb2ef4f7f5b"}
Jul 31 23:28:26.420993 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:26.420978    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-86538af5-758959bf6b-gcxjr"
Jul 31 23:28:26.425977 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:26.425952    2584 scope.go:117] "RemoveContainer" containerID="0cedd3e1ebcb464cfbe91be611301b75a9e34d4f191cdd65cd0adf8b5703a7c4"
Jul 31 23:28:26.426292 ip-10-0-128-243 kubenswrapper[2584]: E0731 23:28:26.426270    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0cedd3e1ebcb464cfbe91be611301b75a9e34d4f191cdd65cd0adf8b5703a7c4\": container with ID starting with 0cedd3e1ebcb464cfbe91be611301b75a9e34d4f191cdd65cd0adf8b5703a7c4 not found: ID does not exist" containerID="0cedd3e1ebcb464cfbe91be611301b75a9e34d4f191cdd65cd0adf8b5703a7c4"
Jul 31 23:28:26.426397 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:26.426298    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0cedd3e1ebcb464cfbe91be611301b75a9e34d4f191cdd65cd0adf8b5703a7c4"} err="failed to get container status \"0cedd3e1ebcb464cfbe91be611301b75a9e34d4f191cdd65cd0adf8b5703a7c4\": rpc error: code = NotFound desc = could not find container \"0cedd3e1ebcb464cfbe91be611301b75a9e34d4f191cdd65cd0adf8b5703a7c4\": container with ID starting with 0cedd3e1ebcb464cfbe91be611301b75a9e34d4f191cdd65cd0adf8b5703a7c4 not found: ID does not exist"
Jul 31 23:28:26.426397 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:26.426315    2584 scope.go:117] "RemoveContainer" containerID="9be4aa8dc4855f2db61b1957c134dd35a1f58246ecb7ba36a23d0d65d26fa2f3"
Jul 31 23:28:26.435950 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:26.435935    2584 scope.go:117] "RemoveContainer" containerID="9be4aa8dc4855f2db61b1957c134dd35a1f58246ecb7ba36a23d0d65d26fa2f3"
Jul 31 23:28:26.436207 ip-10-0-128-243 kubenswrapper[2584]: E0731 23:28:26.436188    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9be4aa8dc4855f2db61b1957c134dd35a1f58246ecb7ba36a23d0d65d26fa2f3\": container with ID starting with 9be4aa8dc4855f2db61b1957c134dd35a1f58246ecb7ba36a23d0d65d26fa2f3 not found: ID does not exist" containerID="9be4aa8dc4855f2db61b1957c134dd35a1f58246ecb7ba36a23d0d65d26fa2f3"
Jul 31 23:28:26.436264 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:26.436213    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9be4aa8dc4855f2db61b1957c134dd35a1f58246ecb7ba36a23d0d65d26fa2f3"} err="failed to get container status \"9be4aa8dc4855f2db61b1957c134dd35a1f58246ecb7ba36a23d0d65d26fa2f3\": rpc error: code = NotFound desc = could not find container \"9be4aa8dc4855f2db61b1957c134dd35a1f58246ecb7ba36a23d0d65d26fa2f3\": container with ID starting with 9be4aa8dc4855f2db61b1957c134dd35a1f58246ecb7ba36a23d0d65d26fa2f3 not found: ID does not exist"
Jul 31 23:28:26.436264 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:26.436231    2584 scope.go:117] "RemoveContainer" containerID="2e931bf243277480c630651b558d8558c2ba9d7615b1e473accb9090c5020b98"
Jul 31 23:28:26.442120 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:26.442097    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-86538af5-7f9476b98d-vfrfs"]
Jul 31 23:28:26.444925 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:26.444905    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-86538af5-7f9476b98d-vfrfs"]
Jul 31 23:28:26.445108 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:26.445090    2584 scope.go:117] "RemoveContainer" containerID="2e931bf243277480c630651b558d8558c2ba9d7615b1e473accb9090c5020b98"
Jul 31 23:28:26.445422 ip-10-0-128-243 kubenswrapper[2584]: E0731 23:28:26.445394    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e931bf243277480c630651b558d8558c2ba9d7615b1e473accb9090c5020b98\": container with ID starting with 2e931bf243277480c630651b558d8558c2ba9d7615b1e473accb9090c5020b98 not found: ID does not exist" containerID="2e931bf243277480c630651b558d8558c2ba9d7615b1e473accb9090c5020b98"
Jul 31 23:28:26.445514 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:26.445432    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e931bf243277480c630651b558d8558c2ba9d7615b1e473accb9090c5020b98"} err="failed to get container status \"2e931bf243277480c630651b558d8558c2ba9d7615b1e473accb9090c5020b98\": rpc error: code = NotFound desc = could not find container \"2e931bf243277480c630651b558d8558c2ba9d7615b1e473accb9090c5020b98\": container with ID starting with 2e931bf243277480c630651b558d8558c2ba9d7615b1e473accb9090c5020b98 not found: ID does not exist"
Jul 31 23:28:26.454584 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:26.454562    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-86538af5-7b9b4bdbf6-wm7xn"]
Jul 31 23:28:26.458131 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:26.458111    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-86538af5-7b9b4bdbf6-wm7xn"]
Jul 31 23:28:26.469462 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:26.469430    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-86538af5-758959bf6b-gcxjr"]
Jul 31 23:28:26.472921 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:26.472899    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-86538af5-758959bf6b-gcxjr"]
Jul 31 23:28:26.891251 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:26.891220    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04fba67b-282e-476d-8cfa-b732b893b610" path="/var/lib/kubelet/pods/04fba67b-282e-476d-8cfa-b732b893b610/volumes"
Jul 31 23:28:26.891624 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:26.891607    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0653682e-d1ad-4023-a8cd-93c0d41c2a7f" path="/var/lib/kubelet/pods/0653682e-d1ad-4023-a8cd-93c0d41c2a7f/volumes"
Jul 31 23:28:26.892013 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:26.891998    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6ccca49-0e98-406b-b45e-ac8c3289879c" path="/var/lib/kubelet/pods/d6ccca49-0e98-406b-b45e-ac8c3289879c/volumes"
Jul 31 23:28:31.193367 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:31.193329    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7056468c-85cb6cf9bd-sg7wm"]
Jul 31 23:28:31.193903 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:31.193884    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0653682e-d1ad-4023-a8cd-93c0d41c2a7f" containerName="payload-pre-processing"
Jul 31 23:28:31.193996 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:31.193906    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="0653682e-d1ad-4023-a8cd-93c0d41c2a7f" containerName="payload-pre-processing"
Jul 31 23:28:31.193996 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:31.193922    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04fba67b-282e-476d-8cfa-b732b893b610" containerName="payload-processing"
Jul 31 23:28:31.193996 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:31.193930    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="04fba67b-282e-476d-8cfa-b732b893b610" containerName="payload-processing"
Jul 31 23:28:31.193996 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:31.193946    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6ccca49-0e98-406b-b45e-ac8c3289879c" containerName="maas-api"
Jul 31 23:28:31.193996 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:31.193955    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6ccca49-0e98-406b-b45e-ac8c3289879c" containerName="maas-api"
Jul 31 23:28:31.194232 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:31.194066    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6ccca49-0e98-406b-b45e-ac8c3289879c" containerName="maas-api"
Jul 31 23:28:31.194232 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:31.194082    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="0653682e-d1ad-4023-a8cd-93c0d41c2a7f" containerName="payload-pre-processing"
Jul 31 23:28:31.194232 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:31.194090    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="04fba67b-282e-476d-8cfa-b732b893b610" containerName="payload-processing"
Jul 31 23:28:31.196973 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:31.196953    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7056468c-85cb6cf9bd-sg7wm"
Jul 31 23:28:31.199473 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:31.199450    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-7056468c-serving-cert\""
Jul 31 23:28:31.207395 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:31.207369    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7056468c-85cb6cf9bd-sg7wm"]
Jul 31 23:28:31.260884 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:31.260849    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4jtl\" (UniqueName: \"kubernetes.io/projected/fc596b41-abbd-4d7e-88a5-f418a9cbabce-kube-api-access-g4jtl\") pod \"maas-api-e2e-derive-7056468c-85cb6cf9bd-sg7wm\" (UID: \"fc596b41-abbd-4d7e-88a5-f418a9cbabce\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7056468c-85cb6cf9bd-sg7wm"
Jul 31 23:28:31.261069 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:31.260908    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc596b41-abbd-4d7e-88a5-f418a9cbabce-maas-api-tls\") pod \"maas-api-e2e-derive-7056468c-85cb6cf9bd-sg7wm\" (UID: \"fc596b41-abbd-4d7e-88a5-f418a9cbabce\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7056468c-85cb6cf9bd-sg7wm"
Jul 31 23:28:31.261069 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:31.260937    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc596b41-abbd-4d7e-88a5-f418a9cbabce-openshift-service-ca\") pod \"maas-api-e2e-derive-7056468c-85cb6cf9bd-sg7wm\" (UID: \"fc596b41-abbd-4d7e-88a5-f418a9cbabce\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7056468c-85cb6cf9bd-sg7wm"
Jul 31 23:28:31.361925 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:31.361891    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc596b41-abbd-4d7e-88a5-f418a9cbabce-maas-api-tls\") pod \"maas-api-e2e-derive-7056468c-85cb6cf9bd-sg7wm\" (UID: \"fc596b41-abbd-4d7e-88a5-f418a9cbabce\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7056468c-85cb6cf9bd-sg7wm"
Jul 31 23:28:31.361925 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:31.361930    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc596b41-abbd-4d7e-88a5-f418a9cbabce-openshift-service-ca\") pod \"maas-api-e2e-derive-7056468c-85cb6cf9bd-sg7wm\" (UID: \"fc596b41-abbd-4d7e-88a5-f418a9cbabce\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7056468c-85cb6cf9bd-sg7wm"
Jul 31 23:28:31.362144 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:31.361994    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4jtl\" (UniqueName: \"kubernetes.io/projected/fc596b41-abbd-4d7e-88a5-f418a9cbabce-kube-api-access-g4jtl\") pod \"maas-api-e2e-derive-7056468c-85cb6cf9bd-sg7wm\" (UID: \"fc596b41-abbd-4d7e-88a5-f418a9cbabce\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7056468c-85cb6cf9bd-sg7wm"
Jul 31 23:28:31.362144 ip-10-0-128-243 kubenswrapper[2584]: E0731 23:28:31.362059    2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-7056468c-serving-cert: secret "maas-api-e2e-derive-7056468c-serving-cert" not found
Jul 31 23:28:31.362209 ip-10-0-128-243 kubenswrapper[2584]: E0731 23:28:31.362144    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc596b41-abbd-4d7e-88a5-f418a9cbabce-maas-api-tls podName:fc596b41-abbd-4d7e-88a5-f418a9cbabce nodeName:}" failed. No retries permitted until 2026-07-31 23:28:31.862122428 +0000 UTC m=+1895.507908717 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fc596b41-abbd-4d7e-88a5-f418a9cbabce-maas-api-tls") pod "maas-api-e2e-derive-7056468c-85cb6cf9bd-sg7wm" (UID: "fc596b41-abbd-4d7e-88a5-f418a9cbabce") : secret "maas-api-e2e-derive-7056468c-serving-cert" not found
Jul 31 23:28:31.362660 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:31.362639    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc596b41-abbd-4d7e-88a5-f418a9cbabce-openshift-service-ca\") pod \"maas-api-e2e-derive-7056468c-85cb6cf9bd-sg7wm\" (UID: \"fc596b41-abbd-4d7e-88a5-f418a9cbabce\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7056468c-85cb6cf9bd-sg7wm"
Jul 31 23:28:31.372433 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:31.372408    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4jtl\" (UniqueName: \"kubernetes.io/projected/fc596b41-abbd-4d7e-88a5-f418a9cbabce-kube-api-access-g4jtl\") pod \"maas-api-e2e-derive-7056468c-85cb6cf9bd-sg7wm\" (UID: \"fc596b41-abbd-4d7e-88a5-f418a9cbabce\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7056468c-85cb6cf9bd-sg7wm"
Jul 31 23:28:31.815719 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:31.815658    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-7056468c-6fdff4765b-5882c"]
Jul 31 23:28:31.819072 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:31.819053    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-7056468c-6fdff4765b-5882c"
Jul 31 23:28:31.821852 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:31.821826    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-7056468c-dockercfg-q7gmd\""
Jul 31 23:28:31.822163 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:31.822146    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-7056468c\""
Jul 31 23:28:31.827672 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:31.827647    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-7056468c-6fdff4765b-5882c"]
Jul 31 23:28:31.846165 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:31.846134    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-7056468c-fcb7d7898-xcv75"]
Jul 31 23:28:31.849545 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:31.849529    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-7056468c-fcb7d7898-xcv75"
Jul 31 23:28:31.857904 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:31.857880    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-7056468c-fcb7d7898-xcv75"]
Jul 31 23:28:31.865794 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:31.865766    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc596b41-abbd-4d7e-88a5-f418a9cbabce-maas-api-tls\") pod \"maas-api-e2e-derive-7056468c-85cb6cf9bd-sg7wm\" (UID: \"fc596b41-abbd-4d7e-88a5-f418a9cbabce\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7056468c-85cb6cf9bd-sg7wm"
Jul 31 23:28:31.865913 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:31.865810    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4f8b773-1ab4-4376-8158-46fd41c3a6e1-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-7056468c-6fdff4765b-5882c\" (UID: \"f4f8b773-1ab4-4376-8158-46fd41c3a6e1\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-7056468c-6fdff4765b-5882c"
Jul 31 23:28:31.865913 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:31.865841    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fk9b8\" (UniqueName: \"kubernetes.io/projected/f4f8b773-1ab4-4376-8158-46fd41c3a6e1-kube-api-access-fk9b8\") pod \"payload-pre-processing-e2e-derive-7056468c-6fdff4765b-5882c\" (UID: \"f4f8b773-1ab4-4376-8158-46fd41c3a6e1\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-7056468c-6fdff4765b-5882c"
Jul 31 23:28:31.868283 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:31.868262    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc596b41-abbd-4d7e-88a5-f418a9cbabce-maas-api-tls\") pod \"maas-api-e2e-derive-7056468c-85cb6cf9bd-sg7wm\" (UID: \"fc596b41-abbd-4d7e-88a5-f418a9cbabce\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7056468c-85cb6cf9bd-sg7wm"
Jul 31 23:28:31.966972 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:31.966934    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6ngd\" (UniqueName: \"kubernetes.io/projected/4efd1495-d519-4ea0-9ee2-a997ca6d2e44-kube-api-access-c6ngd\") pod \"payload-processing-e2e-derive-7056468c-fcb7d7898-xcv75\" (UID: \"4efd1495-d519-4ea0-9ee2-a997ca6d2e44\") " pod="openshift-ingress/payload-processing-e2e-derive-7056468c-fcb7d7898-xcv75"
Jul 31 23:28:31.967188 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:31.967003    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4f8b773-1ab4-4376-8158-46fd41c3a6e1-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-7056468c-6fdff4765b-5882c\" (UID: \"f4f8b773-1ab4-4376-8158-46fd41c3a6e1\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-7056468c-6fdff4765b-5882c"
Jul 31 23:28:31.967188 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:31.967031    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fk9b8\" (UniqueName: \"kubernetes.io/projected/f4f8b773-1ab4-4376-8158-46fd41c3a6e1-kube-api-access-fk9b8\") pod \"payload-pre-processing-e2e-derive-7056468c-6fdff4765b-5882c\" (UID: \"f4f8b773-1ab4-4376-8158-46fd41c3a6e1\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-7056468c-6fdff4765b-5882c"
Jul 31 23:28:31.967188 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:31.967053    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4efd1495-d519-4ea0-9ee2-a997ca6d2e44-plugins-config-volume\") pod \"payload-processing-e2e-derive-7056468c-fcb7d7898-xcv75\" (UID: \"4efd1495-d519-4ea0-9ee2-a997ca6d2e44\") " pod="openshift-ingress/payload-processing-e2e-derive-7056468c-fcb7d7898-xcv75"
Jul 31 23:28:31.967767 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:31.967746    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4f8b773-1ab4-4376-8158-46fd41c3a6e1-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-7056468c-6fdff4765b-5882c\" (UID: \"f4f8b773-1ab4-4376-8158-46fd41c3a6e1\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-7056468c-6fdff4765b-5882c"
Jul 31 23:28:31.975337 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:31.975313    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fk9b8\" (UniqueName: \"kubernetes.io/projected/f4f8b773-1ab4-4376-8158-46fd41c3a6e1-kube-api-access-fk9b8\") pod \"payload-pre-processing-e2e-derive-7056468c-6fdff4765b-5882c\" (UID: \"f4f8b773-1ab4-4376-8158-46fd41c3a6e1\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-7056468c-6fdff4765b-5882c"
Jul 31 23:28:32.068169 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:32.068073    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6ngd\" (UniqueName: \"kubernetes.io/projected/4efd1495-d519-4ea0-9ee2-a997ca6d2e44-kube-api-access-c6ngd\") pod \"payload-processing-e2e-derive-7056468c-fcb7d7898-xcv75\" (UID: \"4efd1495-d519-4ea0-9ee2-a997ca6d2e44\") " pod="openshift-ingress/payload-processing-e2e-derive-7056468c-fcb7d7898-xcv75"
Jul 31 23:28:32.068169 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:32.068119    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4efd1495-d519-4ea0-9ee2-a997ca6d2e44-plugins-config-volume\") pod \"payload-processing-e2e-derive-7056468c-fcb7d7898-xcv75\" (UID: \"4efd1495-d519-4ea0-9ee2-a997ca6d2e44\") " pod="openshift-ingress/payload-processing-e2e-derive-7056468c-fcb7d7898-xcv75"
Jul 31 23:28:32.068838 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:32.068811    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4efd1495-d519-4ea0-9ee2-a997ca6d2e44-plugins-config-volume\") pod \"payload-processing-e2e-derive-7056468c-fcb7d7898-xcv75\" (UID: \"4efd1495-d519-4ea0-9ee2-a997ca6d2e44\") " pod="openshift-ingress/payload-processing-e2e-derive-7056468c-fcb7d7898-xcv75"
Jul 31 23:28:32.075872 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:32.075852    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6ngd\" (UniqueName: \"kubernetes.io/projected/4efd1495-d519-4ea0-9ee2-a997ca6d2e44-kube-api-access-c6ngd\") pod \"payload-processing-e2e-derive-7056468c-fcb7d7898-xcv75\" (UID: \"4efd1495-d519-4ea0-9ee2-a997ca6d2e44\") " pod="openshift-ingress/payload-processing-e2e-derive-7056468c-fcb7d7898-xcv75"
Jul 31 23:28:32.109416 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:32.109377    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7056468c-85cb6cf9bd-sg7wm"
Jul 31 23:28:32.131544 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:32.131513    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-7056468c-6fdff4765b-5882c"
Jul 31 23:28:32.161024 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:32.160985    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-7056468c-fcb7d7898-xcv75"
Jul 31 23:28:32.299649 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:32.299345    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7056468c-85cb6cf9bd-sg7wm"]
Jul 31 23:28:32.303191 ip-10-0-128-243 kubenswrapper[2584]: W0731 23:28:32.303073    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc596b41_abbd_4d7e_88a5_f418a9cbabce.slice/crio-ab96643e36ffe7d56e065b7d6160c8cde90172c22461bd31164a6be6fc919ea5 WatchSource:0}: Error finding container ab96643e36ffe7d56e065b7d6160c8cde90172c22461bd31164a6be6fc919ea5: Status 404 returned error can't find the container with id ab96643e36ffe7d56e065b7d6160c8cde90172c22461bd31164a6be6fc919ea5
Jul 31 23:28:32.452488 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:32.452396    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7056468c-85cb6cf9bd-sg7wm" event={"ID":"fc596b41-abbd-4d7e-88a5-f418a9cbabce","Type":"ContainerStarted","Data":"ab96643e36ffe7d56e065b7d6160c8cde90172c22461bd31164a6be6fc919ea5"}
Jul 31 23:28:32.531546 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:32.531476    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-7056468c-6fdff4765b-5882c"]
Jul 31 23:28:32.534197 ip-10-0-128-243 kubenswrapper[2584]: W0731 23:28:32.534167    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4f8b773_1ab4_4376_8158_46fd41c3a6e1.slice/crio-a1b0837648cb88f2b32c31ff02bf5bc1b6253118b66390642308a36fd4f70613 WatchSource:0}: Error finding container a1b0837648cb88f2b32c31ff02bf5bc1b6253118b66390642308a36fd4f70613: Status 404 returned error can't find the container with id a1b0837648cb88f2b32c31ff02bf5bc1b6253118b66390642308a36fd4f70613
Jul 31 23:28:32.569761 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:32.569739    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-7056468c-fcb7d7898-xcv75"]
Jul 31 23:28:32.572275 ip-10-0-128-243 kubenswrapper[2584]: W0731 23:28:32.572245    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4efd1495_d519_4ea0_9ee2_a997ca6d2e44.slice/crio-82e0d02c2b8ed9f068c42d7a35f244994ed2d3b6a3e53aac332b1ae981d762ed WatchSource:0}: Error finding container 82e0d02c2b8ed9f068c42d7a35f244994ed2d3b6a3e53aac332b1ae981d762ed: Status 404 returned error can't find the container with id 82e0d02c2b8ed9f068c42d7a35f244994ed2d3b6a3e53aac332b1ae981d762ed
Jul 31 23:28:33.458991 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:33.458958    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-7056468c-fcb7d7898-xcv75" event={"ID":"4efd1495-d519-4ea0-9ee2-a997ca6d2e44","Type":"ContainerStarted","Data":"d52279dbff4f173a2433e5564e4b10f86ed9b7f2531b772cddfb341a8bb49016"}
Jul 31 23:28:33.459448 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:33.459016    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-7056468c-fcb7d7898-xcv75" event={"ID":"4efd1495-d519-4ea0-9ee2-a997ca6d2e44","Type":"ContainerStarted","Data":"82e0d02c2b8ed9f068c42d7a35f244994ed2d3b6a3e53aac332b1ae981d762ed"}
Jul 31 23:28:33.459448 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:33.459132    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-7056468c-fcb7d7898-xcv75"
Jul 31 23:28:33.460496 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:33.460469    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7056468c-85cb6cf9bd-sg7wm" event={"ID":"fc596b41-abbd-4d7e-88a5-f418a9cbabce","Type":"ContainerStarted","Data":"c63554c1414df0db1de7983a41f23ebb19318a1536fc1b182e7bb49673e16d4a"}
Jul 31 23:28:33.460616 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:33.460523    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7056468c-85cb6cf9bd-sg7wm"
Jul 31 23:28:33.461944 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:33.461922    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-7056468c-6fdff4765b-5882c" event={"ID":"f4f8b773-1ab4-4376-8158-46fd41c3a6e1","Type":"ContainerStarted","Data":"c27a412be117751725325f38b781aff3b0c0b661e3358ef4a0633818748ca19b"}
Jul 31 23:28:33.462024 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:33.461966    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-7056468c-6fdff4765b-5882c" event={"ID":"f4f8b773-1ab4-4376-8158-46fd41c3a6e1","Type":"ContainerStarted","Data":"a1b0837648cb88f2b32c31ff02bf5bc1b6253118b66390642308a36fd4f70613"}
Jul 31 23:28:33.462024 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:33.461996    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-7056468c-6fdff4765b-5882c"
Jul 31 23:28:33.474615 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:33.474574    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-7056468c-fcb7d7898-xcv75" podStartSLOduration=2.474560627 podStartE2EDuration="2.474560627s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:28:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:28:33.472378526 +0000 UTC m=+1897.118164836" watchObservedRunningTime="2026-07-31 23:28:33.474560627 +0000 UTC m=+1897.120346936"
Jul 31 23:28:33.488248 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:33.488205    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7056468c-85cb6cf9bd-sg7wm" podStartSLOduration=2.178113168 podStartE2EDuration="2.48819143s" totalImagesPullingTime="310.078262ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:28:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:28:33.487284192 +0000 UTC m=+1897.133070503" watchObservedRunningTime="2026-07-31 23:28:33.48819143 +0000 UTC m=+1897.133977739"
Jul 31 23:28:33.501765 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:33.501721    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-7056468c-6fdff4765b-5882c" podStartSLOduration=2.5017102810000003 podStartE2EDuration="2.501710281s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:28:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:28:33.499905911 +0000 UTC m=+1897.145692233" watchObservedRunningTime="2026-07-31 23:28:33.501710281 +0000 UTC m=+1897.147496581"
Jul 31 23:28:39.472863 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:39.472833    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7056468c-85cb6cf9bd-sg7wm"
Jul 31 23:28:44.469428 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:44.469399    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-7056468c-6fdff4765b-5882c"
Jul 31 23:28:44.469809 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:28:44.469474    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-7056468c-fcb7d7898-xcv75"
Jul 31 23:29:01.685956 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:29:01.685919    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7056468c-85cb6cf9bd-sg7wm"]
Jul 31 23:29:01.686509 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:29:01.686204    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7056468c-85cb6cf9bd-sg7wm" podUID="fc596b41-abbd-4d7e-88a5-f418a9cbabce" containerName="maas-api" containerID="cri-o://c63554c1414df0db1de7983a41f23ebb19318a1536fc1b182e7bb49673e16d4a" gracePeriod=30
Jul 31 23:29:01.740347 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:29:01.740314    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-7056468c-fcb7d7898-xcv75"]
Jul 31 23:29:01.740606 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:29:01.740569    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-7056468c-fcb7d7898-xcv75" podUID="4efd1495-d519-4ea0-9ee2-a997ca6d2e44" containerName="payload-processing" containerID="cri-o://d52279dbff4f173a2433e5564e4b10f86ed9b7f2531b772cddfb341a8bb49016" gracePeriod=30
Jul 31 23:29:01.746664 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:29:01.746632    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-7056468c-6fdff4765b-5882c"]
Jul 31 23:29:01.747212 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:29:01.746887    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-7056468c-6fdff4765b-5882c" podUID="f4f8b773-1ab4-4376-8158-46fd41c3a6e1" containerName="payload-pre-processing" containerID="cri-o://c27a412be117751725325f38b781aff3b0c0b661e3358ef4a0633818748ca19b" gracePeriod=30
Jul 31 23:29:02.003987 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:29:02.003961    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-7056468c-fcb7d7898-xcv75"
Jul 31 23:29:02.031931 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:29:02.031897    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7056468c-85cb6cf9bd-sg7wm"
Jul 31 23:29:02.057027 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:29:02.057003    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-7056468c-6fdff4765b-5882c"
Jul 31 23:29:02.130804 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:29:02.130770    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c6ngd\" (UniqueName: \"kubernetes.io/projected/4efd1495-d519-4ea0-9ee2-a997ca6d2e44-kube-api-access-c6ngd\") pod \"4efd1495-d519-4ea0-9ee2-a997ca6d2e44\" (UID: \"4efd1495-d519-4ea0-9ee2-a997ca6d2e44\") "
Jul 31 23:29:02.130995 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:29:02.130838    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4efd1495-d519-4ea0-9ee2-a997ca6d2e44-plugins-config-volume\") pod \"4efd1495-d519-4ea0-9ee2-a997ca6d2e44\" (UID: \"4efd1495-d519-4ea0-9ee2-a997ca6d2e44\") "
Jul 31 23:29:02.131066 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:29:02.131013    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g4jtl\" (UniqueName: \"kubernetes.io/projected/fc596b41-abbd-4d7e-88a5-f418a9cbabce-kube-api-access-g4jtl\") pod \"fc596b41-abbd-4d7e-88a5-f418a9cbabce\" (UID: \"fc596b41-abbd-4d7e-88a5-f418a9cbabce\") "
Jul 31 23:29:02.131140 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:29:02.131122    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc596b41-abbd-4d7e-88a5-f418a9cbabce-maas-api-tls\") pod \"fc596b41-abbd-4d7e-88a5-f418a9cbabce\" (UID: \"fc596b41-abbd-4d7e-88a5-f418a9cbabce\") "
Jul 31 23:29:02.131218 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:29:02.131198    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc596b41-abbd-4d7e-88a5-f418a9cbabce-openshift-service-ca\") pod \"fc596b41-abbd-4d7e-88a5-f418a9cbabce\" (UID: \"fc596b41-abbd-4d7e-88a5-f418a9cbabce\") "
Jul 31 23:29:02.131343 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:29:02.131222    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4efd1495-d519-4ea0-9ee2-a997ca6d2e44-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4efd1495-d519-4ea0-9ee2-a997ca6d2e44" (UID: "4efd1495-d519-4ea0-9ee2-a997ca6d2e44"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:29:02.131477 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:29:02.131461    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4efd1495-d519-4ea0-9ee2-a997ca6d2e44-plugins-config-volume\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:29:02.131540 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:29:02.131490    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc596b41-abbd-4d7e-88a5-f418a9cbabce-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fc596b41-abbd-4d7e-88a5-f418a9cbabce" (UID: "fc596b41-abbd-4d7e-88a5-f418a9cbabce"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:29:02.133072 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:29:02.133043    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4efd1495-d519-4ea0-9ee2-a997ca6d2e44-kube-api-access-c6ngd" (OuterVolumeSpecName: "kube-api-access-c6ngd") pod "4efd1495-d519-4ea0-9ee2-a997ca6d2e44" (UID: "4efd1495-d519-4ea0-9ee2-a997ca6d2e44"). InnerVolumeSpecName "kube-api-access-c6ngd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 23:29:02.133072 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:29:02.133049    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc596b41-abbd-4d7e-88a5-f418a9cbabce-kube-api-access-g4jtl" (OuterVolumeSpecName: "kube-api-access-g4jtl") pod "fc596b41-abbd-4d7e-88a5-f418a9cbabce" (UID: "fc596b41-abbd-4d7e-88a5-f418a9cbabce"). InnerVolumeSpecName "kube-api-access-g4jtl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 23:29:02.133425 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:29:02.133404    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc596b41-abbd-4d7e-88a5-f418a9cbabce-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fc596b41-abbd-4d7e-88a5-f418a9cbabce" (UID: "fc596b41-abbd-4d7e-88a5-f418a9cbabce"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 23:29:02.232580 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:29:02.232489    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4f8b773-1ab4-4376-8158-46fd41c3a6e1-plugins-config-volume\") pod \"f4f8b773-1ab4-4376-8158-46fd41c3a6e1\" (UID: \"f4f8b773-1ab4-4376-8158-46fd41c3a6e1\") "
Jul 31 23:29:02.232781 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:29:02.232607    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fk9b8\" (UniqueName: \"kubernetes.io/projected/f4f8b773-1ab4-4376-8158-46fd41c3a6e1-kube-api-access-fk9b8\") pod \"f4f8b773-1ab4-4376-8158-46fd41c3a6e1\" (UID: \"f4f8b773-1ab4-4376-8158-46fd41c3a6e1\") "
Jul 31 23:29:02.232867 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:29:02.232848    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc596b41-abbd-4d7e-88a5-f418a9cbabce-maas-api-tls\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:29:02.232924 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:29:02.232873    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc596b41-abbd-4d7e-88a5-f418a9cbabce-openshift-service-ca\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:29:02.232924 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:29:02.232888    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c6ngd\" (UniqueName: \"kubernetes.io/projected/4efd1495-d519-4ea0-9ee2-a997ca6d2e44-kube-api-access-c6ngd\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:29:02.232924 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:29:02.232902    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g4jtl\" (UniqueName: \"kubernetes.io/projected/fc596b41-abbd-4d7e-88a5-f418a9cbabce-kube-api-access-g4jtl\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:29:02.233087 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:29:02.232952    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4f8b773-1ab4-4376-8158-46fd41c3a6e1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f4f8b773-1ab4-4376-8158-46fd41c3a6e1" (UID: "f4f8b773-1ab4-4376-8158-46fd41c3a6e1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:29:02.234657 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:29:02.234627    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4f8b773-1ab4-4376-8158-46fd41c3a6e1-kube-api-access-fk9b8" (OuterVolumeSpecName: "kube-api-access-fk9b8") pod "f4f8b773-1ab4-4376-8158-46fd41c3a6e1" (UID: "f4f8b773-1ab4-4376-8158-46fd41c3a6e1"). InnerVolumeSpecName "kube-api-access-fk9b8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 23:29:02.333858 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:29:02.333819    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fk9b8\" (UniqueName: \"kubernetes.io/projected/f4f8b773-1ab4-4376-8158-46fd41c3a6e1-kube-api-access-fk9b8\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:29:02.333858 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:29:02.333849    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4f8b773-1ab4-4376-8158-46fd41c3a6e1-plugins-config-volume\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:29:02.608385 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:29:02.608351    2584 generic.go:356] "Generic (PLEG): container finished" podID="4efd1495-d519-4ea0-9ee2-a997ca6d2e44" containerID="d52279dbff4f173a2433e5564e4b10f86ed9b7f2531b772cddfb341a8bb49016" exitCode=0
Jul 31 23:29:02.608564 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:29:02.608398    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-7056468c-fcb7d7898-xcv75"
Jul 31 23:29:02.608564 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:29:02.608444    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-7056468c-fcb7d7898-xcv75" event={"ID":"4efd1495-d519-4ea0-9ee2-a997ca6d2e44","Type":"ContainerDied","Data":"d52279dbff4f173a2433e5564e4b10f86ed9b7f2531b772cddfb341a8bb49016"}
Jul 31 23:29:02.608564 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:29:02.608498    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-7056468c-fcb7d7898-xcv75" event={"ID":"4efd1495-d519-4ea0-9ee2-a997ca6d2e44","Type":"ContainerDied","Data":"82e0d02c2b8ed9f068c42d7a35f244994ed2d3b6a3e53aac332b1ae981d762ed"}
Jul 31 23:29:02.608564 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:29:02.608513    2584 scope.go:117] "RemoveContainer" containerID="d52279dbff4f173a2433e5564e4b10f86ed9b7f2531b772cddfb341a8bb49016"
Jul 31 23:29:02.609905 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:29:02.609882    2584 generic.go:356] "Generic (PLEG): container finished" podID="fc596b41-abbd-4d7e-88a5-f418a9cbabce" containerID="c63554c1414df0db1de7983a41f23ebb19318a1536fc1b182e7bb49673e16d4a" exitCode=0
Jul 31 23:29:02.610026 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:29:02.609930    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7056468c-85cb6cf9bd-sg7wm"
Jul 31 23:29:02.610026 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:29:02.609966    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7056468c-85cb6cf9bd-sg7wm" event={"ID":"fc596b41-abbd-4d7e-88a5-f418a9cbabce","Type":"ContainerDied","Data":"c63554c1414df0db1de7983a41f23ebb19318a1536fc1b182e7bb49673e16d4a"}
Jul 31 23:29:02.610026 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:29:02.610014    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7056468c-85cb6cf9bd-sg7wm" event={"ID":"fc596b41-abbd-4d7e-88a5-f418a9cbabce","Type":"ContainerDied","Data":"ab96643e36ffe7d56e065b7d6160c8cde90172c22461bd31164a6be6fc919ea5"}
Jul 31 23:29:02.611272 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:29:02.611249    2584 generic.go:356] "Generic (PLEG): container finished" podID="f4f8b773-1ab4-4376-8158-46fd41c3a6e1" containerID="c27a412be117751725325f38b781aff3b0c0b661e3358ef4a0633818748ca19b" exitCode=0
Jul 31 23:29:02.611400 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:29:02.611295    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-7056468c-6fdff4765b-5882c"
Jul 31 23:29:02.611400 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:29:02.611334    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-7056468c-6fdff4765b-5882c" event={"ID":"f4f8b773-1ab4-4376-8158-46fd41c3a6e1","Type":"ContainerDied","Data":"c27a412be117751725325f38b781aff3b0c0b661e3358ef4a0633818748ca19b"}
Jul 31 23:29:02.611400 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:29:02.611388    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-7056468c-6fdff4765b-5882c" event={"ID":"f4f8b773-1ab4-4376-8158-46fd41c3a6e1","Type":"ContainerDied","Data":"a1b0837648cb88f2b32c31ff02bf5bc1b6253118b66390642308a36fd4f70613"}
Jul 31 23:29:02.617893 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:29:02.617871    2584 scope.go:117] "RemoveContainer" containerID="d52279dbff4f173a2433e5564e4b10f86ed9b7f2531b772cddfb341a8bb49016"
Jul 31 23:29:02.618189 ip-10-0-128-243 kubenswrapper[2584]: E0731 23:29:02.618167    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d52279dbff4f173a2433e5564e4b10f86ed9b7f2531b772cddfb341a8bb49016\": container with ID starting with d52279dbff4f173a2433e5564e4b10f86ed9b7f2531b772cddfb341a8bb49016 not found: ID does not exist" containerID="d52279dbff4f173a2433e5564e4b10f86ed9b7f2531b772cddfb341a8bb49016"
Jul 31 23:29:02.618269 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:29:02.618200    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d52279dbff4f173a2433e5564e4b10f86ed9b7f2531b772cddfb341a8bb49016"} err="failed to get container status \"d52279dbff4f173a2433e5564e4b10f86ed9b7f2531b772cddfb341a8bb49016\": rpc error: code = NotFound desc = could not find container \"d52279dbff4f173a2433e5564e4b10f86ed9b7f2531b772cddfb341a8bb49016\": container with ID starting with d52279dbff4f173a2433e5564e4b10f86ed9b7f2531b772cddfb341a8bb49016 not found: ID does not exist"
Jul 31 23:29:02.618269 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:29:02.618222    2584 scope.go:117] "RemoveContainer" containerID="c63554c1414df0db1de7983a41f23ebb19318a1536fc1b182e7bb49673e16d4a"
Jul 31 23:29:02.627899 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:29:02.627883    2584 scope.go:117] "RemoveContainer" containerID="c63554c1414df0db1de7983a41f23ebb19318a1536fc1b182e7bb49673e16d4a"
Jul 31 23:29:02.628166 ip-10-0-128-243 kubenswrapper[2584]: E0731 23:29:02.628145    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c63554c1414df0db1de7983a41f23ebb19318a1536fc1b182e7bb49673e16d4a\": container with ID starting with c63554c1414df0db1de7983a41f23ebb19318a1536fc1b182e7bb49673e16d4a not found: ID does not exist" containerID="c63554c1414df0db1de7983a41f23ebb19318a1536fc1b182e7bb49673e16d4a"
Jul 31 23:29:02.628214 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:29:02.628173    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c63554c1414df0db1de7983a41f23ebb19318a1536fc1b182e7bb49673e16d4a"} err="failed to get container status \"c63554c1414df0db1de7983a41f23ebb19318a1536fc1b182e7bb49673e16d4a\": rpc error: code = NotFound desc = could not find container \"c63554c1414df0db1de7983a41f23ebb19318a1536fc1b182e7bb49673e16d4a\": container with ID starting with c63554c1414df0db1de7983a41f23ebb19318a1536fc1b182e7bb49673e16d4a not found: ID does not exist"
Jul 31 23:29:02.628214 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:29:02.628190    2584 scope.go:117] "RemoveContainer" containerID="c27a412be117751725325f38b781aff3b0c0b661e3358ef4a0633818748ca19b"
Jul 31 23:29:02.636588 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:29:02.636572    2584 scope.go:117] "RemoveContainer" containerID="c27a412be117751725325f38b781aff3b0c0b661e3358ef4a0633818748ca19b"
Jul 31 23:29:02.636896 ip-10-0-128-243 kubenswrapper[2584]: E0731 23:29:02.636878    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c27a412be117751725325f38b781aff3b0c0b661e3358ef4a0633818748ca19b\": container with ID starting with c27a412be117751725325f38b781aff3b0c0b661e3358ef4a0633818748ca19b not found: ID does not exist" containerID="c27a412be117751725325f38b781aff3b0c0b661e3358ef4a0633818748ca19b"
Jul 31 23:29:02.636964 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:29:02.636906    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c27a412be117751725325f38b781aff3b0c0b661e3358ef4a0633818748ca19b"} err="failed to get container status \"c27a412be117751725325f38b781aff3b0c0b661e3358ef4a0633818748ca19b\": rpc error: code = NotFound desc = could not find container \"c27a412be117751725325f38b781aff3b0c0b661e3358ef4a0633818748ca19b\": container with ID starting with c27a412be117751725325f38b781aff3b0c0b661e3358ef4a0633818748ca19b not found: ID does not exist"
Jul 31 23:29:02.643761 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:29:02.643737    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-7056468c-6fdff4765b-5882c"]
Jul 31 23:29:02.651478 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:29:02.651457    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-7056468c-6fdff4765b-5882c"]
Jul 31 23:29:02.670738 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:29:02.670712    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-7056468c-fcb7d7898-xcv75"]
Jul 31 23:29:02.679905 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:29:02.679858    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-7056468c-fcb7d7898-xcv75"]
Jul 31 23:29:02.689074 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:29:02.689051    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7056468c-85cb6cf9bd-sg7wm"]
Jul 31 23:29:02.694891 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:29:02.694872    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7056468c-85cb6cf9bd-sg7wm"]
Jul 31 23:29:02.890558 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:29:02.890477    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4efd1495-d519-4ea0-9ee2-a997ca6d2e44" path="/var/lib/kubelet/pods/4efd1495-d519-4ea0-9ee2-a997ca6d2e44/volumes"
Jul 31 23:29:02.890862 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:29:02.890849    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4f8b773-1ab4-4376-8158-46fd41c3a6e1" path="/var/lib/kubelet/pods/f4f8b773-1ab4-4376-8158-46fd41c3a6e1/volumes"
Jul 31 23:29:02.891201 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:29:02.891188    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc596b41-abbd-4d7e-88a5-f418a9cbabce" path="/var/lib/kubelet/pods/fc596b41-abbd-4d7e-88a5-f418a9cbabce/volumes"
Jul 31 23:31:01.555491 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:01.555457    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-124aaef2-79f664d6f5-m6lxh"]
Jul 31 23:31:01.555966 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:01.555848    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc596b41-abbd-4d7e-88a5-f418a9cbabce" containerName="maas-api"
Jul 31 23:31:01.555966 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:01.555863    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc596b41-abbd-4d7e-88a5-f418a9cbabce" containerName="maas-api"
Jul 31 23:31:01.555966 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:01.555875    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4f8b773-1ab4-4376-8158-46fd41c3a6e1" containerName="payload-pre-processing"
Jul 31 23:31:01.555966 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:01.555881    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4f8b773-1ab4-4376-8158-46fd41c3a6e1" containerName="payload-pre-processing"
Jul 31 23:31:01.555966 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:01.555895    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4efd1495-d519-4ea0-9ee2-a997ca6d2e44" containerName="payload-processing"
Jul 31 23:31:01.555966 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:01.555901    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="4efd1495-d519-4ea0-9ee2-a997ca6d2e44" containerName="payload-processing"
Jul 31 23:31:01.555966 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:01.555954    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="4efd1495-d519-4ea0-9ee2-a997ca6d2e44" containerName="payload-processing"
Jul 31 23:31:01.555966 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:01.555962    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4f8b773-1ab4-4376-8158-46fd41c3a6e1" containerName="payload-pre-processing"
Jul 31 23:31:01.555966 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:01.555968    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc596b41-abbd-4d7e-88a5-f418a9cbabce" containerName="maas-api"
Jul 31 23:31:01.558866 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:01.558843    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-124aaef2-79f664d6f5-m6lxh"
Jul 31 23:31:01.561415 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:01.561393    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-124aaef2-serving-cert\""
Jul 31 23:31:01.565773 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:01.565736    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aeb4fea1-f63d-4d0a-98de-ebe3d0103393-openshift-service-ca\") pod \"maas-api-e2e-mt-124aaef2-79f664d6f5-m6lxh\" (UID: \"aeb4fea1-f63d-4d0a-98de-ebe3d0103393\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-124aaef2-79f664d6f5-m6lxh"
Jul 31 23:31:01.565888 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:01.565805    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnvxm\" (UniqueName: \"kubernetes.io/projected/aeb4fea1-f63d-4d0a-98de-ebe3d0103393-kube-api-access-pnvxm\") pod \"maas-api-e2e-mt-124aaef2-79f664d6f5-m6lxh\" (UID: \"aeb4fea1-f63d-4d0a-98de-ebe3d0103393\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-124aaef2-79f664d6f5-m6lxh"
Jul 31 23:31:01.565888 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:01.565843    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aeb4fea1-f63d-4d0a-98de-ebe3d0103393-maas-api-tls\") pod \"maas-api-e2e-mt-124aaef2-79f664d6f5-m6lxh\" (UID: \"aeb4fea1-f63d-4d0a-98de-ebe3d0103393\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-124aaef2-79f664d6f5-m6lxh"
Jul 31 23:31:01.569087 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:01.569063    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-124aaef2-79f664d6f5-m6lxh"]
Jul 31 23:31:01.666924 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:01.666894    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aeb4fea1-f63d-4d0a-98de-ebe3d0103393-openshift-service-ca\") pod \"maas-api-e2e-mt-124aaef2-79f664d6f5-m6lxh\" (UID: \"aeb4fea1-f63d-4d0a-98de-ebe3d0103393\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-124aaef2-79f664d6f5-m6lxh"
Jul 31 23:31:01.667093 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:01.666942    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnvxm\" (UniqueName: \"kubernetes.io/projected/aeb4fea1-f63d-4d0a-98de-ebe3d0103393-kube-api-access-pnvxm\") pod \"maas-api-e2e-mt-124aaef2-79f664d6f5-m6lxh\" (UID: \"aeb4fea1-f63d-4d0a-98de-ebe3d0103393\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-124aaef2-79f664d6f5-m6lxh"
Jul 31 23:31:01.667093 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:01.666974    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aeb4fea1-f63d-4d0a-98de-ebe3d0103393-maas-api-tls\") pod \"maas-api-e2e-mt-124aaef2-79f664d6f5-m6lxh\" (UID: \"aeb4fea1-f63d-4d0a-98de-ebe3d0103393\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-124aaef2-79f664d6f5-m6lxh"
Jul 31 23:31:01.667184 ip-10-0-128-243 kubenswrapper[2584]: E0731 23:31:01.667110    2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-124aaef2-serving-cert: secret "maas-api-e2e-mt-124aaef2-serving-cert" not found
Jul 31 23:31:01.667184 ip-10-0-128-243 kubenswrapper[2584]: E0731 23:31:01.667172    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aeb4fea1-f63d-4d0a-98de-ebe3d0103393-maas-api-tls podName:aeb4fea1-f63d-4d0a-98de-ebe3d0103393 nodeName:}" failed. No retries permitted until 2026-07-31 23:31:02.167157157 +0000 UTC m=+2045.812943447 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/aeb4fea1-f63d-4d0a-98de-ebe3d0103393-maas-api-tls") pod "maas-api-e2e-mt-124aaef2-79f664d6f5-m6lxh" (UID: "aeb4fea1-f63d-4d0a-98de-ebe3d0103393") : secret "maas-api-e2e-mt-124aaef2-serving-cert" not found
Jul 31 23:31:01.667602 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:01.667571    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aeb4fea1-f63d-4d0a-98de-ebe3d0103393-openshift-service-ca\") pod \"maas-api-e2e-mt-124aaef2-79f664d6f5-m6lxh\" (UID: \"aeb4fea1-f63d-4d0a-98de-ebe3d0103393\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-124aaef2-79f664d6f5-m6lxh"
Jul 31 23:31:01.677462 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:01.677441    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnvxm\" (UniqueName: \"kubernetes.io/projected/aeb4fea1-f63d-4d0a-98de-ebe3d0103393-kube-api-access-pnvxm\") pod \"maas-api-e2e-mt-124aaef2-79f664d6f5-m6lxh\" (UID: \"aeb4fea1-f63d-4d0a-98de-ebe3d0103393\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-124aaef2-79f664d6f5-m6lxh"
Jul 31 23:31:02.171400 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:02.171365    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aeb4fea1-f63d-4d0a-98de-ebe3d0103393-maas-api-tls\") pod \"maas-api-e2e-mt-124aaef2-79f664d6f5-m6lxh\" (UID: \"aeb4fea1-f63d-4d0a-98de-ebe3d0103393\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-124aaef2-79f664d6f5-m6lxh"
Jul 31 23:31:02.173822 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:02.173801    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aeb4fea1-f63d-4d0a-98de-ebe3d0103393-maas-api-tls\") pod \"maas-api-e2e-mt-124aaef2-79f664d6f5-m6lxh\" (UID: \"aeb4fea1-f63d-4d0a-98de-ebe3d0103393\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-124aaef2-79f664d6f5-m6lxh"
Jul 31 23:31:02.272228 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:02.272191    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-124aaef2-7b7cf9b756-fhhw7"]
Jul 31 23:31:02.275710 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:02.275687    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-124aaef2-7b7cf9b756-fhhw7"
Jul 31 23:31:02.278253 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:02.278232    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-124aaef2-dockercfg-wg8g7\""
Jul 31 23:31:02.278411 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:02.278394    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-124aaef2\""
Jul 31 23:31:02.284925 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:02.284897    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-124aaef2-7b7cf9b756-fhhw7"]
Jul 31 23:31:02.306543 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:02.306500    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-124aaef2-f5f9f4c46-brjwf"]
Jul 31 23:31:02.310198 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:02.310165    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-124aaef2-f5f9f4c46-brjwf"
Jul 31 23:31:02.311202 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:02.311149    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-124aaef2-f5f9f4c46-brjwf"]
Jul 31 23:31:02.372798 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:02.372759    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cd66e883-affa-49f3-8b2b-8204aa324987-plugins-config-volume\") pod \"payload-processing-e2e-mt-124aaef2-f5f9f4c46-brjwf\" (UID: \"cd66e883-affa-49f3-8b2b-8204aa324987\") " pod="openshift-ingress/payload-processing-e2e-mt-124aaef2-f5f9f4c46-brjwf"
Jul 31 23:31:02.372798 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:02.372801    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79z5x\" (UniqueName: \"kubernetes.io/projected/7796dc8a-d32b-4117-8234-81378252a31a-kube-api-access-79z5x\") pod \"payload-pre-processing-e2e-mt-124aaef2-7b7cf9b756-fhhw7\" (UID: \"7796dc8a-d32b-4117-8234-81378252a31a\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-124aaef2-7b7cf9b756-fhhw7"
Jul 31 23:31:02.372999 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:02.372819    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhq6j\" (UniqueName: \"kubernetes.io/projected/cd66e883-affa-49f3-8b2b-8204aa324987-kube-api-access-nhq6j\") pod \"payload-processing-e2e-mt-124aaef2-f5f9f4c46-brjwf\" (UID: \"cd66e883-affa-49f3-8b2b-8204aa324987\") " pod="openshift-ingress/payload-processing-e2e-mt-124aaef2-f5f9f4c46-brjwf"
Jul 31 23:31:02.372999 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:02.372906    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7796dc8a-d32b-4117-8234-81378252a31a-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-124aaef2-7b7cf9b756-fhhw7\" (UID: \"7796dc8a-d32b-4117-8234-81378252a31a\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-124aaef2-7b7cf9b756-fhhw7"
Jul 31 23:31:02.470917 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:02.470838    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-124aaef2-79f664d6f5-m6lxh"
Jul 31 23:31:02.473477 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:02.473445    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cd66e883-affa-49f3-8b2b-8204aa324987-plugins-config-volume\") pod \"payload-processing-e2e-mt-124aaef2-f5f9f4c46-brjwf\" (UID: \"cd66e883-affa-49f3-8b2b-8204aa324987\") " pod="openshift-ingress/payload-processing-e2e-mt-124aaef2-f5f9f4c46-brjwf"
Jul 31 23:31:02.473620 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:02.473490    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79z5x\" (UniqueName: \"kubernetes.io/projected/7796dc8a-d32b-4117-8234-81378252a31a-kube-api-access-79z5x\") pod \"payload-pre-processing-e2e-mt-124aaef2-7b7cf9b756-fhhw7\" (UID: \"7796dc8a-d32b-4117-8234-81378252a31a\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-124aaef2-7b7cf9b756-fhhw7"
Jul 31 23:31:02.473620 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:02.473507    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhq6j\" (UniqueName: \"kubernetes.io/projected/cd66e883-affa-49f3-8b2b-8204aa324987-kube-api-access-nhq6j\") pod \"payload-processing-e2e-mt-124aaef2-f5f9f4c46-brjwf\" (UID: \"cd66e883-affa-49f3-8b2b-8204aa324987\") " pod="openshift-ingress/payload-processing-e2e-mt-124aaef2-f5f9f4c46-brjwf"
Jul 31 23:31:02.473620 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:02.473538    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7796dc8a-d32b-4117-8234-81378252a31a-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-124aaef2-7b7cf9b756-fhhw7\" (UID: \"7796dc8a-d32b-4117-8234-81378252a31a\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-124aaef2-7b7cf9b756-fhhw7"
Jul 31 23:31:02.474301 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:02.474190    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cd66e883-affa-49f3-8b2b-8204aa324987-plugins-config-volume\") pod \"payload-processing-e2e-mt-124aaef2-f5f9f4c46-brjwf\" (UID: \"cd66e883-affa-49f3-8b2b-8204aa324987\") " pod="openshift-ingress/payload-processing-e2e-mt-124aaef2-f5f9f4c46-brjwf"
Jul 31 23:31:02.474301 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:02.474217    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7796dc8a-d32b-4117-8234-81378252a31a-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-124aaef2-7b7cf9b756-fhhw7\" (UID: \"7796dc8a-d32b-4117-8234-81378252a31a\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-124aaef2-7b7cf9b756-fhhw7"
Jul 31 23:31:02.482521 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:02.482492    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhq6j\" (UniqueName: \"kubernetes.io/projected/cd66e883-affa-49f3-8b2b-8204aa324987-kube-api-access-nhq6j\") pod \"payload-processing-e2e-mt-124aaef2-f5f9f4c46-brjwf\" (UID: \"cd66e883-affa-49f3-8b2b-8204aa324987\") " pod="openshift-ingress/payload-processing-e2e-mt-124aaef2-f5f9f4c46-brjwf"
Jul 31 23:31:02.482612 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:02.482586    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-79z5x\" (UniqueName: \"kubernetes.io/projected/7796dc8a-d32b-4117-8234-81378252a31a-kube-api-access-79z5x\") pod \"payload-pre-processing-e2e-mt-124aaef2-7b7cf9b756-fhhw7\" (UID: \"7796dc8a-d32b-4117-8234-81378252a31a\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-124aaef2-7b7cf9b756-fhhw7"
Jul 31 23:31:02.587237 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:02.587202    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-124aaef2-7b7cf9b756-fhhw7"
Jul 31 23:31:02.598309 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:02.598279    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-124aaef2-79f664d6f5-m6lxh"]
Jul 31 23:31:02.600817 ip-10-0-128-243 kubenswrapper[2584]: W0731 23:31:02.600764    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaeb4fea1_f63d_4d0a_98de_ebe3d0103393.slice/crio-0747c4575bb8a3919e6d14c3ff21e90dad20468bbd269f8128199b48ebbe4fc8 WatchSource:0}: Error finding container 0747c4575bb8a3919e6d14c3ff21e90dad20468bbd269f8128199b48ebbe4fc8: Status 404 returned error can't find the container with id 0747c4575bb8a3919e6d14c3ff21e90dad20468bbd269f8128199b48ebbe4fc8
Jul 31 23:31:02.623333 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:02.623302    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-124aaef2-f5f9f4c46-brjwf"
Jul 31 23:31:02.736133 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:02.736065    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-124aaef2-7b7cf9b756-fhhw7"]
Jul 31 23:31:02.739076 ip-10-0-128-243 kubenswrapper[2584]: W0731 23:31:02.739032    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7796dc8a_d32b_4117_8234_81378252a31a.slice/crio-2ee0fddb9d01c1be51bec79832d5dfd8a699a32bafa02cd73f4f941490f2468e WatchSource:0}: Error finding container 2ee0fddb9d01c1be51bec79832d5dfd8a699a32bafa02cd73f4f941490f2468e: Status 404 returned error can't find the container with id 2ee0fddb9d01c1be51bec79832d5dfd8a699a32bafa02cd73f4f941490f2468e
Jul 31 23:31:02.772750 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:02.772727    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-124aaef2-f5f9f4c46-brjwf"]
Jul 31 23:31:02.780873 ip-10-0-128-243 kubenswrapper[2584]: W0731 23:31:02.780849    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd66e883_affa_49f3_8b2b_8204aa324987.slice/crio-9bcf84df3b9e52dbb6d7adb5dac4eaa64f65272c44eb9c43a97759539674151a WatchSource:0}: Error finding container 9bcf84df3b9e52dbb6d7adb5dac4eaa64f65272c44eb9c43a97759539674151a: Status 404 returned error can't find the container with id 9bcf84df3b9e52dbb6d7adb5dac4eaa64f65272c44eb9c43a97759539674151a
Jul 31 23:31:03.175089 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:03.175059    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-124aaef2-79f664d6f5-m6lxh" event={"ID":"aeb4fea1-f63d-4d0a-98de-ebe3d0103393","Type":"ContainerStarted","Data":"8b9b10acfbe8593d7f0af4757e4f6caa0e9e50be97441c3c0659429fcf38260a"}
Jul 31 23:31:03.175262 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:03.175112    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-124aaef2-79f664d6f5-m6lxh" event={"ID":"aeb4fea1-f63d-4d0a-98de-ebe3d0103393","Type":"ContainerStarted","Data":"0747c4575bb8a3919e6d14c3ff21e90dad20468bbd269f8128199b48ebbe4fc8"}
Jul 31 23:31:03.175262 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:03.175177    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-124aaef2-79f664d6f5-m6lxh"
Jul 31 23:31:03.176895 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:03.176868    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-124aaef2-f5f9f4c46-brjwf" event={"ID":"cd66e883-affa-49f3-8b2b-8204aa324987","Type":"ContainerStarted","Data":"30bef96f2e7ab1941064b4cafe12d65dcc33a47edfdd1ad3d55855aa9a4ce7d6"}
Jul 31 23:31:03.177021 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:03.176913    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-124aaef2-f5f9f4c46-brjwf" event={"ID":"cd66e883-affa-49f3-8b2b-8204aa324987","Type":"ContainerStarted","Data":"9bcf84df3b9e52dbb6d7adb5dac4eaa64f65272c44eb9c43a97759539674151a"}
Jul 31 23:31:03.177021 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:03.176962    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-124aaef2-f5f9f4c46-brjwf"
Jul 31 23:31:03.178551 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:03.178526    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-124aaef2-7b7cf9b756-fhhw7" event={"ID":"7796dc8a-d32b-4117-8234-81378252a31a","Type":"ContainerStarted","Data":"4dbb77a4d8efc1dff6bee000de8ad7fec6933b8f89d55ac741990852def4c65e"}
Jul 31 23:31:03.178694 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:03.178569    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-124aaef2-7b7cf9b756-fhhw7"
Jul 31 23:31:03.178694 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:03.178586    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-124aaef2-7b7cf9b756-fhhw7" event={"ID":"7796dc8a-d32b-4117-8234-81378252a31a","Type":"ContainerStarted","Data":"2ee0fddb9d01c1be51bec79832d5dfd8a699a32bafa02cd73f4f941490f2468e"}
Jul 31 23:31:03.191012 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:03.190970    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-124aaef2-79f664d6f5-m6lxh" podStartSLOduration=1.723574516 podStartE2EDuration="2.190958276s" totalImagesPullingTime="467.38376ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:31:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:31:03.18941927 +0000 UTC m=+2046.835205578" watchObservedRunningTime="2026-07-31 23:31:03.190958276 +0000 UTC m=+2046.836744585"
Jul 31 23:31:03.205161 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:03.205125    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-124aaef2-7b7cf9b756-fhhw7" podStartSLOduration=1.205113404 podStartE2EDuration="1.205113404s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:31:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:31:03.203064734 +0000 UTC m=+2046.848851043" watchObservedRunningTime="2026-07-31 23:31:03.205113404 +0000 UTC m=+2046.850899712"
Jul 31 23:31:03.218163 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:03.218125    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-124aaef2-f5f9f4c46-brjwf" podStartSLOduration=1.218113575 podStartE2EDuration="1.218113575s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:31:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:31:03.216970033 +0000 UTC m=+2046.862756345" watchObservedRunningTime="2026-07-31 23:31:03.218113575 +0000 UTC m=+2046.863899884"
Jul 31 23:31:09.190026 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:09.189991    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-124aaef2-79f664d6f5-m6lxh"
Jul 31 23:31:14.187239 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:14.187208    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-124aaef2-f5f9f4c46-brjwf"
Jul 31 23:31:14.187680 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:14.187366    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-124aaef2-7b7cf9b756-fhhw7"
Jul 31 23:31:43.408178 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:43.408138    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-124aaef2-79f664d6f5-m6lxh"]
Jul 31 23:31:43.408659 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:43.408396    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-124aaef2-79f664d6f5-m6lxh" podUID="aeb4fea1-f63d-4d0a-98de-ebe3d0103393" containerName="maas-api" containerID="cri-o://8b9b10acfbe8593d7f0af4757e4f6caa0e9e50be97441c3c0659429fcf38260a" gracePeriod=30
Jul 31 23:31:43.446093 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:43.446018    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-124aaef2-f5f9f4c46-brjwf"]
Jul 31 23:31:43.446306 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:43.446266    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-124aaef2-f5f9f4c46-brjwf" podUID="cd66e883-affa-49f3-8b2b-8204aa324987" containerName="payload-processing" containerID="cri-o://30bef96f2e7ab1941064b4cafe12d65dcc33a47edfdd1ad3d55855aa9a4ce7d6" gracePeriod=30
Jul 31 23:31:43.451128 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:43.451103    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-124aaef2-7b7cf9b756-fhhw7"]
Jul 31 23:31:43.451390 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:43.451342    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-124aaef2-7b7cf9b756-fhhw7" podUID="7796dc8a-d32b-4117-8234-81378252a31a" containerName="payload-pre-processing" containerID="cri-o://4dbb77a4d8efc1dff6bee000de8ad7fec6933b8f89d55ac741990852def4c65e" gracePeriod=30
Jul 31 23:31:43.705947 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:43.705912    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-124aaef2-79f664d6f5-m6lxh"
Jul 31 23:31:43.729247 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:43.729218    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-124aaef2-7b7cf9b756-fhhw7"
Jul 31 23:31:43.732363 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:43.732342    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-124aaef2-f5f9f4c46-brjwf"
Jul 31 23:31:43.839394 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:43.839360    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pnvxm\" (UniqueName: \"kubernetes.io/projected/aeb4fea1-f63d-4d0a-98de-ebe3d0103393-kube-api-access-pnvxm\") pod \"aeb4fea1-f63d-4d0a-98de-ebe3d0103393\" (UID: \"aeb4fea1-f63d-4d0a-98de-ebe3d0103393\") "
Jul 31 23:31:43.839569 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:43.839492    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cd66e883-affa-49f3-8b2b-8204aa324987-plugins-config-volume\") pod \"cd66e883-affa-49f3-8b2b-8204aa324987\" (UID: \"cd66e883-affa-49f3-8b2b-8204aa324987\") "
Jul 31 23:31:43.839569 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:43.839520    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7796dc8a-d32b-4117-8234-81378252a31a-plugins-config-volume\") pod \"7796dc8a-d32b-4117-8234-81378252a31a\" (UID: \"7796dc8a-d32b-4117-8234-81378252a31a\") "
Jul 31 23:31:43.839569 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:43.839545    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aeb4fea1-f63d-4d0a-98de-ebe3d0103393-maas-api-tls\") pod \"aeb4fea1-f63d-4d0a-98de-ebe3d0103393\" (UID: \"aeb4fea1-f63d-4d0a-98de-ebe3d0103393\") "
Jul 31 23:31:43.839786 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:43.839571    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aeb4fea1-f63d-4d0a-98de-ebe3d0103393-openshift-service-ca\") pod \"aeb4fea1-f63d-4d0a-98de-ebe3d0103393\" (UID: \"aeb4fea1-f63d-4d0a-98de-ebe3d0103393\") "
Jul 31 23:31:43.839786 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:43.839603    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nhq6j\" (UniqueName: \"kubernetes.io/projected/cd66e883-affa-49f3-8b2b-8204aa324987-kube-api-access-nhq6j\") pod \"cd66e883-affa-49f3-8b2b-8204aa324987\" (UID: \"cd66e883-affa-49f3-8b2b-8204aa324987\") "
Jul 31 23:31:43.839786 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:43.839653    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-79z5x\" (UniqueName: \"kubernetes.io/projected/7796dc8a-d32b-4117-8234-81378252a31a-kube-api-access-79z5x\") pod \"7796dc8a-d32b-4117-8234-81378252a31a\" (UID: \"7796dc8a-d32b-4117-8234-81378252a31a\") "
Jul 31 23:31:43.840002 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:43.839953    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7796dc8a-d32b-4117-8234-81378252a31a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7796dc8a-d32b-4117-8234-81378252a31a" (UID: "7796dc8a-d32b-4117-8234-81378252a31a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:31:43.840002 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:43.839984    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aeb4fea1-f63d-4d0a-98de-ebe3d0103393-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "aeb4fea1-f63d-4d0a-98de-ebe3d0103393" (UID: "aeb4fea1-f63d-4d0a-98de-ebe3d0103393"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:31:43.840197 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:43.839998    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd66e883-affa-49f3-8b2b-8204aa324987-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "cd66e883-affa-49f3-8b2b-8204aa324987" (UID: "cd66e883-affa-49f3-8b2b-8204aa324987"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:31:43.841751 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:43.841719    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aeb4fea1-f63d-4d0a-98de-ebe3d0103393-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "aeb4fea1-f63d-4d0a-98de-ebe3d0103393" (UID: "aeb4fea1-f63d-4d0a-98de-ebe3d0103393"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 23:31:43.841751 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:43.841725    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aeb4fea1-f63d-4d0a-98de-ebe3d0103393-kube-api-access-pnvxm" (OuterVolumeSpecName: "kube-api-access-pnvxm") pod "aeb4fea1-f63d-4d0a-98de-ebe3d0103393" (UID: "aeb4fea1-f63d-4d0a-98de-ebe3d0103393"). InnerVolumeSpecName "kube-api-access-pnvxm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 23:31:43.841927 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:43.841807    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd66e883-affa-49f3-8b2b-8204aa324987-kube-api-access-nhq6j" (OuterVolumeSpecName: "kube-api-access-nhq6j") pod "cd66e883-affa-49f3-8b2b-8204aa324987" (UID: "cd66e883-affa-49f3-8b2b-8204aa324987"). InnerVolumeSpecName "kube-api-access-nhq6j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 23:31:43.841927 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:43.841899    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7796dc8a-d32b-4117-8234-81378252a31a-kube-api-access-79z5x" (OuterVolumeSpecName: "kube-api-access-79z5x") pod "7796dc8a-d32b-4117-8234-81378252a31a" (UID: "7796dc8a-d32b-4117-8234-81378252a31a"). InnerVolumeSpecName "kube-api-access-79z5x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 23:31:43.940721 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:43.940616    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-79z5x\" (UniqueName: \"kubernetes.io/projected/7796dc8a-d32b-4117-8234-81378252a31a-kube-api-access-79z5x\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:31:43.940721 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:43.940643    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pnvxm\" (UniqueName: \"kubernetes.io/projected/aeb4fea1-f63d-4d0a-98de-ebe3d0103393-kube-api-access-pnvxm\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:31:43.940721 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:43.940652    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cd66e883-affa-49f3-8b2b-8204aa324987-plugins-config-volume\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:31:43.940721 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:43.940662    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7796dc8a-d32b-4117-8234-81378252a31a-plugins-config-volume\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:31:43.940721 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:43.940671    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aeb4fea1-f63d-4d0a-98de-ebe3d0103393-maas-api-tls\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:31:43.940721 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:43.940679    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aeb4fea1-f63d-4d0a-98de-ebe3d0103393-openshift-service-ca\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:31:43.940721 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:43.940687    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nhq6j\" (UniqueName: \"kubernetes.io/projected/cd66e883-affa-49f3-8b2b-8204aa324987-kube-api-access-nhq6j\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:31:44.377191 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:44.377156    2584 generic.go:356] "Generic (PLEG): container finished" podID="aeb4fea1-f63d-4d0a-98de-ebe3d0103393" containerID="8b9b10acfbe8593d7f0af4757e4f6caa0e9e50be97441c3c0659429fcf38260a" exitCode=0
Jul 31 23:31:44.377442 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:44.377197    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-124aaef2-79f664d6f5-m6lxh"
Jul 31 23:31:44.377442 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:44.377245    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-124aaef2-79f664d6f5-m6lxh" event={"ID":"aeb4fea1-f63d-4d0a-98de-ebe3d0103393","Type":"ContainerDied","Data":"8b9b10acfbe8593d7f0af4757e4f6caa0e9e50be97441c3c0659429fcf38260a"}
Jul 31 23:31:44.377442 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:44.377300    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-124aaef2-79f664d6f5-m6lxh" event={"ID":"aeb4fea1-f63d-4d0a-98de-ebe3d0103393","Type":"ContainerDied","Data":"0747c4575bb8a3919e6d14c3ff21e90dad20468bbd269f8128199b48ebbe4fc8"}
Jul 31 23:31:44.377442 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:44.377323    2584 scope.go:117] "RemoveContainer" containerID="8b9b10acfbe8593d7f0af4757e4f6caa0e9e50be97441c3c0659429fcf38260a"
Jul 31 23:31:44.378678 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:44.378655    2584 generic.go:356] "Generic (PLEG): container finished" podID="cd66e883-affa-49f3-8b2b-8204aa324987" containerID="30bef96f2e7ab1941064b4cafe12d65dcc33a47edfdd1ad3d55855aa9a4ce7d6" exitCode=0
Jul 31 23:31:44.378811 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:44.378691    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-124aaef2-f5f9f4c46-brjwf"
Jul 31 23:31:44.378811 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:44.378734    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-124aaef2-f5f9f4c46-brjwf" event={"ID":"cd66e883-affa-49f3-8b2b-8204aa324987","Type":"ContainerDied","Data":"30bef96f2e7ab1941064b4cafe12d65dcc33a47edfdd1ad3d55855aa9a4ce7d6"}
Jul 31 23:31:44.378811 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:44.378779    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-124aaef2-f5f9f4c46-brjwf" event={"ID":"cd66e883-affa-49f3-8b2b-8204aa324987","Type":"ContainerDied","Data":"9bcf84df3b9e52dbb6d7adb5dac4eaa64f65272c44eb9c43a97759539674151a"}
Jul 31 23:31:44.380152 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:44.380129    2584 generic.go:356] "Generic (PLEG): container finished" podID="7796dc8a-d32b-4117-8234-81378252a31a" containerID="4dbb77a4d8efc1dff6bee000de8ad7fec6933b8f89d55ac741990852def4c65e" exitCode=0
Jul 31 23:31:44.380280 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:44.380170    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-124aaef2-7b7cf9b756-fhhw7"
Jul 31 23:31:44.380280 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:44.380216    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-124aaef2-7b7cf9b756-fhhw7" event={"ID":"7796dc8a-d32b-4117-8234-81378252a31a","Type":"ContainerDied","Data":"4dbb77a4d8efc1dff6bee000de8ad7fec6933b8f89d55ac741990852def4c65e"}
Jul 31 23:31:44.380280 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:44.380265    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-124aaef2-7b7cf9b756-fhhw7" event={"ID":"7796dc8a-d32b-4117-8234-81378252a31a","Type":"ContainerDied","Data":"2ee0fddb9d01c1be51bec79832d5dfd8a699a32bafa02cd73f4f941490f2468e"}
Jul 31 23:31:44.388040 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:44.387985    2584 scope.go:117] "RemoveContainer" containerID="8b9b10acfbe8593d7f0af4757e4f6caa0e9e50be97441c3c0659429fcf38260a"
Jul 31 23:31:44.388400 ip-10-0-128-243 kubenswrapper[2584]: E0731 23:31:44.388348    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b9b10acfbe8593d7f0af4757e4f6caa0e9e50be97441c3c0659429fcf38260a\": container with ID starting with 8b9b10acfbe8593d7f0af4757e4f6caa0e9e50be97441c3c0659429fcf38260a not found: ID does not exist" containerID="8b9b10acfbe8593d7f0af4757e4f6caa0e9e50be97441c3c0659429fcf38260a"
Jul 31 23:31:44.388502 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:44.388413    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b9b10acfbe8593d7f0af4757e4f6caa0e9e50be97441c3c0659429fcf38260a"} err="failed to get container status \"8b9b10acfbe8593d7f0af4757e4f6caa0e9e50be97441c3c0659429fcf38260a\": rpc error: code = NotFound desc = could not find container \"8b9b10acfbe8593d7f0af4757e4f6caa0e9e50be97441c3c0659429fcf38260a\": container with ID starting with 8b9b10acfbe8593d7f0af4757e4f6caa0e9e50be97441c3c0659429fcf38260a not found: ID does not exist"
Jul 31 23:31:44.388502 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:44.388456    2584 scope.go:117] "RemoveContainer" containerID="30bef96f2e7ab1941064b4cafe12d65dcc33a47edfdd1ad3d55855aa9a4ce7d6"
Jul 31 23:31:44.397535 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:44.397509    2584 scope.go:117] "RemoveContainer" containerID="30bef96f2e7ab1941064b4cafe12d65dcc33a47edfdd1ad3d55855aa9a4ce7d6"
Jul 31 23:31:44.397858 ip-10-0-128-243 kubenswrapper[2584]: E0731 23:31:44.397836    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30bef96f2e7ab1941064b4cafe12d65dcc33a47edfdd1ad3d55855aa9a4ce7d6\": container with ID starting with 30bef96f2e7ab1941064b4cafe12d65dcc33a47edfdd1ad3d55855aa9a4ce7d6 not found: ID does not exist" containerID="30bef96f2e7ab1941064b4cafe12d65dcc33a47edfdd1ad3d55855aa9a4ce7d6"
Jul 31 23:31:44.397950 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:44.397869    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30bef96f2e7ab1941064b4cafe12d65dcc33a47edfdd1ad3d55855aa9a4ce7d6"} err="failed to get container status \"30bef96f2e7ab1941064b4cafe12d65dcc33a47edfdd1ad3d55855aa9a4ce7d6\": rpc error: code = NotFound desc = could not find container \"30bef96f2e7ab1941064b4cafe12d65dcc33a47edfdd1ad3d55855aa9a4ce7d6\": container with ID starting with 30bef96f2e7ab1941064b4cafe12d65dcc33a47edfdd1ad3d55855aa9a4ce7d6 not found: ID does not exist"
Jul 31 23:31:44.397950 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:44.397892    2584 scope.go:117] "RemoveContainer" containerID="4dbb77a4d8efc1dff6bee000de8ad7fec6933b8f89d55ac741990852def4c65e"
Jul 31 23:31:44.402621 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:44.402599    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-124aaef2-79f664d6f5-m6lxh"]
Jul 31 23:31:44.406520 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:44.406495    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-124aaef2-79f664d6f5-m6lxh"]
Jul 31 23:31:44.407079 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:44.407061    2584 scope.go:117] "RemoveContainer" containerID="4dbb77a4d8efc1dff6bee000de8ad7fec6933b8f89d55ac741990852def4c65e"
Jul 31 23:31:44.407407 ip-10-0-128-243 kubenswrapper[2584]: E0731 23:31:44.407388    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4dbb77a4d8efc1dff6bee000de8ad7fec6933b8f89d55ac741990852def4c65e\": container with ID starting with 4dbb77a4d8efc1dff6bee000de8ad7fec6933b8f89d55ac741990852def4c65e not found: ID does not exist" containerID="4dbb77a4d8efc1dff6bee000de8ad7fec6933b8f89d55ac741990852def4c65e"
Jul 31 23:31:44.407480 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:44.407418    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4dbb77a4d8efc1dff6bee000de8ad7fec6933b8f89d55ac741990852def4c65e"} err="failed to get container status \"4dbb77a4d8efc1dff6bee000de8ad7fec6933b8f89d55ac741990852def4c65e\": rpc error: code = NotFound desc = could not find container \"4dbb77a4d8efc1dff6bee000de8ad7fec6933b8f89d55ac741990852def4c65e\": container with ID starting with 4dbb77a4d8efc1dff6bee000de8ad7fec6933b8f89d55ac741990852def4c65e not found: ID does not exist"
Jul 31 23:31:44.416102 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:44.416080    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-124aaef2-7b7cf9b756-fhhw7"]
Jul 31 23:31:44.419515 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:44.419494    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-124aaef2-7b7cf9b756-fhhw7"]
Jul 31 23:31:44.430103 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:44.430050    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-124aaef2-f5f9f4c46-brjwf"]
Jul 31 23:31:44.432208 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:44.432184    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-124aaef2-f5f9f4c46-brjwf"]
Jul 31 23:31:44.890496 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:44.890462    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7796dc8a-d32b-4117-8234-81378252a31a" path="/var/lib/kubelet/pods/7796dc8a-d32b-4117-8234-81378252a31a/volumes"
Jul 31 23:31:44.890864 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:44.890850    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aeb4fea1-f63d-4d0a-98de-ebe3d0103393" path="/var/lib/kubelet/pods/aeb4fea1-f63d-4d0a-98de-ebe3d0103393/volumes"
Jul 31 23:31:44.891161 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:31:44.891149    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd66e883-affa-49f3-8b2b-8204aa324987" path="/var/lib/kubelet/pods/cd66e883-affa-49f3-8b2b-8204aa324987/volumes"
Jul 31 23:34:08.676947 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:34:08.676914    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0a726cba-7f5698d95c-qbd6l"]
Jul 31 23:34:08.677353 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:34:08.677281    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd66e883-affa-49f3-8b2b-8204aa324987" containerName="payload-processing"
Jul 31 23:34:08.677353 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:34:08.677292    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd66e883-affa-49f3-8b2b-8204aa324987" containerName="payload-processing"
Jul 31 23:34:08.677353 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:34:08.677305    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7796dc8a-d32b-4117-8234-81378252a31a" containerName="payload-pre-processing"
Jul 31 23:34:08.677353 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:34:08.677311    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="7796dc8a-d32b-4117-8234-81378252a31a" containerName="payload-pre-processing"
Jul 31 23:34:08.677353 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:34:08.677319    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aeb4fea1-f63d-4d0a-98de-ebe3d0103393" containerName="maas-api"
Jul 31 23:34:08.677353 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:34:08.677325    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="aeb4fea1-f63d-4d0a-98de-ebe3d0103393" containerName="maas-api"
Jul 31 23:34:08.677545 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:34:08.677384    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd66e883-affa-49f3-8b2b-8204aa324987" containerName="payload-processing"
Jul 31 23:34:08.677545 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:34:08.677397    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="aeb4fea1-f63d-4d0a-98de-ebe3d0103393" containerName="maas-api"
Jul 31 23:34:08.677545 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:34:08.677405    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="7796dc8a-d32b-4117-8234-81378252a31a" containerName="payload-pre-processing"
Jul 31 23:34:08.680429 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:34:08.680412    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0a726cba-7f5698d95c-qbd6l"
Jul 31 23:34:08.682885 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:34:08.682861    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-0a726cba-serving-cert\""
Jul 31 23:34:08.689509 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:34:08.689489    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0a726cba-7f5698d95c-qbd6l"]
Jul 31 23:34:08.773959 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:34:08.773916    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7abe4d0-392a-4364-8963-b7f7f57d229b-maas-api-tls\") pod \"maas-api-e2e-mt-0a726cba-7f5698d95c-qbd6l\" (UID: \"a7abe4d0-392a-4364-8963-b7f7f57d229b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0a726cba-7f5698d95c-qbd6l"
Jul 31 23:34:08.773959 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:34:08.773954    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7abe4d0-392a-4364-8963-b7f7f57d229b-openshift-service-ca\") pod \"maas-api-e2e-mt-0a726cba-7f5698d95c-qbd6l\" (UID: \"a7abe4d0-392a-4364-8963-b7f7f57d229b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0a726cba-7f5698d95c-qbd6l"
Jul 31 23:34:08.774187 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:34:08.774039    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lblj2\" (UniqueName: \"kubernetes.io/projected/a7abe4d0-392a-4364-8963-b7f7f57d229b-kube-api-access-lblj2\") pod \"maas-api-e2e-mt-0a726cba-7f5698d95c-qbd6l\" (UID: \"a7abe4d0-392a-4364-8963-b7f7f57d229b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0a726cba-7f5698d95c-qbd6l"
Jul 31 23:34:08.874756 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:34:08.874716    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lblj2\" (UniqueName: \"kubernetes.io/projected/a7abe4d0-392a-4364-8963-b7f7f57d229b-kube-api-access-lblj2\") pod \"maas-api-e2e-mt-0a726cba-7f5698d95c-qbd6l\" (UID: \"a7abe4d0-392a-4364-8963-b7f7f57d229b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0a726cba-7f5698d95c-qbd6l"
Jul 31 23:34:08.874944 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:34:08.874804    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7abe4d0-392a-4364-8963-b7f7f57d229b-maas-api-tls\") pod \"maas-api-e2e-mt-0a726cba-7f5698d95c-qbd6l\" (UID: \"a7abe4d0-392a-4364-8963-b7f7f57d229b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0a726cba-7f5698d95c-qbd6l"
Jul 31 23:34:08.874944 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:34:08.874824    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7abe4d0-392a-4364-8963-b7f7f57d229b-openshift-service-ca\") pod \"maas-api-e2e-mt-0a726cba-7f5698d95c-qbd6l\" (UID: \"a7abe4d0-392a-4364-8963-b7f7f57d229b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0a726cba-7f5698d95c-qbd6l"
Jul 31 23:34:08.875020 ip-10-0-128-243 kubenswrapper[2584]: E0731 23:34:08.874957    2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-0a726cba-serving-cert: secret "maas-api-e2e-mt-0a726cba-serving-cert" not found
Jul 31 23:34:08.875054 ip-10-0-128-243 kubenswrapper[2584]: E0731 23:34:08.875035    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7abe4d0-392a-4364-8963-b7f7f57d229b-maas-api-tls podName:a7abe4d0-392a-4364-8963-b7f7f57d229b nodeName:}" failed. No retries permitted until 2026-07-31 23:34:09.375017255 +0000 UTC m=+2233.020803545 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a7abe4d0-392a-4364-8963-b7f7f57d229b-maas-api-tls") pod "maas-api-e2e-mt-0a726cba-7f5698d95c-qbd6l" (UID: "a7abe4d0-392a-4364-8963-b7f7f57d229b") : secret "maas-api-e2e-mt-0a726cba-serving-cert" not found
Jul 31 23:34:08.875436 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:34:08.875419    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7abe4d0-392a-4364-8963-b7f7f57d229b-openshift-service-ca\") pod \"maas-api-e2e-mt-0a726cba-7f5698d95c-qbd6l\" (UID: \"a7abe4d0-392a-4364-8963-b7f7f57d229b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0a726cba-7f5698d95c-qbd6l"
Jul 31 23:34:08.887576 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:34:08.887550    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lblj2\" (UniqueName: \"kubernetes.io/projected/a7abe4d0-392a-4364-8963-b7f7f57d229b-kube-api-access-lblj2\") pod \"maas-api-e2e-mt-0a726cba-7f5698d95c-qbd6l\" (UID: \"a7abe4d0-392a-4364-8963-b7f7f57d229b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0a726cba-7f5698d95c-qbd6l"
Jul 31 23:34:09.379346 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:34:09.379313    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7abe4d0-392a-4364-8963-b7f7f57d229b-maas-api-tls\") pod \"maas-api-e2e-mt-0a726cba-7f5698d95c-qbd6l\" (UID: \"a7abe4d0-392a-4364-8963-b7f7f57d229b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0a726cba-7f5698d95c-qbd6l"
Jul 31 23:34:09.381788 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:34:09.381766    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7abe4d0-392a-4364-8963-b7f7f57d229b-maas-api-tls\") pod \"maas-api-e2e-mt-0a726cba-7f5698d95c-qbd6l\" (UID: \"a7abe4d0-392a-4364-8963-b7f7f57d229b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0a726cba-7f5698d95c-qbd6l"
Jul 31 23:34:09.416069 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:34:09.416033    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-0a726cba-6b599fbfd-g9rg5"]
Jul 31 23:34:09.419556 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:34:09.419539    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-0a726cba-6b599fbfd-g9rg5"
Jul 31 23:34:09.422020 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:34:09.421999    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-0a726cba\""
Jul 31 23:34:09.422158 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:34:09.422142    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-0a726cba-dockercfg-tbkcd\""
Jul 31 23:34:09.429557 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:34:09.429526    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-0a726cba-6b599fbfd-g9rg5"]
Jul 31 23:34:09.451092 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:34:09.451056    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-0a726cba-6bb554847b-6ms5t"]
Jul 31 23:34:09.454399 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:34:09.454382    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-0a726cba-6bb554847b-6ms5t"
Jul 31 23:34:09.463773 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:34:09.463745    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-0a726cba-6bb554847b-6ms5t"]
Jul 31 23:34:09.480641 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:34:09.480606    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0ab92f3a-b957-4f1d-be02-8fbf44b03291-plugins-config-volume\") pod \"payload-processing-e2e-mt-0a726cba-6b599fbfd-g9rg5\" (UID: \"0ab92f3a-b957-4f1d-be02-8fbf44b03291\") " pod="openshift-ingress/payload-processing-e2e-mt-0a726cba-6b599fbfd-g9rg5"
Jul 31 23:34:09.480984 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:34:09.480952    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7xv5\" (UniqueName: \"kubernetes.io/projected/0ab92f3a-b957-4f1d-be02-8fbf44b03291-kube-api-access-x7xv5\") pod \"payload-processing-e2e-mt-0a726cba-6b599fbfd-g9rg5\" (UID: \"0ab92f3a-b957-4f1d-be02-8fbf44b03291\") " pod="openshift-ingress/payload-processing-e2e-mt-0a726cba-6b599fbfd-g9rg5"
Jul 31 23:34:09.581660 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:34:09.581624    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58246e62-471b-4507-9ac3-69f3a95d578a-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-0a726cba-6bb554847b-6ms5t\" (UID: \"58246e62-471b-4507-9ac3-69f3a95d578a\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-0a726cba-6bb554847b-6ms5t"
Jul 31 23:34:09.581660 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:34:09.581665    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0ab92f3a-b957-4f1d-be02-8fbf44b03291-plugins-config-volume\") pod \"payload-processing-e2e-mt-0a726cba-6b599fbfd-g9rg5\" (UID: \"0ab92f3a-b957-4f1d-be02-8fbf44b03291\") " pod="openshift-ingress/payload-processing-e2e-mt-0a726cba-6b599fbfd-g9rg5"
Jul 31 23:34:09.581922 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:34:09.581793    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7xv5\" (UniqueName: \"kubernetes.io/projected/0ab92f3a-b957-4f1d-be02-8fbf44b03291-kube-api-access-x7xv5\") pod \"payload-processing-e2e-mt-0a726cba-6b599fbfd-g9rg5\" (UID: \"0ab92f3a-b957-4f1d-be02-8fbf44b03291\") " pod="openshift-ingress/payload-processing-e2e-mt-0a726cba-6b599fbfd-g9rg5"
Jul 31 23:34:09.581922 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:34:09.581903    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvlqd\" (UniqueName: \"kubernetes.io/projected/58246e62-471b-4507-9ac3-69f3a95d578a-kube-api-access-vvlqd\") pod \"payload-pre-processing-e2e-mt-0a726cba-6bb554847b-6ms5t\" (UID: \"58246e62-471b-4507-9ac3-69f3a95d578a\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-0a726cba-6bb554847b-6ms5t"
Jul 31 23:34:09.582454 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:34:09.582435    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0ab92f3a-b957-4f1d-be02-8fbf44b03291-plugins-config-volume\") pod \"payload-processing-e2e-mt-0a726cba-6b599fbfd-g9rg5\" (UID: \"0ab92f3a-b957-4f1d-be02-8fbf44b03291\") " pod="openshift-ingress/payload-processing-e2e-mt-0a726cba-6b599fbfd-g9rg5"
Jul 31 23:34:09.589382 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:34:09.589363    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7xv5\" (UniqueName: \"kubernetes.io/projected/0ab92f3a-b957-4f1d-be02-8fbf44b03291-kube-api-access-x7xv5\") pod \"payload-processing-e2e-mt-0a726cba-6b599fbfd-g9rg5\" (UID: \"0ab92f3a-b957-4f1d-be02-8fbf44b03291\") " pod="openshift-ingress/payload-processing-e2e-mt-0a726cba-6b599fbfd-g9rg5"
Jul 31 23:34:09.592206 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:34:09.592184    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0a726cba-7f5698d95c-qbd6l"
Jul 31 23:34:09.683100 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:34:09.682657    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvlqd\" (UniqueName: \"kubernetes.io/projected/58246e62-471b-4507-9ac3-69f3a95d578a-kube-api-access-vvlqd\") pod \"payload-pre-processing-e2e-mt-0a726cba-6bb554847b-6ms5t\" (UID: \"58246e62-471b-4507-9ac3-69f3a95d578a\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-0a726cba-6bb554847b-6ms5t"
Jul 31 23:34:09.683100 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:34:09.682725    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58246e62-471b-4507-9ac3-69f3a95d578a-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-0a726cba-6bb554847b-6ms5t\" (UID: \"58246e62-471b-4507-9ac3-69f3a95d578a\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-0a726cba-6bb554847b-6ms5t"
Jul 31 23:34:09.683680 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:34:09.683655    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58246e62-471b-4507-9ac3-69f3a95d578a-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-0a726cba-6bb554847b-6ms5t\" (UID: \"58246e62-471b-4507-9ac3-69f3a95d578a\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-0a726cba-6bb554847b-6ms5t"
Jul 31 23:34:09.691029 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:34:09.690989    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvlqd\" (UniqueName: \"kubernetes.io/projected/58246e62-471b-4507-9ac3-69f3a95d578a-kube-api-access-vvlqd\") pod \"payload-pre-processing-e2e-mt-0a726cba-6bb554847b-6ms5t\" (UID: \"58246e62-471b-4507-9ac3-69f3a95d578a\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-0a726cba-6bb554847b-6ms5t"
Jul 31 23:34:09.731543 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:34:09.731510    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-0a726cba-6b599fbfd-g9rg5"
Jul 31 23:34:09.735380 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:34:09.735322    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0a726cba-7f5698d95c-qbd6l"]
Jul 31 23:34:09.740948 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:34:09.740928    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 31 23:34:09.766033 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:34:09.766002    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-0a726cba-6bb554847b-6ms5t"
Jul 31 23:34:09.888581 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:34:09.887470    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-0a726cba-6b599fbfd-g9rg5"]
Jul 31 23:34:09.890233 ip-10-0-128-243 kubenswrapper[2584]: W0731 23:34:09.890199    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ab92f3a_b957_4f1d_be02_8fbf44b03291.slice/crio-8086f7a484bd4e0653470871fefc88a58047cfc9d7bad27c999c055430d3cfc6 WatchSource:0}: Error finding container 8086f7a484bd4e0653470871fefc88a58047cfc9d7bad27c999c055430d3cfc6: Status 404 returned error can't find the container with id 8086f7a484bd4e0653470871fefc88a58047cfc9d7bad27c999c055430d3cfc6
Jul 31 23:34:09.914689 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:34:09.914666    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-0a726cba-6bb554847b-6ms5t"]
Jul 31 23:34:09.916413 ip-10-0-128-243 kubenswrapper[2584]: W0731 23:34:09.916385    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58246e62_471b_4507_9ac3_69f3a95d578a.slice/crio-5562dbb74b83f4d7704968f0b842dd123c365527ea8e21af4449dfc97bb76072 WatchSource:0}: Error finding container 5562dbb74b83f4d7704968f0b842dd123c365527ea8e21af4449dfc97bb76072: Status 404 returned error can't find the container with id 5562dbb74b83f4d7704968f0b842dd123c365527ea8e21af4449dfc97bb76072
Jul 31 23:34:10.035786 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:34:10.035740    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0a726cba-7f5698d95c-qbd6l" event={"ID":"a7abe4d0-392a-4364-8963-b7f7f57d229b","Type":"ContainerStarted","Data":"837d859c398a70bdd52f6f57f27f9adbaf8c19b597ea14df6d93e8f0cfbbf70d"}
Jul 31 23:34:10.037607 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:34:10.037576    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-0a726cba-6bb554847b-6ms5t" event={"ID":"58246e62-471b-4507-9ac3-69f3a95d578a","Type":"ContainerStarted","Data":"7b66e423f6d3c600afc18da053f784c5f3e744006369c31ebd70cd532b1f2eff"}
Jul 31 23:34:10.037787 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:34:10.037684    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-0a726cba-6bb554847b-6ms5t" event={"ID":"58246e62-471b-4507-9ac3-69f3a95d578a","Type":"ContainerStarted","Data":"5562dbb74b83f4d7704968f0b842dd123c365527ea8e21af4449dfc97bb76072"}
Jul 31 23:34:10.037787 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:34:10.037750    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-0a726cba-6bb554847b-6ms5t"
Jul 31 23:34:10.039408 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:34:10.039376    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-0a726cba-6b599fbfd-g9rg5" event={"ID":"0ab92f3a-b957-4f1d-be02-8fbf44b03291","Type":"ContainerStarted","Data":"5c581a7444f5b77110da634529f618cdc7f090ba9dbd0ffa77aeadf5d28f3256"}
Jul 31 23:34:10.039514 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:34:10.039430    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-0a726cba-6b599fbfd-g9rg5" event={"ID":"0ab92f3a-b957-4f1d-be02-8fbf44b03291","Type":"ContainerStarted","Data":"8086f7a484bd4e0653470871fefc88a58047cfc9d7bad27c999c055430d3cfc6"}
Jul 31 23:34:10.039560 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:34:10.039511    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-0a726cba-6b599fbfd-g9rg5"
Jul 31 23:34:10.053072 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:34:10.053022    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-0a726cba-6bb554847b-6ms5t" podStartSLOduration=1.053005596 podStartE2EDuration="1.053005596s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:34:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:34:10.052325246 +0000 UTC m=+2233.698111554" watchObservedRunningTime="2026-07-31 23:34:10.053005596 +0000 UTC m=+2233.698791904"
Jul 31 23:34:10.068652 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:34:10.068597    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-0a726cba-6b599fbfd-g9rg5" podStartSLOduration=1.06858311 podStartE2EDuration="1.06858311s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:34:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:34:10.067244977 +0000 UTC m=+2233.713031296" watchObservedRunningTime="2026-07-31 23:34:10.06858311 +0000 UTC m=+2233.714369397"
Jul 31 23:34:11.046215 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:34:11.046175    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0a726cba-7f5698d95c-qbd6l" event={"ID":"a7abe4d0-392a-4364-8963-b7f7f57d229b","Type":"ContainerStarted","Data":"a7c5796bd74762211108d2dbb8e73a235c61cbd37386f35bd01239d5f1188318"}
Jul 31 23:34:11.061991 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:34:11.061945    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0a726cba-7f5698d95c-qbd6l" podStartSLOduration=2.594601457 podStartE2EDuration="3.061914918s" totalImagesPullingTime="467.313461ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:34:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:34:11.060587145 +0000 UTC m=+2234.706373459" watchObservedRunningTime="2026-07-31 23:34:11.061914918 +0000 UTC m=+2234.707701226"
Jul 31 23:34:12.052571 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:34:12.052538    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0a726cba-7f5698d95c-qbd6l"
Jul 31 23:34:18.062860 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:34:18.062830    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0a726cba-7f5698d95c-qbd6l"
Jul 31 23:34:21.048534 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:34:21.048460    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-0a726cba-6b599fbfd-g9rg5"
Jul 31 23:34:21.048924 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:34:21.048540    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-0a726cba-6bb554847b-6ms5t"
Jul 31 23:35:32.367274 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:35:32.367235    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0a726cba-7f5698d95c-qbd6l"]
Jul 31 23:35:32.367860 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:35:32.367480    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0a726cba-7f5698d95c-qbd6l" podUID="a7abe4d0-392a-4364-8963-b7f7f57d229b" containerName="maas-api" containerID="cri-o://a7c5796bd74762211108d2dbb8e73a235c61cbd37386f35bd01239d5f1188318" gracePeriod=30
Jul 31 23:35:32.415139 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:35:32.415110    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-0a726cba-6b599fbfd-g9rg5"]
Jul 31 23:35:32.415354 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:35:32.415330    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-0a726cba-6b599fbfd-g9rg5" podUID="0ab92f3a-b957-4f1d-be02-8fbf44b03291" containerName="payload-processing" containerID="cri-o://5c581a7444f5b77110da634529f618cdc7f090ba9dbd0ffa77aeadf5d28f3256" gracePeriod=30
Jul 31 23:35:32.421670 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:35:32.421644    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-0a726cba-6bb554847b-6ms5t"]
Jul 31 23:35:32.421914 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:35:32.421886    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-0a726cba-6bb554847b-6ms5t" podUID="58246e62-471b-4507-9ac3-69f3a95d578a" containerName="payload-pre-processing" containerID="cri-o://7b66e423f6d3c600afc18da053f784c5f3e744006369c31ebd70cd532b1f2eff" gracePeriod=30
Jul 31 23:35:32.691291 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:35:32.691265    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-0a726cba-6b599fbfd-g9rg5"
Jul 31 23:35:32.694249 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:35:32.694223    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0a726cba-7f5698d95c-qbd6l"
Jul 31 23:35:32.719621 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:35:32.719596    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-0a726cba-6bb554847b-6ms5t"
Jul 31 23:35:32.735026 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:35:32.734944    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7abe4d0-392a-4364-8963-b7f7f57d229b-openshift-service-ca\") pod \"a7abe4d0-392a-4364-8963-b7f7f57d229b\" (UID: \"a7abe4d0-392a-4364-8963-b7f7f57d229b\") "
Jul 31 23:35:32.735026 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:35:32.734992    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lblj2\" (UniqueName: \"kubernetes.io/projected/a7abe4d0-392a-4364-8963-b7f7f57d229b-kube-api-access-lblj2\") pod \"a7abe4d0-392a-4364-8963-b7f7f57d229b\" (UID: \"a7abe4d0-392a-4364-8963-b7f7f57d229b\") "
Jul 31 23:35:32.735026 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:35:32.735025    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7abe4d0-392a-4364-8963-b7f7f57d229b-maas-api-tls\") pod \"a7abe4d0-392a-4364-8963-b7f7f57d229b\" (UID: \"a7abe4d0-392a-4364-8963-b7f7f57d229b\") "
Jul 31 23:35:32.735742 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:35:32.735052    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0ab92f3a-b957-4f1d-be02-8fbf44b03291-plugins-config-volume\") pod \"0ab92f3a-b957-4f1d-be02-8fbf44b03291\" (UID: \"0ab92f3a-b957-4f1d-be02-8fbf44b03291\") "
Jul 31 23:35:32.735742 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:35:32.735097    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x7xv5\" (UniqueName: \"kubernetes.io/projected/0ab92f3a-b957-4f1d-be02-8fbf44b03291-kube-api-access-x7xv5\") pod \"0ab92f3a-b957-4f1d-be02-8fbf44b03291\" (UID: \"0ab92f3a-b957-4f1d-be02-8fbf44b03291\") "
Jul 31 23:35:32.735742 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:35:32.735402    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7abe4d0-392a-4364-8963-b7f7f57d229b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a7abe4d0-392a-4364-8963-b7f7f57d229b" (UID: "a7abe4d0-392a-4364-8963-b7f7f57d229b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:35:32.735742 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:35:32.735476    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7abe4d0-392a-4364-8963-b7f7f57d229b-openshift-service-ca\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:35:32.736951 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:35:32.736906    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ab92f3a-b957-4f1d-be02-8fbf44b03291-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0ab92f3a-b957-4f1d-be02-8fbf44b03291" (UID: "0ab92f3a-b957-4f1d-be02-8fbf44b03291"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:35:32.738258 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:35:32.738217    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ab92f3a-b957-4f1d-be02-8fbf44b03291-kube-api-access-x7xv5" (OuterVolumeSpecName: "kube-api-access-x7xv5") pod "0ab92f3a-b957-4f1d-be02-8fbf44b03291" (UID: "0ab92f3a-b957-4f1d-be02-8fbf44b03291"). InnerVolumeSpecName "kube-api-access-x7xv5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 23:35:32.738374 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:35:32.738331    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7abe4d0-392a-4364-8963-b7f7f57d229b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a7abe4d0-392a-4364-8963-b7f7f57d229b" (UID: "a7abe4d0-392a-4364-8963-b7f7f57d229b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 23:35:32.739038 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:35:32.739016    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7abe4d0-392a-4364-8963-b7f7f57d229b-kube-api-access-lblj2" (OuterVolumeSpecName: "kube-api-access-lblj2") pod "a7abe4d0-392a-4364-8963-b7f7f57d229b" (UID: "a7abe4d0-392a-4364-8963-b7f7f57d229b"). InnerVolumeSpecName "kube-api-access-lblj2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 23:35:32.836346 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:35:32.836310    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vvlqd\" (UniqueName: \"kubernetes.io/projected/58246e62-471b-4507-9ac3-69f3a95d578a-kube-api-access-vvlqd\") pod \"58246e62-471b-4507-9ac3-69f3a95d578a\" (UID: \"58246e62-471b-4507-9ac3-69f3a95d578a\") "
Jul 31 23:35:32.836346 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:35:32.836349    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58246e62-471b-4507-9ac3-69f3a95d578a-plugins-config-volume\") pod \"58246e62-471b-4507-9ac3-69f3a95d578a\" (UID: \"58246e62-471b-4507-9ac3-69f3a95d578a\") "
Jul 31 23:35:32.836545 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:35:32.836531    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lblj2\" (UniqueName: \"kubernetes.io/projected/a7abe4d0-392a-4364-8963-b7f7f57d229b-kube-api-access-lblj2\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:35:32.836545 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:35:32.836541    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7abe4d0-392a-4364-8963-b7f7f57d229b-maas-api-tls\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:35:32.836617 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:35:32.836550    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0ab92f3a-b957-4f1d-be02-8fbf44b03291-plugins-config-volume\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:35:32.836617 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:35:32.836561    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x7xv5\" (UniqueName: \"kubernetes.io/projected/0ab92f3a-b957-4f1d-be02-8fbf44b03291-kube-api-access-x7xv5\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:35:32.836898 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:35:32.836861    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58246e62-471b-4507-9ac3-69f3a95d578a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "58246e62-471b-4507-9ac3-69f3a95d578a" (UID: "58246e62-471b-4507-9ac3-69f3a95d578a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:35:32.838384 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:35:32.838364    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58246e62-471b-4507-9ac3-69f3a95d578a-kube-api-access-vvlqd" (OuterVolumeSpecName: "kube-api-access-vvlqd") pod "58246e62-471b-4507-9ac3-69f3a95d578a" (UID: "58246e62-471b-4507-9ac3-69f3a95d578a"). InnerVolumeSpecName "kube-api-access-vvlqd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 23:35:32.937749 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:35:32.937501    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vvlqd\" (UniqueName: \"kubernetes.io/projected/58246e62-471b-4507-9ac3-69f3a95d578a-kube-api-access-vvlqd\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:35:32.937749 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:35:32.937533    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58246e62-471b-4507-9ac3-69f3a95d578a-plugins-config-volume\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:35:33.445527 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:35:33.445493    2584 generic.go:356] "Generic (PLEG): container finished" podID="0ab92f3a-b957-4f1d-be02-8fbf44b03291" containerID="5c581a7444f5b77110da634529f618cdc7f090ba9dbd0ffa77aeadf5d28f3256" exitCode=0
Jul 31 23:35:33.445980 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:35:33.445552    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-0a726cba-6b599fbfd-g9rg5"
Jul 31 23:35:33.445980 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:35:33.445583    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-0a726cba-6b599fbfd-g9rg5" event={"ID":"0ab92f3a-b957-4f1d-be02-8fbf44b03291","Type":"ContainerDied","Data":"5c581a7444f5b77110da634529f618cdc7f090ba9dbd0ffa77aeadf5d28f3256"}
Jul 31 23:35:33.445980 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:35:33.445636    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-0a726cba-6b599fbfd-g9rg5" event={"ID":"0ab92f3a-b957-4f1d-be02-8fbf44b03291","Type":"ContainerDied","Data":"8086f7a484bd4e0653470871fefc88a58047cfc9d7bad27c999c055430d3cfc6"}
Jul 31 23:35:33.445980 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:35:33.445667    2584 scope.go:117] "RemoveContainer" containerID="5c581a7444f5b77110da634529f618cdc7f090ba9dbd0ffa77aeadf5d28f3256"
Jul 31 23:35:33.447257 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:35:33.447232    2584 generic.go:356] "Generic (PLEG): container finished" podID="a7abe4d0-392a-4364-8963-b7f7f57d229b" containerID="a7c5796bd74762211108d2dbb8e73a235c61cbd37386f35bd01239d5f1188318" exitCode=0
Jul 31 23:35:33.447362 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:35:33.447285    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0a726cba-7f5698d95c-qbd6l"
Jul 31 23:35:33.447362 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:35:33.447316    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0a726cba-7f5698d95c-qbd6l" event={"ID":"a7abe4d0-392a-4364-8963-b7f7f57d229b","Type":"ContainerDied","Data":"a7c5796bd74762211108d2dbb8e73a235c61cbd37386f35bd01239d5f1188318"}
Jul 31 23:35:33.447362 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:35:33.447357    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0a726cba-7f5698d95c-qbd6l" event={"ID":"a7abe4d0-392a-4364-8963-b7f7f57d229b","Type":"ContainerDied","Data":"837d859c398a70bdd52f6f57f27f9adbaf8c19b597ea14df6d93e8f0cfbbf70d"}
Jul 31 23:35:33.448896 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:35:33.448867    2584 generic.go:356] "Generic (PLEG): container finished" podID="58246e62-471b-4507-9ac3-69f3a95d578a" containerID="7b66e423f6d3c600afc18da053f784c5f3e744006369c31ebd70cd532b1f2eff" exitCode=0
Jul 31 23:35:33.449021 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:35:33.448907    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-0a726cba-6bb554847b-6ms5t"
Jul 31 23:35:33.449021 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:35:33.448928    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-0a726cba-6bb554847b-6ms5t" event={"ID":"58246e62-471b-4507-9ac3-69f3a95d578a","Type":"ContainerDied","Data":"7b66e423f6d3c600afc18da053f784c5f3e744006369c31ebd70cd532b1f2eff"}
Jul 31 23:35:33.449021 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:35:33.448965    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-0a726cba-6bb554847b-6ms5t" event={"ID":"58246e62-471b-4507-9ac3-69f3a95d578a","Type":"ContainerDied","Data":"5562dbb74b83f4d7704968f0b842dd123c365527ea8e21af4449dfc97bb76072"}
Jul 31 23:35:33.455595 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:35:33.455578    2584 scope.go:117] "RemoveContainer" containerID="5c581a7444f5b77110da634529f618cdc7f090ba9dbd0ffa77aeadf5d28f3256"
Jul 31 23:35:33.455871 ip-10-0-128-243 kubenswrapper[2584]: E0731 23:35:33.455854    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c581a7444f5b77110da634529f618cdc7f090ba9dbd0ffa77aeadf5d28f3256\": container with ID starting with 5c581a7444f5b77110da634529f618cdc7f090ba9dbd0ffa77aeadf5d28f3256 not found: ID does not exist" containerID="5c581a7444f5b77110da634529f618cdc7f090ba9dbd0ffa77aeadf5d28f3256"
Jul 31 23:35:33.455924 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:35:33.455879    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c581a7444f5b77110da634529f618cdc7f090ba9dbd0ffa77aeadf5d28f3256"} err="failed to get container status \"5c581a7444f5b77110da634529f618cdc7f090ba9dbd0ffa77aeadf5d28f3256\": rpc error: code = NotFound desc = could not find container \"5c581a7444f5b77110da634529f618cdc7f090ba9dbd0ffa77aeadf5d28f3256\": container with ID starting with 5c581a7444f5b77110da634529f618cdc7f090ba9dbd0ffa77aeadf5d28f3256 not found: ID does not exist"
Jul 31 23:35:33.455924 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:35:33.455897    2584 scope.go:117] "RemoveContainer" containerID="a7c5796bd74762211108d2dbb8e73a235c61cbd37386f35bd01239d5f1188318"
Jul 31 23:35:33.465376 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:35:33.465345    2584 scope.go:117] "RemoveContainer" containerID="a7c5796bd74762211108d2dbb8e73a235c61cbd37386f35bd01239d5f1188318"
Jul 31 23:35:33.465794 ip-10-0-128-243 kubenswrapper[2584]: E0731 23:35:33.465776    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7c5796bd74762211108d2dbb8e73a235c61cbd37386f35bd01239d5f1188318\": container with ID starting with a7c5796bd74762211108d2dbb8e73a235c61cbd37386f35bd01239d5f1188318 not found: ID does not exist" containerID="a7c5796bd74762211108d2dbb8e73a235c61cbd37386f35bd01239d5f1188318"
Jul 31 23:35:33.465903 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:35:33.465803    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7c5796bd74762211108d2dbb8e73a235c61cbd37386f35bd01239d5f1188318"} err="failed to get container status \"a7c5796bd74762211108d2dbb8e73a235c61cbd37386f35bd01239d5f1188318\": rpc error: code = NotFound desc = could not find container \"a7c5796bd74762211108d2dbb8e73a235c61cbd37386f35bd01239d5f1188318\": container with ID starting with a7c5796bd74762211108d2dbb8e73a235c61cbd37386f35bd01239d5f1188318 not found: ID does not exist"
Jul 31 23:35:33.465903 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:35:33.465825    2584 scope.go:117] "RemoveContainer" containerID="7b66e423f6d3c600afc18da053f784c5f3e744006369c31ebd70cd532b1f2eff"
Jul 31 23:35:33.469468 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:35:33.469446    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-0a726cba-6bb554847b-6ms5t"]
Jul 31 23:35:33.473252 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:35:33.473210    2584 scope.go:117] "RemoveContainer" containerID="7b66e423f6d3c600afc18da053f784c5f3e744006369c31ebd70cd532b1f2eff"
Jul 31 23:35:33.473512 ip-10-0-128-243 kubenswrapper[2584]: E0731 23:35:33.473493    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b66e423f6d3c600afc18da053f784c5f3e744006369c31ebd70cd532b1f2eff\": container with ID starting with 7b66e423f6d3c600afc18da053f784c5f3e744006369c31ebd70cd532b1f2eff not found: ID does not exist" containerID="7b66e423f6d3c600afc18da053f784c5f3e744006369c31ebd70cd532b1f2eff"
Jul 31 23:35:33.473589 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:35:33.473519    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b66e423f6d3c600afc18da053f784c5f3e744006369c31ebd70cd532b1f2eff"} err="failed to get container status \"7b66e423f6d3c600afc18da053f784c5f3e744006369c31ebd70cd532b1f2eff\": rpc error: code = NotFound desc = could not find container \"7b66e423f6d3c600afc18da053f784c5f3e744006369c31ebd70cd532b1f2eff\": container with ID starting with 7b66e423f6d3c600afc18da053f784c5f3e744006369c31ebd70cd532b1f2eff not found: ID does not exist"
Jul 31 23:35:33.475089 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:35:33.475070    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-0a726cba-6bb554847b-6ms5t"]
Jul 31 23:35:33.484557 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:35:33.484538    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-0a726cba-6b599fbfd-g9rg5"]
Jul 31 23:35:33.487891 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:35:33.487872    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-0a726cba-6b599fbfd-g9rg5"]
Jul 31 23:35:33.497838 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:35:33.497816    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0a726cba-7f5698d95c-qbd6l"]
Jul 31 23:35:33.501166 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:35:33.501148    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0a726cba-7f5698d95c-qbd6l"]
Jul 31 23:35:34.891534 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:35:34.891498    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ab92f3a-b957-4f1d-be02-8fbf44b03291" path="/var/lib/kubelet/pods/0ab92f3a-b957-4f1d-be02-8fbf44b03291/volumes"
Jul 31 23:35:34.891902 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:35:34.891869    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58246e62-471b-4507-9ac3-69f3a95d578a" path="/var/lib/kubelet/pods/58246e62-471b-4507-9ac3-69f3a95d578a/volumes"
Jul 31 23:35:34.892215 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:35:34.892201    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7abe4d0-392a-4364-8963-b7f7f57d229b" path="/var/lib/kubelet/pods/a7abe4d0-392a-4364-8963-b7f7f57d229b/volumes"
Jul 31 23:37:32.684979 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:37:32.684893    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-02002b-79f6759b5c-vdzrc"]
Jul 31 23:37:32.685394 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:37:32.685304    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7abe4d0-392a-4364-8963-b7f7f57d229b" containerName="maas-api"
Jul 31 23:37:32.685394 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:37:32.685318    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7abe4d0-392a-4364-8963-b7f7f57d229b" containerName="maas-api"
Jul 31 23:37:32.685394 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:37:32.685335    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58246e62-471b-4507-9ac3-69f3a95d578a" containerName="payload-pre-processing"
Jul 31 23:37:32.685394 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:37:32.685340    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="58246e62-471b-4507-9ac3-69f3a95d578a" containerName="payload-pre-processing"
Jul 31 23:37:32.685394 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:37:32.685354    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ab92f3a-b957-4f1d-be02-8fbf44b03291" containerName="payload-processing"
Jul 31 23:37:32.685394 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:37:32.685360    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ab92f3a-b957-4f1d-be02-8fbf44b03291" containerName="payload-processing"
Jul 31 23:37:32.685632 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:37:32.685416    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="58246e62-471b-4507-9ac3-69f3a95d578a" containerName="payload-pre-processing"
Jul 31 23:37:32.685632 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:37:32.685426    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7abe4d0-392a-4364-8963-b7f7f57d229b" containerName="maas-api"
Jul 31 23:37:32.685632 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:37:32.685440    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ab92f3a-b957-4f1d-be02-8fbf44b03291" containerName="payload-processing"
Jul 31 23:37:32.688447 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:37:32.688430    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-02002b-79f6759b5c-vdzrc"
Jul 31 23:37:32.690966 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:37:32.690945    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-02002b-serving-cert\""
Jul 31 23:37:32.699644 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:37:32.699614    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-02002b-79f6759b5c-vdzrc"]
Jul 31 23:37:32.816322 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:37:32.816279    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e95c8bf2-987c-4924-99df-4fda74671c2d-openshift-service-ca\") pod \"maas-api-e2e-inf-a-02002b-79f6759b5c-vdzrc\" (UID: \"e95c8bf2-987c-4924-99df-4fda74671c2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-02002b-79f6759b5c-vdzrc"
Jul 31 23:37:32.816515 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:37:32.816371    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e95c8bf2-987c-4924-99df-4fda74671c2d-maas-api-tls\") pod \"maas-api-e2e-inf-a-02002b-79f6759b5c-vdzrc\" (UID: \"e95c8bf2-987c-4924-99df-4fda74671c2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-02002b-79f6759b5c-vdzrc"
Jul 31 23:37:32.816515 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:37:32.816394    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvxmf\" (UniqueName: \"kubernetes.io/projected/e95c8bf2-987c-4924-99df-4fda74671c2d-kube-api-access-jvxmf\") pod \"maas-api-e2e-inf-a-02002b-79f6759b5c-vdzrc\" (UID: \"e95c8bf2-987c-4924-99df-4fda74671c2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-02002b-79f6759b5c-vdzrc"
Jul 31 23:37:32.917816 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:37:32.917782    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e95c8bf2-987c-4924-99df-4fda74671c2d-maas-api-tls\") pod \"maas-api-e2e-inf-a-02002b-79f6759b5c-vdzrc\" (UID: \"e95c8bf2-987c-4924-99df-4fda74671c2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-02002b-79f6759b5c-vdzrc"
Jul 31 23:37:32.917816 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:37:32.917822    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvxmf\" (UniqueName: \"kubernetes.io/projected/e95c8bf2-987c-4924-99df-4fda74671c2d-kube-api-access-jvxmf\") pod \"maas-api-e2e-inf-a-02002b-79f6759b5c-vdzrc\" (UID: \"e95c8bf2-987c-4924-99df-4fda74671c2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-02002b-79f6759b5c-vdzrc"
Jul 31 23:37:32.918044 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:37:32.917871    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e95c8bf2-987c-4924-99df-4fda74671c2d-openshift-service-ca\") pod \"maas-api-e2e-inf-a-02002b-79f6759b5c-vdzrc\" (UID: \"e95c8bf2-987c-4924-99df-4fda74671c2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-02002b-79f6759b5c-vdzrc"
Jul 31 23:37:32.918044 ip-10-0-128-243 kubenswrapper[2584]: E0731 23:37:32.917960    2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-02002b-serving-cert: secret "maas-api-e2e-inf-a-02002b-serving-cert" not found
Jul 31 23:37:32.918044 ip-10-0-128-243 kubenswrapper[2584]: E0731 23:37:32.918039    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e95c8bf2-987c-4924-99df-4fda74671c2d-maas-api-tls podName:e95c8bf2-987c-4924-99df-4fda74671c2d nodeName:}" failed. No retries permitted until 2026-07-31 23:37:33.41801646 +0000 UTC m=+2437.063802747 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e95c8bf2-987c-4924-99df-4fda74671c2d-maas-api-tls") pod "maas-api-e2e-inf-a-02002b-79f6759b5c-vdzrc" (UID: "e95c8bf2-987c-4924-99df-4fda74671c2d") : secret "maas-api-e2e-inf-a-02002b-serving-cert" not found
Jul 31 23:37:32.918583 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:37:32.918565    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e95c8bf2-987c-4924-99df-4fda74671c2d-openshift-service-ca\") pod \"maas-api-e2e-inf-a-02002b-79f6759b5c-vdzrc\" (UID: \"e95c8bf2-987c-4924-99df-4fda74671c2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-02002b-79f6759b5c-vdzrc"
Jul 31 23:37:32.929267 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:37:32.929231    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvxmf\" (UniqueName: \"kubernetes.io/projected/e95c8bf2-987c-4924-99df-4fda74671c2d-kube-api-access-jvxmf\") pod \"maas-api-e2e-inf-a-02002b-79f6759b5c-vdzrc\" (UID: \"e95c8bf2-987c-4924-99df-4fda74671c2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-02002b-79f6759b5c-vdzrc"
Jul 31 23:37:33.274892 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:37:33.274858    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-02002b-77697f6d68-zbbvr"]
Jul 31 23:37:33.278237 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:37:33.278216    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-02002b-77697f6d68-zbbvr"
Jul 31 23:37:33.284279 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:37:33.284251    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-02002b-dockercfg-x2nqw\""
Jul 31 23:37:33.284279 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:37:33.284275    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-02002b\""
Jul 31 23:37:33.293092 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:37:33.293068    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-02002b-77697f6d68-zbbvr"]
Jul 31 23:37:33.310280 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:37:33.310248    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-02002b-5667b6bfc8-pksxt"]
Jul 31 23:37:33.313715 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:37:33.313680    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-02002b-5667b6bfc8-pksxt"
Jul 31 23:37:33.325458 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:37:33.325431    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-02002b-5667b6bfc8-pksxt"]
Jul 31 23:37:33.422709 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:37:33.422676    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e95c8bf2-987c-4924-99df-4fda74671c2d-maas-api-tls\") pod \"maas-api-e2e-inf-a-02002b-79f6759b5c-vdzrc\" (UID: \"e95c8bf2-987c-4924-99df-4fda74671c2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-02002b-79f6759b5c-vdzrc"
Jul 31 23:37:33.422903 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:37:33.422781    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c841a60-7bfa-4149-aa1b-b7546e2cccd1-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-02002b-5667b6bfc8-pksxt\" (UID: \"0c841a60-7bfa-4149-aa1b-b7546e2cccd1\") " pod="openshift-ingress/payload-processing-e2e-inf-a-02002b-5667b6bfc8-pksxt"
Jul 31 23:37:33.422903 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:37:33.422881    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvfk9\" (UniqueName: \"kubernetes.io/projected/f4f19fc4-3f03-41e2-8467-4cb8fb3c4a50-kube-api-access-lvfk9\") pod \"payload-pre-processing-e2e-inf-a-02002b-77697f6d68-zbbvr\" (UID: \"f4f19fc4-3f03-41e2-8467-4cb8fb3c4a50\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-02002b-77697f6d68-zbbvr"
Jul 31 23:37:33.423004 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:37:33.422918    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4f19fc4-3f03-41e2-8467-4cb8fb3c4a50-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-02002b-77697f6d68-zbbvr\" (UID: \"f4f19fc4-3f03-41e2-8467-4cb8fb3c4a50\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-02002b-77697f6d68-zbbvr"
Jul 31 23:37:33.423004 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:37:33.422938    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xn87x\" (UniqueName: \"kubernetes.io/projected/0c841a60-7bfa-4149-aa1b-b7546e2cccd1-kube-api-access-xn87x\") pod \"payload-processing-e2e-inf-a-02002b-5667b6bfc8-pksxt\" (UID: \"0c841a60-7bfa-4149-aa1b-b7546e2cccd1\") " pod="openshift-ingress/payload-processing-e2e-inf-a-02002b-5667b6bfc8-pksxt"
Jul 31 23:37:33.425377 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:37:33.425346    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e95c8bf2-987c-4924-99df-4fda74671c2d-maas-api-tls\") pod \"maas-api-e2e-inf-a-02002b-79f6759b5c-vdzrc\" (UID: \"e95c8bf2-987c-4924-99df-4fda74671c2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-02002b-79f6759b5c-vdzrc"
Jul 31 23:37:33.523907 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:37:33.523876    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c841a60-7bfa-4149-aa1b-b7546e2cccd1-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-02002b-5667b6bfc8-pksxt\" (UID: \"0c841a60-7bfa-4149-aa1b-b7546e2cccd1\") " pod="openshift-ingress/payload-processing-e2e-inf-a-02002b-5667b6bfc8-pksxt"
Jul 31 23:37:33.524118 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:37:33.523932    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvfk9\" (UniqueName: \"kubernetes.io/projected/f4f19fc4-3f03-41e2-8467-4cb8fb3c4a50-kube-api-access-lvfk9\") pod \"payload-pre-processing-e2e-inf-a-02002b-77697f6d68-zbbvr\" (UID: \"f4f19fc4-3f03-41e2-8467-4cb8fb3c4a50\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-02002b-77697f6d68-zbbvr"
Jul 31 23:37:33.524118 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:37:33.523966    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4f19fc4-3f03-41e2-8467-4cb8fb3c4a50-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-02002b-77697f6d68-zbbvr\" (UID: \"f4f19fc4-3f03-41e2-8467-4cb8fb3c4a50\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-02002b-77697f6d68-zbbvr"
Jul 31 23:37:33.524118 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:37:33.524016    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xn87x\" (UniqueName: \"kubernetes.io/projected/0c841a60-7bfa-4149-aa1b-b7546e2cccd1-kube-api-access-xn87x\") pod \"payload-processing-e2e-inf-a-02002b-5667b6bfc8-pksxt\" (UID: \"0c841a60-7bfa-4149-aa1b-b7546e2cccd1\") " pod="openshift-ingress/payload-processing-e2e-inf-a-02002b-5667b6bfc8-pksxt"
Jul 31 23:37:33.524650 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:37:33.524628    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4f19fc4-3f03-41e2-8467-4cb8fb3c4a50-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-02002b-77697f6d68-zbbvr\" (UID: \"f4f19fc4-3f03-41e2-8467-4cb8fb3c4a50\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-02002b-77697f6d68-zbbvr"
Jul 31 23:37:33.524725 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:37:33.524685    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c841a60-7bfa-4149-aa1b-b7546e2cccd1-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-02002b-5667b6bfc8-pksxt\" (UID: \"0c841a60-7bfa-4149-aa1b-b7546e2cccd1\") " pod="openshift-ingress/payload-processing-e2e-inf-a-02002b-5667b6bfc8-pksxt"
Jul 31 23:37:33.532246 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:37:33.532173    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xn87x\" (UniqueName: \"kubernetes.io/projected/0c841a60-7bfa-4149-aa1b-b7546e2cccd1-kube-api-access-xn87x\") pod \"payload-processing-e2e-inf-a-02002b-5667b6bfc8-pksxt\" (UID: \"0c841a60-7bfa-4149-aa1b-b7546e2cccd1\") " pod="openshift-ingress/payload-processing-e2e-inf-a-02002b-5667b6bfc8-pksxt"
Jul 31 23:37:33.532374 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:37:33.532324    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvfk9\" (UniqueName: \"kubernetes.io/projected/f4f19fc4-3f03-41e2-8467-4cb8fb3c4a50-kube-api-access-lvfk9\") pod \"payload-pre-processing-e2e-inf-a-02002b-77697f6d68-zbbvr\" (UID: \"f4f19fc4-3f03-41e2-8467-4cb8fb3c4a50\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-02002b-77697f6d68-zbbvr"
Jul 31 23:37:33.588678 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:37:33.588639    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-02002b-77697f6d68-zbbvr"
Jul 31 23:37:33.600255 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:37:33.600224    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-02002b-79f6759b5c-vdzrc"
Jul 31 23:37:33.623739 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:37:33.623687    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-02002b-5667b6bfc8-pksxt"
Jul 31 23:37:33.759327 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:37:33.759286    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-02002b-77697f6d68-zbbvr"]
Jul 31 23:37:33.764942 ip-10-0-128-243 kubenswrapper[2584]: W0731 23:37:33.764835    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4f19fc4_3f03_41e2_8467_4cb8fb3c4a50.slice/crio-0402701ea2363b75b46eab6c3ab126ced9ee1e82924afd90b1d710f9a4037345 WatchSource:0}: Error finding container 0402701ea2363b75b46eab6c3ab126ced9ee1e82924afd90b1d710f9a4037345: Status 404 returned error can't find the container with id 0402701ea2363b75b46eab6c3ab126ced9ee1e82924afd90b1d710f9a4037345
Jul 31 23:37:33.782417 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:37:33.782354    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-02002b-79f6759b5c-vdzrc"]
Jul 31 23:37:33.785041 ip-10-0-128-243 kubenswrapper[2584]: W0731 23:37:33.784999    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode95c8bf2_987c_4924_99df_4fda74671c2d.slice/crio-f7977818429a7469929648efe6d90ed0f44b41ce4898a0be48b200759cd0a438 WatchSource:0}: Error finding container f7977818429a7469929648efe6d90ed0f44b41ce4898a0be48b200759cd0a438: Status 404 returned error can't find the container with id f7977818429a7469929648efe6d90ed0f44b41ce4898a0be48b200759cd0a438
Jul 31 23:37:33.814198 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:37:33.814019    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-02002b-5667b6bfc8-pksxt"]
Jul 31 23:37:33.816346 ip-10-0-128-243 kubenswrapper[2584]: W0731 23:37:33.816321    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c841a60_7bfa_4149_aa1b_b7546e2cccd1.slice/crio-aa4afd2b6d73ea0957b5b631cffaa5ee58cc3aeb757e34c008c4083712054f7f WatchSource:0}: Error finding container aa4afd2b6d73ea0957b5b631cffaa5ee58cc3aeb757e34c008c4083712054f7f: Status 404 returned error can't find the container with id aa4afd2b6d73ea0957b5b631cffaa5ee58cc3aeb757e34c008c4083712054f7f
Jul 31 23:37:34.023242 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:37:34.023205    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-02002b-79f6759b5c-vdzrc" event={"ID":"e95c8bf2-987c-4924-99df-4fda74671c2d","Type":"ContainerStarted","Data":"f7977818429a7469929648efe6d90ed0f44b41ce4898a0be48b200759cd0a438"}
Jul 31 23:37:34.024528 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:37:34.024502    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-02002b-5667b6bfc8-pksxt" event={"ID":"0c841a60-7bfa-4149-aa1b-b7546e2cccd1","Type":"ContainerStarted","Data":"01f7f21d72be8a678b63a75d3e852269722f1381d39d6f70169f35d2dafc644a"}
Jul 31 23:37:34.024649 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:37:34.024550    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-02002b-5667b6bfc8-pksxt" event={"ID":"0c841a60-7bfa-4149-aa1b-b7546e2cccd1","Type":"ContainerStarted","Data":"aa4afd2b6d73ea0957b5b631cffaa5ee58cc3aeb757e34c008c4083712054f7f"}
Jul 31 23:37:34.024649 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:37:34.024639    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-02002b-5667b6bfc8-pksxt"
Jul 31 23:37:34.026075 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:37:34.026055    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-02002b-77697f6d68-zbbvr" event={"ID":"f4f19fc4-3f03-41e2-8467-4cb8fb3c4a50","Type":"ContainerStarted","Data":"dd1236b7c424650c9b908e9426699310ee32a4c8b23b45f6880badfc3eec8d69"}
Jul 31 23:37:34.026151 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:37:34.026091    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-02002b-77697f6d68-zbbvr" event={"ID":"f4f19fc4-3f03-41e2-8467-4cb8fb3c4a50","Type":"ContainerStarted","Data":"0402701ea2363b75b46eab6c3ab126ced9ee1e82924afd90b1d710f9a4037345"}
Jul 31 23:37:34.026151 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:37:34.026130    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-02002b-77697f6d68-zbbvr"
Jul 31 23:37:34.039909 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:37:34.039818    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-02002b-5667b6bfc8-pksxt" podStartSLOduration=1.039802714 podStartE2EDuration="1.039802714s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:37:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:37:34.038566531 +0000 UTC m=+2437.684352842" watchObservedRunningTime="2026-07-31 23:37:34.039802714 +0000 UTC m=+2437.685589014"
Jul 31 23:37:34.055074 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:37:34.055022    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-02002b-77697f6d68-zbbvr" podStartSLOduration=1.055008117 podStartE2EDuration="1.055008117s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:37:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:37:34.053100852 +0000 UTC m=+2437.698887167" watchObservedRunningTime="2026-07-31 23:37:34.055008117 +0000 UTC m=+2437.700794427"
Jul 31 23:37:35.033200 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:37:35.033161    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-02002b-79f6759b5c-vdzrc" event={"ID":"e95c8bf2-987c-4924-99df-4fda74671c2d","Type":"ContainerStarted","Data":"a14824c0d097b7bf6485a0c6ef8a90de5af3872faab3a2e5fc83e4d90ec77f72"}
Jul 31 23:37:35.033638 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:37:35.033386    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-02002b-79f6759b5c-vdzrc"
Jul 31 23:37:35.048953 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:37:35.048902    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-02002b-79f6759b5c-vdzrc" podStartSLOduration=2.542530687 podStartE2EDuration="3.048885515s" totalImagesPullingTime="506.354828ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:37:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:37:35.047242714 +0000 UTC m=+2438.693029017" watchObservedRunningTime="2026-07-31 23:37:35.048885515 +0000 UTC m=+2438.694671821"
Jul 31 23:37:41.043752 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:37:41.043718    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-02002b-79f6759b5c-vdzrc"
Jul 31 23:37:45.035312 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:37:45.035275    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-02002b-5667b6bfc8-pksxt"
Jul 31 23:37:45.035804 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:37:45.035782    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-02002b-77697f6d68-zbbvr"
Jul 31 23:38:00.865758 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:38:00.864374    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3c3302-7cc485d58b-gcfs5"]
Jul 31 23:38:00.870734 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:38:00.870681    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3c3302-7cc485d58b-gcfs5"
Jul 31 23:38:00.873177 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:38:00.873150    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-3c3302-serving-cert\""
Jul 31 23:38:00.879209 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:38:00.879146    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3c3302-7cc485d58b-gcfs5"]
Jul 31 23:38:00.967874 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:38:00.967836    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgwpf\" (UniqueName: \"kubernetes.io/projected/390df8c2-d64e-4f6b-9703-d75767fa6e29-kube-api-access-dgwpf\") pod \"maas-api-e2e-inf-b-3c3302-7cc485d58b-gcfs5\" (UID: \"390df8c2-d64e-4f6b-9703-d75767fa6e29\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3c3302-7cc485d58b-gcfs5"
Jul 31 23:38:00.968066 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:38:00.967889    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/390df8c2-d64e-4f6b-9703-d75767fa6e29-openshift-service-ca\") pod \"maas-api-e2e-inf-b-3c3302-7cc485d58b-gcfs5\" (UID: \"390df8c2-d64e-4f6b-9703-d75767fa6e29\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3c3302-7cc485d58b-gcfs5"
Jul 31 23:38:00.968066 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:38:00.967950    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/390df8c2-d64e-4f6b-9703-d75767fa6e29-maas-api-tls\") pod \"maas-api-e2e-inf-b-3c3302-7cc485d58b-gcfs5\" (UID: \"390df8c2-d64e-4f6b-9703-d75767fa6e29\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3c3302-7cc485d58b-gcfs5"
Jul 31 23:38:01.069299 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:38:01.069256    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgwpf\" (UniqueName: \"kubernetes.io/projected/390df8c2-d64e-4f6b-9703-d75767fa6e29-kube-api-access-dgwpf\") pod \"maas-api-e2e-inf-b-3c3302-7cc485d58b-gcfs5\" (UID: \"390df8c2-d64e-4f6b-9703-d75767fa6e29\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3c3302-7cc485d58b-gcfs5"
Jul 31 23:38:01.069299 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:38:01.069303    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/390df8c2-d64e-4f6b-9703-d75767fa6e29-openshift-service-ca\") pod \"maas-api-e2e-inf-b-3c3302-7cc485d58b-gcfs5\" (UID: \"390df8c2-d64e-4f6b-9703-d75767fa6e29\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3c3302-7cc485d58b-gcfs5"
Jul 31 23:38:01.069548 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:38:01.069349    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/390df8c2-d64e-4f6b-9703-d75767fa6e29-maas-api-tls\") pod \"maas-api-e2e-inf-b-3c3302-7cc485d58b-gcfs5\" (UID: \"390df8c2-d64e-4f6b-9703-d75767fa6e29\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3c3302-7cc485d58b-gcfs5"
Jul 31 23:38:01.069548 ip-10-0-128-243 kubenswrapper[2584]: E0731 23:38:01.069508    2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-3c3302-serving-cert: secret "maas-api-e2e-inf-b-3c3302-serving-cert" not found
Jul 31 23:38:01.069635 ip-10-0-128-243 kubenswrapper[2584]: E0731 23:38:01.069589    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/390df8c2-d64e-4f6b-9703-d75767fa6e29-maas-api-tls podName:390df8c2-d64e-4f6b-9703-d75767fa6e29 nodeName:}" failed. No retries permitted until 2026-07-31 23:38:01.569568139 +0000 UTC m=+2465.215354431 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/390df8c2-d64e-4f6b-9703-d75767fa6e29-maas-api-tls") pod "maas-api-e2e-inf-b-3c3302-7cc485d58b-gcfs5" (UID: "390df8c2-d64e-4f6b-9703-d75767fa6e29") : secret "maas-api-e2e-inf-b-3c3302-serving-cert" not found
Jul 31 23:38:01.069999 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:38:01.069981    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/390df8c2-d64e-4f6b-9703-d75767fa6e29-openshift-service-ca\") pod \"maas-api-e2e-inf-b-3c3302-7cc485d58b-gcfs5\" (UID: \"390df8c2-d64e-4f6b-9703-d75767fa6e29\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3c3302-7cc485d58b-gcfs5"
Jul 31 23:38:01.077474 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:38:01.077450    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgwpf\" (UniqueName: \"kubernetes.io/projected/390df8c2-d64e-4f6b-9703-d75767fa6e29-kube-api-access-dgwpf\") pod \"maas-api-e2e-inf-b-3c3302-7cc485d58b-gcfs5\" (UID: \"390df8c2-d64e-4f6b-9703-d75767fa6e29\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3c3302-7cc485d58b-gcfs5"
Jul 31 23:38:01.463236 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:38:01.463204    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-3c3302-568f85dd-q2l8w"]
Jul 31 23:38:01.466924 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:38:01.466907    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3c3302-568f85dd-q2l8w"
Jul 31 23:38:01.469225 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:38:01.469204    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-3c3302-dockercfg-v67zp\""
Jul 31 23:38:01.469225 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:38:01.469224    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-3c3302\""
Jul 31 23:38:01.476909 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:38:01.476883    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-3c3302-568f85dd-q2l8w"]
Jul 31 23:38:01.513275 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:38:01.513242    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-3c3302-5b95f4f6f6-br6mb"]
Jul 31 23:38:01.516815 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:38:01.516798    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-3c3302-5b95f4f6f6-br6mb"
Jul 31 23:38:01.526146 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:38:01.526119    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-3c3302-5b95f4f6f6-br6mb"]
Jul 31 23:38:01.574852 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:38:01.574820    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/68763429-4fb7-44fc-8cf0-07dc06d1f099-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-3c3302-568f85dd-q2l8w\" (UID: \"68763429-4fb7-44fc-8cf0-07dc06d1f099\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3c3302-568f85dd-q2l8w"
Jul 31 23:38:01.575051 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:38:01.574889    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56hsg\" (UniqueName: \"kubernetes.io/projected/68763429-4fb7-44fc-8cf0-07dc06d1f099-kube-api-access-56hsg\") pod \"payload-pre-processing-e2e-inf-b-3c3302-568f85dd-q2l8w\" (UID: \"68763429-4fb7-44fc-8cf0-07dc06d1f099\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3c3302-568f85dd-q2l8w"
Jul 31 23:38:01.575051 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:38:01.575015    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/390df8c2-d64e-4f6b-9703-d75767fa6e29-maas-api-tls\") pod \"maas-api-e2e-inf-b-3c3302-7cc485d58b-gcfs5\" (UID: \"390df8c2-d64e-4f6b-9703-d75767fa6e29\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3c3302-7cc485d58b-gcfs5"
Jul 31 23:38:01.575175 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:38:01.575057    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f4b8003-8661-480e-b795-61d107ea9a75-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-3c3302-5b95f4f6f6-br6mb\" (UID: \"0f4b8003-8661-480e-b795-61d107ea9a75\") " pod="openshift-ingress/payload-processing-e2e-inf-b-3c3302-5b95f4f6f6-br6mb"
Jul 31 23:38:01.575175 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:38:01.575097    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvhwt\" (UniqueName: \"kubernetes.io/projected/0f4b8003-8661-480e-b795-61d107ea9a75-kube-api-access-qvhwt\") pod \"payload-processing-e2e-inf-b-3c3302-5b95f4f6f6-br6mb\" (UID: \"0f4b8003-8661-480e-b795-61d107ea9a75\") " pod="openshift-ingress/payload-processing-e2e-inf-b-3c3302-5b95f4f6f6-br6mb"
Jul 31 23:38:01.577686 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:38:01.577663    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/390df8c2-d64e-4f6b-9703-d75767fa6e29-maas-api-tls\") pod \"maas-api-e2e-inf-b-3c3302-7cc485d58b-gcfs5\" (UID: \"390df8c2-d64e-4f6b-9703-d75767fa6e29\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3c3302-7cc485d58b-gcfs5"
Jul 31 23:38:01.676514 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:38:01.676475    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56hsg\" (UniqueName: \"kubernetes.io/projected/68763429-4fb7-44fc-8cf0-07dc06d1f099-kube-api-access-56hsg\") pod \"payload-pre-processing-e2e-inf-b-3c3302-568f85dd-q2l8w\" (UID: \"68763429-4fb7-44fc-8cf0-07dc06d1f099\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3c3302-568f85dd-q2l8w"
Jul 31 23:38:01.676746 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:38:01.676558    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f4b8003-8661-480e-b795-61d107ea9a75-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-3c3302-5b95f4f6f6-br6mb\" (UID: \"0f4b8003-8661-480e-b795-61d107ea9a75\") " pod="openshift-ingress/payload-processing-e2e-inf-b-3c3302-5b95f4f6f6-br6mb"
Jul 31 23:38:01.676746 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:38:01.676593    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvhwt\" (UniqueName: \"kubernetes.io/projected/0f4b8003-8661-480e-b795-61d107ea9a75-kube-api-access-qvhwt\") pod \"payload-processing-e2e-inf-b-3c3302-5b95f4f6f6-br6mb\" (UID: \"0f4b8003-8661-480e-b795-61d107ea9a75\") " pod="openshift-ingress/payload-processing-e2e-inf-b-3c3302-5b95f4f6f6-br6mb"
Jul 31 23:38:01.676746 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:38:01.676618    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/68763429-4fb7-44fc-8cf0-07dc06d1f099-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-3c3302-568f85dd-q2l8w\" (UID: \"68763429-4fb7-44fc-8cf0-07dc06d1f099\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3c3302-568f85dd-q2l8w"
Jul 31 23:38:01.677305 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:38:01.677281    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f4b8003-8661-480e-b795-61d107ea9a75-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-3c3302-5b95f4f6f6-br6mb\" (UID: \"0f4b8003-8661-480e-b795-61d107ea9a75\") " pod="openshift-ingress/payload-processing-e2e-inf-b-3c3302-5b95f4f6f6-br6mb"
Jul 31 23:38:01.677486 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:38:01.677459    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/68763429-4fb7-44fc-8cf0-07dc06d1f099-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-3c3302-568f85dd-q2l8w\" (UID: \"68763429-4fb7-44fc-8cf0-07dc06d1f099\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3c3302-568f85dd-q2l8w"
Jul 31 23:38:01.684185 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:38:01.684157    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-56hsg\" (UniqueName: \"kubernetes.io/projected/68763429-4fb7-44fc-8cf0-07dc06d1f099-kube-api-access-56hsg\") pod \"payload-pre-processing-e2e-inf-b-3c3302-568f85dd-q2l8w\" (UID: \"68763429-4fb7-44fc-8cf0-07dc06d1f099\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3c3302-568f85dd-q2l8w"
Jul 31 23:38:01.684268 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:38:01.684228    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvhwt\" (UniqueName: \"kubernetes.io/projected/0f4b8003-8661-480e-b795-61d107ea9a75-kube-api-access-qvhwt\") pod \"payload-processing-e2e-inf-b-3c3302-5b95f4f6f6-br6mb\" (UID: \"0f4b8003-8661-480e-b795-61d107ea9a75\") " pod="openshift-ingress/payload-processing-e2e-inf-b-3c3302-5b95f4f6f6-br6mb"
Jul 31 23:38:01.779285 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:38:01.779203    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3c3302-568f85dd-q2l8w"
Jul 31 23:38:01.787301 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:38:01.787283    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3c3302-7cc485d58b-gcfs5"
Jul 31 23:38:01.828232 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:38:01.828198    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-3c3302-5b95f4f6f6-br6mb"
Jul 31 23:38:01.936509 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:38:01.936452    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-3c3302-568f85dd-q2l8w"]
Jul 31 23:38:01.939498 ip-10-0-128-243 kubenswrapper[2584]: W0731 23:38:01.939455    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68763429_4fb7_44fc_8cf0_07dc06d1f099.slice/crio-cd3a9ea306b0f92172316f9bc3ed2ee87db4239f29e2fbb70b6242c601d8bed7 WatchSource:0}: Error finding container cd3a9ea306b0f92172316f9bc3ed2ee87db4239f29e2fbb70b6242c601d8bed7: Status 404 returned error can't find the container with id cd3a9ea306b0f92172316f9bc3ed2ee87db4239f29e2fbb70b6242c601d8bed7
Jul 31 23:38:01.961321 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:38:01.961294    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3c3302-7cc485d58b-gcfs5"]
Jul 31 23:38:01.964055 ip-10-0-128-243 kubenswrapper[2584]: W0731 23:38:01.964029    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod390df8c2_d64e_4f6b_9703_d75767fa6e29.slice/crio-2735c3f1ff0de2f290d792ab939ca72b7657c71a8020174e4e4237e01af05987 WatchSource:0}: Error finding container 2735c3f1ff0de2f290d792ab939ca72b7657c71a8020174e4e4237e01af05987: Status 404 returned error can't find the container with id 2735c3f1ff0de2f290d792ab939ca72b7657c71a8020174e4e4237e01af05987
Jul 31 23:38:01.995883 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:38:01.995860    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-3c3302-5b95f4f6f6-br6mb"]
Jul 31 23:38:02.001842 ip-10-0-128-243 kubenswrapper[2584]: W0731 23:38:02.001809    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f4b8003_8661_480e_b795_61d107ea9a75.slice/crio-b2da33538900b04e19195a7951392e6844f946358bdcb223fd1f532d1ca80b46 WatchSource:0}: Error finding container b2da33538900b04e19195a7951392e6844f946358bdcb223fd1f532d1ca80b46: Status 404 returned error can't find the container with id b2da33538900b04e19195a7951392e6844f946358bdcb223fd1f532d1ca80b46
Jul 31 23:38:02.168396 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:38:02.168351    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-3c3302-5b95f4f6f6-br6mb" event={"ID":"0f4b8003-8661-480e-b795-61d107ea9a75","Type":"ContainerStarted","Data":"7e323d4c8ba3e07d651886e07eaf639ac50f50c3de88a1ff9af2efc548b22f3d"}
Jul 31 23:38:02.168564 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:38:02.168417    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-3c3302-5b95f4f6f6-br6mb" event={"ID":"0f4b8003-8661-480e-b795-61d107ea9a75","Type":"ContainerStarted","Data":"b2da33538900b04e19195a7951392e6844f946358bdcb223fd1f532d1ca80b46"}
Jul 31 23:38:02.168564 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:38:02.168446    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-3c3302-5b95f4f6f6-br6mb"
Jul 31 23:38:02.170132 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:38:02.170099    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3c3302-568f85dd-q2l8w" event={"ID":"68763429-4fb7-44fc-8cf0-07dc06d1f099","Type":"ContainerStarted","Data":"b589ecb096d628c1a584b624bf5994ae2066bef9295ce026c068199732fe6569"}
Jul 31 23:38:02.170271 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:38:02.170161    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3c3302-568f85dd-q2l8w"
Jul 31 23:38:02.170271 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:38:02.170185    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3c3302-568f85dd-q2l8w" event={"ID":"68763429-4fb7-44fc-8cf0-07dc06d1f099","Type":"ContainerStarted","Data":"cd3a9ea306b0f92172316f9bc3ed2ee87db4239f29e2fbb70b6242c601d8bed7"}
Jul 31 23:38:02.171298 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:38:02.171269    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3c3302-7cc485d58b-gcfs5" event={"ID":"390df8c2-d64e-4f6b-9703-d75767fa6e29","Type":"ContainerStarted","Data":"2735c3f1ff0de2f290d792ab939ca72b7657c71a8020174e4e4237e01af05987"}
Jul 31 23:38:02.184253 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:38:02.184206    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-3c3302-5b95f4f6f6-br6mb" podStartSLOduration=1.184191277 podStartE2EDuration="1.184191277s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:38:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:38:02.183097259 +0000 UTC m=+2465.828883569" watchObservedRunningTime="2026-07-31 23:38:02.184191277 +0000 UTC m=+2465.829977564"
Jul 31 23:38:02.198105 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:38:02.198052    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3c3302-568f85dd-q2l8w" podStartSLOduration=1.198036629 podStartE2EDuration="1.198036629s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:38:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:38:02.196865912 +0000 UTC m=+2465.842652220" watchObservedRunningTime="2026-07-31 23:38:02.198036629 +0000 UTC m=+2465.843822939"
Jul 31 23:38:03.178111 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:38:03.178071    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3c3302-7cc485d58b-gcfs5" event={"ID":"390df8c2-d64e-4f6b-9703-d75767fa6e29","Type":"ContainerStarted","Data":"9e7fb59349686c629e0e36527dfce1f76f76abd65d6d0b92203ce75a865b334d"}
Jul 31 23:38:03.178559 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:38:03.178352    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3c3302-7cc485d58b-gcfs5"
Jul 31 23:38:03.194348 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:38:03.194302    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3c3302-7cc485d58b-gcfs5" podStartSLOduration=2.664649441 podStartE2EDuration="3.194288727s" totalImagesPullingTime="529.639286ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:38:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:38:03.192994111 +0000 UTC m=+2466.838780417" watchObservedRunningTime="2026-07-31 23:38:03.194288727 +0000 UTC m=+2466.840075036"
Jul 31 23:38:09.189011 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:38:09.188982    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3c3302-7cc485d58b-gcfs5"
Jul 31 23:38:13.180490 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:38:13.180457    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-3c3302-5b95f4f6f6-br6mb"
Jul 31 23:38:13.180979 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:38:13.180668    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3c3302-568f85dd-q2l8w"
Jul 31 23:39:56.058200 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:39:56.058167    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-02002b-79f6759b5c-vdzrc"]
Jul 31 23:39:56.058778 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:39:56.058427    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-02002b-79f6759b5c-vdzrc" podUID="e95c8bf2-987c-4924-99df-4fda74671c2d" containerName="maas-api" containerID="cri-o://a14824c0d097b7bf6485a0c6ef8a90de5af3872faab3a2e5fc83e4d90ec77f72" gracePeriod=30
Jul 31 23:39:56.114374 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:39:56.114342    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-02002b-5667b6bfc8-pksxt"]
Jul 31 23:39:56.114595 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:39:56.114555    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-02002b-5667b6bfc8-pksxt" podUID="0c841a60-7bfa-4149-aa1b-b7546e2cccd1" containerName="payload-processing" containerID="cri-o://01f7f21d72be8a678b63a75d3e852269722f1381d39d6f70169f35d2dafc644a" gracePeriod=30
Jul 31 23:39:56.120097 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:39:56.120066    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-02002b-77697f6d68-zbbvr"]
Jul 31 23:39:56.120338 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:39:56.120286    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-02002b-77697f6d68-zbbvr" podUID="f4f19fc4-3f03-41e2-8467-4cb8fb3c4a50" containerName="payload-pre-processing" containerID="cri-o://dd1236b7c424650c9b908e9426699310ee32a4c8b23b45f6880badfc3eec8d69" gracePeriod=30
Jul 31 23:39:56.390068 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:39:56.390041    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-02002b-5667b6bfc8-pksxt"
Jul 31 23:39:56.410881 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:39:56.410853    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xn87x\" (UniqueName: \"kubernetes.io/projected/0c841a60-7bfa-4149-aa1b-b7546e2cccd1-kube-api-access-xn87x\") pod \"0c841a60-7bfa-4149-aa1b-b7546e2cccd1\" (UID: \"0c841a60-7bfa-4149-aa1b-b7546e2cccd1\") "
Jul 31 23:39:56.410979 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:39:56.410949    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c841a60-7bfa-4149-aa1b-b7546e2cccd1-plugins-config-volume\") pod \"0c841a60-7bfa-4149-aa1b-b7546e2cccd1\" (UID: \"0c841a60-7bfa-4149-aa1b-b7546e2cccd1\") "
Jul 31 23:39:56.411537 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:39:56.411508    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c841a60-7bfa-4149-aa1b-b7546e2cccd1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0c841a60-7bfa-4149-aa1b-b7546e2cccd1" (UID: "0c841a60-7bfa-4149-aa1b-b7546e2cccd1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:39:56.413943 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:39:56.413919    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c841a60-7bfa-4149-aa1b-b7546e2cccd1-kube-api-access-xn87x" (OuterVolumeSpecName: "kube-api-access-xn87x") pod "0c841a60-7bfa-4149-aa1b-b7546e2cccd1" (UID: "0c841a60-7bfa-4149-aa1b-b7546e2cccd1"). InnerVolumeSpecName "kube-api-access-xn87x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 23:39:56.415961 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:39:56.415941    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-02002b-79f6759b5c-vdzrc"
Jul 31 23:39:56.419583 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:39:56.419565    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-02002b-77697f6d68-zbbvr"
Jul 31 23:39:56.512233 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:39:56.512205    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lvfk9\" (UniqueName: \"kubernetes.io/projected/f4f19fc4-3f03-41e2-8467-4cb8fb3c4a50-kube-api-access-lvfk9\") pod \"f4f19fc4-3f03-41e2-8467-4cb8fb3c4a50\" (UID: \"f4f19fc4-3f03-41e2-8467-4cb8fb3c4a50\") "
Jul 31 23:39:56.512411 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:39:56.512262    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jvxmf\" (UniqueName: \"kubernetes.io/projected/e95c8bf2-987c-4924-99df-4fda74671c2d-kube-api-access-jvxmf\") pod \"e95c8bf2-987c-4924-99df-4fda74671c2d\" (UID: \"e95c8bf2-987c-4924-99df-4fda74671c2d\") "
Jul 31 23:39:56.512411 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:39:56.512284    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e95c8bf2-987c-4924-99df-4fda74671c2d-openshift-service-ca\") pod \"e95c8bf2-987c-4924-99df-4fda74671c2d\" (UID: \"e95c8bf2-987c-4924-99df-4fda74671c2d\") "
Jul 31 23:39:56.512411 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:39:56.512324    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4f19fc4-3f03-41e2-8467-4cb8fb3c4a50-plugins-config-volume\") pod \"f4f19fc4-3f03-41e2-8467-4cb8fb3c4a50\" (UID: \"f4f19fc4-3f03-41e2-8467-4cb8fb3c4a50\") "
Jul 31 23:39:56.512411 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:39:56.512345    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e95c8bf2-987c-4924-99df-4fda74671c2d-maas-api-tls\") pod \"e95c8bf2-987c-4924-99df-4fda74671c2d\" (UID: \"e95c8bf2-987c-4924-99df-4fda74671c2d\") "
Jul 31 23:39:56.512633 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:39:56.512590    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c841a60-7bfa-4149-aa1b-b7546e2cccd1-plugins-config-volume\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:39:56.512633 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:39:56.512609    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xn87x\" (UniqueName: \"kubernetes.io/projected/0c841a60-7bfa-4149-aa1b-b7546e2cccd1-kube-api-access-xn87x\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:39:56.512762 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:39:56.512648    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e95c8bf2-987c-4924-99df-4fda74671c2d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e95c8bf2-987c-4924-99df-4fda74671c2d" (UID: "e95c8bf2-987c-4924-99df-4fda74671c2d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:39:56.512762 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:39:56.512731    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4f19fc4-3f03-41e2-8467-4cb8fb3c4a50-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f4f19fc4-3f03-41e2-8467-4cb8fb3c4a50" (UID: "f4f19fc4-3f03-41e2-8467-4cb8fb3c4a50"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:39:56.514460 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:39:56.514429    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e95c8bf2-987c-4924-99df-4fda74671c2d-kube-api-access-jvxmf" (OuterVolumeSpecName: "kube-api-access-jvxmf") pod "e95c8bf2-987c-4924-99df-4fda74671c2d" (UID: "e95c8bf2-987c-4924-99df-4fda74671c2d"). InnerVolumeSpecName "kube-api-access-jvxmf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 23:39:56.514460 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:39:56.514449    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e95c8bf2-987c-4924-99df-4fda74671c2d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e95c8bf2-987c-4924-99df-4fda74671c2d" (UID: "e95c8bf2-987c-4924-99df-4fda74671c2d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 23:39:56.514460 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:39:56.514452    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4f19fc4-3f03-41e2-8467-4cb8fb3c4a50-kube-api-access-lvfk9" (OuterVolumeSpecName: "kube-api-access-lvfk9") pod "f4f19fc4-3f03-41e2-8467-4cb8fb3c4a50" (UID: "f4f19fc4-3f03-41e2-8467-4cb8fb3c4a50"). InnerVolumeSpecName "kube-api-access-lvfk9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 23:39:56.613474 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:39:56.613412    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4f19fc4-3f03-41e2-8467-4cb8fb3c4a50-plugins-config-volume\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:39:56.613474 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:39:56.613433    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e95c8bf2-987c-4924-99df-4fda74671c2d-maas-api-tls\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:39:56.613474 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:39:56.613442    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lvfk9\" (UniqueName: \"kubernetes.io/projected/f4f19fc4-3f03-41e2-8467-4cb8fb3c4a50-kube-api-access-lvfk9\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:39:56.613474 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:39:56.613452    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jvxmf\" (UniqueName: \"kubernetes.io/projected/e95c8bf2-987c-4924-99df-4fda74671c2d-kube-api-access-jvxmf\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:39:56.613474 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:39:56.613462    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e95c8bf2-987c-4924-99df-4fda74671c2d-openshift-service-ca\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:39:56.717476 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:39:56.717434    2584 generic.go:356] "Generic (PLEG): container finished" podID="e95c8bf2-987c-4924-99df-4fda74671c2d" containerID="a14824c0d097b7bf6485a0c6ef8a90de5af3872faab3a2e5fc83e4d90ec77f72" exitCode=0
Jul 31 23:39:56.717476 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:39:56.717476    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-02002b-79f6759b5c-vdzrc"
Jul 31 23:39:56.717678 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:39:56.717515    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-02002b-79f6759b5c-vdzrc" event={"ID":"e95c8bf2-987c-4924-99df-4fda74671c2d","Type":"ContainerDied","Data":"a14824c0d097b7bf6485a0c6ef8a90de5af3872faab3a2e5fc83e4d90ec77f72"}
Jul 31 23:39:56.717678 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:39:56.717561    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-02002b-79f6759b5c-vdzrc" event={"ID":"e95c8bf2-987c-4924-99df-4fda74671c2d","Type":"ContainerDied","Data":"f7977818429a7469929648efe6d90ed0f44b41ce4898a0be48b200759cd0a438"}
Jul 31 23:39:56.717678 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:39:56.717588    2584 scope.go:117] "RemoveContainer" containerID="a14824c0d097b7bf6485a0c6ef8a90de5af3872faab3a2e5fc83e4d90ec77f72"
Jul 31 23:39:56.718788 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:39:56.718760    2584 generic.go:356] "Generic (PLEG): container finished" podID="0c841a60-7bfa-4149-aa1b-b7546e2cccd1" containerID="01f7f21d72be8a678b63a75d3e852269722f1381d39d6f70169f35d2dafc644a" exitCode=0
Jul 31 23:39:56.718872 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:39:56.718800    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-02002b-5667b6bfc8-pksxt"
Jul 31 23:39:56.718872 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:39:56.718828    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-02002b-5667b6bfc8-pksxt" event={"ID":"0c841a60-7bfa-4149-aa1b-b7546e2cccd1","Type":"ContainerDied","Data":"01f7f21d72be8a678b63a75d3e852269722f1381d39d6f70169f35d2dafc644a"}
Jul 31 23:39:56.718872 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:39:56.718862    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-02002b-5667b6bfc8-pksxt" event={"ID":"0c841a60-7bfa-4149-aa1b-b7546e2cccd1","Type":"ContainerDied","Data":"aa4afd2b6d73ea0957b5b631cffaa5ee58cc3aeb757e34c008c4083712054f7f"}
Jul 31 23:39:56.720268 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:39:56.720242    2584 generic.go:356] "Generic (PLEG): container finished" podID="f4f19fc4-3f03-41e2-8467-4cb8fb3c4a50" containerID="dd1236b7c424650c9b908e9426699310ee32a4c8b23b45f6880badfc3eec8d69" exitCode=0
Jul 31 23:39:56.720387 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:39:56.720283    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-02002b-77697f6d68-zbbvr" event={"ID":"f4f19fc4-3f03-41e2-8467-4cb8fb3c4a50","Type":"ContainerDied","Data":"dd1236b7c424650c9b908e9426699310ee32a4c8b23b45f6880badfc3eec8d69"}
Jul 31 23:39:56.720387 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:39:56.720294    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-02002b-77697f6d68-zbbvr"
Jul 31 23:39:56.720387 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:39:56.720317    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-02002b-77697f6d68-zbbvr" event={"ID":"f4f19fc4-3f03-41e2-8467-4cb8fb3c4a50","Type":"ContainerDied","Data":"0402701ea2363b75b46eab6c3ab126ced9ee1e82924afd90b1d710f9a4037345"}
Jul 31 23:39:56.741191 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:39:56.741173    2584 scope.go:117] "RemoveContainer" containerID="a14824c0d097b7bf6485a0c6ef8a90de5af3872faab3a2e5fc83e4d90ec77f72"
Jul 31 23:39:56.741478 ip-10-0-128-243 kubenswrapper[2584]: E0731 23:39:56.741460    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a14824c0d097b7bf6485a0c6ef8a90de5af3872faab3a2e5fc83e4d90ec77f72\": container with ID starting with a14824c0d097b7bf6485a0c6ef8a90de5af3872faab3a2e5fc83e4d90ec77f72 not found: ID does not exist" containerID="a14824c0d097b7bf6485a0c6ef8a90de5af3872faab3a2e5fc83e4d90ec77f72"
Jul 31 23:39:56.741524 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:39:56.741488    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a14824c0d097b7bf6485a0c6ef8a90de5af3872faab3a2e5fc83e4d90ec77f72"} err="failed to get container status \"a14824c0d097b7bf6485a0c6ef8a90de5af3872faab3a2e5fc83e4d90ec77f72\": rpc error: code = NotFound desc = could not find container \"a14824c0d097b7bf6485a0c6ef8a90de5af3872faab3a2e5fc83e4d90ec77f72\": container with ID starting with a14824c0d097b7bf6485a0c6ef8a90de5af3872faab3a2e5fc83e4d90ec77f72 not found: ID does not exist"
Jul 31 23:39:56.741524 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:39:56.741508    2584 scope.go:117] "RemoveContainer" containerID="01f7f21d72be8a678b63a75d3e852269722f1381d39d6f70169f35d2dafc644a"
Jul 31 23:39:56.749111 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:39:56.749090    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-02002b-79f6759b5c-vdzrc"]
Jul 31 23:39:56.749394 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:39:56.749378    2584 scope.go:117] "RemoveContainer" containerID="01f7f21d72be8a678b63a75d3e852269722f1381d39d6f70169f35d2dafc644a"
Jul 31 23:39:56.749677 ip-10-0-128-243 kubenswrapper[2584]: E0731 23:39:56.749659    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01f7f21d72be8a678b63a75d3e852269722f1381d39d6f70169f35d2dafc644a\": container with ID starting with 01f7f21d72be8a678b63a75d3e852269722f1381d39d6f70169f35d2dafc644a not found: ID does not exist" containerID="01f7f21d72be8a678b63a75d3e852269722f1381d39d6f70169f35d2dafc644a"
Jul 31 23:39:56.749757 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:39:56.749683    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01f7f21d72be8a678b63a75d3e852269722f1381d39d6f70169f35d2dafc644a"} err="failed to get container status \"01f7f21d72be8a678b63a75d3e852269722f1381d39d6f70169f35d2dafc644a\": rpc error: code = NotFound desc = could not find container \"01f7f21d72be8a678b63a75d3e852269722f1381d39d6f70169f35d2dafc644a\": container with ID starting with 01f7f21d72be8a678b63a75d3e852269722f1381d39d6f70169f35d2dafc644a not found: ID does not exist"
Jul 31 23:39:56.749757 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:39:56.749727    2584 scope.go:117] "RemoveContainer" containerID="dd1236b7c424650c9b908e9426699310ee32a4c8b23b45f6880badfc3eec8d69"
Jul 31 23:39:56.755294 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:39:56.755276    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-02002b-79f6759b5c-vdzrc"]
Jul 31 23:39:56.757490 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:39:56.757477    2584 scope.go:117] "RemoveContainer" containerID="dd1236b7c424650c9b908e9426699310ee32a4c8b23b45f6880badfc3eec8d69"
Jul 31 23:39:56.757779 ip-10-0-128-243 kubenswrapper[2584]: E0731 23:39:56.757759    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd1236b7c424650c9b908e9426699310ee32a4c8b23b45f6880badfc3eec8d69\": container with ID starting with dd1236b7c424650c9b908e9426699310ee32a4c8b23b45f6880badfc3eec8d69 not found: ID does not exist" containerID="dd1236b7c424650c9b908e9426699310ee32a4c8b23b45f6880badfc3eec8d69"
Jul 31 23:39:56.757834 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:39:56.757785    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd1236b7c424650c9b908e9426699310ee32a4c8b23b45f6880badfc3eec8d69"} err="failed to get container status \"dd1236b7c424650c9b908e9426699310ee32a4c8b23b45f6880badfc3eec8d69\": rpc error: code = NotFound desc = could not find container \"dd1236b7c424650c9b908e9426699310ee32a4c8b23b45f6880badfc3eec8d69\": container with ID starting with dd1236b7c424650c9b908e9426699310ee32a4c8b23b45f6880badfc3eec8d69 not found: ID does not exist"
Jul 31 23:39:56.765346 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:39:56.765328    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-02002b-5667b6bfc8-pksxt"]
Jul 31 23:39:56.771164 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:39:56.771144    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-02002b-5667b6bfc8-pksxt"]
Jul 31 23:39:56.781862 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:39:56.781837    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-02002b-77697f6d68-zbbvr"]
Jul 31 23:39:56.783570 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:39:56.783551    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-02002b-77697f6d68-zbbvr"]
Jul 31 23:39:56.892130 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:39:56.892050    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c841a60-7bfa-4149-aa1b-b7546e2cccd1" path="/var/lib/kubelet/pods/0c841a60-7bfa-4149-aa1b-b7546e2cccd1/volumes"
Jul 31 23:39:56.892436 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:39:56.892423    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e95c8bf2-987c-4924-99df-4fda74671c2d" path="/var/lib/kubelet/pods/e95c8bf2-987c-4924-99df-4fda74671c2d/volumes"
Jul 31 23:39:56.892766 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:39:56.892753    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4f19fc4-3f03-41e2-8467-4cb8fb3c4a50" path="/var/lib/kubelet/pods/f4f19fc4-3f03-41e2-8467-4cb8fb3c4a50/volumes"
Jul 31 23:40:13.487182 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:13.487136    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3c3302-7cc485d58b-gcfs5"]
Jul 31 23:40:13.487668 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:13.487398    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3c3302-7cc485d58b-gcfs5" podUID="390df8c2-d64e-4f6b-9703-d75767fa6e29" containerName="maas-api" containerID="cri-o://9e7fb59349686c629e0e36527dfce1f76f76abd65d6d0b92203ce75a865b334d" gracePeriod=30
Jul 31 23:40:13.544069 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:13.544031    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-3c3302-5b95f4f6f6-br6mb"]
Jul 31 23:40:13.544305 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:13.544255    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-3c3302-5b95f4f6f6-br6mb" podUID="0f4b8003-8661-480e-b795-61d107ea9a75" containerName="payload-processing" containerID="cri-o://7e323d4c8ba3e07d651886e07eaf639ac50f50c3de88a1ff9af2efc548b22f3d" gracePeriod=30
Jul 31 23:40:13.549863 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:13.549836    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-3c3302-568f85dd-q2l8w"]
Jul 31 23:40:13.550123 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:13.550099    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3c3302-568f85dd-q2l8w" podUID="68763429-4fb7-44fc-8cf0-07dc06d1f099" containerName="payload-pre-processing" containerID="cri-o://b589ecb096d628c1a584b624bf5994ae2066bef9295ce026c068199732fe6569" gracePeriod=30
Jul 31 23:40:13.804233 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:13.804180    2584 generic.go:356] "Generic (PLEG): container finished" podID="0f4b8003-8661-480e-b795-61d107ea9a75" containerID="7e323d4c8ba3e07d651886e07eaf639ac50f50c3de88a1ff9af2efc548b22f3d" exitCode=0
Jul 31 23:40:13.804450 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:13.804262    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-3c3302-5b95f4f6f6-br6mb" event={"ID":"0f4b8003-8661-480e-b795-61d107ea9a75","Type":"ContainerDied","Data":"7e323d4c8ba3e07d651886e07eaf639ac50f50c3de88a1ff9af2efc548b22f3d"}
Jul 31 23:40:13.806007 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:13.805987    2584 generic.go:356] "Generic (PLEG): container finished" podID="68763429-4fb7-44fc-8cf0-07dc06d1f099" containerID="b589ecb096d628c1a584b624bf5994ae2066bef9295ce026c068199732fe6569" exitCode=0
Jul 31 23:40:13.806124 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:13.806057    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3c3302-568f85dd-q2l8w" event={"ID":"68763429-4fb7-44fc-8cf0-07dc06d1f099","Type":"ContainerDied","Data":"b589ecb096d628c1a584b624bf5994ae2066bef9295ce026c068199732fe6569"}
Jul 31 23:40:13.807532 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:13.807510    2584 generic.go:356] "Generic (PLEG): container finished" podID="390df8c2-d64e-4f6b-9703-d75767fa6e29" containerID="9e7fb59349686c629e0e36527dfce1f76f76abd65d6d0b92203ce75a865b334d" exitCode=0
Jul 31 23:40:13.807637 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:13.807584    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3c3302-7cc485d58b-gcfs5" event={"ID":"390df8c2-d64e-4f6b-9703-d75767fa6e29","Type":"ContainerDied","Data":"9e7fb59349686c629e0e36527dfce1f76f76abd65d6d0b92203ce75a865b334d"}
Jul 31 23:40:13.826908 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:13.826884    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3c3302-7cc485d58b-gcfs5"
Jul 31 23:40:13.838421 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:13.838398    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3c3302-568f85dd-q2l8w"
Jul 31 23:40:13.842600 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:13.842581    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-3c3302-5b95f4f6f6-br6mb"
Jul 31 23:40:13.982607 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:13.982571    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/390df8c2-d64e-4f6b-9703-d75767fa6e29-maas-api-tls\") pod \"390df8c2-d64e-4f6b-9703-d75767fa6e29\" (UID: \"390df8c2-d64e-4f6b-9703-d75767fa6e29\") "
Jul 31 23:40:13.982785 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:13.982620    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qvhwt\" (UniqueName: \"kubernetes.io/projected/0f4b8003-8661-480e-b795-61d107ea9a75-kube-api-access-qvhwt\") pod \"0f4b8003-8661-480e-b795-61d107ea9a75\" (UID: \"0f4b8003-8661-480e-b795-61d107ea9a75\") "
Jul 31 23:40:13.982785 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:13.982668    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dgwpf\" (UniqueName: \"kubernetes.io/projected/390df8c2-d64e-4f6b-9703-d75767fa6e29-kube-api-access-dgwpf\") pod \"390df8c2-d64e-4f6b-9703-d75767fa6e29\" (UID: \"390df8c2-d64e-4f6b-9703-d75767fa6e29\") "
Jul 31 23:40:13.982939 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:13.982777    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/390df8c2-d64e-4f6b-9703-d75767fa6e29-openshift-service-ca\") pod \"390df8c2-d64e-4f6b-9703-d75767fa6e29\" (UID: \"390df8c2-d64e-4f6b-9703-d75767fa6e29\") "
Jul 31 23:40:13.982939 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:13.982833    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-56hsg\" (UniqueName: \"kubernetes.io/projected/68763429-4fb7-44fc-8cf0-07dc06d1f099-kube-api-access-56hsg\") pod \"68763429-4fb7-44fc-8cf0-07dc06d1f099\" (UID: \"68763429-4fb7-44fc-8cf0-07dc06d1f099\") "
Jul 31 23:40:13.982939 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:13.982863    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f4b8003-8661-480e-b795-61d107ea9a75-plugins-config-volume\") pod \"0f4b8003-8661-480e-b795-61d107ea9a75\" (UID: \"0f4b8003-8661-480e-b795-61d107ea9a75\") "
Jul 31 23:40:13.982939 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:13.982886    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/68763429-4fb7-44fc-8cf0-07dc06d1f099-plugins-config-volume\") pod \"68763429-4fb7-44fc-8cf0-07dc06d1f099\" (UID: \"68763429-4fb7-44fc-8cf0-07dc06d1f099\") "
Jul 31 23:40:13.983267 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:13.983190    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/390df8c2-d64e-4f6b-9703-d75767fa6e29-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "390df8c2-d64e-4f6b-9703-d75767fa6e29" (UID: "390df8c2-d64e-4f6b-9703-d75767fa6e29"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:40:13.983341 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:13.983314    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f4b8003-8661-480e-b795-61d107ea9a75-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0f4b8003-8661-480e-b795-61d107ea9a75" (UID: "0f4b8003-8661-480e-b795-61d107ea9a75"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:40:13.983381 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:13.983343    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68763429-4fb7-44fc-8cf0-07dc06d1f099-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "68763429-4fb7-44fc-8cf0-07dc06d1f099" (UID: "68763429-4fb7-44fc-8cf0-07dc06d1f099"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:40:13.985303 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:13.985277    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/390df8c2-d64e-4f6b-9703-d75767fa6e29-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "390df8c2-d64e-4f6b-9703-d75767fa6e29" (UID: "390df8c2-d64e-4f6b-9703-d75767fa6e29"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 23:40:13.985303 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:13.985287    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f4b8003-8661-480e-b795-61d107ea9a75-kube-api-access-qvhwt" (OuterVolumeSpecName: "kube-api-access-qvhwt") pod "0f4b8003-8661-480e-b795-61d107ea9a75" (UID: "0f4b8003-8661-480e-b795-61d107ea9a75"). InnerVolumeSpecName "kube-api-access-qvhwt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 23:40:13.985486 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:13.985466    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68763429-4fb7-44fc-8cf0-07dc06d1f099-kube-api-access-56hsg" (OuterVolumeSpecName: "kube-api-access-56hsg") pod "68763429-4fb7-44fc-8cf0-07dc06d1f099" (UID: "68763429-4fb7-44fc-8cf0-07dc06d1f099"). InnerVolumeSpecName "kube-api-access-56hsg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 23:40:13.985537 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:13.985483    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/390df8c2-d64e-4f6b-9703-d75767fa6e29-kube-api-access-dgwpf" (OuterVolumeSpecName: "kube-api-access-dgwpf") pod "390df8c2-d64e-4f6b-9703-d75767fa6e29" (UID: "390df8c2-d64e-4f6b-9703-d75767fa6e29"). InnerVolumeSpecName "kube-api-access-dgwpf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 23:40:14.084264 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:14.084228    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qvhwt\" (UniqueName: \"kubernetes.io/projected/0f4b8003-8661-480e-b795-61d107ea9a75-kube-api-access-qvhwt\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:40:14.084264 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:14.084262    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dgwpf\" (UniqueName: \"kubernetes.io/projected/390df8c2-d64e-4f6b-9703-d75767fa6e29-kube-api-access-dgwpf\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:40:14.084460 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:14.084271    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/390df8c2-d64e-4f6b-9703-d75767fa6e29-openshift-service-ca\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:40:14.084460 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:14.084279    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-56hsg\" (UniqueName: \"kubernetes.io/projected/68763429-4fb7-44fc-8cf0-07dc06d1f099-kube-api-access-56hsg\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:40:14.084460 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:14.084288    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f4b8003-8661-480e-b795-61d107ea9a75-plugins-config-volume\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:40:14.084460 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:14.084296    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/68763429-4fb7-44fc-8cf0-07dc06d1f099-plugins-config-volume\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:40:14.084460 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:14.084305    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/390df8c2-d64e-4f6b-9703-d75767fa6e29-maas-api-tls\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:40:14.814278 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:14.814245    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-3c3302-5b95f4f6f6-br6mb" event={"ID":"0f4b8003-8661-480e-b795-61d107ea9a75","Type":"ContainerDied","Data":"b2da33538900b04e19195a7951392e6844f946358bdcb223fd1f532d1ca80b46"}
Jul 31 23:40:14.814752 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:14.814304    2584 scope.go:117] "RemoveContainer" containerID="7e323d4c8ba3e07d651886e07eaf639ac50f50c3de88a1ff9af2efc548b22f3d"
Jul 31 23:40:14.814752 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:14.814249    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-3c3302-5b95f4f6f6-br6mb"
Jul 31 23:40:14.815586 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:14.815570    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3c3302-568f85dd-q2l8w"
Jul 31 23:40:14.815655 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:14.815602    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3c3302-568f85dd-q2l8w" event={"ID":"68763429-4fb7-44fc-8cf0-07dc06d1f099","Type":"ContainerDied","Data":"cd3a9ea306b0f92172316f9bc3ed2ee87db4239f29e2fbb70b6242c601d8bed7"}
Jul 31 23:40:14.816961 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:14.816942    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3c3302-7cc485d58b-gcfs5" event={"ID":"390df8c2-d64e-4f6b-9703-d75767fa6e29","Type":"ContainerDied","Data":"2735c3f1ff0de2f290d792ab939ca72b7657c71a8020174e4e4237e01af05987"}
Jul 31 23:40:14.817058 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:14.816950    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3c3302-7cc485d58b-gcfs5"
Jul 31 23:40:14.825392 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:14.825376    2584 scope.go:117] "RemoveContainer" containerID="b589ecb096d628c1a584b624bf5994ae2066bef9295ce026c068199732fe6569"
Jul 31 23:40:14.833779 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:14.833758    2584 scope.go:117] "RemoveContainer" containerID="9e7fb59349686c629e0e36527dfce1f76f76abd65d6d0b92203ce75a865b334d"
Jul 31 23:40:14.839063 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:14.839040    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-3c3302-5b95f4f6f6-br6mb"]
Jul 31 23:40:14.841653 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:14.841631    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-3c3302-5b95f4f6f6-br6mb"]
Jul 31 23:40:14.857607 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:14.857576    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-3c3302-568f85dd-q2l8w"]
Jul 31 23:40:14.862071 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:14.862050    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-3c3302-568f85dd-q2l8w"]
Jul 31 23:40:14.872191 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:14.872154    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3c3302-7cc485d58b-gcfs5"]
Jul 31 23:40:14.876080 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:14.876058    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3c3302-7cc485d58b-gcfs5"]
Jul 31 23:40:14.891427 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:14.891397    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f4b8003-8661-480e-b795-61d107ea9a75" path="/var/lib/kubelet/pods/0f4b8003-8661-480e-b795-61d107ea9a75/volumes"
Jul 31 23:40:14.891824 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:14.891809    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="390df8c2-d64e-4f6b-9703-d75767fa6e29" path="/var/lib/kubelet/pods/390df8c2-d64e-4f6b-9703-d75767fa6e29/volumes"
Jul 31 23:40:14.892146 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:14.892134    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68763429-4fb7-44fc-8cf0-07dc06d1f099" path="/var/lib/kubelet/pods/68763429-4fb7-44fc-8cf0-07dc06d1f099/volumes"
Jul 31 23:40:25.722027 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:25.721939    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-4734d6-6b9fdb7b4b-555pv"]
Jul 31 23:40:25.722472 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:25.722445    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68763429-4fb7-44fc-8cf0-07dc06d1f099" containerName="payload-pre-processing"
Jul 31 23:40:25.722472 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:25.722463    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="68763429-4fb7-44fc-8cf0-07dc06d1f099" containerName="payload-pre-processing"
Jul 31 23:40:25.722601 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:25.722482    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="390df8c2-d64e-4f6b-9703-d75767fa6e29" containerName="maas-api"
Jul 31 23:40:25.722601 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:25.722492    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="390df8c2-d64e-4f6b-9703-d75767fa6e29" containerName="maas-api"
Jul 31 23:40:25.722601 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:25.722510    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4f19fc4-3f03-41e2-8467-4cb8fb3c4a50" containerName="payload-pre-processing"
Jul 31 23:40:25.722601 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:25.722519    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4f19fc4-3f03-41e2-8467-4cb8fb3c4a50" containerName="payload-pre-processing"
Jul 31 23:40:25.722601 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:25.722531    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c841a60-7bfa-4149-aa1b-b7546e2cccd1" containerName="payload-processing"
Jul 31 23:40:25.722601 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:25.722543    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c841a60-7bfa-4149-aa1b-b7546e2cccd1" containerName="payload-processing"
Jul 31 23:40:25.722601 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:25.722556    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f4b8003-8661-480e-b795-61d107ea9a75" containerName="payload-processing"
Jul 31 23:40:25.722601 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:25.722565    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f4b8003-8661-480e-b795-61d107ea9a75" containerName="payload-processing"
Jul 31 23:40:25.722601 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:25.722580    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e95c8bf2-987c-4924-99df-4fda74671c2d" containerName="maas-api"
Jul 31 23:40:25.722601 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:25.722589    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="e95c8bf2-987c-4924-99df-4fda74671c2d" containerName="maas-api"
Jul 31 23:40:25.723103 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:25.722679    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="68763429-4fb7-44fc-8cf0-07dc06d1f099" containerName="payload-pre-processing"
Jul 31 23:40:25.723103 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:25.722693    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="e95c8bf2-987c-4924-99df-4fda74671c2d" containerName="maas-api"
Jul 31 23:40:25.723103 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:25.722735    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4f19fc4-3f03-41e2-8467-4cb8fb3c4a50" containerName="payload-pre-processing"
Jul 31 23:40:25.723103 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:25.722745    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f4b8003-8661-480e-b795-61d107ea9a75" containerName="payload-processing"
Jul 31 23:40:25.723103 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:25.722755    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="390df8c2-d64e-4f6b-9703-d75767fa6e29" containerName="maas-api"
Jul 31 23:40:25.723103 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:25.722765    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c841a60-7bfa-4149-aa1b-b7546e2cccd1" containerName="payload-processing"
Jul 31 23:40:25.727739 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:25.727716    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4734d6-6b9fdb7b4b-555pv"
Jul 31 23:40:25.730169 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:25.730138    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-4734d6-serving-cert\""
Jul 31 23:40:25.737601 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:25.737570    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-4734d6-6b9fdb7b4b-555pv"]
Jul 31 23:40:25.888045 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:25.888000    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f36049b-c544-4b5d-8372-5c567e1c28a1-openshift-service-ca\") pod \"maas-api-e2e-api-a-4734d6-6b9fdb7b4b-555pv\" (UID: \"6f36049b-c544-4b5d-8372-5c567e1c28a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4734d6-6b9fdb7b4b-555pv"
Jul 31 23:40:25.888237 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:25.888106    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f36049b-c544-4b5d-8372-5c567e1c28a1-maas-api-tls\") pod \"maas-api-e2e-api-a-4734d6-6b9fdb7b4b-555pv\" (UID: \"6f36049b-c544-4b5d-8372-5c567e1c28a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4734d6-6b9fdb7b4b-555pv"
Jul 31 23:40:25.888237 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:25.888154    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6k5qq\" (UniqueName: \"kubernetes.io/projected/6f36049b-c544-4b5d-8372-5c567e1c28a1-kube-api-access-6k5qq\") pod \"maas-api-e2e-api-a-4734d6-6b9fdb7b4b-555pv\" (UID: \"6f36049b-c544-4b5d-8372-5c567e1c28a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4734d6-6b9fdb7b4b-555pv"
Jul 31 23:40:25.989026 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:25.988934    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f36049b-c544-4b5d-8372-5c567e1c28a1-openshift-service-ca\") pod \"maas-api-e2e-api-a-4734d6-6b9fdb7b4b-555pv\" (UID: \"6f36049b-c544-4b5d-8372-5c567e1c28a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4734d6-6b9fdb7b4b-555pv"
Jul 31 23:40:25.989188 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:25.989032    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f36049b-c544-4b5d-8372-5c567e1c28a1-maas-api-tls\") pod \"maas-api-e2e-api-a-4734d6-6b9fdb7b4b-555pv\" (UID: \"6f36049b-c544-4b5d-8372-5c567e1c28a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4734d6-6b9fdb7b4b-555pv"
Jul 31 23:40:25.989188 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:25.989082    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6k5qq\" (UniqueName: \"kubernetes.io/projected/6f36049b-c544-4b5d-8372-5c567e1c28a1-kube-api-access-6k5qq\") pod \"maas-api-e2e-api-a-4734d6-6b9fdb7b4b-555pv\" (UID: \"6f36049b-c544-4b5d-8372-5c567e1c28a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4734d6-6b9fdb7b4b-555pv"
Jul 31 23:40:25.989317 ip-10-0-128-243 kubenswrapper[2584]: E0731 23:40:25.989179    2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-4734d6-serving-cert: secret "maas-api-e2e-api-a-4734d6-serving-cert" not found
Jul 31 23:40:25.989317 ip-10-0-128-243 kubenswrapper[2584]: E0731 23:40:25.989270    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f36049b-c544-4b5d-8372-5c567e1c28a1-maas-api-tls podName:6f36049b-c544-4b5d-8372-5c567e1c28a1 nodeName:}" failed. No retries permitted until 2026-07-31 23:40:26.489247619 +0000 UTC m=+2610.135033906 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6f36049b-c544-4b5d-8372-5c567e1c28a1-maas-api-tls") pod "maas-api-e2e-api-a-4734d6-6b9fdb7b4b-555pv" (UID: "6f36049b-c544-4b5d-8372-5c567e1c28a1") : secret "maas-api-e2e-api-a-4734d6-serving-cert" not found
Jul 31 23:40:25.989651 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:25.989631    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f36049b-c544-4b5d-8372-5c567e1c28a1-openshift-service-ca\") pod \"maas-api-e2e-api-a-4734d6-6b9fdb7b4b-555pv\" (UID: \"6f36049b-c544-4b5d-8372-5c567e1c28a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4734d6-6b9fdb7b4b-555pv"
Jul 31 23:40:25.997600 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:25.997574    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6k5qq\" (UniqueName: \"kubernetes.io/projected/6f36049b-c544-4b5d-8372-5c567e1c28a1-kube-api-access-6k5qq\") pod \"maas-api-e2e-api-a-4734d6-6b9fdb7b4b-555pv\" (UID: \"6f36049b-c544-4b5d-8372-5c567e1c28a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4734d6-6b9fdb7b4b-555pv"
Jul 31 23:40:26.424342 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:26.424298    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-4734d6-84f7cbdfc6-trsjc"]
Jul 31 23:40:26.427925 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:26.427901    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4734d6-84f7cbdfc6-trsjc"
Jul 31 23:40:26.430229 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:26.430209    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-4734d6\""
Jul 31 23:40:26.438335 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:26.438311    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-4734d6-84f7cbdfc6-trsjc"]
Jul 31 23:40:26.476912 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:26.476874    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-4734d6-d784db66d-tj2f8"]
Jul 31 23:40:26.480155 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:26.480138    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-4734d6-d784db66d-tj2f8"
Jul 31 23:40:26.482498 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:26.482474    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-4734d6-dockercfg-vvlv4\""
Jul 31 23:40:26.489636 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:26.489556    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-4734d6-d784db66d-tj2f8"]
Jul 31 23:40:26.494517 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:26.494493    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f36049b-c544-4b5d-8372-5c567e1c28a1-maas-api-tls\") pod \"maas-api-e2e-api-a-4734d6-6b9fdb7b4b-555pv\" (UID: \"6f36049b-c544-4b5d-8372-5c567e1c28a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4734d6-6b9fdb7b4b-555pv"
Jul 31 23:40:26.497140 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:26.497115    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f36049b-c544-4b5d-8372-5c567e1c28a1-maas-api-tls\") pod \"maas-api-e2e-api-a-4734d6-6b9fdb7b4b-555pv\" (UID: \"6f36049b-c544-4b5d-8372-5c567e1c28a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4734d6-6b9fdb7b4b-555pv"
Jul 31 23:40:26.595547 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:26.595512    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvhqg\" (UniqueName: \"kubernetes.io/projected/25d1e458-7d50-49cc-9a48-196a9d204686-kube-api-access-mvhqg\") pod \"payload-pre-processing-e2e-api-a-4734d6-84f7cbdfc6-trsjc\" (UID: \"25d1e458-7d50-49cc-9a48-196a9d204686\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-4734d6-84f7cbdfc6-trsjc"
Jul 31 23:40:26.595547 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:26.595557    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39e931f8-a795-4fa9-ab16-b5c8253b9fee-plugins-config-volume\") pod \"payload-processing-e2e-api-a-4734d6-d784db66d-tj2f8\" (UID: \"39e931f8-a795-4fa9-ab16-b5c8253b9fee\") " pod="openshift-ingress/payload-processing-e2e-api-a-4734d6-d784db66d-tj2f8"
Jul 31 23:40:26.595856 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:26.595613    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/25d1e458-7d50-49cc-9a48-196a9d204686-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-4734d6-84f7cbdfc6-trsjc\" (UID: \"25d1e458-7d50-49cc-9a48-196a9d204686\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-4734d6-84f7cbdfc6-trsjc"
Jul 31 23:40:26.595856 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:26.595731    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7rkb\" (UniqueName: \"kubernetes.io/projected/39e931f8-a795-4fa9-ab16-b5c8253b9fee-kube-api-access-d7rkb\") pod \"payload-processing-e2e-api-a-4734d6-d784db66d-tj2f8\" (UID: \"39e931f8-a795-4fa9-ab16-b5c8253b9fee\") " pod="openshift-ingress/payload-processing-e2e-api-a-4734d6-d784db66d-tj2f8"
Jul 31 23:40:26.639670 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:26.639634    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4734d6-6b9fdb7b4b-555pv"
Jul 31 23:40:26.698060 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:26.697038    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/25d1e458-7d50-49cc-9a48-196a9d204686-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-4734d6-84f7cbdfc6-trsjc\" (UID: \"25d1e458-7d50-49cc-9a48-196a9d204686\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-4734d6-84f7cbdfc6-trsjc"
Jul 31 23:40:26.698060 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:26.697091    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7rkb\" (UniqueName: \"kubernetes.io/projected/39e931f8-a795-4fa9-ab16-b5c8253b9fee-kube-api-access-d7rkb\") pod \"payload-processing-e2e-api-a-4734d6-d784db66d-tj2f8\" (UID: \"39e931f8-a795-4fa9-ab16-b5c8253b9fee\") " pod="openshift-ingress/payload-processing-e2e-api-a-4734d6-d784db66d-tj2f8"
Jul 31 23:40:26.698060 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:26.697172    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvhqg\" (UniqueName: \"kubernetes.io/projected/25d1e458-7d50-49cc-9a48-196a9d204686-kube-api-access-mvhqg\") pod \"payload-pre-processing-e2e-api-a-4734d6-84f7cbdfc6-trsjc\" (UID: \"25d1e458-7d50-49cc-9a48-196a9d204686\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-4734d6-84f7cbdfc6-trsjc"
Jul 31 23:40:26.698060 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:26.697208    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39e931f8-a795-4fa9-ab16-b5c8253b9fee-plugins-config-volume\") pod \"payload-processing-e2e-api-a-4734d6-d784db66d-tj2f8\" (UID: \"39e931f8-a795-4fa9-ab16-b5c8253b9fee\") " pod="openshift-ingress/payload-processing-e2e-api-a-4734d6-d784db66d-tj2f8"
Jul 31 23:40:26.698060 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:26.698009    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39e931f8-a795-4fa9-ab16-b5c8253b9fee-plugins-config-volume\") pod \"payload-processing-e2e-api-a-4734d6-d784db66d-tj2f8\" (UID: \"39e931f8-a795-4fa9-ab16-b5c8253b9fee\") " pod="openshift-ingress/payload-processing-e2e-api-a-4734d6-d784db66d-tj2f8"
Jul 31 23:40:26.698427 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:26.698326    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/25d1e458-7d50-49cc-9a48-196a9d204686-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-4734d6-84f7cbdfc6-trsjc\" (UID: \"25d1e458-7d50-49cc-9a48-196a9d204686\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-4734d6-84f7cbdfc6-trsjc"
Jul 31 23:40:26.707398 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:26.707371    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvhqg\" (UniqueName: \"kubernetes.io/projected/25d1e458-7d50-49cc-9a48-196a9d204686-kube-api-access-mvhqg\") pod \"payload-pre-processing-e2e-api-a-4734d6-84f7cbdfc6-trsjc\" (UID: \"25d1e458-7d50-49cc-9a48-196a9d204686\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-4734d6-84f7cbdfc6-trsjc"
Jul 31 23:40:26.707543 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:26.707472    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7rkb\" (UniqueName: \"kubernetes.io/projected/39e931f8-a795-4fa9-ab16-b5c8253b9fee-kube-api-access-d7rkb\") pod \"payload-processing-e2e-api-a-4734d6-d784db66d-tj2f8\" (UID: \"39e931f8-a795-4fa9-ab16-b5c8253b9fee\") " pod="openshift-ingress/payload-processing-e2e-api-a-4734d6-d784db66d-tj2f8"
Jul 31 23:40:26.741197 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:26.741167    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4734d6-84f7cbdfc6-trsjc"
Jul 31 23:40:26.779283 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:26.779253    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-4734d6-6b9fdb7b4b-555pv"]
Jul 31 23:40:26.780796 ip-10-0-128-243 kubenswrapper[2584]: W0731 23:40:26.780760    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f36049b_c544_4b5d_8372_5c567e1c28a1.slice/crio-21040871a2dcbad85110a2ddc8ad710be64e1dfa543f86d22798acfe47febb44 WatchSource:0}: Error finding container 21040871a2dcbad85110a2ddc8ad710be64e1dfa543f86d22798acfe47febb44: Status 404 returned error can't find the container with id 21040871a2dcbad85110a2ddc8ad710be64e1dfa543f86d22798acfe47febb44
Jul 31 23:40:26.782257 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:26.782237    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 31 23:40:26.792021 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:26.791992    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-4734d6-d784db66d-tj2f8"
Jul 31 23:40:26.882006 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:26.881948    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4734d6-6b9fdb7b4b-555pv" event={"ID":"6f36049b-c544-4b5d-8372-5c567e1c28a1","Type":"ContainerStarted","Data":"21040871a2dcbad85110a2ddc8ad710be64e1dfa543f86d22798acfe47febb44"}
Jul 31 23:40:26.893650 ip-10-0-128-243 kubenswrapper[2584]: W0731 23:40:26.893612    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25d1e458_7d50_49cc_9a48_196a9d204686.slice/crio-cb6c64b096b6102828cad3a4e3a3c342125438a8894751200399011d634a863e WatchSource:0}: Error finding container cb6c64b096b6102828cad3a4e3a3c342125438a8894751200399011d634a863e: Status 404 returned error can't find the container with id cb6c64b096b6102828cad3a4e3a3c342125438a8894751200399011d634a863e
Jul 31 23:40:26.894961 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:26.894929    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-4734d6-84f7cbdfc6-trsjc"]
Jul 31 23:40:26.943154 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:26.943127    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-4734d6-d784db66d-tj2f8"]
Jul 31 23:40:26.945783 ip-10-0-128-243 kubenswrapper[2584]: W0731 23:40:26.945760    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39e931f8_a795_4fa9_ab16_b5c8253b9fee.slice/crio-d65c8dff59ecfd6036e5b15084160eae19a19814463f7a32a96bc9d3714559ad WatchSource:0}: Error finding container d65c8dff59ecfd6036e5b15084160eae19a19814463f7a32a96bc9d3714559ad: Status 404 returned error can't find the container with id d65c8dff59ecfd6036e5b15084160eae19a19814463f7a32a96bc9d3714559ad
Jul 31 23:40:27.888149 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:27.888107    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-4734d6-d784db66d-tj2f8" event={"ID":"39e931f8-a795-4fa9-ab16-b5c8253b9fee","Type":"ContainerStarted","Data":"e86d74dbcbbc194ddc82c60a5456fe5a822e7675d80ce5e365218f693044a431"}
Jul 31 23:40:27.888540 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:27.888201    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-4734d6-d784db66d-tj2f8" event={"ID":"39e931f8-a795-4fa9-ab16-b5c8253b9fee","Type":"ContainerStarted","Data":"d65c8dff59ecfd6036e5b15084160eae19a19814463f7a32a96bc9d3714559ad"}
Jul 31 23:40:27.888540 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:27.888231    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-4734d6-d784db66d-tj2f8"
Jul 31 23:40:27.889513 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:27.889485    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4734d6-6b9fdb7b4b-555pv" event={"ID":"6f36049b-c544-4b5d-8372-5c567e1c28a1","Type":"ContainerStarted","Data":"1baadadf742c313badff9938122725e25843723bb6f82461565841213adaaeb6"}
Jul 31 23:40:27.889624 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:27.889591    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4734d6-6b9fdb7b4b-555pv"
Jul 31 23:40:27.890989 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:27.890965    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4734d6-84f7cbdfc6-trsjc" event={"ID":"25d1e458-7d50-49cc-9a48-196a9d204686","Type":"ContainerStarted","Data":"e356c6fd0e08a4772999bb515403ad40c6f81159f9383caa522e69581a9f90a6"}
Jul 31 23:40:27.891090 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:27.891004    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4734d6-84f7cbdfc6-trsjc" event={"ID":"25d1e458-7d50-49cc-9a48-196a9d204686","Type":"ContainerStarted","Data":"cb6c64b096b6102828cad3a4e3a3c342125438a8894751200399011d634a863e"}
Jul 31 23:40:27.891090 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:27.891056    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4734d6-84f7cbdfc6-trsjc"
Jul 31 23:40:27.906107 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:27.906065    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-4734d6-d784db66d-tj2f8" podStartSLOduration=1.906052418 podStartE2EDuration="1.906052418s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:40:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:40:27.904357788 +0000 UTC m=+2611.550144110" watchObservedRunningTime="2026-07-31 23:40:27.906052418 +0000 UTC m=+2611.551838705"
Jul 31 23:40:27.920804 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:27.920762    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4734d6-6b9fdb7b4b-555pv" podStartSLOduration=2.489017942 podStartE2EDuration="2.920749456s" totalImagesPullingTime="431.731514ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:40:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:40:27.919234182 +0000 UTC m=+2611.565020502" watchObservedRunningTime="2026-07-31 23:40:27.920749456 +0000 UTC m=+2611.566535764"
Jul 31 23:40:27.935181 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:27.935134    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4734d6-84f7cbdfc6-trsjc" podStartSLOduration=1.935119686 podStartE2EDuration="1.935119686s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:40:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:40:27.933723367 +0000 UTC m=+2611.579509674" watchObservedRunningTime="2026-07-31 23:40:27.935119686 +0000 UTC m=+2611.580905994"
Jul 31 23:40:33.900568 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:33.900528    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4734d6-6b9fdb7b4b-555pv"
Jul 31 23:40:38.897630 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:38.897603    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4734d6-84f7cbdfc6-trsjc"
Jul 31 23:40:38.898231 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:38.897676    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-4734d6-d784db66d-tj2f8"
Jul 31 23:40:53.878908 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:53.878876    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f7d39b-79b689d4d4-z9m6t"]
Jul 31 23:40:53.882447 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:53.882428    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f7d39b-79b689d4d4-z9m6t"
Jul 31 23:40:53.885688 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:53.885660    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-f7d39b-serving-cert\""
Jul 31 23:40:53.894436 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:53.894410    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f7d39b-79b689d4d4-z9m6t"]
Jul 31 23:40:53.940475 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:53.940438    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rs6fp\" (UniqueName: \"kubernetes.io/projected/1cef4c8e-16bc-4ea1-869c-c18075e8b953-kube-api-access-rs6fp\") pod \"maas-api-e2e-api-b-f7d39b-79b689d4d4-z9m6t\" (UID: \"1cef4c8e-16bc-4ea1-869c-c18075e8b953\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f7d39b-79b689d4d4-z9m6t"
Jul 31 23:40:53.940637 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:53.940494    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1cef4c8e-16bc-4ea1-869c-c18075e8b953-maas-api-tls\") pod \"maas-api-e2e-api-b-f7d39b-79b689d4d4-z9m6t\" (UID: \"1cef4c8e-16bc-4ea1-869c-c18075e8b953\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f7d39b-79b689d4d4-z9m6t"
Jul 31 23:40:53.940637 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:53.940549    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cef4c8e-16bc-4ea1-869c-c18075e8b953-openshift-service-ca\") pod \"maas-api-e2e-api-b-f7d39b-79b689d4d4-z9m6t\" (UID: \"1cef4c8e-16bc-4ea1-869c-c18075e8b953\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f7d39b-79b689d4d4-z9m6t"
Jul 31 23:40:54.041146 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:54.041112    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rs6fp\" (UniqueName: \"kubernetes.io/projected/1cef4c8e-16bc-4ea1-869c-c18075e8b953-kube-api-access-rs6fp\") pod \"maas-api-e2e-api-b-f7d39b-79b689d4d4-z9m6t\" (UID: \"1cef4c8e-16bc-4ea1-869c-c18075e8b953\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f7d39b-79b689d4d4-z9m6t"
Jul 31 23:40:54.041333 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:54.041170    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1cef4c8e-16bc-4ea1-869c-c18075e8b953-maas-api-tls\") pod \"maas-api-e2e-api-b-f7d39b-79b689d4d4-z9m6t\" (UID: \"1cef4c8e-16bc-4ea1-869c-c18075e8b953\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f7d39b-79b689d4d4-z9m6t"
Jul 31 23:40:54.041333 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:54.041204    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cef4c8e-16bc-4ea1-869c-c18075e8b953-openshift-service-ca\") pod \"maas-api-e2e-api-b-f7d39b-79b689d4d4-z9m6t\" (UID: \"1cef4c8e-16bc-4ea1-869c-c18075e8b953\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f7d39b-79b689d4d4-z9m6t"
Jul 31 23:40:54.041906 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:54.041885    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cef4c8e-16bc-4ea1-869c-c18075e8b953-openshift-service-ca\") pod \"maas-api-e2e-api-b-f7d39b-79b689d4d4-z9m6t\" (UID: \"1cef4c8e-16bc-4ea1-869c-c18075e8b953\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f7d39b-79b689d4d4-z9m6t"
Jul 31 23:40:54.043605 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:54.043585    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1cef4c8e-16bc-4ea1-869c-c18075e8b953-maas-api-tls\") pod \"maas-api-e2e-api-b-f7d39b-79b689d4d4-z9m6t\" (UID: \"1cef4c8e-16bc-4ea1-869c-c18075e8b953\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f7d39b-79b689d4d4-z9m6t"
Jul 31 23:40:54.048617 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:54.048600    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rs6fp\" (UniqueName: \"kubernetes.io/projected/1cef4c8e-16bc-4ea1-869c-c18075e8b953-kube-api-access-rs6fp\") pod \"maas-api-e2e-api-b-f7d39b-79b689d4d4-z9m6t\" (UID: \"1cef4c8e-16bc-4ea1-869c-c18075e8b953\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f7d39b-79b689d4d4-z9m6t"
Jul 31 23:40:54.194995 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:54.194899    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f7d39b-79b689d4d4-z9m6t"
Jul 31 23:40:54.331630 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:54.331598    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f7d39b-79b689d4d4-z9m6t"]
Jul 31 23:40:54.333766 ip-10-0-128-243 kubenswrapper[2584]: W0731 23:40:54.333677    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cef4c8e_16bc_4ea1_869c_c18075e8b953.slice/crio-96402da0aa80483cebb28075dd6adbdeac6a33b59ea14a2a49b24c9e7caaf217 WatchSource:0}: Error finding container 96402da0aa80483cebb28075dd6adbdeac6a33b59ea14a2a49b24c9e7caaf217: Status 404 returned error can't find the container with id 96402da0aa80483cebb28075dd6adbdeac6a33b59ea14a2a49b24c9e7caaf217
Jul 31 23:40:54.502787 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:54.502693    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-f7d39b-855748d956-7sd5q"]
Jul 31 23:40:54.507200 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:54.507182    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f7d39b-855748d956-7sd5q"
Jul 31 23:40:54.509727 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:54.509683    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-f7d39b-dockercfg-2h5xb\""
Jul 31 23:40:54.510147 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:54.510129    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-f7d39b\""
Jul 31 23:40:54.518379 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:54.518358    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-f7d39b-855748d956-7sd5q"]
Jul 31 23:40:54.531179 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:54.531153    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-f7d39b-588d44c784-mhd7p"]
Jul 31 23:40:54.534979 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:54.534952    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-f7d39b-588d44c784-mhd7p"
Jul 31 23:40:54.543059 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:54.542958    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-f7d39b-588d44c784-mhd7p"]
Jul 31 23:40:54.544633 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:54.544613    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmhjv\" (UniqueName: \"kubernetes.io/projected/3e92ca9f-6ca2-4252-8b9c-c323f42b5ed8-kube-api-access-vmhjv\") pod \"payload-pre-processing-e2e-api-b-f7d39b-855748d956-7sd5q\" (UID: \"3e92ca9f-6ca2-4252-8b9c-c323f42b5ed8\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-f7d39b-855748d956-7sd5q"
Jul 31 23:40:54.544773 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:54.544655    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e92ca9f-6ca2-4252-8b9c-c323f42b5ed8-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-f7d39b-855748d956-7sd5q\" (UID: \"3e92ca9f-6ca2-4252-8b9c-c323f42b5ed8\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-f7d39b-855748d956-7sd5q"
Jul 31 23:40:54.645629 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:54.645598    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vsjlj\" (UniqueName: \"kubernetes.io/projected/8a171bee-5577-46b0-bc5c-b4f4c917d848-kube-api-access-vsjlj\") pod \"payload-processing-e2e-api-b-f7d39b-588d44c784-mhd7p\" (UID: \"8a171bee-5577-46b0-bc5c-b4f4c917d848\") " pod="openshift-ingress/payload-processing-e2e-api-b-f7d39b-588d44c784-mhd7p"
Jul 31 23:40:54.645827 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:54.645633    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a171bee-5577-46b0-bc5c-b4f4c917d848-plugins-config-volume\") pod \"payload-processing-e2e-api-b-f7d39b-588d44c784-mhd7p\" (UID: \"8a171bee-5577-46b0-bc5c-b4f4c917d848\") " pod="openshift-ingress/payload-processing-e2e-api-b-f7d39b-588d44c784-mhd7p"
Jul 31 23:40:54.645827 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:54.645662    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmhjv\" (UniqueName: \"kubernetes.io/projected/3e92ca9f-6ca2-4252-8b9c-c323f42b5ed8-kube-api-access-vmhjv\") pod \"payload-pre-processing-e2e-api-b-f7d39b-855748d956-7sd5q\" (UID: \"3e92ca9f-6ca2-4252-8b9c-c323f42b5ed8\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-f7d39b-855748d956-7sd5q"
Jul 31 23:40:54.645827 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:54.645692    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e92ca9f-6ca2-4252-8b9c-c323f42b5ed8-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-f7d39b-855748d956-7sd5q\" (UID: \"3e92ca9f-6ca2-4252-8b9c-c323f42b5ed8\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-f7d39b-855748d956-7sd5q"
Jul 31 23:40:54.646393 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:54.646369    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e92ca9f-6ca2-4252-8b9c-c323f42b5ed8-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-f7d39b-855748d956-7sd5q\" (UID: \"3e92ca9f-6ca2-4252-8b9c-c323f42b5ed8\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-f7d39b-855748d956-7sd5q"
Jul 31 23:40:54.653590 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:54.653567    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmhjv\" (UniqueName: \"kubernetes.io/projected/3e92ca9f-6ca2-4252-8b9c-c323f42b5ed8-kube-api-access-vmhjv\") pod \"payload-pre-processing-e2e-api-b-f7d39b-855748d956-7sd5q\" (UID: \"3e92ca9f-6ca2-4252-8b9c-c323f42b5ed8\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-f7d39b-855748d956-7sd5q"
Jul 31 23:40:54.746610 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:54.746579    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsjlj\" (UniqueName: \"kubernetes.io/projected/8a171bee-5577-46b0-bc5c-b4f4c917d848-kube-api-access-vsjlj\") pod \"payload-processing-e2e-api-b-f7d39b-588d44c784-mhd7p\" (UID: \"8a171bee-5577-46b0-bc5c-b4f4c917d848\") " pod="openshift-ingress/payload-processing-e2e-api-b-f7d39b-588d44c784-mhd7p"
Jul 31 23:40:54.746740 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:54.746615    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a171bee-5577-46b0-bc5c-b4f4c917d848-plugins-config-volume\") pod \"payload-processing-e2e-api-b-f7d39b-588d44c784-mhd7p\" (UID: \"8a171bee-5577-46b0-bc5c-b4f4c917d848\") " pod="openshift-ingress/payload-processing-e2e-api-b-f7d39b-588d44c784-mhd7p"
Jul 31 23:40:54.747284 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:54.747266    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a171bee-5577-46b0-bc5c-b4f4c917d848-plugins-config-volume\") pod \"payload-processing-e2e-api-b-f7d39b-588d44c784-mhd7p\" (UID: \"8a171bee-5577-46b0-bc5c-b4f4c917d848\") " pod="openshift-ingress/payload-processing-e2e-api-b-f7d39b-588d44c784-mhd7p"
Jul 31 23:40:54.758070 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:54.758048    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vsjlj\" (UniqueName: \"kubernetes.io/projected/8a171bee-5577-46b0-bc5c-b4f4c917d848-kube-api-access-vsjlj\") pod \"payload-processing-e2e-api-b-f7d39b-588d44c784-mhd7p\" (UID: \"8a171bee-5577-46b0-bc5c-b4f4c917d848\") " pod="openshift-ingress/payload-processing-e2e-api-b-f7d39b-588d44c784-mhd7p"
Jul 31 23:40:54.818383 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:54.818355    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f7d39b-855748d956-7sd5q"
Jul 31 23:40:54.848450 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:54.848419    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-f7d39b-588d44c784-mhd7p"
Jul 31 23:40:54.971939 ip-10-0-128-243 kubenswrapper[2584]: W0731 23:40:54.971878    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e92ca9f_6ca2_4252_8b9c_c323f42b5ed8.slice/crio-f0bf258826840d967e52588fef8e6d6fdefc096e6888809047aab10c0741c7dd WatchSource:0}: Error finding container f0bf258826840d967e52588fef8e6d6fdefc096e6888809047aab10c0741c7dd: Status 404 returned error can't find the container with id f0bf258826840d967e52588fef8e6d6fdefc096e6888809047aab10c0741c7dd
Jul 31 23:40:54.972450 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:54.972063    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-f7d39b-855748d956-7sd5q"]
Jul 31 23:40:55.025201 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:55.025171    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f7d39b-79b689d4d4-z9m6t" event={"ID":"1cef4c8e-16bc-4ea1-869c-c18075e8b953","Type":"ContainerStarted","Data":"dba888ae81927b244e136e8b2e9bbba00fcea44c60ee2b7f1e0d876dc8f8e556"}
Jul 31 23:40:55.025331 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:55.025233    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f7d39b-79b689d4d4-z9m6t" event={"ID":"1cef4c8e-16bc-4ea1-869c-c18075e8b953","Type":"ContainerStarted","Data":"96402da0aa80483cebb28075dd6adbdeac6a33b59ea14a2a49b24c9e7caaf217"}
Jul 31 23:40:55.025331 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:55.025264    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f7d39b-79b689d4d4-z9m6t"
Jul 31 23:40:55.026383 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:55.026355    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f7d39b-855748d956-7sd5q" event={"ID":"3e92ca9f-6ca2-4252-8b9c-c323f42b5ed8","Type":"ContainerStarted","Data":"f0bf258826840d967e52588fef8e6d6fdefc096e6888809047aab10c0741c7dd"}
Jul 31 23:40:55.044779 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:55.044738    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f7d39b-79b689d4d4-z9m6t" podStartSLOduration=1.67156471 podStartE2EDuration="2.044723108s" totalImagesPullingTime="373.158398ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:40:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:40:55.043484661 +0000 UTC m=+2638.689270973" watchObservedRunningTime="2026-07-31 23:40:55.044723108 +0000 UTC m=+2638.690509419"
Jul 31 23:40:55.210163 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:55.210137    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-f7d39b-588d44c784-mhd7p"]
Jul 31 23:40:55.212388 ip-10-0-128-243 kubenswrapper[2584]: W0731 23:40:55.212344    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a171bee_5577_46b0_bc5c_b4f4c917d848.slice/crio-a031d3697ceef618c6fea75b3daf4c74fb66dc80e2c8495f6bb96c02623478a5 WatchSource:0}: Error finding container a031d3697ceef618c6fea75b3daf4c74fb66dc80e2c8495f6bb96c02623478a5: Status 404 returned error can't find the container with id a031d3697ceef618c6fea75b3daf4c74fb66dc80e2c8495f6bb96c02623478a5
Jul 31 23:40:56.036260 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:56.036211    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f7d39b-855748d956-7sd5q" event={"ID":"3e92ca9f-6ca2-4252-8b9c-c323f42b5ed8","Type":"ContainerStarted","Data":"6c8b7ee6aad76ef561ad49df916e7a54633590fca42d6230f2ca2812753011cd"}
Jul 31 23:40:56.036739 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:56.036312    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f7d39b-855748d956-7sd5q"
Jul 31 23:40:56.038039 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:56.038007    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-f7d39b-588d44c784-mhd7p" event={"ID":"8a171bee-5577-46b0-bc5c-b4f4c917d848","Type":"ContainerStarted","Data":"bcefb3d4b26fc7e3b2a68703572db736c7d4e680881270c0cf5fcf5ba2bed112"}
Jul 31 23:40:56.038174 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:56.038062    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-f7d39b-588d44c784-mhd7p" event={"ID":"8a171bee-5577-46b0-bc5c-b4f4c917d848","Type":"ContainerStarted","Data":"a031d3697ceef618c6fea75b3daf4c74fb66dc80e2c8495f6bb96c02623478a5"}
Jul 31 23:40:56.052802 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:56.052753    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f7d39b-855748d956-7sd5q" podStartSLOduration=2.052739672 podStartE2EDuration="2.052739672s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:40:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:40:56.051513712 +0000 UTC m=+2639.697300021" watchObservedRunningTime="2026-07-31 23:40:56.052739672 +0000 UTC m=+2639.698525977"
Jul 31 23:40:56.067205 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:56.067167    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-f7d39b-588d44c784-mhd7p" podStartSLOduration=2.067153755 podStartE2EDuration="2.067153755s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:40:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:40:56.065435642 +0000 UTC m=+2639.711221952" watchObservedRunningTime="2026-07-31 23:40:56.067153755 +0000 UTC m=+2639.712940063"
Jul 31 23:40:57.043401 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:40:57.043369    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-f7d39b-588d44c784-mhd7p"
Jul 31 23:41:01.043290 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:01.043260    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f7d39b-79b689d4d4-z9m6t"
Jul 31 23:41:07.046155 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:07.046125    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f7d39b-855748d956-7sd5q"
Jul 31 23:41:08.050930 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:08.050897    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-f7d39b-588d44c784-mhd7p"
Jul 31 23:41:35.965095 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:35.965057    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-4734d6-6b9fdb7b4b-555pv"]
Jul 31 23:41:35.965755 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:35.965305    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4734d6-6b9fdb7b4b-555pv" podUID="6f36049b-c544-4b5d-8372-5c567e1c28a1" containerName="maas-api" containerID="cri-o://1baadadf742c313badff9938122725e25843723bb6f82461565841213adaaeb6" gracePeriod=30
Jul 31 23:41:36.016623 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:36.016594    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-4734d6-d784db66d-tj2f8"]
Jul 31 23:41:36.016875 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:36.016849    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-4734d6-d784db66d-tj2f8" podUID="39e931f8-a795-4fa9-ab16-b5c8253b9fee" containerName="payload-processing" containerID="cri-o://e86d74dbcbbc194ddc82c60a5456fe5a822e7675d80ce5e365218f693044a431" gracePeriod=30
Jul 31 23:41:36.024562 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:36.024531    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-4734d6-84f7cbdfc6-trsjc"]
Jul 31 23:41:36.024870 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:36.024819    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4734d6-84f7cbdfc6-trsjc" podUID="25d1e458-7d50-49cc-9a48-196a9d204686" containerName="payload-pre-processing" containerID="cri-o://e356c6fd0e08a4772999bb515403ad40c6f81159f9383caa522e69581a9f90a6" gracePeriod=30
Jul 31 23:41:36.233332 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:36.233110    2584 generic.go:356] "Generic (PLEG): container finished" podID="39e931f8-a795-4fa9-ab16-b5c8253b9fee" containerID="e86d74dbcbbc194ddc82c60a5456fe5a822e7675d80ce5e365218f693044a431" exitCode=0
Jul 31 23:41:36.233332 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:36.233215    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-4734d6-d784db66d-tj2f8" event={"ID":"39e931f8-a795-4fa9-ab16-b5c8253b9fee","Type":"ContainerDied","Data":"e86d74dbcbbc194ddc82c60a5456fe5a822e7675d80ce5e365218f693044a431"}
Jul 31 23:41:36.235319 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:36.235296    2584 generic.go:356] "Generic (PLEG): container finished" podID="6f36049b-c544-4b5d-8372-5c567e1c28a1" containerID="1baadadf742c313badff9938122725e25843723bb6f82461565841213adaaeb6" exitCode=0
Jul 31 23:41:36.235442 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:36.235399    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4734d6-6b9fdb7b4b-555pv" event={"ID":"6f36049b-c544-4b5d-8372-5c567e1c28a1","Type":"ContainerDied","Data":"1baadadf742c313badff9938122725e25843723bb6f82461565841213adaaeb6"}
Jul 31 23:41:36.237070 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:36.237042    2584 generic.go:356] "Generic (PLEG): container finished" podID="25d1e458-7d50-49cc-9a48-196a9d204686" containerID="e356c6fd0e08a4772999bb515403ad40c6f81159f9383caa522e69581a9f90a6" exitCode=0
Jul 31 23:41:36.237170 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:36.237065    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4734d6-84f7cbdfc6-trsjc" event={"ID":"25d1e458-7d50-49cc-9a48-196a9d204686","Type":"ContainerDied","Data":"e356c6fd0e08a4772999bb515403ad40c6f81159f9383caa522e69581a9f90a6"}
Jul 31 23:41:36.288325 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:36.288298    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-4734d6-d784db66d-tj2f8"
Jul 31 23:41:36.334469 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:36.334217    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39e931f8-a795-4fa9-ab16-b5c8253b9fee-plugins-config-volume\") pod \"39e931f8-a795-4fa9-ab16-b5c8253b9fee\" (UID: \"39e931f8-a795-4fa9-ab16-b5c8253b9fee\") "
Jul 31 23:41:36.334469 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:36.334267    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4734d6-84f7cbdfc6-trsjc"
Jul 31 23:41:36.334469 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:36.334330    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d7rkb\" (UniqueName: \"kubernetes.io/projected/39e931f8-a795-4fa9-ab16-b5c8253b9fee-kube-api-access-d7rkb\") pod \"39e931f8-a795-4fa9-ab16-b5c8253b9fee\" (UID: \"39e931f8-a795-4fa9-ab16-b5c8253b9fee\") "
Jul 31 23:41:36.334752 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:36.334612    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39e931f8-a795-4fa9-ab16-b5c8253b9fee-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "39e931f8-a795-4fa9-ab16-b5c8253b9fee" (UID: "39e931f8-a795-4fa9-ab16-b5c8253b9fee"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:41:36.336664 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:36.336635    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39e931f8-a795-4fa9-ab16-b5c8253b9fee-kube-api-access-d7rkb" (OuterVolumeSpecName: "kube-api-access-d7rkb") pod "39e931f8-a795-4fa9-ab16-b5c8253b9fee" (UID: "39e931f8-a795-4fa9-ab16-b5c8253b9fee"). InnerVolumeSpecName "kube-api-access-d7rkb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 23:41:36.338103 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:36.338081    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4734d6-6b9fdb7b4b-555pv"
Jul 31 23:41:36.435189 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:36.435161    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mvhqg\" (UniqueName: \"kubernetes.io/projected/25d1e458-7d50-49cc-9a48-196a9d204686-kube-api-access-mvhqg\") pod \"25d1e458-7d50-49cc-9a48-196a9d204686\" (UID: \"25d1e458-7d50-49cc-9a48-196a9d204686\") "
Jul 31 23:41:36.435377 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:36.435197    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f36049b-c544-4b5d-8372-5c567e1c28a1-openshift-service-ca\") pod \"6f36049b-c544-4b5d-8372-5c567e1c28a1\" (UID: \"6f36049b-c544-4b5d-8372-5c567e1c28a1\") "
Jul 31 23:41:36.435377 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:36.435244    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6k5qq\" (UniqueName: \"kubernetes.io/projected/6f36049b-c544-4b5d-8372-5c567e1c28a1-kube-api-access-6k5qq\") pod \"6f36049b-c544-4b5d-8372-5c567e1c28a1\" (UID: \"6f36049b-c544-4b5d-8372-5c567e1c28a1\") "
Jul 31 23:41:36.435377 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:36.435323    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f36049b-c544-4b5d-8372-5c567e1c28a1-maas-api-tls\") pod \"6f36049b-c544-4b5d-8372-5c567e1c28a1\" (UID: \"6f36049b-c544-4b5d-8372-5c567e1c28a1\") "
Jul 31 23:41:36.435377 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:36.435349    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/25d1e458-7d50-49cc-9a48-196a9d204686-plugins-config-volume\") pod \"25d1e458-7d50-49cc-9a48-196a9d204686\" (UID: \"25d1e458-7d50-49cc-9a48-196a9d204686\") "
Jul 31 23:41:36.435606 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:36.435569    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39e931f8-a795-4fa9-ab16-b5c8253b9fee-plugins-config-volume\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:41:36.435606 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:36.435587    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d7rkb\" (UniqueName: \"kubernetes.io/projected/39e931f8-a795-4fa9-ab16-b5c8253b9fee-kube-api-access-d7rkb\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:41:36.435606 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:36.435591    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f36049b-c544-4b5d-8372-5c567e1c28a1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6f36049b-c544-4b5d-8372-5c567e1c28a1" (UID: "6f36049b-c544-4b5d-8372-5c567e1c28a1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:41:36.435852 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:36.435828    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25d1e458-7d50-49cc-9a48-196a9d204686-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "25d1e458-7d50-49cc-9a48-196a9d204686" (UID: "25d1e458-7d50-49cc-9a48-196a9d204686"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:41:36.437432 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:36.437405    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f36049b-c544-4b5d-8372-5c567e1c28a1-kube-api-access-6k5qq" (OuterVolumeSpecName: "kube-api-access-6k5qq") pod "6f36049b-c544-4b5d-8372-5c567e1c28a1" (UID: "6f36049b-c544-4b5d-8372-5c567e1c28a1"). InnerVolumeSpecName "kube-api-access-6k5qq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 23:41:36.437540 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:36.437450    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25d1e458-7d50-49cc-9a48-196a9d204686-kube-api-access-mvhqg" (OuterVolumeSpecName: "kube-api-access-mvhqg") pod "25d1e458-7d50-49cc-9a48-196a9d204686" (UID: "25d1e458-7d50-49cc-9a48-196a9d204686"). InnerVolumeSpecName "kube-api-access-mvhqg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 23:41:36.437540 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:36.437456    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f36049b-c544-4b5d-8372-5c567e1c28a1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6f36049b-c544-4b5d-8372-5c567e1c28a1" (UID: "6f36049b-c544-4b5d-8372-5c567e1c28a1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 23:41:36.536556 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:36.536480    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mvhqg\" (UniqueName: \"kubernetes.io/projected/25d1e458-7d50-49cc-9a48-196a9d204686-kube-api-access-mvhqg\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:41:36.536556 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:36.536516    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f36049b-c544-4b5d-8372-5c567e1c28a1-openshift-service-ca\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:41:36.536556 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:36.536531    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6k5qq\" (UniqueName: \"kubernetes.io/projected/6f36049b-c544-4b5d-8372-5c567e1c28a1-kube-api-access-6k5qq\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:41:36.536556 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:36.536545    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f36049b-c544-4b5d-8372-5c567e1c28a1-maas-api-tls\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:41:36.536821 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:36.536559    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/25d1e458-7d50-49cc-9a48-196a9d204686-plugins-config-volume\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:41:37.243600 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:37.243514    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-4734d6-d784db66d-tj2f8"
Jul 31 23:41:37.244050 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:37.243517    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-4734d6-d784db66d-tj2f8" event={"ID":"39e931f8-a795-4fa9-ab16-b5c8253b9fee","Type":"ContainerDied","Data":"d65c8dff59ecfd6036e5b15084160eae19a19814463f7a32a96bc9d3714559ad"}
Jul 31 23:41:37.244050 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:37.243673    2584 scope.go:117] "RemoveContainer" containerID="e86d74dbcbbc194ddc82c60a5456fe5a822e7675d80ce5e365218f693044a431"
Jul 31 23:41:37.245055 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:37.245037    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4734d6-6b9fdb7b4b-555pv"
Jul 31 23:41:37.245225 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:37.245052    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4734d6-6b9fdb7b4b-555pv" event={"ID":"6f36049b-c544-4b5d-8372-5c567e1c28a1","Type":"ContainerDied","Data":"21040871a2dcbad85110a2ddc8ad710be64e1dfa543f86d22798acfe47febb44"}
Jul 31 23:41:37.246626 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:37.246569    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4734d6-84f7cbdfc6-trsjc"
Jul 31 23:41:37.246626 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:37.246602    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4734d6-84f7cbdfc6-trsjc" event={"ID":"25d1e458-7d50-49cc-9a48-196a9d204686","Type":"ContainerDied","Data":"cb6c64b096b6102828cad3a4e3a3c342125438a8894751200399011d634a863e"}
Jul 31 23:41:37.253317 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:37.253300    2584 scope.go:117] "RemoveContainer" containerID="1baadadf742c313badff9938122725e25843723bb6f82461565841213adaaeb6"
Jul 31 23:41:37.260882 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:37.260866    2584 scope.go:117] "RemoveContainer" containerID="e356c6fd0e08a4772999bb515403ad40c6f81159f9383caa522e69581a9f90a6"
Jul 31 23:41:37.267047 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:37.267022    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-4734d6-84f7cbdfc6-trsjc"]
Jul 31 23:41:37.269057 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:37.269040    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-4734d6-84f7cbdfc6-trsjc"]
Jul 31 23:41:37.277493 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:37.277473    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-4734d6-6b9fdb7b4b-555pv"]
Jul 31 23:41:37.280973 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:37.280952    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-4734d6-6b9fdb7b4b-555pv"]
Jul 31 23:41:37.289967 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:37.289935    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-4734d6-d784db66d-tj2f8"]
Jul 31 23:41:37.297675 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:37.293309    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-4734d6-d784db66d-tj2f8"]
Jul 31 23:41:38.891309 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:38.891275    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="25d1e458-7d50-49cc-9a48-196a9d204686" path="/var/lib/kubelet/pods/25d1e458-7d50-49cc-9a48-196a9d204686/volumes"
Jul 31 23:41:38.891849 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:38.891828    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39e931f8-a795-4fa9-ab16-b5c8253b9fee" path="/var/lib/kubelet/pods/39e931f8-a795-4fa9-ab16-b5c8253b9fee/volumes"
Jul 31 23:41:38.892260 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:38.892242    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f36049b-c544-4b5d-8372-5c567e1c28a1" path="/var/lib/kubelet/pods/6f36049b-c544-4b5d-8372-5c567e1c28a1/volumes"
Jul 31 23:41:58.169145 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:58.169109    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f7d39b-79b689d4d4-z9m6t"]
Jul 31 23:41:58.169647 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:58.169405    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f7d39b-79b689d4d4-z9m6t" podUID="1cef4c8e-16bc-4ea1-869c-c18075e8b953" containerName="maas-api" containerID="cri-o://dba888ae81927b244e136e8b2e9bbba00fcea44c60ee2b7f1e0d876dc8f8e556" gracePeriod=30
Jul 31 23:41:58.218423 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:58.218388    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-f7d39b-588d44c784-mhd7p"]
Jul 31 23:41:58.218761 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:58.218664    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-f7d39b-588d44c784-mhd7p" podUID="8a171bee-5577-46b0-bc5c-b4f4c917d848" containerName="payload-processing" containerID="cri-o://bcefb3d4b26fc7e3b2a68703572db736c7d4e680881270c0cf5fcf5ba2bed112" gracePeriod=30
Jul 31 23:41:58.223572 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:58.223538    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-f7d39b-855748d956-7sd5q"]
Jul 31 23:41:58.223950 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:58.223770    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f7d39b-855748d956-7sd5q" podUID="3e92ca9f-6ca2-4252-8b9c-c323f42b5ed8" containerName="payload-pre-processing" containerID="cri-o://6c8b7ee6aad76ef561ad49df916e7a54633590fca42d6230f2ca2812753011cd" gracePeriod=30
Jul 31 23:41:58.353227 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:58.353111    2584 generic.go:356] "Generic (PLEG): container finished" podID="3e92ca9f-6ca2-4252-8b9c-c323f42b5ed8" containerID="6c8b7ee6aad76ef561ad49df916e7a54633590fca42d6230f2ca2812753011cd" exitCode=0
Jul 31 23:41:58.353227 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:58.353172    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f7d39b-855748d956-7sd5q" event={"ID":"3e92ca9f-6ca2-4252-8b9c-c323f42b5ed8","Type":"ContainerDied","Data":"6c8b7ee6aad76ef561ad49df916e7a54633590fca42d6230f2ca2812753011cd"}
Jul 31 23:41:58.355828 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:58.355794    2584 generic.go:356] "Generic (PLEG): container finished" podID="1cef4c8e-16bc-4ea1-869c-c18075e8b953" containerID="dba888ae81927b244e136e8b2e9bbba00fcea44c60ee2b7f1e0d876dc8f8e556" exitCode=0
Jul 31 23:41:58.355828 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:58.355817    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f7d39b-79b689d4d4-z9m6t" event={"ID":"1cef4c8e-16bc-4ea1-869c-c18075e8b953","Type":"ContainerDied","Data":"dba888ae81927b244e136e8b2e9bbba00fcea44c60ee2b7f1e0d876dc8f8e556"}
Jul 31 23:41:58.357685 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:58.357653    2584 generic.go:356] "Generic (PLEG): container finished" podID="8a171bee-5577-46b0-bc5c-b4f4c917d848" containerID="bcefb3d4b26fc7e3b2a68703572db736c7d4e680881270c0cf5fcf5ba2bed112" exitCode=0
Jul 31 23:41:58.357841 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:58.357738    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-f7d39b-588d44c784-mhd7p" event={"ID":"8a171bee-5577-46b0-bc5c-b4f4c917d848","Type":"ContainerDied","Data":"bcefb3d4b26fc7e3b2a68703572db736c7d4e680881270c0cf5fcf5ba2bed112"}
Jul 31 23:41:58.484242 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:58.484216    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-f7d39b-588d44c784-mhd7p"
Jul 31 23:41:58.496724 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:58.496688    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f7d39b-79b689d4d4-z9m6t"
Jul 31 23:41:58.533593 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:58.533566    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f7d39b-855748d956-7sd5q"
Jul 31 23:41:58.617493 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:58.617460    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vmhjv\" (UniqueName: \"kubernetes.io/projected/3e92ca9f-6ca2-4252-8b9c-c323f42b5ed8-kube-api-access-vmhjv\") pod \"3e92ca9f-6ca2-4252-8b9c-c323f42b5ed8\" (UID: \"3e92ca9f-6ca2-4252-8b9c-c323f42b5ed8\") "
Jul 31 23:41:58.617493 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:58.617496    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a171bee-5577-46b0-bc5c-b4f4c917d848-plugins-config-volume\") pod \"8a171bee-5577-46b0-bc5c-b4f4c917d848\" (UID: \"8a171bee-5577-46b0-bc5c-b4f4c917d848\") "
Jul 31 23:41:58.617766 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:58.617517    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vsjlj\" (UniqueName: \"kubernetes.io/projected/8a171bee-5577-46b0-bc5c-b4f4c917d848-kube-api-access-vsjlj\") pod \"8a171bee-5577-46b0-bc5c-b4f4c917d848\" (UID: \"8a171bee-5577-46b0-bc5c-b4f4c917d848\") "
Jul 31 23:41:58.617766 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:58.617558    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1cef4c8e-16bc-4ea1-869c-c18075e8b953-maas-api-tls\") pod \"1cef4c8e-16bc-4ea1-869c-c18075e8b953\" (UID: \"1cef4c8e-16bc-4ea1-869c-c18075e8b953\") "
Jul 31 23:41:58.617766 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:58.617574    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cef4c8e-16bc-4ea1-869c-c18075e8b953-openshift-service-ca\") pod \"1cef4c8e-16bc-4ea1-869c-c18075e8b953\" (UID: \"1cef4c8e-16bc-4ea1-869c-c18075e8b953\") "
Jul 31 23:41:58.617766 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:58.617620    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rs6fp\" (UniqueName: \"kubernetes.io/projected/1cef4c8e-16bc-4ea1-869c-c18075e8b953-kube-api-access-rs6fp\") pod \"1cef4c8e-16bc-4ea1-869c-c18075e8b953\" (UID: \"1cef4c8e-16bc-4ea1-869c-c18075e8b953\") "
Jul 31 23:41:58.617766 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:58.617691    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e92ca9f-6ca2-4252-8b9c-c323f42b5ed8-plugins-config-volume\") pod \"3e92ca9f-6ca2-4252-8b9c-c323f42b5ed8\" (UID: \"3e92ca9f-6ca2-4252-8b9c-c323f42b5ed8\") "
Jul 31 23:41:58.618089 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:58.618057    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1cef4c8e-16bc-4ea1-869c-c18075e8b953-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1cef4c8e-16bc-4ea1-869c-c18075e8b953" (UID: "1cef4c8e-16bc-4ea1-869c-c18075e8b953"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:41:58.618355 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:58.618180    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e92ca9f-6ca2-4252-8b9c-c323f42b5ed8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3e92ca9f-6ca2-4252-8b9c-c323f42b5ed8" (UID: "3e92ca9f-6ca2-4252-8b9c-c323f42b5ed8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:41:58.618489 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:58.618393    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a171bee-5577-46b0-bc5c-b4f4c917d848-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8a171bee-5577-46b0-bc5c-b4f4c917d848" (UID: "8a171bee-5577-46b0-bc5c-b4f4c917d848"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:41:58.619799 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:58.619770    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a171bee-5577-46b0-bc5c-b4f4c917d848-kube-api-access-vsjlj" (OuterVolumeSpecName: "kube-api-access-vsjlj") pod "8a171bee-5577-46b0-bc5c-b4f4c917d848" (UID: "8a171bee-5577-46b0-bc5c-b4f4c917d848"). InnerVolumeSpecName "kube-api-access-vsjlj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 23:41:58.619893 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:58.619808    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e92ca9f-6ca2-4252-8b9c-c323f42b5ed8-kube-api-access-vmhjv" (OuterVolumeSpecName: "kube-api-access-vmhjv") pod "3e92ca9f-6ca2-4252-8b9c-c323f42b5ed8" (UID: "3e92ca9f-6ca2-4252-8b9c-c323f42b5ed8"). InnerVolumeSpecName "kube-api-access-vmhjv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 23:41:58.620069 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:58.620053    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1cef4c8e-16bc-4ea1-869c-c18075e8b953-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1cef4c8e-16bc-4ea1-869c-c18075e8b953" (UID: "1cef4c8e-16bc-4ea1-869c-c18075e8b953"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 23:41:58.620125 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:58.620074    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1cef4c8e-16bc-4ea1-869c-c18075e8b953-kube-api-access-rs6fp" (OuterVolumeSpecName: "kube-api-access-rs6fp") pod "1cef4c8e-16bc-4ea1-869c-c18075e8b953" (UID: "1cef4c8e-16bc-4ea1-869c-c18075e8b953"). InnerVolumeSpecName "kube-api-access-rs6fp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 23:41:58.718676 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:58.718592    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vmhjv\" (UniqueName: \"kubernetes.io/projected/3e92ca9f-6ca2-4252-8b9c-c323f42b5ed8-kube-api-access-vmhjv\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:41:58.718676 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:58.718622    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a171bee-5577-46b0-bc5c-b4f4c917d848-plugins-config-volume\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:41:58.718676 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:58.718631    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vsjlj\" (UniqueName: \"kubernetes.io/projected/8a171bee-5577-46b0-bc5c-b4f4c917d848-kube-api-access-vsjlj\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:41:58.718676 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:58.718665    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1cef4c8e-16bc-4ea1-869c-c18075e8b953-maas-api-tls\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:41:58.718676 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:58.718674    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cef4c8e-16bc-4ea1-869c-c18075e8b953-openshift-service-ca\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:41:58.718676 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:58.718682    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rs6fp\" (UniqueName: \"kubernetes.io/projected/1cef4c8e-16bc-4ea1-869c-c18075e8b953-kube-api-access-rs6fp\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:41:58.719007 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:58.718691    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e92ca9f-6ca2-4252-8b9c-c323f42b5ed8-plugins-config-volume\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:41:59.363873 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:59.363848    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f7d39b-855748d956-7sd5q"
Jul 31 23:41:59.364311 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:59.363879    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f7d39b-855748d956-7sd5q" event={"ID":"3e92ca9f-6ca2-4252-8b9c-c323f42b5ed8","Type":"ContainerDied","Data":"f0bf258826840d967e52588fef8e6d6fdefc096e6888809047aab10c0741c7dd"}
Jul 31 23:41:59.364311 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:59.363950    2584 scope.go:117] "RemoveContainer" containerID="6c8b7ee6aad76ef561ad49df916e7a54633590fca42d6230f2ca2812753011cd"
Jul 31 23:41:59.365286 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:59.365268    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f7d39b-79b689d4d4-z9m6t"
Jul 31 23:41:59.365366 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:59.365279    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f7d39b-79b689d4d4-z9m6t" event={"ID":"1cef4c8e-16bc-4ea1-869c-c18075e8b953","Type":"ContainerDied","Data":"96402da0aa80483cebb28075dd6adbdeac6a33b59ea14a2a49b24c9e7caaf217"}
Jul 31 23:41:59.366857 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:59.366827    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-f7d39b-588d44c784-mhd7p"
Jul 31 23:41:59.366857 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:59.366849    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-f7d39b-588d44c784-mhd7p" event={"ID":"8a171bee-5577-46b0-bc5c-b4f4c917d848","Type":"ContainerDied","Data":"a031d3697ceef618c6fea75b3daf4c74fb66dc80e2c8495f6bb96c02623478a5"}
Jul 31 23:41:59.373755 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:59.373737    2584 scope.go:117] "RemoveContainer" containerID="dba888ae81927b244e136e8b2e9bbba00fcea44c60ee2b7f1e0d876dc8f8e556"
Jul 31 23:41:59.381288 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:59.381271    2584 scope.go:117] "RemoveContainer" containerID="bcefb3d4b26fc7e3b2a68703572db736c7d4e680881270c0cf5fcf5ba2bed112"
Jul 31 23:41:59.386674 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:59.386647    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-f7d39b-855748d956-7sd5q"]
Jul 31 23:41:59.390749 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:59.390727    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-f7d39b-855748d956-7sd5q"]
Jul 31 23:41:59.404028 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:59.403996    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f7d39b-79b689d4d4-z9m6t"]
Jul 31 23:41:59.407948 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:59.407920    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f7d39b-79b689d4d4-z9m6t"]
Jul 31 23:41:59.417274 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:59.417249    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-f7d39b-588d44c784-mhd7p"]
Jul 31 23:41:59.419421 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:41:59.419400    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-f7d39b-588d44c784-mhd7p"]
Jul 31 23:42:00.892491 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:00.892455    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1cef4c8e-16bc-4ea1-869c-c18075e8b953" path="/var/lib/kubelet/pods/1cef4c8e-16bc-4ea1-869c-c18075e8b953/volumes"
Jul 31 23:42:00.892874 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:00.892856    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e92ca9f-6ca2-4252-8b9c-c323f42b5ed8" path="/var/lib/kubelet/pods/3e92ca9f-6ca2-4252-8b9c-c323f42b5ed8/volumes"
Jul 31 23:42:00.893204 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:00.893190    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a171bee-5577-46b0-bc5c-b4f4c917d848" path="/var/lib/kubelet/pods/8a171bee-5577-46b0-bc5c-b4f4c917d848/volumes"
Jul 31 23:42:09.354383 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:09.354348    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-99d0e4-7d4d6f455b-p8s4s"]
Jul 31 23:42:09.354904 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:09.354714    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39e931f8-a795-4fa9-ab16-b5c8253b9fee" containerName="payload-processing"
Jul 31 23:42:09.354904 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:09.354726    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="39e931f8-a795-4fa9-ab16-b5c8253b9fee" containerName="payload-processing"
Jul 31 23:42:09.354904 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:09.354736    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f36049b-c544-4b5d-8372-5c567e1c28a1" containerName="maas-api"
Jul 31 23:42:09.354904 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:09.354742    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f36049b-c544-4b5d-8372-5c567e1c28a1" containerName="maas-api"
Jul 31 23:42:09.354904 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:09.354753    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a171bee-5577-46b0-bc5c-b4f4c917d848" containerName="payload-processing"
Jul 31 23:42:09.354904 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:09.354759    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a171bee-5577-46b0-bc5c-b4f4c917d848" containerName="payload-processing"
Jul 31 23:42:09.354904 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:09.354767    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25d1e458-7d50-49cc-9a48-196a9d204686" containerName="payload-pre-processing"
Jul 31 23:42:09.354904 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:09.354772    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="25d1e458-7d50-49cc-9a48-196a9d204686" containerName="payload-pre-processing"
Jul 31 23:42:09.354904 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:09.354778    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cef4c8e-16bc-4ea1-869c-c18075e8b953" containerName="maas-api"
Jul 31 23:42:09.354904 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:09.354783    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cef4c8e-16bc-4ea1-869c-c18075e8b953" containerName="maas-api"
Jul 31 23:42:09.354904 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:09.354790    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e92ca9f-6ca2-4252-8b9c-c323f42b5ed8" containerName="payload-pre-processing"
Jul 31 23:42:09.354904 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:09.354796    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e92ca9f-6ca2-4252-8b9c-c323f42b5ed8" containerName="payload-pre-processing"
Jul 31 23:42:09.354904 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:09.354847    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="25d1e458-7d50-49cc-9a48-196a9d204686" containerName="payload-pre-processing"
Jul 31 23:42:09.354904 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:09.354870    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e92ca9f-6ca2-4252-8b9c-c323f42b5ed8" containerName="payload-pre-processing"
Jul 31 23:42:09.354904 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:09.354876    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f36049b-c544-4b5d-8372-5c567e1c28a1" containerName="maas-api"
Jul 31 23:42:09.354904 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:09.354885    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="1cef4c8e-16bc-4ea1-869c-c18075e8b953" containerName="maas-api"
Jul 31 23:42:09.354904 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:09.354896    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="39e931f8-a795-4fa9-ab16-b5c8253b9fee" containerName="payload-processing"
Jul 31 23:42:09.354904 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:09.354904    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a171bee-5577-46b0-bc5c-b4f4c917d848" containerName="payload-processing"
Jul 31 23:42:09.357941 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:09.357920    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-99d0e4-7d4d6f455b-p8s4s"
Jul 31 23:42:09.360422 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:09.360399    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-99d0e4-serving-cert\""
Jul 31 23:42:09.368391 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:09.368370    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-99d0e4-7d4d6f455b-p8s4s"]
Jul 31 23:42:09.514655 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:09.514612    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f0556ab-9fae-434e-ba95-3f3ad3f55d8d-maas-api-tls\") pod \"maas-api-e2e-shared-a-99d0e4-7d4d6f455b-p8s4s\" (UID: \"2f0556ab-9fae-434e-ba95-3f3ad3f55d8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-99d0e4-7d4d6f455b-p8s4s"
Jul 31 23:42:09.514846 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:09.514745    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f0556ab-9fae-434e-ba95-3f3ad3f55d8d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-99d0e4-7d4d6f455b-p8s4s\" (UID: \"2f0556ab-9fae-434e-ba95-3f3ad3f55d8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-99d0e4-7d4d6f455b-p8s4s"
Jul 31 23:42:09.514846 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:09.514800    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qj6wd\" (UniqueName: \"kubernetes.io/projected/2f0556ab-9fae-434e-ba95-3f3ad3f55d8d-kube-api-access-qj6wd\") pod \"maas-api-e2e-shared-a-99d0e4-7d4d6f455b-p8s4s\" (UID: \"2f0556ab-9fae-434e-ba95-3f3ad3f55d8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-99d0e4-7d4d6f455b-p8s4s"
Jul 31 23:42:09.616353 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:09.616267    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f0556ab-9fae-434e-ba95-3f3ad3f55d8d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-99d0e4-7d4d6f455b-p8s4s\" (UID: \"2f0556ab-9fae-434e-ba95-3f3ad3f55d8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-99d0e4-7d4d6f455b-p8s4s"
Jul 31 23:42:09.616353 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:09.616309    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qj6wd\" (UniqueName: \"kubernetes.io/projected/2f0556ab-9fae-434e-ba95-3f3ad3f55d8d-kube-api-access-qj6wd\") pod \"maas-api-e2e-shared-a-99d0e4-7d4d6f455b-p8s4s\" (UID: \"2f0556ab-9fae-434e-ba95-3f3ad3f55d8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-99d0e4-7d4d6f455b-p8s4s"
Jul 31 23:42:09.616353 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:09.616338    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f0556ab-9fae-434e-ba95-3f3ad3f55d8d-maas-api-tls\") pod \"maas-api-e2e-shared-a-99d0e4-7d4d6f455b-p8s4s\" (UID: \"2f0556ab-9fae-434e-ba95-3f3ad3f55d8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-99d0e4-7d4d6f455b-p8s4s"
Jul 31 23:42:09.616577 ip-10-0-128-243 kubenswrapper[2584]: E0731 23:42:09.616429    2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-99d0e4-serving-cert: secret "maas-api-e2e-shared-a-99d0e4-serving-cert" not found
Jul 31 23:42:09.616577 ip-10-0-128-243 kubenswrapper[2584]: E0731 23:42:09.616482    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f0556ab-9fae-434e-ba95-3f3ad3f55d8d-maas-api-tls podName:2f0556ab-9fae-434e-ba95-3f3ad3f55d8d nodeName:}" failed. No retries permitted until 2026-07-31 23:42:10.116466258 +0000 UTC m=+2713.762252545 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2f0556ab-9fae-434e-ba95-3f3ad3f55d8d-maas-api-tls") pod "maas-api-e2e-shared-a-99d0e4-7d4d6f455b-p8s4s" (UID: "2f0556ab-9fae-434e-ba95-3f3ad3f55d8d") : secret "maas-api-e2e-shared-a-99d0e4-serving-cert" not found
Jul 31 23:42:09.616917 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:09.616897    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f0556ab-9fae-434e-ba95-3f3ad3f55d8d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-99d0e4-7d4d6f455b-p8s4s\" (UID: \"2f0556ab-9fae-434e-ba95-3f3ad3f55d8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-99d0e4-7d4d6f455b-p8s4s"
Jul 31 23:42:09.627271 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:09.627242    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qj6wd\" (UniqueName: \"kubernetes.io/projected/2f0556ab-9fae-434e-ba95-3f3ad3f55d8d-kube-api-access-qj6wd\") pod \"maas-api-e2e-shared-a-99d0e4-7d4d6f455b-p8s4s\" (UID: \"2f0556ab-9fae-434e-ba95-3f3ad3f55d8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-99d0e4-7d4d6f455b-p8s4s"
Jul 31 23:42:10.120875 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:10.120831    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f0556ab-9fae-434e-ba95-3f3ad3f55d8d-maas-api-tls\") pod \"maas-api-e2e-shared-a-99d0e4-7d4d6f455b-p8s4s\" (UID: \"2f0556ab-9fae-434e-ba95-3f3ad3f55d8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-99d0e4-7d4d6f455b-p8s4s"
Jul 31 23:42:10.123448 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:10.123415    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f0556ab-9fae-434e-ba95-3f3ad3f55d8d-maas-api-tls\") pod \"maas-api-e2e-shared-a-99d0e4-7d4d6f455b-p8s4s\" (UID: \"2f0556ab-9fae-434e-ba95-3f3ad3f55d8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-99d0e4-7d4d6f455b-p8s4s"
Jul 31 23:42:10.166331 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:10.166291    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-99d0e4-d7d547fbd-j7dfx"]
Jul 31 23:42:10.169951 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:10.169930    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-99d0e4-d7d547fbd-j7dfx"
Jul 31 23:42:10.172268 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:10.172244    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-99d0e4-5ff6d9c7f4-qwzd2"]
Jul 31 23:42:10.172364 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:10.172307    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-99d0e4-dockercfg-nc589\""
Jul 31 23:42:10.172364 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:10.172317    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-99d0e4\""
Jul 31 23:42:10.175440 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:10.175421    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-99d0e4-5ff6d9c7f4-qwzd2"
Jul 31 23:42:10.179626 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:10.179601    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-99d0e4-d7d547fbd-j7dfx"]
Jul 31 23:42:10.186231 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:10.186204    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-99d0e4-5ff6d9c7f4-qwzd2"]
Jul 31 23:42:10.268653 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:10.268621    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-99d0e4-7d4d6f455b-p8s4s"
Jul 31 23:42:10.322485 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:10.322452    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d600216e-7383-4606-b644-1ef7258899c6-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-99d0e4-d7d547fbd-j7dfx\" (UID: \"d600216e-7383-4606-b644-1ef7258899c6\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-99d0e4-d7d547fbd-j7dfx"
Jul 31 23:42:10.322687 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:10.322501    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfs4l\" (UniqueName: \"kubernetes.io/projected/2cbdb4d2-c06b-4e03-bbff-44e4085444fc-kube-api-access-bfs4l\") pod \"payload-processing-e2e-shared-a-99d0e4-5ff6d9c7f4-qwzd2\" (UID: \"2cbdb4d2-c06b-4e03-bbff-44e4085444fc\") " pod="openshift-ingress/payload-processing-e2e-shared-a-99d0e4-5ff6d9c7f4-qwzd2"
Jul 31 23:42:10.322687 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:10.322553    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2cbdb4d2-c06b-4e03-bbff-44e4085444fc-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-99d0e4-5ff6d9c7f4-qwzd2\" (UID: \"2cbdb4d2-c06b-4e03-bbff-44e4085444fc\") " pod="openshift-ingress/payload-processing-e2e-shared-a-99d0e4-5ff6d9c7f4-qwzd2"
Jul 31 23:42:10.322687 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:10.322631    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mw6qk\" (UniqueName: \"kubernetes.io/projected/d600216e-7383-4606-b644-1ef7258899c6-kube-api-access-mw6qk\") pod \"payload-pre-processing-e2e-shared-a-99d0e4-d7d547fbd-j7dfx\" (UID: \"d600216e-7383-4606-b644-1ef7258899c6\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-99d0e4-d7d547fbd-j7dfx"
Jul 31 23:42:10.409206 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:10.409120    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-99d0e4-7d4d6f455b-p8s4s"]
Jul 31 23:42:10.411913 ip-10-0-128-243 kubenswrapper[2584]: W0731 23:42:10.411884    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f0556ab_9fae_434e_ba95_3f3ad3f55d8d.slice/crio-9dcab5a6abad02d6fb9d0ff47267e5af0293f721d31acfd806049685b4999419 WatchSource:0}: Error finding container 9dcab5a6abad02d6fb9d0ff47267e5af0293f721d31acfd806049685b4999419: Status 404 returned error can't find the container with id 9dcab5a6abad02d6fb9d0ff47267e5af0293f721d31acfd806049685b4999419
Jul 31 23:42:10.423134 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:10.423108    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d600216e-7383-4606-b644-1ef7258899c6-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-99d0e4-d7d547fbd-j7dfx\" (UID: \"d600216e-7383-4606-b644-1ef7258899c6\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-99d0e4-d7d547fbd-j7dfx"
Jul 31 23:42:10.423232 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:10.423153    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfs4l\" (UniqueName: \"kubernetes.io/projected/2cbdb4d2-c06b-4e03-bbff-44e4085444fc-kube-api-access-bfs4l\") pod \"payload-processing-e2e-shared-a-99d0e4-5ff6d9c7f4-qwzd2\" (UID: \"2cbdb4d2-c06b-4e03-bbff-44e4085444fc\") " pod="openshift-ingress/payload-processing-e2e-shared-a-99d0e4-5ff6d9c7f4-qwzd2"
Jul 31 23:42:10.423232 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:10.423194    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2cbdb4d2-c06b-4e03-bbff-44e4085444fc-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-99d0e4-5ff6d9c7f4-qwzd2\" (UID: \"2cbdb4d2-c06b-4e03-bbff-44e4085444fc\") " pod="openshift-ingress/payload-processing-e2e-shared-a-99d0e4-5ff6d9c7f4-qwzd2"
Jul 31 23:42:10.423357 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:10.423267    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mw6qk\" (UniqueName: \"kubernetes.io/projected/d600216e-7383-4606-b644-1ef7258899c6-kube-api-access-mw6qk\") pod \"payload-pre-processing-e2e-shared-a-99d0e4-d7d547fbd-j7dfx\" (UID: \"d600216e-7383-4606-b644-1ef7258899c6\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-99d0e4-d7d547fbd-j7dfx"
Jul 31 23:42:10.423986 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:10.423965    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2cbdb4d2-c06b-4e03-bbff-44e4085444fc-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-99d0e4-5ff6d9c7f4-qwzd2\" (UID: \"2cbdb4d2-c06b-4e03-bbff-44e4085444fc\") " pod="openshift-ingress/payload-processing-e2e-shared-a-99d0e4-5ff6d9c7f4-qwzd2"
Jul 31 23:42:10.423986 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:10.423983    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d600216e-7383-4606-b644-1ef7258899c6-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-99d0e4-d7d547fbd-j7dfx\" (UID: \"d600216e-7383-4606-b644-1ef7258899c6\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-99d0e4-d7d547fbd-j7dfx"
Jul 31 23:42:10.431145 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:10.431122    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfs4l\" (UniqueName: \"kubernetes.io/projected/2cbdb4d2-c06b-4e03-bbff-44e4085444fc-kube-api-access-bfs4l\") pod \"payload-processing-e2e-shared-a-99d0e4-5ff6d9c7f4-qwzd2\" (UID: \"2cbdb4d2-c06b-4e03-bbff-44e4085444fc\") " pod="openshift-ingress/payload-processing-e2e-shared-a-99d0e4-5ff6d9c7f4-qwzd2"
Jul 31 23:42:10.431268 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:10.431214    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mw6qk\" (UniqueName: \"kubernetes.io/projected/d600216e-7383-4606-b644-1ef7258899c6-kube-api-access-mw6qk\") pod \"payload-pre-processing-e2e-shared-a-99d0e4-d7d547fbd-j7dfx\" (UID: \"d600216e-7383-4606-b644-1ef7258899c6\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-99d0e4-d7d547fbd-j7dfx"
Jul 31 23:42:10.431396 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:10.431377    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-99d0e4-7d4d6f455b-p8s4s" event={"ID":"2f0556ab-9fae-434e-ba95-3f3ad3f55d8d","Type":"ContainerStarted","Data":"9dcab5a6abad02d6fb9d0ff47267e5af0293f721d31acfd806049685b4999419"}
Jul 31 23:42:10.483831 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:10.483795    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-99d0e4-d7d547fbd-j7dfx"
Jul 31 23:42:10.492942 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:10.492915    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-99d0e4-5ff6d9c7f4-qwzd2"
Jul 31 23:42:10.628813 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:10.628779    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-99d0e4-d7d547fbd-j7dfx"]
Jul 31 23:42:10.632008 ip-10-0-128-243 kubenswrapper[2584]: W0731 23:42:10.631668    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd600216e_7383_4606_b644_1ef7258899c6.slice/crio-9aed5846846503b22ca6d696a14d824e5f59f821b4bc0ccb401addefdacbd811 WatchSource:0}: Error finding container 9aed5846846503b22ca6d696a14d824e5f59f821b4bc0ccb401addefdacbd811: Status 404 returned error can't find the container with id 9aed5846846503b22ca6d696a14d824e5f59f821b4bc0ccb401addefdacbd811
Jul 31 23:42:10.661813 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:10.661789    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-99d0e4-5ff6d9c7f4-qwzd2"]
Jul 31 23:42:10.672014 ip-10-0-128-243 kubenswrapper[2584]: W0731 23:42:10.671974    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cbdb4d2_c06b_4e03_bbff_44e4085444fc.slice/crio-8ef71504b44fe9ef2183dc93167ae8fc5e1911b530ee9ae6f13bff9bd9d0262d WatchSource:0}: Error finding container 8ef71504b44fe9ef2183dc93167ae8fc5e1911b530ee9ae6f13bff9bd9d0262d: Status 404 returned error can't find the container with id 8ef71504b44fe9ef2183dc93167ae8fc5e1911b530ee9ae6f13bff9bd9d0262d
Jul 31 23:42:11.438861 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:11.438829    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-99d0e4-7d4d6f455b-p8s4s" event={"ID":"2f0556ab-9fae-434e-ba95-3f3ad3f55d8d","Type":"ContainerStarted","Data":"e24739b8ac6412d8c4277e3b2e13bf3214a6fb1af93af9dc7d0d60649382327d"}
Jul 31 23:42:11.439352 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:11.438899    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-99d0e4-7d4d6f455b-p8s4s"
Jul 31 23:42:11.440535 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:11.440508    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-99d0e4-5ff6d9c7f4-qwzd2" event={"ID":"2cbdb4d2-c06b-4e03-bbff-44e4085444fc","Type":"ContainerStarted","Data":"d577b95a0f1323756ac33b31f2f0c1d979538f45750865bf334cb6c0e9cde1f7"}
Jul 31 23:42:11.440682 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:11.440568    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-99d0e4-5ff6d9c7f4-qwzd2" event={"ID":"2cbdb4d2-c06b-4e03-bbff-44e4085444fc","Type":"ContainerStarted","Data":"8ef71504b44fe9ef2183dc93167ae8fc5e1911b530ee9ae6f13bff9bd9d0262d"}
Jul 31 23:42:11.440682 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:11.440595    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-99d0e4-5ff6d9c7f4-qwzd2"
Jul 31 23:42:11.442291 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:11.442266    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-99d0e4-d7d547fbd-j7dfx" event={"ID":"d600216e-7383-4606-b644-1ef7258899c6","Type":"ContainerStarted","Data":"d7ee50b8cf71b01858d27a6770a975b5afa05f581bb38add083c5e47397a7976"}
Jul 31 23:42:11.442406 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:11.442312    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-99d0e4-d7d547fbd-j7dfx" event={"ID":"d600216e-7383-4606-b644-1ef7258899c6","Type":"ContainerStarted","Data":"9aed5846846503b22ca6d696a14d824e5f59f821b4bc0ccb401addefdacbd811"}
Jul 31 23:42:11.442406 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:11.442351    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-99d0e4-d7d547fbd-j7dfx"
Jul 31 23:42:11.455982 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:11.455933    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-99d0e4-7d4d6f455b-p8s4s" podStartSLOduration=2.051283233 podStartE2EDuration="2.455919972s" totalImagesPullingTime="404.636739ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:42:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:42:11.45459609 +0000 UTC m=+2715.100382399" watchObservedRunningTime="2026-07-31 23:42:11.455919972 +0000 UTC m=+2715.101706280"
Jul 31 23:42:11.470604 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:11.470559    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-99d0e4-5ff6d9c7f4-qwzd2" podStartSLOduration=1.470544326 podStartE2EDuration="1.470544326s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:42:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:42:11.469515005 +0000 UTC m=+2715.115301319" watchObservedRunningTime="2026-07-31 23:42:11.470544326 +0000 UTC m=+2715.116330612"
Jul 31 23:42:11.484712 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:11.484655    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-99d0e4-d7d547fbd-j7dfx" podStartSLOduration=1.484640163 podStartE2EDuration="1.484640163s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:42:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:42:11.483275327 +0000 UTC m=+2715.129061648" watchObservedRunningTime="2026-07-31 23:42:11.484640163 +0000 UTC m=+2715.130426471"
Jul 31 23:42:17.454621 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:17.454590    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-99d0e4-7d4d6f455b-p8s4s"
Jul 31 23:42:22.451443 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:22.451411    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-99d0e4-5ff6d9c7f4-qwzd2"
Jul 31 23:42:22.451900 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:22.451498    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-99d0e4-d7d547fbd-j7dfx"
Jul 31 23:42:37.570555 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:37.570518    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-3d60f7-8455c9d5d6-wztsz"]
Jul 31 23:42:37.574352 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:37.574331    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3d60f7-8455c9d5d6-wztsz"
Jul 31 23:42:37.576724 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:37.576689    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-3d60f7-serving-cert\""
Jul 31 23:42:37.582580 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:37.582548    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-3d60f7-8455c9d5d6-wztsz"]
Jul 31 23:42:37.659794 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:37.659759    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4ca8cdda-d121-4034-9121-c64a7dcf257d-maas-api-tls\") pod \"maas-api-e2e-shared-b-3d60f7-8455c9d5d6-wztsz\" (UID: \"4ca8cdda-d121-4034-9121-c64a7dcf257d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3d60f7-8455c9d5d6-wztsz"
Jul 31 23:42:37.659794 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:37.659793    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqjd9\" (UniqueName: \"kubernetes.io/projected/4ca8cdda-d121-4034-9121-c64a7dcf257d-kube-api-access-dqjd9\") pod \"maas-api-e2e-shared-b-3d60f7-8455c9d5d6-wztsz\" (UID: \"4ca8cdda-d121-4034-9121-c64a7dcf257d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3d60f7-8455c9d5d6-wztsz"
Jul 31 23:42:37.660001 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:37.659878    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ca8cdda-d121-4034-9121-c64a7dcf257d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-3d60f7-8455c9d5d6-wztsz\" (UID: \"4ca8cdda-d121-4034-9121-c64a7dcf257d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3d60f7-8455c9d5d6-wztsz"
Jul 31 23:42:37.760851 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:37.760818    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ca8cdda-d121-4034-9121-c64a7dcf257d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-3d60f7-8455c9d5d6-wztsz\" (UID: \"4ca8cdda-d121-4034-9121-c64a7dcf257d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3d60f7-8455c9d5d6-wztsz"
Jul 31 23:42:37.761022 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:37.760874    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4ca8cdda-d121-4034-9121-c64a7dcf257d-maas-api-tls\") pod \"maas-api-e2e-shared-b-3d60f7-8455c9d5d6-wztsz\" (UID: \"4ca8cdda-d121-4034-9121-c64a7dcf257d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3d60f7-8455c9d5d6-wztsz"
Jul 31 23:42:37.761022 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:37.760893    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqjd9\" (UniqueName: \"kubernetes.io/projected/4ca8cdda-d121-4034-9121-c64a7dcf257d-kube-api-access-dqjd9\") pod \"maas-api-e2e-shared-b-3d60f7-8455c9d5d6-wztsz\" (UID: \"4ca8cdda-d121-4034-9121-c64a7dcf257d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3d60f7-8455c9d5d6-wztsz"
Jul 31 23:42:37.761095 ip-10-0-128-243 kubenswrapper[2584]: E0731 23:42:37.761059    2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-3d60f7-serving-cert: secret "maas-api-e2e-shared-b-3d60f7-serving-cert" not found
Jul 31 23:42:37.761149 ip-10-0-128-243 kubenswrapper[2584]: E0731 23:42:37.761136    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ca8cdda-d121-4034-9121-c64a7dcf257d-maas-api-tls podName:4ca8cdda-d121-4034-9121-c64a7dcf257d nodeName:}" failed. No retries permitted until 2026-07-31 23:42:38.261113668 +0000 UTC m=+2741.906899959 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4ca8cdda-d121-4034-9121-c64a7dcf257d-maas-api-tls") pod "maas-api-e2e-shared-b-3d60f7-8455c9d5d6-wztsz" (UID: "4ca8cdda-d121-4034-9121-c64a7dcf257d") : secret "maas-api-e2e-shared-b-3d60f7-serving-cert" not found
Jul 31 23:42:37.761557 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:37.761539    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ca8cdda-d121-4034-9121-c64a7dcf257d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-3d60f7-8455c9d5d6-wztsz\" (UID: \"4ca8cdda-d121-4034-9121-c64a7dcf257d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3d60f7-8455c9d5d6-wztsz"
Jul 31 23:42:37.769991 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:37.769971    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqjd9\" (UniqueName: \"kubernetes.io/projected/4ca8cdda-d121-4034-9121-c64a7dcf257d-kube-api-access-dqjd9\") pod \"maas-api-e2e-shared-b-3d60f7-8455c9d5d6-wztsz\" (UID: \"4ca8cdda-d121-4034-9121-c64a7dcf257d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3d60f7-8455c9d5d6-wztsz"
Jul 31 23:42:38.203932 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:38.203898    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-3d60f7-6dd78b98c8-twrbh"]
Jul 31 23:42:38.207553 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:38.207536    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-3d60f7-6dd78b98c8-twrbh"
Jul 31 23:42:38.209974 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:38.209951    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-3d60f7\""
Jul 31 23:42:38.217571 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:38.217548    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-3d60f7-6dd78b98c8-twrbh"]
Jul 31 23:42:38.260336 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:38.260301    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-3d60f7-bdc4c97d4-8dqv7"]
Jul 31 23:42:38.263892 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:38.263871    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-3d60f7-bdc4c97d4-8dqv7"
Jul 31 23:42:38.265158 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:38.265138    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c52bc313-e332-4809-bd37-3bdbf62bdc06-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-3d60f7-6dd78b98c8-twrbh\" (UID: \"c52bc313-e332-4809-bd37-3bdbf62bdc06\") " pod="openshift-ingress/payload-processing-e2e-shared-b-3d60f7-6dd78b98c8-twrbh"
Jul 31 23:42:38.265253 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:38.265168    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjgzp\" (UniqueName: \"kubernetes.io/projected/c52bc313-e332-4809-bd37-3bdbf62bdc06-kube-api-access-sjgzp\") pod \"payload-processing-e2e-shared-b-3d60f7-6dd78b98c8-twrbh\" (UID: \"c52bc313-e332-4809-bd37-3bdbf62bdc06\") " pod="openshift-ingress/payload-processing-e2e-shared-b-3d60f7-6dd78b98c8-twrbh"
Jul 31 23:42:38.265253 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:38.265208    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4ca8cdda-d121-4034-9121-c64a7dcf257d-maas-api-tls\") pod \"maas-api-e2e-shared-b-3d60f7-8455c9d5d6-wztsz\" (UID: \"4ca8cdda-d121-4034-9121-c64a7dcf257d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3d60f7-8455c9d5d6-wztsz"
Jul 31 23:42:38.265326 ip-10-0-128-243 kubenswrapper[2584]: E0731 23:42:38.265294    2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-3d60f7-serving-cert: secret "maas-api-e2e-shared-b-3d60f7-serving-cert" not found
Jul 31 23:42:38.265360 ip-10-0-128-243 kubenswrapper[2584]: E0731 23:42:38.265343    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ca8cdda-d121-4034-9121-c64a7dcf257d-maas-api-tls podName:4ca8cdda-d121-4034-9121-c64a7dcf257d nodeName:}" failed. No retries permitted until 2026-07-31 23:42:39.265326998 +0000 UTC m=+2742.911113286 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4ca8cdda-d121-4034-9121-c64a7dcf257d-maas-api-tls") pod "maas-api-e2e-shared-b-3d60f7-8455c9d5d6-wztsz" (UID: "4ca8cdda-d121-4034-9121-c64a7dcf257d") : secret "maas-api-e2e-shared-b-3d60f7-serving-cert" not found
Jul 31 23:42:38.266432 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:38.266414    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-3d60f7-dockercfg-29qcm\""
Jul 31 23:42:38.273510 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:38.273487    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-3d60f7-bdc4c97d4-8dqv7"]
Jul 31 23:42:38.366086 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:38.366055    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fh58p\" (UniqueName: \"kubernetes.io/projected/a658ca8a-339e-414a-837a-11e522529476-kube-api-access-fh58p\") pod \"payload-pre-processing-e2e-shared-b-3d60f7-bdc4c97d4-8dqv7\" (UID: \"a658ca8a-339e-414a-837a-11e522529476\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-3d60f7-bdc4c97d4-8dqv7"
Jul 31 23:42:38.366278 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:38.366114    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c52bc313-e332-4809-bd37-3bdbf62bdc06-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-3d60f7-6dd78b98c8-twrbh\" (UID: \"c52bc313-e332-4809-bd37-3bdbf62bdc06\") " pod="openshift-ingress/payload-processing-e2e-shared-b-3d60f7-6dd78b98c8-twrbh"
Jul 31 23:42:38.366278 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:38.366136    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjgzp\" (UniqueName: \"kubernetes.io/projected/c52bc313-e332-4809-bd37-3bdbf62bdc06-kube-api-access-sjgzp\") pod \"payload-processing-e2e-shared-b-3d60f7-6dd78b98c8-twrbh\" (UID: \"c52bc313-e332-4809-bd37-3bdbf62bdc06\") " pod="openshift-ingress/payload-processing-e2e-shared-b-3d60f7-6dd78b98c8-twrbh"
Jul 31 23:42:38.366278 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:38.366174    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a658ca8a-339e-414a-837a-11e522529476-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-3d60f7-bdc4c97d4-8dqv7\" (UID: \"a658ca8a-339e-414a-837a-11e522529476\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-3d60f7-bdc4c97d4-8dqv7"
Jul 31 23:42:38.366878 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:38.366859    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c52bc313-e332-4809-bd37-3bdbf62bdc06-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-3d60f7-6dd78b98c8-twrbh\" (UID: \"c52bc313-e332-4809-bd37-3bdbf62bdc06\") " pod="openshift-ingress/payload-processing-e2e-shared-b-3d60f7-6dd78b98c8-twrbh"
Jul 31 23:42:38.374414 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:38.374389    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjgzp\" (UniqueName: \"kubernetes.io/projected/c52bc313-e332-4809-bd37-3bdbf62bdc06-kube-api-access-sjgzp\") pod \"payload-processing-e2e-shared-b-3d60f7-6dd78b98c8-twrbh\" (UID: \"c52bc313-e332-4809-bd37-3bdbf62bdc06\") " pod="openshift-ingress/payload-processing-e2e-shared-b-3d60f7-6dd78b98c8-twrbh"
Jul 31 23:42:38.467767 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:38.467655    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a658ca8a-339e-414a-837a-11e522529476-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-3d60f7-bdc4c97d4-8dqv7\" (UID: \"a658ca8a-339e-414a-837a-11e522529476\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-3d60f7-bdc4c97d4-8dqv7"
Jul 31 23:42:38.467916 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:38.467774    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fh58p\" (UniqueName: \"kubernetes.io/projected/a658ca8a-339e-414a-837a-11e522529476-kube-api-access-fh58p\") pod \"payload-pre-processing-e2e-shared-b-3d60f7-bdc4c97d4-8dqv7\" (UID: \"a658ca8a-339e-414a-837a-11e522529476\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-3d60f7-bdc4c97d4-8dqv7"
Jul 31 23:42:38.468384 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:38.468364    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a658ca8a-339e-414a-837a-11e522529476-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-3d60f7-bdc4c97d4-8dqv7\" (UID: \"a658ca8a-339e-414a-837a-11e522529476\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-3d60f7-bdc4c97d4-8dqv7"
Jul 31 23:42:38.476735 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:38.476695    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fh58p\" (UniqueName: \"kubernetes.io/projected/a658ca8a-339e-414a-837a-11e522529476-kube-api-access-fh58p\") pod \"payload-pre-processing-e2e-shared-b-3d60f7-bdc4c97d4-8dqv7\" (UID: \"a658ca8a-339e-414a-837a-11e522529476\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-3d60f7-bdc4c97d4-8dqv7"
Jul 31 23:42:38.519449 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:38.519423    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-3d60f7-6dd78b98c8-twrbh"
Jul 31 23:42:38.574850 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:38.574816    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-3d60f7-bdc4c97d4-8dqv7"
Jul 31 23:42:38.724353 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:38.724324    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-3d60f7-bdc4c97d4-8dqv7"]
Jul 31 23:42:38.726142 ip-10-0-128-243 kubenswrapper[2584]: W0731 23:42:38.726115    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda658ca8a_339e_414a_837a_11e522529476.slice/crio-c3c2c47055c90da14545d81e95f87bd6716daf0aa626c4f6709bcd409a504df6 WatchSource:0}: Error finding container c3c2c47055c90da14545d81e95f87bd6716daf0aa626c4f6709bcd409a504df6: Status 404 returned error can't find the container with id c3c2c47055c90da14545d81e95f87bd6716daf0aa626c4f6709bcd409a504df6
Jul 31 23:42:38.873603 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:38.873578    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-3d60f7-6dd78b98c8-twrbh"]
Jul 31 23:42:38.876033 ip-10-0-128-243 kubenswrapper[2584]: W0731 23:42:38.876006    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc52bc313_e332_4809_bd37_3bdbf62bdc06.slice/crio-94dae7b455befcd268dfdda8b1e29cd88ba6fd6b9c7ca283b7795a33a67d3203 WatchSource:0}: Error finding container 94dae7b455befcd268dfdda8b1e29cd88ba6fd6b9c7ca283b7795a33a67d3203: Status 404 returned error can't find the container with id 94dae7b455befcd268dfdda8b1e29cd88ba6fd6b9c7ca283b7795a33a67d3203
Jul 31 23:42:39.275500 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:39.275453    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4ca8cdda-d121-4034-9121-c64a7dcf257d-maas-api-tls\") pod \"maas-api-e2e-shared-b-3d60f7-8455c9d5d6-wztsz\" (UID: \"4ca8cdda-d121-4034-9121-c64a7dcf257d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3d60f7-8455c9d5d6-wztsz"
Jul 31 23:42:39.277942 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:39.277916    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4ca8cdda-d121-4034-9121-c64a7dcf257d-maas-api-tls\") pod \"maas-api-e2e-shared-b-3d60f7-8455c9d5d6-wztsz\" (UID: \"4ca8cdda-d121-4034-9121-c64a7dcf257d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3d60f7-8455c9d5d6-wztsz"
Jul 31 23:42:39.387712 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:39.387675    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3d60f7-8455c9d5d6-wztsz"
Jul 31 23:42:39.521115 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:39.521089    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-3d60f7-8455c9d5d6-wztsz"]
Jul 31 23:42:39.523436 ip-10-0-128-243 kubenswrapper[2584]: W0731 23:42:39.523406    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ca8cdda_d121_4034_9121_c64a7dcf257d.slice/crio-80f22b87322a7ff55daf06c2009d2077968b229ffb1abafb73b3c06ce659491c WatchSource:0}: Error finding container 80f22b87322a7ff55daf06c2009d2077968b229ffb1abafb73b3c06ce659491c: Status 404 returned error can't find the container with id 80f22b87322a7ff55daf06c2009d2077968b229ffb1abafb73b3c06ce659491c
Jul 31 23:42:39.597463 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:39.597425    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-3d60f7-bdc4c97d4-8dqv7" event={"ID":"a658ca8a-339e-414a-837a-11e522529476","Type":"ContainerStarted","Data":"03da00a7437b8df37cdb65a245e3d848ff5896726386389e3869d86a89290971"}
Jul 31 23:42:39.597946 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:39.597494    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-3d60f7-bdc4c97d4-8dqv7" event={"ID":"a658ca8a-339e-414a-837a-11e522529476","Type":"ContainerStarted","Data":"c3c2c47055c90da14545d81e95f87bd6716daf0aa626c4f6709bcd409a504df6"}
Jul 31 23:42:39.597946 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:39.597600    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-3d60f7-bdc4c97d4-8dqv7"
Jul 31 23:42:39.598779 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:39.598728    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3d60f7-8455c9d5d6-wztsz" event={"ID":"4ca8cdda-d121-4034-9121-c64a7dcf257d","Type":"ContainerStarted","Data":"80f22b87322a7ff55daf06c2009d2077968b229ffb1abafb73b3c06ce659491c"}
Jul 31 23:42:39.600089 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:39.600068    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-3d60f7-6dd78b98c8-twrbh" event={"ID":"c52bc313-e332-4809-bd37-3bdbf62bdc06","Type":"ContainerStarted","Data":"c9b509e2d1161ececee7e923621e1c618d16f717449dfeb55ce4e1934d94f115"}
Jul 31 23:42:39.600273 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:39.600112    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-3d60f7-6dd78b98c8-twrbh" event={"ID":"c52bc313-e332-4809-bd37-3bdbf62bdc06","Type":"ContainerStarted","Data":"94dae7b455befcd268dfdda8b1e29cd88ba6fd6b9c7ca283b7795a33a67d3203"}
Jul 31 23:42:39.600273 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:39.600148    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-3d60f7-6dd78b98c8-twrbh"
Jul 31 23:42:39.612451 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:39.612397    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-3d60f7-bdc4c97d4-8dqv7" podStartSLOduration=1.612383736 podStartE2EDuration="1.612383736s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:42:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:42:39.611010561 +0000 UTC m=+2743.256796879" watchObservedRunningTime="2026-07-31 23:42:39.612383736 +0000 UTC m=+2743.258170022"
Jul 31 23:42:39.626393 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:39.626345    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-3d60f7-6dd78b98c8-twrbh" podStartSLOduration=1.626328787 podStartE2EDuration="1.626328787s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:42:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:42:39.625214987 +0000 UTC m=+2743.271001295" watchObservedRunningTime="2026-07-31 23:42:39.626328787 +0000 UTC m=+2743.272115096"
Jul 31 23:42:40.606553 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:40.606516    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3d60f7-8455c9d5d6-wztsz" event={"ID":"4ca8cdda-d121-4034-9121-c64a7dcf257d","Type":"ContainerStarted","Data":"85d329d4a9cc3ba30ac0582153bebc5d6ca85773fb6ac7e687d94feaeae610a5"}
Jul 31 23:42:40.607041 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:40.606890    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3d60f7-8455c9d5d6-wztsz"
Jul 31 23:42:40.622476 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:40.622424    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3d60f7-8455c9d5d6-wztsz" podStartSLOduration=3.166100695 podStartE2EDuration="3.622408413s" totalImagesPullingTime="456.307718ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:42:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:42:40.621457587 +0000 UTC m=+2744.267243913" watchObservedRunningTime="2026-07-31 23:42:40.622408413 +0000 UTC m=+2744.268194723"
Jul 31 23:42:46.617393 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:46.617361    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3d60f7-8455c9d5d6-wztsz"
Jul 31 23:42:50.608910 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:50.608877    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-3d60f7-bdc4c97d4-8dqv7"
Jul 31 23:42:50.609296 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:42:50.608951    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-3d60f7-6dd78b98c8-twrbh"
Jul 31 23:50:43.800287 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:50:43.800250    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-c16217-74f9dd6987-f876j"]
Jul 31 23:50:43.804024 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:50:43.804000    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c16217-74f9dd6987-f876j"
Jul 31 23:50:43.806395 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:50:43.806376    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-c16217-serving-cert\""
Jul 31 23:50:43.811922 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:50:43.811893    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-c16217-74f9dd6987-f876j"]
Jul 31 23:50:43.887887 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:50:43.887824    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd76d76e-27c6-4fda-b5b2-1253e05ea281-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-c16217-74f9dd6987-f876j\" (UID: \"bd76d76e-27c6-4fda-b5b2-1253e05ea281\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c16217-74f9dd6987-f876j"
Jul 31 23:50:43.888063 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:50:43.887974    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4b75\" (UniqueName: \"kubernetes.io/projected/bd76d76e-27c6-4fda-b5b2-1253e05ea281-kube-api-access-f4b75\") pod \"maas-api-e2e-ipp-a-c16217-74f9dd6987-f876j\" (UID: \"bd76d76e-27c6-4fda-b5b2-1253e05ea281\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c16217-74f9dd6987-f876j"
Jul 31 23:50:43.888063 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:50:43.888049    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd76d76e-27c6-4fda-b5b2-1253e05ea281-maas-api-tls\") pod \"maas-api-e2e-ipp-a-c16217-74f9dd6987-f876j\" (UID: \"bd76d76e-27c6-4fda-b5b2-1253e05ea281\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c16217-74f9dd6987-f876j"
Jul 31 23:50:43.988758 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:50:43.988723    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd76d76e-27c6-4fda-b5b2-1253e05ea281-maas-api-tls\") pod \"maas-api-e2e-ipp-a-c16217-74f9dd6987-f876j\" (UID: \"bd76d76e-27c6-4fda-b5b2-1253e05ea281\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c16217-74f9dd6987-f876j"
Jul 31 23:50:43.988944 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:50:43.988781    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd76d76e-27c6-4fda-b5b2-1253e05ea281-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-c16217-74f9dd6987-f876j\" (UID: \"bd76d76e-27c6-4fda-b5b2-1253e05ea281\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c16217-74f9dd6987-f876j"
Jul 31 23:50:43.988944 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:50:43.988814    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4b75\" (UniqueName: \"kubernetes.io/projected/bd76d76e-27c6-4fda-b5b2-1253e05ea281-kube-api-access-f4b75\") pod \"maas-api-e2e-ipp-a-c16217-74f9dd6987-f876j\" (UID: \"bd76d76e-27c6-4fda-b5b2-1253e05ea281\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c16217-74f9dd6987-f876j"
Jul 31 23:50:43.989588 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:50:43.989568    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd76d76e-27c6-4fda-b5b2-1253e05ea281-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-c16217-74f9dd6987-f876j\" (UID: \"bd76d76e-27c6-4fda-b5b2-1253e05ea281\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c16217-74f9dd6987-f876j"
Jul 31 23:50:43.991120 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:50:43.991098    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd76d76e-27c6-4fda-b5b2-1253e05ea281-maas-api-tls\") pod \"maas-api-e2e-ipp-a-c16217-74f9dd6987-f876j\" (UID: \"bd76d76e-27c6-4fda-b5b2-1253e05ea281\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c16217-74f9dd6987-f876j"
Jul 31 23:50:43.997342 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:50:43.997318    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4b75\" (UniqueName: \"kubernetes.io/projected/bd76d76e-27c6-4fda-b5b2-1253e05ea281-kube-api-access-f4b75\") pod \"maas-api-e2e-ipp-a-c16217-74f9dd6987-f876j\" (UID: \"bd76d76e-27c6-4fda-b5b2-1253e05ea281\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c16217-74f9dd6987-f876j"
Jul 31 23:50:44.114641 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:50:44.114612    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c16217-74f9dd6987-f876j"
Jul 31 23:50:44.248047 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:50:44.248013    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-c16217-74f9dd6987-f876j"]
Jul 31 23:50:44.250997 ip-10-0-128-243 kubenswrapper[2584]: W0731 23:50:44.250964    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd76d76e_27c6_4fda_b5b2_1253e05ea281.slice/crio-e6cf7e97c2dc2ec92ec2dc51aa7e01962e1b21ab40d2dce6c57014e859a43d61 WatchSource:0}: Error finding container e6cf7e97c2dc2ec92ec2dc51aa7e01962e1b21ab40d2dce6c57014e859a43d61: Status 404 returned error can't find the container with id e6cf7e97c2dc2ec92ec2dc51aa7e01962e1b21ab40d2dce6c57014e859a43d61
Jul 31 23:50:44.252363 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:50:44.252346    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 31 23:50:44.924438 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:50:44.924403    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c16217-74f9dd6987-f876j" event={"ID":"bd76d76e-27c6-4fda-b5b2-1253e05ea281","Type":"ContainerStarted","Data":"a0f5d38debee187f28bd27d89df4c361bc47a63e2c4276b59cd08c0ce52c521e"}
Jul 31 23:50:44.924866 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:50:44.924457    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c16217-74f9dd6987-f876j" event={"ID":"bd76d76e-27c6-4fda-b5b2-1253e05ea281","Type":"ContainerStarted","Data":"e6cf7e97c2dc2ec92ec2dc51aa7e01962e1b21ab40d2dce6c57014e859a43d61"}
Jul 31 23:50:44.924866 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:50:44.924499    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c16217-74f9dd6987-f876j"
Jul 31 23:50:44.941252 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:50:44.941197    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c16217-74f9dd6987-f876j" podStartSLOduration=1.460439785 podStartE2EDuration="1.941178967s" totalImagesPullingTime="480.739182ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:50:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:50:44.939285526 +0000 UTC m=+3228.585071838" watchObservedRunningTime="2026-07-31 23:50:44.941178967 +0000 UTC m=+3228.586965345"
Jul 31 23:50:50.934580 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:50:50.934505    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c16217-74f9dd6987-f876j"
Jul 31 23:51:12.931258 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:51:12.931223    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-5a93d4-7bfbd454c8-4n6qh"]
Jul 31 23:51:12.934927 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:51:12.934906    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5a93d4-7bfbd454c8-4n6qh"
Jul 31 23:51:12.937878 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:51:12.937858    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-5a93d4-serving-cert\""
Jul 31 23:51:12.946302 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:51:12.946274    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-5a93d4-7bfbd454c8-4n6qh"]
Jul 31 23:51:13.052952 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:51:13.052919    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9vp5\" (UniqueName: \"kubernetes.io/projected/76714e3b-4bab-476a-ab5b-1bb95995b0e1-kube-api-access-z9vp5\") pod \"maas-api-e2e-ipp-b-5a93d4-7bfbd454c8-4n6qh\" (UID: \"76714e3b-4bab-476a-ab5b-1bb95995b0e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5a93d4-7bfbd454c8-4n6qh"
Jul 31 23:51:13.053157 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:51:13.053063    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/76714e3b-4bab-476a-ab5b-1bb95995b0e1-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-5a93d4-7bfbd454c8-4n6qh\" (UID: \"76714e3b-4bab-476a-ab5b-1bb95995b0e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5a93d4-7bfbd454c8-4n6qh"
Jul 31 23:51:13.053157 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:51:13.053137    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/76714e3b-4bab-476a-ab5b-1bb95995b0e1-maas-api-tls\") pod \"maas-api-e2e-ipp-b-5a93d4-7bfbd454c8-4n6qh\" (UID: \"76714e3b-4bab-476a-ab5b-1bb95995b0e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5a93d4-7bfbd454c8-4n6qh"
Jul 31 23:51:13.153774 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:51:13.153728    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/76714e3b-4bab-476a-ab5b-1bb95995b0e1-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-5a93d4-7bfbd454c8-4n6qh\" (UID: \"76714e3b-4bab-476a-ab5b-1bb95995b0e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5a93d4-7bfbd454c8-4n6qh"
Jul 31 23:51:13.153973 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:51:13.153795    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/76714e3b-4bab-476a-ab5b-1bb95995b0e1-maas-api-tls\") pod \"maas-api-e2e-ipp-b-5a93d4-7bfbd454c8-4n6qh\" (UID: \"76714e3b-4bab-476a-ab5b-1bb95995b0e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5a93d4-7bfbd454c8-4n6qh"
Jul 31 23:51:13.153973 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:51:13.153826    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9vp5\" (UniqueName: \"kubernetes.io/projected/76714e3b-4bab-476a-ab5b-1bb95995b0e1-kube-api-access-z9vp5\") pod \"maas-api-e2e-ipp-b-5a93d4-7bfbd454c8-4n6qh\" (UID: \"76714e3b-4bab-476a-ab5b-1bb95995b0e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5a93d4-7bfbd454c8-4n6qh"
Jul 31 23:51:13.154104 ip-10-0-128-243 kubenswrapper[2584]: E0731 23:51:13.153979    2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-5a93d4-serving-cert: secret "maas-api-e2e-ipp-b-5a93d4-serving-cert" not found
Jul 31 23:51:13.154104 ip-10-0-128-243 kubenswrapper[2584]: E0731 23:51:13.154054    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76714e3b-4bab-476a-ab5b-1bb95995b0e1-maas-api-tls podName:76714e3b-4bab-476a-ab5b-1bb95995b0e1 nodeName:}" failed. No retries permitted until 2026-07-31 23:51:13.654032321 +0000 UTC m=+3257.299818612 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/76714e3b-4bab-476a-ab5b-1bb95995b0e1-maas-api-tls") pod "maas-api-e2e-ipp-b-5a93d4-7bfbd454c8-4n6qh" (UID: "76714e3b-4bab-476a-ab5b-1bb95995b0e1") : secret "maas-api-e2e-ipp-b-5a93d4-serving-cert" not found
Jul 31 23:51:13.154359 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:51:13.154338    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/76714e3b-4bab-476a-ab5b-1bb95995b0e1-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-5a93d4-7bfbd454c8-4n6qh\" (UID: \"76714e3b-4bab-476a-ab5b-1bb95995b0e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5a93d4-7bfbd454c8-4n6qh"
Jul 31 23:51:13.162759 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:51:13.162731    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9vp5\" (UniqueName: \"kubernetes.io/projected/76714e3b-4bab-476a-ab5b-1bb95995b0e1-kube-api-access-z9vp5\") pod \"maas-api-e2e-ipp-b-5a93d4-7bfbd454c8-4n6qh\" (UID: \"76714e3b-4bab-476a-ab5b-1bb95995b0e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5a93d4-7bfbd454c8-4n6qh"
Jul 31 23:51:13.659042 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:51:13.659009    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/76714e3b-4bab-476a-ab5b-1bb95995b0e1-maas-api-tls\") pod \"maas-api-e2e-ipp-b-5a93d4-7bfbd454c8-4n6qh\" (UID: \"76714e3b-4bab-476a-ab5b-1bb95995b0e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5a93d4-7bfbd454c8-4n6qh"
Jul 31 23:51:13.661406 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:51:13.661376    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/76714e3b-4bab-476a-ab5b-1bb95995b0e1-maas-api-tls\") pod \"maas-api-e2e-ipp-b-5a93d4-7bfbd454c8-4n6qh\" (UID: \"76714e3b-4bab-476a-ab5b-1bb95995b0e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5a93d4-7bfbd454c8-4n6qh"
Jul 31 23:51:13.705037 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:51:13.705008    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-5a93d4-b6bd494d8-f2prm"]
Jul 31 23:51:13.708566 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:51:13.708549    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5a93d4-b6bd494d8-f2prm"
Jul 31 23:51:13.712290 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:51:13.712270    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-5a93d4\""
Jul 31 23:51:13.712462 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:51:13.712446    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-5a93d4-dockercfg-h444w\""
Jul 31 23:51:13.721015 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:51:13.720990    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-5a93d4-b6bd494d8-f2prm"]
Jul 31 23:51:13.737552 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:51:13.737515    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-5a93d4-75b48cf6c6-dgrf8"]
Jul 31 23:51:13.740903 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:51:13.740883    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-5a93d4-75b48cf6c6-dgrf8"
Jul 31 23:51:13.751096 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:51:13.751069    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-5a93d4-75b48cf6c6-dgrf8"]
Jul 31 23:51:13.850315 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:51:13.850279    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5a93d4-7bfbd454c8-4n6qh"
Jul 31 23:51:13.861663 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:51:13.861631    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9h6q6\" (UniqueName: \"kubernetes.io/projected/2708980c-46e9-4bfd-b55e-149d9e3267d3-kube-api-access-9h6q6\") pod \"payload-processing-e2e-ipp-b-5a93d4-75b48cf6c6-dgrf8\" (UID: \"2708980c-46e9-4bfd-b55e-149d9e3267d3\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-5a93d4-75b48cf6c6-dgrf8"
Jul 31 23:51:13.861790 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:51:13.861728    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87czp\" (UniqueName: \"kubernetes.io/projected/c5da16a9-2cbe-46cc-afdc-76af2b1d878d-kube-api-access-87czp\") pod \"payload-pre-processing-e2e-ipp-b-5a93d4-b6bd494d8-f2prm\" (UID: \"c5da16a9-2cbe-46cc-afdc-76af2b1d878d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5a93d4-b6bd494d8-f2prm"
Jul 31 23:51:13.861790 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:51:13.861776    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c5da16a9-2cbe-46cc-afdc-76af2b1d878d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-5a93d4-b6bd494d8-f2prm\" (UID: \"c5da16a9-2cbe-46cc-afdc-76af2b1d878d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5a93d4-b6bd494d8-f2prm"
Jul 31 23:51:13.861943 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:51:13.861824    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2708980c-46e9-4bfd-b55e-149d9e3267d3-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-5a93d4-75b48cf6c6-dgrf8\" (UID: \"2708980c-46e9-4bfd-b55e-149d9e3267d3\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-5a93d4-75b48cf6c6-dgrf8"
Jul 31 23:51:13.964119 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:51:13.962913    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9h6q6\" (UniqueName: \"kubernetes.io/projected/2708980c-46e9-4bfd-b55e-149d9e3267d3-kube-api-access-9h6q6\") pod \"payload-processing-e2e-ipp-b-5a93d4-75b48cf6c6-dgrf8\" (UID: \"2708980c-46e9-4bfd-b55e-149d9e3267d3\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-5a93d4-75b48cf6c6-dgrf8"
Jul 31 23:51:13.964119 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:51:13.962989    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87czp\" (UniqueName: \"kubernetes.io/projected/c5da16a9-2cbe-46cc-afdc-76af2b1d878d-kube-api-access-87czp\") pod \"payload-pre-processing-e2e-ipp-b-5a93d4-b6bd494d8-f2prm\" (UID: \"c5da16a9-2cbe-46cc-afdc-76af2b1d878d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5a93d4-b6bd494d8-f2prm"
Jul 31 23:51:13.964119 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:51:13.963027    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c5da16a9-2cbe-46cc-afdc-76af2b1d878d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-5a93d4-b6bd494d8-f2prm\" (UID: \"c5da16a9-2cbe-46cc-afdc-76af2b1d878d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5a93d4-b6bd494d8-f2prm"
Jul 31 23:51:13.964119 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:51:13.963061    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2708980c-46e9-4bfd-b55e-149d9e3267d3-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-5a93d4-75b48cf6c6-dgrf8\" (UID: \"2708980c-46e9-4bfd-b55e-149d9e3267d3\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-5a93d4-75b48cf6c6-dgrf8"
Jul 31 23:51:13.964119 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:51:13.963950    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2708980c-46e9-4bfd-b55e-149d9e3267d3-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-5a93d4-75b48cf6c6-dgrf8\" (UID: \"2708980c-46e9-4bfd-b55e-149d9e3267d3\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-5a93d4-75b48cf6c6-dgrf8"
Jul 31 23:51:13.965127 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:51:13.965105    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c5da16a9-2cbe-46cc-afdc-76af2b1d878d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-5a93d4-b6bd494d8-f2prm\" (UID: \"c5da16a9-2cbe-46cc-afdc-76af2b1d878d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5a93d4-b6bd494d8-f2prm"
Jul 31 23:51:13.972264 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:51:13.972221    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-87czp\" (UniqueName: \"kubernetes.io/projected/c5da16a9-2cbe-46cc-afdc-76af2b1d878d-kube-api-access-87czp\") pod \"payload-pre-processing-e2e-ipp-b-5a93d4-b6bd494d8-f2prm\" (UID: \"c5da16a9-2cbe-46cc-afdc-76af2b1d878d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5a93d4-b6bd494d8-f2prm"
Jul 31 23:51:13.972416 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:51:13.972399    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9h6q6\" (UniqueName: \"kubernetes.io/projected/2708980c-46e9-4bfd-b55e-149d9e3267d3-kube-api-access-9h6q6\") pod \"payload-processing-e2e-ipp-b-5a93d4-75b48cf6c6-dgrf8\" (UID: \"2708980c-46e9-4bfd-b55e-149d9e3267d3\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-5a93d4-75b48cf6c6-dgrf8"
Jul 31 23:51:13.989622 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:51:13.989595    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-5a93d4-7bfbd454c8-4n6qh"]
Jul 31 23:51:13.992119 ip-10-0-128-243 kubenswrapper[2584]: W0731 23:51:13.992087    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76714e3b_4bab_476a_ab5b_1bb95995b0e1.slice/crio-5a3bff790358ce43d13d5835b0c0d56c5188d6e5f35976f2a3a42a56c947529e WatchSource:0}: Error finding container 5a3bff790358ce43d13d5835b0c0d56c5188d6e5f35976f2a3a42a56c947529e: Status 404 returned error can't find the container with id 5a3bff790358ce43d13d5835b0c0d56c5188d6e5f35976f2a3a42a56c947529e
Jul 31 23:51:14.020322 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:51:14.020299    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5a93d4-b6bd494d8-f2prm"
Jul 31 23:51:14.054410 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:51:14.054383    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-5a93d4-75b48cf6c6-dgrf8"
Jul 31 23:51:14.067200 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:51:14.067177    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5a93d4-7bfbd454c8-4n6qh" event={"ID":"76714e3b-4bab-476a-ab5b-1bb95995b0e1","Type":"ContainerStarted","Data":"5a3bff790358ce43d13d5835b0c0d56c5188d6e5f35976f2a3a42a56c947529e"}
Jul 31 23:51:14.182680 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:51:14.182648    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-5a93d4-b6bd494d8-f2prm"]
Jul 31 23:51:14.188087 ip-10-0-128-243 kubenswrapper[2584]: W0731 23:51:14.187975    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5da16a9_2cbe_46cc_afdc_76af2b1d878d.slice/crio-f617ea0f3306e49ad6d7be1529c77356eb0ed5ea8a45fe4eaa815adb6b793a80 WatchSource:0}: Error finding container f617ea0f3306e49ad6d7be1529c77356eb0ed5ea8a45fe4eaa815adb6b793a80: Status 404 returned error can't find the container with id f617ea0f3306e49ad6d7be1529c77356eb0ed5ea8a45fe4eaa815adb6b793a80
Jul 31 23:51:14.216817 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:51:14.216790    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-5a93d4-75b48cf6c6-dgrf8"]
Jul 31 23:51:14.217797 ip-10-0-128-243 kubenswrapper[2584]: W0731 23:51:14.217765    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2708980c_46e9_4bfd_b55e_149d9e3267d3.slice/crio-1c4951a517d1eed50a9d4120272aa2f0545330f53323227fe546e056ea0aa059 WatchSource:0}: Error finding container 1c4951a517d1eed50a9d4120272aa2f0545330f53323227fe546e056ea0aa059: Status 404 returned error can't find the container with id 1c4951a517d1eed50a9d4120272aa2f0545330f53323227fe546e056ea0aa059
Jul 31 23:51:15.074158 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:51:15.074119    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-5a93d4-75b48cf6c6-dgrf8" event={"ID":"2708980c-46e9-4bfd-b55e-149d9e3267d3","Type":"ContainerStarted","Data":"f0742bbb47f869245933bc1cc3baa21d867230725d7dadaafe486ba841d7fc84"}
Jul 31 23:51:15.074603 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:51:15.074183    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-5a93d4-75b48cf6c6-dgrf8" event={"ID":"2708980c-46e9-4bfd-b55e-149d9e3267d3","Type":"ContainerStarted","Data":"1c4951a517d1eed50a9d4120272aa2f0545330f53323227fe546e056ea0aa059"}
Jul 31 23:51:15.074603 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:51:15.074232    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-5a93d4-75b48cf6c6-dgrf8"
Jul 31 23:51:15.075756 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:51:15.075728    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5a93d4-b6bd494d8-f2prm" event={"ID":"c5da16a9-2cbe-46cc-afdc-76af2b1d878d","Type":"ContainerStarted","Data":"f53e81b3911de77129682721ad6f8acb7a2e2fd10f100b3b5be19d1ee697364b"}
Jul 31 23:51:15.075889 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:51:15.075812    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5a93d4-b6bd494d8-f2prm"
Jul 31 23:51:15.075889 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:51:15.075840    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5a93d4-b6bd494d8-f2prm" event={"ID":"c5da16a9-2cbe-46cc-afdc-76af2b1d878d","Type":"ContainerStarted","Data":"f617ea0f3306e49ad6d7be1529c77356eb0ed5ea8a45fe4eaa815adb6b793a80"}
Jul 31 23:51:15.077124 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:51:15.077100    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5a93d4-7bfbd454c8-4n6qh" event={"ID":"76714e3b-4bab-476a-ab5b-1bb95995b0e1","Type":"ContainerStarted","Data":"0e2e34e0c75ceb0a1d5ee4a9a921c2491783446d51199d778902703a4c455209"}
Jul 31 23:51:15.077231 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:51:15.077184    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5a93d4-7bfbd454c8-4n6qh"
Jul 31 23:51:15.091031 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:51:15.090992    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-5a93d4-75b48cf6c6-dgrf8" podStartSLOduration=2.090980644 podStartE2EDuration="2.090980644s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:51:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:51:15.090200136 +0000 UTC m=+3258.735986444" watchObservedRunningTime="2026-07-31 23:51:15.090980644 +0000 UTC m=+3258.736766953"
Jul 31 23:51:15.108204 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:51:15.108155    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5a93d4-7bfbd454c8-4n6qh" podStartSLOduration=2.671348386 podStartE2EDuration="3.108141467s" totalImagesPullingTime="436.793081ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:51:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:51:15.106130434 +0000 UTC m=+3258.751916746" watchObservedRunningTime="2026-07-31 23:51:15.108141467 +0000 UTC m=+3258.753927776"
Jul 31 23:51:15.122867 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:51:15.122805    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5a93d4-b6bd494d8-f2prm" podStartSLOduration=2.122788791 podStartE2EDuration="2.122788791s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:51:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:51:15.120847908 +0000 UTC m=+3258.766634218" watchObservedRunningTime="2026-07-31 23:51:15.122788791 +0000 UTC m=+3258.768575100"
Jul 31 23:51:21.088321 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:51:21.088290    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5a93d4-7bfbd454c8-4n6qh"
Jul 31 23:51:26.084818 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:51:26.084782    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5a93d4-b6bd494d8-f2prm"
Jul 31 23:51:26.085283 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:51:26.085193    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-5a93d4-75b48cf6c6-dgrf8"
Jul 31 23:51:43.327963 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:51:43.327923    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-29phv_f3023383-4445-4c1e-86a8-446b2e1307fa/payload-processing/0.log"
Jul 31 23:51:43.435912 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:51:43.435881    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-5a93d4-75b48cf6c6-dgrf8_2708980c-46e9-4bfd-b55e-149d9e3267d3/payload-processing/0.log"
Jul 31 23:52:38.600691 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:52:38.600653    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-29phv_f3023383-4445-4c1e-86a8-446b2e1307fa/payload-processing/0.log"
Jul 31 23:52:38.721830 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:52:38.721803    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-5a93d4-75b48cf6c6-dgrf8_2708980c-46e9-4bfd-b55e-149d9e3267d3/payload-processing/0.log"
Jul 31 23:52:43.480819 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:52:43.480789    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-74ebf4-59b5968868-nx8ql"]
Jul 31 23:52:43.484406 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:52:43.484385    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-74ebf4-59b5968868-nx8ql"
Jul 31 23:52:43.486822 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:52:43.486800    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-74ebf4-serving-cert\""
Jul 31 23:52:43.494206 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:52:43.494183    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-74ebf4-59b5968868-nx8ql"]
Jul 31 23:52:43.580014 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:52:43.579985    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad55314b-fbb0-4edc-a27c-550e3a4f90a1-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-74ebf4-59b5968868-nx8ql\" (UID: \"ad55314b-fbb0-4edc-a27c-550e3a4f90a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-74ebf4-59b5968868-nx8ql"
Jul 31 23:52:43.580196 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:52:43.580046    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ad55314b-fbb0-4edc-a27c-550e3a4f90a1-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-74ebf4-59b5968868-nx8ql\" (UID: \"ad55314b-fbb0-4edc-a27c-550e3a4f90a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-74ebf4-59b5968868-nx8ql"
Jul 31 23:52:43.580196 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:52:43.580168    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tm4ld\" (UniqueName: \"kubernetes.io/projected/ad55314b-fbb0-4edc-a27c-550e3a4f90a1-kube-api-access-tm4ld\") pod \"maas-api-e2e-ipp-cleanup-74ebf4-59b5968868-nx8ql\" (UID: \"ad55314b-fbb0-4edc-a27c-550e3a4f90a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-74ebf4-59b5968868-nx8ql"
Jul 31 23:52:43.681037 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:52:43.681001    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad55314b-fbb0-4edc-a27c-550e3a4f90a1-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-74ebf4-59b5968868-nx8ql\" (UID: \"ad55314b-fbb0-4edc-a27c-550e3a4f90a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-74ebf4-59b5968868-nx8ql"
Jul 31 23:52:43.681201 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:52:43.681065    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ad55314b-fbb0-4edc-a27c-550e3a4f90a1-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-74ebf4-59b5968868-nx8ql\" (UID: \"ad55314b-fbb0-4edc-a27c-550e3a4f90a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-74ebf4-59b5968868-nx8ql"
Jul 31 23:52:43.681201 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:52:43.681109    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tm4ld\" (UniqueName: \"kubernetes.io/projected/ad55314b-fbb0-4edc-a27c-550e3a4f90a1-kube-api-access-tm4ld\") pod \"maas-api-e2e-ipp-cleanup-74ebf4-59b5968868-nx8ql\" (UID: \"ad55314b-fbb0-4edc-a27c-550e3a4f90a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-74ebf4-59b5968868-nx8ql"
Jul 31 23:52:43.681644 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:52:43.681618    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad55314b-fbb0-4edc-a27c-550e3a4f90a1-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-74ebf4-59b5968868-nx8ql\" (UID: \"ad55314b-fbb0-4edc-a27c-550e3a4f90a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-74ebf4-59b5968868-nx8ql"
Jul 31 23:52:43.683472 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:52:43.683450    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ad55314b-fbb0-4edc-a27c-550e3a4f90a1-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-74ebf4-59b5968868-nx8ql\" (UID: \"ad55314b-fbb0-4edc-a27c-550e3a4f90a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-74ebf4-59b5968868-nx8ql"
Jul 31 23:52:43.689102 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:52:43.689080    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tm4ld\" (UniqueName: \"kubernetes.io/projected/ad55314b-fbb0-4edc-a27c-550e3a4f90a1-kube-api-access-tm4ld\") pod \"maas-api-e2e-ipp-cleanup-74ebf4-59b5968868-nx8ql\" (UID: \"ad55314b-fbb0-4edc-a27c-550e3a4f90a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-74ebf4-59b5968868-nx8ql"
Jul 31 23:52:43.796663 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:52:43.796587    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-74ebf4-59b5968868-nx8ql"
Jul 31 23:52:44.115115 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:52:44.115080    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-74ebf4-55fb6b469b-59f7m"]
Jul 31 23:52:44.119264 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:52:44.119238    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-74ebf4-55fb6b469b-59f7m"
Jul 31 23:52:44.122033 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:52:44.122010    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-74ebf4-dockercfg-hvql9\""
Jul 31 23:52:44.122171 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:52:44.122048    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-74ebf4\""
Jul 31 23:52:44.128888 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:52:44.128864    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-74ebf4-55fb6b469b-59f7m"]
Jul 31 23:52:44.156254 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:52:44.156230    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-74ebf4-59b5968868-nx8ql"]
Jul 31 23:52:44.158392 ip-10-0-128-243 kubenswrapper[2584]: W0731 23:52:44.158363    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad55314b_fbb0_4edc_a27c_550e3a4f90a1.slice/crio-d88fa74f063dd4ae0fd9830f3d1c45f87977ae487fda0b269d9fcc2572bb757f WatchSource:0}: Error finding container d88fa74f063dd4ae0fd9830f3d1c45f87977ae487fda0b269d9fcc2572bb757f: Status 404 returned error can't find the container with id d88fa74f063dd4ae0fd9830f3d1c45f87977ae487fda0b269d9fcc2572bb757f
Jul 31 23:52:44.287879 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:52:44.287836    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nflm9\" (UniqueName: \"kubernetes.io/projected/f6be327e-fe3d-470c-986e-76aeef34cee3-kube-api-access-nflm9\") pod \"payload-pre-processing-e2e-ipp-cleanup-74ebf4-55fb6b469b-59f7m\" (UID: \"f6be327e-fe3d-470c-986e-76aeef34cee3\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-74ebf4-55fb6b469b-59f7m"
Jul 31 23:52:44.288040 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:52:44.288008    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6be327e-fe3d-470c-986e-76aeef34cee3-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-74ebf4-55fb6b469b-59f7m\" (UID: \"f6be327e-fe3d-470c-986e-76aeef34cee3\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-74ebf4-55fb6b469b-59f7m"
Jul 31 23:52:44.388594 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:52:44.388503    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nflm9\" (UniqueName: \"kubernetes.io/projected/f6be327e-fe3d-470c-986e-76aeef34cee3-kube-api-access-nflm9\") pod \"payload-pre-processing-e2e-ipp-cleanup-74ebf4-55fb6b469b-59f7m\" (UID: \"f6be327e-fe3d-470c-986e-76aeef34cee3\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-74ebf4-55fb6b469b-59f7m"
Jul 31 23:52:44.388798 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:52:44.388610    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6be327e-fe3d-470c-986e-76aeef34cee3-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-74ebf4-55fb6b469b-59f7m\" (UID: \"f6be327e-fe3d-470c-986e-76aeef34cee3\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-74ebf4-55fb6b469b-59f7m"
Jul 31 23:52:44.389393 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:52:44.389369    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6be327e-fe3d-470c-986e-76aeef34cee3-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-74ebf4-55fb6b469b-59f7m\" (UID: \"f6be327e-fe3d-470c-986e-76aeef34cee3\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-74ebf4-55fb6b469b-59f7m"
Jul 31 23:52:44.396643 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:52:44.396622    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nflm9\" (UniqueName: \"kubernetes.io/projected/f6be327e-fe3d-470c-986e-76aeef34cee3-kube-api-access-nflm9\") pod \"payload-pre-processing-e2e-ipp-cleanup-74ebf4-55fb6b469b-59f7m\" (UID: \"f6be327e-fe3d-470c-986e-76aeef34cee3\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-74ebf4-55fb6b469b-59f7m"
Jul 31 23:52:44.431617 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:52:44.431588    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-74ebf4-55fb6b469b-59f7m"
Jul 31 23:52:44.535316 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:52:44.535285    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-74ebf4-59b5968868-nx8ql" event={"ID":"ad55314b-fbb0-4edc-a27c-550e3a4f90a1","Type":"ContainerStarted","Data":"d88fa74f063dd4ae0fd9830f3d1c45f87977ae487fda0b269d9fcc2572bb757f"}
Jul 31 23:52:44.570915 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:52:44.570885    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-74ebf4-55fb6b469b-59f7m"]
Jul 31 23:52:44.572374 ip-10-0-128-243 kubenswrapper[2584]: W0731 23:52:44.572349    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6be327e_fe3d_470c_986e_76aeef34cee3.slice/crio-508b5bfc0155536e3a6161e61ba16054deb4bf78c9c2a574410baefaadb3a285 WatchSource:0}: Error finding container 508b5bfc0155536e3a6161e61ba16054deb4bf78c9c2a574410baefaadb3a285: Status 404 returned error can't find the container with id 508b5bfc0155536e3a6161e61ba16054deb4bf78c9c2a574410baefaadb3a285
Jul 31 23:52:45.543037 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:52:45.543002    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-74ebf4-59b5968868-nx8ql" event={"ID":"ad55314b-fbb0-4edc-a27c-550e3a4f90a1","Type":"ContainerStarted","Data":"f38a88975456f92ab235c49977ec6130d57eef94eb766f765cc71a018aaf5055"}
Jul 31 23:52:45.543518 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:52:45.543121    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-74ebf4-59b5968868-nx8ql"
Jul 31 23:52:45.544576 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:52:45.544547    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-74ebf4-55fb6b469b-59f7m" event={"ID":"f6be327e-fe3d-470c-986e-76aeef34cee3","Type":"ContainerStarted","Data":"8ae60324d460329a9ae2519e90d5abf0be537fefd42fb251ac43530f24e465cb"}
Jul 31 23:52:45.544775 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:52:45.544598    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-74ebf4-55fb6b469b-59f7m" event={"ID":"f6be327e-fe3d-470c-986e-76aeef34cee3","Type":"ContainerStarted","Data":"508b5bfc0155536e3a6161e61ba16054deb4bf78c9c2a574410baefaadb3a285"}
Jul 31 23:52:45.544775 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:52:45.544627    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-74ebf4-55fb6b469b-59f7m"
Jul 31 23:52:45.558705 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:52:45.558661    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-74ebf4-59b5968868-nx8ql" podStartSLOduration=2.134948429 podStartE2EDuration="2.558645473s" totalImagesPullingTime="423.697044ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:52:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:52:45.557823913 +0000 UTC m=+3349.203610221" watchObservedRunningTime="2026-07-31 23:52:45.558645473 +0000 UTC m=+3349.204431781"
Jul 31 23:52:45.573309 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:52:45.573262    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-74ebf4-55fb6b469b-59f7m" podStartSLOduration=1.573247536 podStartE2EDuration="1.573247536s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:52:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:52:45.57266786 +0000 UTC m=+3349.218454185" watchObservedRunningTime="2026-07-31 23:52:45.573247536 +0000 UTC m=+3349.219033844"
Jul 31 23:52:51.559204 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:52:51.559173    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-74ebf4-59b5968868-nx8ql"
Jul 31 23:52:56.555425 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:52:56.555393    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-74ebf4-55fb6b469b-59f7m"
Jul 31 23:53:14.176928 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:53:14.176895    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-74ebf4-59b5968868-nx8ql"]
Jul 31 23:53:14.177491 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:53:14.177116    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-74ebf4-59b5968868-nx8ql" podUID="ad55314b-fbb0-4edc-a27c-550e3a4f90a1" containerName="maas-api" containerID="cri-o://f38a88975456f92ab235c49977ec6130d57eef94eb766f765cc71a018aaf5055" gracePeriod=30
Jul 31 23:53:14.237509 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:53:14.237464    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-74ebf4-55fb6b469b-59f7m"]
Jul 31 23:53:14.238399 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:53:14.238188    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-74ebf4-55fb6b469b-59f7m" podUID="f6be327e-fe3d-470c-986e-76aeef34cee3" containerName="payload-pre-processing" containerID="cri-o://8ae60324d460329a9ae2519e90d5abf0be537fefd42fb251ac43530f24e465cb" gracePeriod=30
Jul 31 23:53:14.499642 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:53:14.499612    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-74ebf4-59b5968868-nx8ql"
Jul 31 23:53:14.530523 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:53:14.530499    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-74ebf4-55fb6b469b-59f7m"
Jul 31 23:53:14.555070 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:53:14.555038    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad55314b-fbb0-4edc-a27c-550e3a4f90a1-openshift-service-ca\") pod \"ad55314b-fbb0-4edc-a27c-550e3a4f90a1\" (UID: \"ad55314b-fbb0-4edc-a27c-550e3a4f90a1\") "
Jul 31 23:53:14.555219 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:53:14.555155    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tm4ld\" (UniqueName: \"kubernetes.io/projected/ad55314b-fbb0-4edc-a27c-550e3a4f90a1-kube-api-access-tm4ld\") pod \"ad55314b-fbb0-4edc-a27c-550e3a4f90a1\" (UID: \"ad55314b-fbb0-4edc-a27c-550e3a4f90a1\") "
Jul 31 23:53:14.555275 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:53:14.555218    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ad55314b-fbb0-4edc-a27c-550e3a4f90a1-maas-api-tls\") pod \"ad55314b-fbb0-4edc-a27c-550e3a4f90a1\" (UID: \"ad55314b-fbb0-4edc-a27c-550e3a4f90a1\") "
Jul 31 23:53:14.555440 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:53:14.555408    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad55314b-fbb0-4edc-a27c-550e3a4f90a1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ad55314b-fbb0-4edc-a27c-550e3a4f90a1" (UID: "ad55314b-fbb0-4edc-a27c-550e3a4f90a1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:53:14.555588 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:53:14.555566    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad55314b-fbb0-4edc-a27c-550e3a4f90a1-openshift-service-ca\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:53:14.557296 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:53:14.557270    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad55314b-fbb0-4edc-a27c-550e3a4f90a1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ad55314b-fbb0-4edc-a27c-550e3a4f90a1" (UID: "ad55314b-fbb0-4edc-a27c-550e3a4f90a1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 23:53:14.557393 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:53:14.557355    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad55314b-fbb0-4edc-a27c-550e3a4f90a1-kube-api-access-tm4ld" (OuterVolumeSpecName: "kube-api-access-tm4ld") pod "ad55314b-fbb0-4edc-a27c-550e3a4f90a1" (UID: "ad55314b-fbb0-4edc-a27c-550e3a4f90a1"). InnerVolumeSpecName "kube-api-access-tm4ld". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 23:53:14.656313 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:53:14.656288    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nflm9\" (UniqueName: \"kubernetes.io/projected/f6be327e-fe3d-470c-986e-76aeef34cee3-kube-api-access-nflm9\") pod \"f6be327e-fe3d-470c-986e-76aeef34cee3\" (UID: \"f6be327e-fe3d-470c-986e-76aeef34cee3\") "
Jul 31 23:53:14.656441 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:53:14.656352    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6be327e-fe3d-470c-986e-76aeef34cee3-plugins-config-volume\") pod \"f6be327e-fe3d-470c-986e-76aeef34cee3\" (UID: \"f6be327e-fe3d-470c-986e-76aeef34cee3\") "
Jul 31 23:53:14.656562 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:53:14.656552    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ad55314b-fbb0-4edc-a27c-550e3a4f90a1-maas-api-tls\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:53:14.656602 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:53:14.656566    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tm4ld\" (UniqueName: \"kubernetes.io/projected/ad55314b-fbb0-4edc-a27c-550e3a4f90a1-kube-api-access-tm4ld\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:53:14.656757 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:53:14.656733    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6be327e-fe3d-470c-986e-76aeef34cee3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f6be327e-fe3d-470c-986e-76aeef34cee3" (UID: "f6be327e-fe3d-470c-986e-76aeef34cee3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:53:14.658257 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:53:14.658239    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6be327e-fe3d-470c-986e-76aeef34cee3-kube-api-access-nflm9" (OuterVolumeSpecName: "kube-api-access-nflm9") pod "f6be327e-fe3d-470c-986e-76aeef34cee3" (UID: "f6be327e-fe3d-470c-986e-76aeef34cee3"). InnerVolumeSpecName "kube-api-access-nflm9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 23:53:14.704199 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:53:14.704160    2584 generic.go:356] "Generic (PLEG): container finished" podID="ad55314b-fbb0-4edc-a27c-550e3a4f90a1" containerID="f38a88975456f92ab235c49977ec6130d57eef94eb766f765cc71a018aaf5055" exitCode=0
Jul 31 23:53:14.704374 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:53:14.704209    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-74ebf4-59b5968868-nx8ql"
Jul 31 23:53:14.704374 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:53:14.704245    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-74ebf4-59b5968868-nx8ql" event={"ID":"ad55314b-fbb0-4edc-a27c-550e3a4f90a1","Type":"ContainerDied","Data":"f38a88975456f92ab235c49977ec6130d57eef94eb766f765cc71a018aaf5055"}
Jul 31 23:53:14.704374 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:53:14.704299    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-74ebf4-59b5968868-nx8ql" event={"ID":"ad55314b-fbb0-4edc-a27c-550e3a4f90a1","Type":"ContainerDied","Data":"d88fa74f063dd4ae0fd9830f3d1c45f87977ae487fda0b269d9fcc2572bb757f"}
Jul 31 23:53:14.704374 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:53:14.704321    2584 scope.go:117] "RemoveContainer" containerID="f38a88975456f92ab235c49977ec6130d57eef94eb766f765cc71a018aaf5055"
Jul 31 23:53:14.705562 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:53:14.705539    2584 generic.go:356] "Generic (PLEG): container finished" podID="f6be327e-fe3d-470c-986e-76aeef34cee3" containerID="8ae60324d460329a9ae2519e90d5abf0be537fefd42fb251ac43530f24e465cb" exitCode=0
Jul 31 23:53:14.705681 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:53:14.705589    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-74ebf4-55fb6b469b-59f7m"
Jul 31 23:53:14.705681 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:53:14.705598    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-74ebf4-55fb6b469b-59f7m" event={"ID":"f6be327e-fe3d-470c-986e-76aeef34cee3","Type":"ContainerDied","Data":"8ae60324d460329a9ae2519e90d5abf0be537fefd42fb251ac43530f24e465cb"}
Jul 31 23:53:14.705681 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:53:14.705630    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-74ebf4-55fb6b469b-59f7m" event={"ID":"f6be327e-fe3d-470c-986e-76aeef34cee3","Type":"ContainerDied","Data":"508b5bfc0155536e3a6161e61ba16054deb4bf78c9c2a574410baefaadb3a285"}
Jul 31 23:53:14.713301 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:53:14.713267    2584 scope.go:117] "RemoveContainer" containerID="f38a88975456f92ab235c49977ec6130d57eef94eb766f765cc71a018aaf5055"
Jul 31 23:53:14.713629 ip-10-0-128-243 kubenswrapper[2584]: E0731 23:53:14.713610    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f38a88975456f92ab235c49977ec6130d57eef94eb766f765cc71a018aaf5055\": container with ID starting with f38a88975456f92ab235c49977ec6130d57eef94eb766f765cc71a018aaf5055 not found: ID does not exist" containerID="f38a88975456f92ab235c49977ec6130d57eef94eb766f765cc71a018aaf5055"
Jul 31 23:53:14.713752 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:53:14.713636    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f38a88975456f92ab235c49977ec6130d57eef94eb766f765cc71a018aaf5055"} err="failed to get container status \"f38a88975456f92ab235c49977ec6130d57eef94eb766f765cc71a018aaf5055\": rpc error: code = NotFound desc = could not find container \"f38a88975456f92ab235c49977ec6130d57eef94eb766f765cc71a018aaf5055\": container with ID starting with f38a88975456f92ab235c49977ec6130d57eef94eb766f765cc71a018aaf5055 not found: ID does not exist"
Jul 31 23:53:14.713752 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:53:14.713653    2584 scope.go:117] "RemoveContainer" containerID="8ae60324d460329a9ae2519e90d5abf0be537fefd42fb251ac43530f24e465cb"
Jul 31 23:53:14.721789 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:53:14.721773    2584 scope.go:117] "RemoveContainer" containerID="8ae60324d460329a9ae2519e90d5abf0be537fefd42fb251ac43530f24e465cb"
Jul 31 23:53:14.722037 ip-10-0-128-243 kubenswrapper[2584]: E0731 23:53:14.722019    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ae60324d460329a9ae2519e90d5abf0be537fefd42fb251ac43530f24e465cb\": container with ID starting with 8ae60324d460329a9ae2519e90d5abf0be537fefd42fb251ac43530f24e465cb not found: ID does not exist" containerID="8ae60324d460329a9ae2519e90d5abf0be537fefd42fb251ac43530f24e465cb"
Jul 31 23:53:14.722108 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:53:14.722046    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ae60324d460329a9ae2519e90d5abf0be537fefd42fb251ac43530f24e465cb"} err="failed to get container status \"8ae60324d460329a9ae2519e90d5abf0be537fefd42fb251ac43530f24e465cb\": rpc error: code = NotFound desc = could not find container \"8ae60324d460329a9ae2519e90d5abf0be537fefd42fb251ac43530f24e465cb\": container with ID starting with 8ae60324d460329a9ae2519e90d5abf0be537fefd42fb251ac43530f24e465cb not found: ID does not exist"
Jul 31 23:53:14.730648 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:53:14.730592    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-74ebf4-59b5968868-nx8ql"]
Jul 31 23:53:14.734732 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:53:14.734687    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-74ebf4-59b5968868-nx8ql"]
Jul 31 23:53:14.744566 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:53:14.744544    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-74ebf4-55fb6b469b-59f7m"]
Jul 31 23:53:14.748685 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:53:14.748661    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-74ebf4-55fb6b469b-59f7m"]
Jul 31 23:53:14.758088 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:53:14.758065    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6be327e-fe3d-470c-986e-76aeef34cee3-plugins-config-volume\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:53:14.758172 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:53:14.758090    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nflm9\" (UniqueName: \"kubernetes.io/projected/f6be327e-fe3d-470c-986e-76aeef34cee3-kube-api-access-nflm9\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:53:14.891190 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:53:14.891157    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad55314b-fbb0-4edc-a27c-550e3a4f90a1" path="/var/lib/kubelet/pods/ad55314b-fbb0-4edc-a27c-550e3a4f90a1/volumes"
Jul 31 23:53:14.891528 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:53:14.891514    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6be327e-fe3d-470c-986e-76aeef34cee3" path="/var/lib/kubelet/pods/f6be327e-fe3d-470c-986e-76aeef34cee3/volumes"
Jul 31 23:53:39.721685 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:53:39.721652    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-c16217-74f9dd6987-f876j"]
Jul 31 23:53:39.722167 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:53:39.721934    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c16217-74f9dd6987-f876j" podUID="bd76d76e-27c6-4fda-b5b2-1253e05ea281" containerName="maas-api" containerID="cri-o://a0f5d38debee187f28bd27d89df4c361bc47a63e2c4276b59cd08c0ce52c521e" gracePeriod=30
Jul 31 23:53:39.841962 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:53:39.841933    2584 generic.go:356] "Generic (PLEG): container finished" podID="bd76d76e-27c6-4fda-b5b2-1253e05ea281" containerID="a0f5d38debee187f28bd27d89df4c361bc47a63e2c4276b59cd08c0ce52c521e" exitCode=0
Jul 31 23:53:39.842091 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:53:39.842008    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c16217-74f9dd6987-f876j" event={"ID":"bd76d76e-27c6-4fda-b5b2-1253e05ea281","Type":"ContainerDied","Data":"a0f5d38debee187f28bd27d89df4c361bc47a63e2c4276b59cd08c0ce52c521e"}
Jul 31 23:53:39.996084 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:53:39.996060    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c16217-74f9dd6987-f876j"
Jul 31 23:53:40.092157 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:53:40.092120    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd76d76e-27c6-4fda-b5b2-1253e05ea281-maas-api-tls\") pod \"bd76d76e-27c6-4fda-b5b2-1253e05ea281\" (UID: \"bd76d76e-27c6-4fda-b5b2-1253e05ea281\") "
Jul 31 23:53:40.092340 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:53:40.092190    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f4b75\" (UniqueName: \"kubernetes.io/projected/bd76d76e-27c6-4fda-b5b2-1253e05ea281-kube-api-access-f4b75\") pod \"bd76d76e-27c6-4fda-b5b2-1253e05ea281\" (UID: \"bd76d76e-27c6-4fda-b5b2-1253e05ea281\") "
Jul 31 23:53:40.092340 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:53:40.092232    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd76d76e-27c6-4fda-b5b2-1253e05ea281-openshift-service-ca\") pod \"bd76d76e-27c6-4fda-b5b2-1253e05ea281\" (UID: \"bd76d76e-27c6-4fda-b5b2-1253e05ea281\") "
Jul 31 23:53:40.092636 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:53:40.092610    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd76d76e-27c6-4fda-b5b2-1253e05ea281-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bd76d76e-27c6-4fda-b5b2-1253e05ea281" (UID: "bd76d76e-27c6-4fda-b5b2-1253e05ea281"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:53:40.094327 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:53:40.094299    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd76d76e-27c6-4fda-b5b2-1253e05ea281-kube-api-access-f4b75" (OuterVolumeSpecName: "kube-api-access-f4b75") pod "bd76d76e-27c6-4fda-b5b2-1253e05ea281" (UID: "bd76d76e-27c6-4fda-b5b2-1253e05ea281"). InnerVolumeSpecName "kube-api-access-f4b75". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 23:53:40.094327 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:53:40.094302    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd76d76e-27c6-4fda-b5b2-1253e05ea281-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bd76d76e-27c6-4fda-b5b2-1253e05ea281" (UID: "bd76d76e-27c6-4fda-b5b2-1253e05ea281"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 23:53:40.193174 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:53:40.193138    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f4b75\" (UniqueName: \"kubernetes.io/projected/bd76d76e-27c6-4fda-b5b2-1253e05ea281-kube-api-access-f4b75\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:53:40.193174 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:53:40.193169    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd76d76e-27c6-4fda-b5b2-1253e05ea281-openshift-service-ca\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:53:40.193174 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:53:40.193181    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd76d76e-27c6-4fda-b5b2-1253e05ea281-maas-api-tls\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:53:40.848985 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:53:40.848948    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c16217-74f9dd6987-f876j" event={"ID":"bd76d76e-27c6-4fda-b5b2-1253e05ea281","Type":"ContainerDied","Data":"e6cf7e97c2dc2ec92ec2dc51aa7e01962e1b21ab40d2dce6c57014e859a43d61"}
Jul 31 23:53:40.849375 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:53:40.849010    2584 scope.go:117] "RemoveContainer" containerID="a0f5d38debee187f28bd27d89df4c361bc47a63e2c4276b59cd08c0ce52c521e"
Jul 31 23:53:40.849375 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:53:40.848960    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c16217-74f9dd6987-f876j"
Jul 31 23:53:40.869370 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:53:40.869344    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-c16217-74f9dd6987-f876j"]
Jul 31 23:53:40.873114 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:53:40.873092    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-c16217-74f9dd6987-f876j"]
Jul 31 23:53:40.892011 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:53:40.891989    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd76d76e-27c6-4fda-b5b2-1253e05ea281" path="/var/lib/kubelet/pods/bd76d76e-27c6-4fda-b5b2-1253e05ea281/volumes"
Jul 31 23:54:39.099166 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:39.099128    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-5a93d4-7bfbd454c8-4n6qh"]
Jul 31 23:54:39.100453 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:39.100389    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5a93d4-7bfbd454c8-4n6qh" podUID="76714e3b-4bab-476a-ab5b-1bb95995b0e1" containerName="maas-api" containerID="cri-o://0e2e34e0c75ceb0a1d5ee4a9a921c2491783446d51199d778902703a4c455209" gracePeriod=30
Jul 31 23:54:39.155472 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:39.155440    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-5a93d4-75b48cf6c6-dgrf8"]
Jul 31 23:54:39.155727 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:39.155661    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-5a93d4-75b48cf6c6-dgrf8" podUID="2708980c-46e9-4bfd-b55e-149d9e3267d3" containerName="payload-processing" containerID="cri-o://f0742bbb47f869245933bc1cc3baa21d867230725d7dadaafe486ba841d7fc84" gracePeriod=30
Jul 31 23:54:39.163483 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:39.163451    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-5a93d4-b6bd494d8-f2prm"]
Jul 31 23:54:39.163720 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:39.163678    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5a93d4-b6bd494d8-f2prm" podUID="c5da16a9-2cbe-46cc-afdc-76af2b1d878d" containerName="payload-pre-processing" containerID="cri-o://f53e81b3911de77129682721ad6f8acb7a2e2fd10f100b3b5be19d1ee697364b" gracePeriod=30
Jul 31 23:54:39.450671 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:39.450643    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5a93d4-7bfbd454c8-4n6qh"
Jul 31 23:54:39.469828 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:39.469768    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-5a93d4-75b48cf6c6-dgrf8"
Jul 31 23:54:39.481248 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:39.481224    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5a93d4-b6bd494d8-f2prm"
Jul 31 23:54:39.541921 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:39.541887    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9h6q6\" (UniqueName: \"kubernetes.io/projected/2708980c-46e9-4bfd-b55e-149d9e3267d3-kube-api-access-9h6q6\") pod \"2708980c-46e9-4bfd-b55e-149d9e3267d3\" (UID: \"2708980c-46e9-4bfd-b55e-149d9e3267d3\") "
Jul 31 23:54:39.541921 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:39.541925    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/76714e3b-4bab-476a-ab5b-1bb95995b0e1-maas-api-tls\") pod \"76714e3b-4bab-476a-ab5b-1bb95995b0e1\" (UID: \"76714e3b-4bab-476a-ab5b-1bb95995b0e1\") "
Jul 31 23:54:39.542173 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:39.541947    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-87czp\" (UniqueName: \"kubernetes.io/projected/c5da16a9-2cbe-46cc-afdc-76af2b1d878d-kube-api-access-87czp\") pod \"c5da16a9-2cbe-46cc-afdc-76af2b1d878d\" (UID: \"c5da16a9-2cbe-46cc-afdc-76af2b1d878d\") "
Jul 31 23:54:39.542173 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:39.541963    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2708980c-46e9-4bfd-b55e-149d9e3267d3-plugins-config-volume\") pod \"2708980c-46e9-4bfd-b55e-149d9e3267d3\" (UID: \"2708980c-46e9-4bfd-b55e-149d9e3267d3\") "
Jul 31 23:54:39.542173 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:39.541990    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c5da16a9-2cbe-46cc-afdc-76af2b1d878d-plugins-config-volume\") pod \"c5da16a9-2cbe-46cc-afdc-76af2b1d878d\" (UID: \"c5da16a9-2cbe-46cc-afdc-76af2b1d878d\") "
Jul 31 23:54:39.542173 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:39.542029    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/76714e3b-4bab-476a-ab5b-1bb95995b0e1-openshift-service-ca\") pod \"76714e3b-4bab-476a-ab5b-1bb95995b0e1\" (UID: \"76714e3b-4bab-476a-ab5b-1bb95995b0e1\") "
Jul 31 23:54:39.542173 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:39.542089    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z9vp5\" (UniqueName: \"kubernetes.io/projected/76714e3b-4bab-476a-ab5b-1bb95995b0e1-kube-api-access-z9vp5\") pod \"76714e3b-4bab-476a-ab5b-1bb95995b0e1\" (UID: \"76714e3b-4bab-476a-ab5b-1bb95995b0e1\") "
Jul 31 23:54:39.542462 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:39.542435    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76714e3b-4bab-476a-ab5b-1bb95995b0e1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "76714e3b-4bab-476a-ab5b-1bb95995b0e1" (UID: "76714e3b-4bab-476a-ab5b-1bb95995b0e1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:54:39.542525 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:39.542454    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c5da16a9-2cbe-46cc-afdc-76af2b1d878d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c5da16a9-2cbe-46cc-afdc-76af2b1d878d" (UID: "c5da16a9-2cbe-46cc-afdc-76af2b1d878d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:54:39.542525 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:39.542454    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2708980c-46e9-4bfd-b55e-149d9e3267d3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2708980c-46e9-4bfd-b55e-149d9e3267d3" (UID: "2708980c-46e9-4bfd-b55e-149d9e3267d3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:54:39.544200 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:39.544165    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2708980c-46e9-4bfd-b55e-149d9e3267d3-kube-api-access-9h6q6" (OuterVolumeSpecName: "kube-api-access-9h6q6") pod "2708980c-46e9-4bfd-b55e-149d9e3267d3" (UID: "2708980c-46e9-4bfd-b55e-149d9e3267d3"). InnerVolumeSpecName "kube-api-access-9h6q6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 23:54:39.544200 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:39.544187    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/76714e3b-4bab-476a-ab5b-1bb95995b0e1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "76714e3b-4bab-476a-ab5b-1bb95995b0e1" (UID: "76714e3b-4bab-476a-ab5b-1bb95995b0e1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 23:54:39.544351 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:39.544244    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76714e3b-4bab-476a-ab5b-1bb95995b0e1-kube-api-access-z9vp5" (OuterVolumeSpecName: "kube-api-access-z9vp5") pod "76714e3b-4bab-476a-ab5b-1bb95995b0e1" (UID: "76714e3b-4bab-476a-ab5b-1bb95995b0e1"). InnerVolumeSpecName "kube-api-access-z9vp5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 23:54:39.544351 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:39.544267    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5da16a9-2cbe-46cc-afdc-76af2b1d878d-kube-api-access-87czp" (OuterVolumeSpecName: "kube-api-access-87czp") pod "c5da16a9-2cbe-46cc-afdc-76af2b1d878d" (UID: "c5da16a9-2cbe-46cc-afdc-76af2b1d878d"). InnerVolumeSpecName "kube-api-access-87czp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 23:54:39.643788 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:39.643690    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z9vp5\" (UniqueName: \"kubernetes.io/projected/76714e3b-4bab-476a-ab5b-1bb95995b0e1-kube-api-access-z9vp5\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:54:39.643788 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:39.643732    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9h6q6\" (UniqueName: \"kubernetes.io/projected/2708980c-46e9-4bfd-b55e-149d9e3267d3-kube-api-access-9h6q6\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:54:39.643788 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:39.643747    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/76714e3b-4bab-476a-ab5b-1bb95995b0e1-maas-api-tls\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:54:39.643788 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:39.643759    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-87czp\" (UniqueName: \"kubernetes.io/projected/c5da16a9-2cbe-46cc-afdc-76af2b1d878d-kube-api-access-87czp\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:54:39.643788 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:39.643771    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2708980c-46e9-4bfd-b55e-149d9e3267d3-plugins-config-volume\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:54:39.643788 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:39.643783    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c5da16a9-2cbe-46cc-afdc-76af2b1d878d-plugins-config-volume\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:54:39.644095 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:39.643796    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/76714e3b-4bab-476a-ab5b-1bb95995b0e1-openshift-service-ca\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:54:40.152970 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:40.152936    2584 generic.go:356] "Generic (PLEG): container finished" podID="2708980c-46e9-4bfd-b55e-149d9e3267d3" containerID="f0742bbb47f869245933bc1cc3baa21d867230725d7dadaafe486ba841d7fc84" exitCode=0
Jul 31 23:54:40.153403 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:40.152981    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-5a93d4-75b48cf6c6-dgrf8"
Jul 31 23:54:40.153403 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:40.153021    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-5a93d4-75b48cf6c6-dgrf8" event={"ID":"2708980c-46e9-4bfd-b55e-149d9e3267d3","Type":"ContainerDied","Data":"f0742bbb47f869245933bc1cc3baa21d867230725d7dadaafe486ba841d7fc84"}
Jul 31 23:54:40.153403 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:40.153081    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-5a93d4-75b48cf6c6-dgrf8" event={"ID":"2708980c-46e9-4bfd-b55e-149d9e3267d3","Type":"ContainerDied","Data":"1c4951a517d1eed50a9d4120272aa2f0545330f53323227fe546e056ea0aa059"}
Jul 31 23:54:40.153403 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:40.153115    2584 scope.go:117] "RemoveContainer" containerID="f0742bbb47f869245933bc1cc3baa21d867230725d7dadaafe486ba841d7fc84"
Jul 31 23:54:40.154401 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:40.154378    2584 generic.go:356] "Generic (PLEG): container finished" podID="c5da16a9-2cbe-46cc-afdc-76af2b1d878d" containerID="f53e81b3911de77129682721ad6f8acb7a2e2fd10f100b3b5be19d1ee697364b" exitCode=0
Jul 31 23:54:40.154490 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:40.154412    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5a93d4-b6bd494d8-f2prm" event={"ID":"c5da16a9-2cbe-46cc-afdc-76af2b1d878d","Type":"ContainerDied","Data":"f53e81b3911de77129682721ad6f8acb7a2e2fd10f100b3b5be19d1ee697364b"}
Jul 31 23:54:40.154490 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:40.154460    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5a93d4-b6bd494d8-f2prm" event={"ID":"c5da16a9-2cbe-46cc-afdc-76af2b1d878d","Type":"ContainerDied","Data":"f617ea0f3306e49ad6d7be1529c77356eb0ed5ea8a45fe4eaa815adb6b793a80"}
Jul 31 23:54:40.154490 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:40.154434    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5a93d4-b6bd494d8-f2prm"
Jul 31 23:54:40.155755 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:40.155729    2584 generic.go:356] "Generic (PLEG): container finished" podID="76714e3b-4bab-476a-ab5b-1bb95995b0e1" containerID="0e2e34e0c75ceb0a1d5ee4a9a921c2491783446d51199d778902703a4c455209" exitCode=0
Jul 31 23:54:40.155866 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:40.155770    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5a93d4-7bfbd454c8-4n6qh"
Jul 31 23:54:40.155866 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:40.155765    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5a93d4-7bfbd454c8-4n6qh" event={"ID":"76714e3b-4bab-476a-ab5b-1bb95995b0e1","Type":"ContainerDied","Data":"0e2e34e0c75ceb0a1d5ee4a9a921c2491783446d51199d778902703a4c455209"}
Jul 31 23:54:40.155866 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:40.155827    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5a93d4-7bfbd454c8-4n6qh" event={"ID":"76714e3b-4bab-476a-ab5b-1bb95995b0e1","Type":"ContainerDied","Data":"5a3bff790358ce43d13d5835b0c0d56c5188d6e5f35976f2a3a42a56c947529e"}
Jul 31 23:54:40.162945 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:40.162928    2584 scope.go:117] "RemoveContainer" containerID="f0742bbb47f869245933bc1cc3baa21d867230725d7dadaafe486ba841d7fc84"
Jul 31 23:54:40.163288 ip-10-0-128-243 kubenswrapper[2584]: E0731 23:54:40.163258    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0742bbb47f869245933bc1cc3baa21d867230725d7dadaafe486ba841d7fc84\": container with ID starting with f0742bbb47f869245933bc1cc3baa21d867230725d7dadaafe486ba841d7fc84 not found: ID does not exist" containerID="f0742bbb47f869245933bc1cc3baa21d867230725d7dadaafe486ba841d7fc84"
Jul 31 23:54:40.163385 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:40.163299    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0742bbb47f869245933bc1cc3baa21d867230725d7dadaafe486ba841d7fc84"} err="failed to get container status \"f0742bbb47f869245933bc1cc3baa21d867230725d7dadaafe486ba841d7fc84\": rpc error: code = NotFound desc = could not find container \"f0742bbb47f869245933bc1cc3baa21d867230725d7dadaafe486ba841d7fc84\": container with ID starting with f0742bbb47f869245933bc1cc3baa21d867230725d7dadaafe486ba841d7fc84 not found: ID does not exist"
Jul 31 23:54:40.163385 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:40.163325    2584 scope.go:117] "RemoveContainer" containerID="f53e81b3911de77129682721ad6f8acb7a2e2fd10f100b3b5be19d1ee697364b"
Jul 31 23:54:40.172751 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:40.172728    2584 scope.go:117] "RemoveContainer" containerID="f53e81b3911de77129682721ad6f8acb7a2e2fd10f100b3b5be19d1ee697364b"
Jul 31 23:54:40.172981 ip-10-0-128-243 kubenswrapper[2584]: E0731 23:54:40.172964    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f53e81b3911de77129682721ad6f8acb7a2e2fd10f100b3b5be19d1ee697364b\": container with ID starting with f53e81b3911de77129682721ad6f8acb7a2e2fd10f100b3b5be19d1ee697364b not found: ID does not exist" containerID="f53e81b3911de77129682721ad6f8acb7a2e2fd10f100b3b5be19d1ee697364b"
Jul 31 23:54:40.173031 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:40.172987    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f53e81b3911de77129682721ad6f8acb7a2e2fd10f100b3b5be19d1ee697364b"} err="failed to get container status \"f53e81b3911de77129682721ad6f8acb7a2e2fd10f100b3b5be19d1ee697364b\": rpc error: code = NotFound desc = could not find container \"f53e81b3911de77129682721ad6f8acb7a2e2fd10f100b3b5be19d1ee697364b\": container with ID starting with f53e81b3911de77129682721ad6f8acb7a2e2fd10f100b3b5be19d1ee697364b not found: ID does not exist"
Jul 31 23:54:40.173031 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:40.173003    2584 scope.go:117] "RemoveContainer" containerID="0e2e34e0c75ceb0a1d5ee4a9a921c2491783446d51199d778902703a4c455209"
Jul 31 23:54:40.180641 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:40.180622    2584 scope.go:117] "RemoveContainer" containerID="0e2e34e0c75ceb0a1d5ee4a9a921c2491783446d51199d778902703a4c455209"
Jul 31 23:54:40.180912 ip-10-0-128-243 kubenswrapper[2584]: E0731 23:54:40.180894    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e2e34e0c75ceb0a1d5ee4a9a921c2491783446d51199d778902703a4c455209\": container with ID starting with 0e2e34e0c75ceb0a1d5ee4a9a921c2491783446d51199d778902703a4c455209 not found: ID does not exist" containerID="0e2e34e0c75ceb0a1d5ee4a9a921c2491783446d51199d778902703a4c455209"
Jul 31 23:54:40.180981 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:40.180918    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e2e34e0c75ceb0a1d5ee4a9a921c2491783446d51199d778902703a4c455209"} err="failed to get container status \"0e2e34e0c75ceb0a1d5ee4a9a921c2491783446d51199d778902703a4c455209\": rpc error: code = NotFound desc = could not find container \"0e2e34e0c75ceb0a1d5ee4a9a921c2491783446d51199d778902703a4c455209\": container with ID starting with 0e2e34e0c75ceb0a1d5ee4a9a921c2491783446d51199d778902703a4c455209 not found: ID does not exist"
Jul 31 23:54:40.184478 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:40.184455    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-5a93d4-75b48cf6c6-dgrf8"]
Jul 31 23:54:40.187757 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:40.187736    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-5a93d4-75b48cf6c6-dgrf8"]
Jul 31 23:54:40.200988 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:40.200961    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-5a93d4-7bfbd454c8-4n6qh"]
Jul 31 23:54:40.204791 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:40.204770    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-5a93d4-7bfbd454c8-4n6qh"]
Jul 31 23:54:40.214424 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:40.214401    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-5a93d4-b6bd494d8-f2prm"]
Jul 31 23:54:40.216358 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:40.216337    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-5a93d4-b6bd494d8-f2prm"]
Jul 31 23:54:40.895829 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:40.895795    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2708980c-46e9-4bfd-b55e-149d9e3267d3" path="/var/lib/kubelet/pods/2708980c-46e9-4bfd-b55e-149d9e3267d3/volumes"
Jul 31 23:54:40.896210 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:40.896196    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76714e3b-4bab-476a-ab5b-1bb95995b0e1" path="/var/lib/kubelet/pods/76714e3b-4bab-476a-ab5b-1bb95995b0e1/volumes"
Jul 31 23:54:40.896527 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:40.896515    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c5da16a9-2cbe-46cc-afdc-76af2b1d878d" path="/var/lib/kubelet/pods/c5da16a9-2cbe-46cc-afdc-76af2b1d878d/volumes"
Jul 31 23:54:56.458497 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:56.458463    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-99d0e4-7d4d6f455b-p8s4s"]
Jul 31 23:54:56.458990 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:56.458681    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-99d0e4-7d4d6f455b-p8s4s" podUID="2f0556ab-9fae-434e-ba95-3f3ad3f55d8d" containerName="maas-api" containerID="cri-o://e24739b8ac6412d8c4277e3b2e13bf3214a6fb1af93af9dc7d0d60649382327d" gracePeriod=30
Jul 31 23:54:56.518300 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:56.518272    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-99d0e4-5ff6d9c7f4-qwzd2"]
Jul 31 23:54:56.518491 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:56.518466    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-99d0e4-5ff6d9c7f4-qwzd2" podUID="2cbdb4d2-c06b-4e03-bbff-44e4085444fc" containerName="payload-processing" containerID="cri-o://d577b95a0f1323756ac33b31f2f0c1d979538f45750865bf334cb6c0e9cde1f7" gracePeriod=30
Jul 31 23:54:56.524209 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:56.524140    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-99d0e4-d7d547fbd-j7dfx"]
Jul 31 23:54:56.524396 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:56.524348    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-99d0e4-d7d547fbd-j7dfx" podUID="d600216e-7383-4606-b644-1ef7258899c6" containerName="payload-pre-processing" containerID="cri-o://d7ee50b8cf71b01858d27a6770a975b5afa05f581bb38add083c5e47397a7976" gracePeriod=30
Jul 31 23:54:56.802050 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:56.802026    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-99d0e4-7d4d6f455b-p8s4s"
Jul 31 23:54:56.827416 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:56.827376    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-99d0e4-5ff6d9c7f4-qwzd2"
Jul 31 23:54:56.829384 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:56.829362    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-99d0e4-d7d547fbd-j7dfx"
Jul 31 23:54:56.894763 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:56.894726    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f0556ab-9fae-434e-ba95-3f3ad3f55d8d-openshift-service-ca\") pod \"2f0556ab-9fae-434e-ba95-3f3ad3f55d8d\" (UID: \"2f0556ab-9fae-434e-ba95-3f3ad3f55d8d\") "
Jul 31 23:54:56.894900 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:56.894815    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f0556ab-9fae-434e-ba95-3f3ad3f55d8d-maas-api-tls\") pod \"2f0556ab-9fae-434e-ba95-3f3ad3f55d8d\" (UID: \"2f0556ab-9fae-434e-ba95-3f3ad3f55d8d\") "
Jul 31 23:54:56.894900 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:56.894855    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qj6wd\" (UniqueName: \"kubernetes.io/projected/2f0556ab-9fae-434e-ba95-3f3ad3f55d8d-kube-api-access-qj6wd\") pod \"2f0556ab-9fae-434e-ba95-3f3ad3f55d8d\" (UID: \"2f0556ab-9fae-434e-ba95-3f3ad3f55d8d\") "
Jul 31 23:54:56.894900 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:56.894881    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d600216e-7383-4606-b644-1ef7258899c6-plugins-config-volume\") pod \"d600216e-7383-4606-b644-1ef7258899c6\" (UID: \"d600216e-7383-4606-b644-1ef7258899c6\") "
Jul 31 23:54:56.895075 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:56.894941    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bfs4l\" (UniqueName: \"kubernetes.io/projected/2cbdb4d2-c06b-4e03-bbff-44e4085444fc-kube-api-access-bfs4l\") pod \"2cbdb4d2-c06b-4e03-bbff-44e4085444fc\" (UID: \"2cbdb4d2-c06b-4e03-bbff-44e4085444fc\") "
Jul 31 23:54:56.895075 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:56.894964    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2cbdb4d2-c06b-4e03-bbff-44e4085444fc-plugins-config-volume\") pod \"2cbdb4d2-c06b-4e03-bbff-44e4085444fc\" (UID: \"2cbdb4d2-c06b-4e03-bbff-44e4085444fc\") "
Jul 31 23:54:56.895075 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:56.895050    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mw6qk\" (UniqueName: \"kubernetes.io/projected/d600216e-7383-4606-b644-1ef7258899c6-kube-api-access-mw6qk\") pod \"d600216e-7383-4606-b644-1ef7258899c6\" (UID: \"d600216e-7383-4606-b644-1ef7258899c6\") "
Jul 31 23:54:56.895234 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:56.895069    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f0556ab-9fae-434e-ba95-3f3ad3f55d8d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2f0556ab-9fae-434e-ba95-3f3ad3f55d8d" (UID: "2f0556ab-9fae-434e-ba95-3f3ad3f55d8d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:54:56.895319 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:56.895300    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f0556ab-9fae-434e-ba95-3f3ad3f55d8d-openshift-service-ca\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:54:56.895379 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:56.895347    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d600216e-7383-4606-b644-1ef7258899c6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d600216e-7383-4606-b644-1ef7258899c6" (UID: "d600216e-7383-4606-b644-1ef7258899c6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:54:56.895427 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:56.895371    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2cbdb4d2-c06b-4e03-bbff-44e4085444fc-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2cbdb4d2-c06b-4e03-bbff-44e4085444fc" (UID: "2cbdb4d2-c06b-4e03-bbff-44e4085444fc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:54:56.897155 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:56.897121    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f0556ab-9fae-434e-ba95-3f3ad3f55d8d-kube-api-access-qj6wd" (OuterVolumeSpecName: "kube-api-access-qj6wd") pod "2f0556ab-9fae-434e-ba95-3f3ad3f55d8d" (UID: "2f0556ab-9fae-434e-ba95-3f3ad3f55d8d"). InnerVolumeSpecName "kube-api-access-qj6wd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 23:54:56.897155 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:56.897154    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d600216e-7383-4606-b644-1ef7258899c6-kube-api-access-mw6qk" (OuterVolumeSpecName: "kube-api-access-mw6qk") pod "d600216e-7383-4606-b644-1ef7258899c6" (UID: "d600216e-7383-4606-b644-1ef7258899c6"). InnerVolumeSpecName "kube-api-access-mw6qk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 23:54:56.897311 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:56.897158    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f0556ab-9fae-434e-ba95-3f3ad3f55d8d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2f0556ab-9fae-434e-ba95-3f3ad3f55d8d" (UID: "2f0556ab-9fae-434e-ba95-3f3ad3f55d8d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 23:54:56.897311 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:56.897227    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2cbdb4d2-c06b-4e03-bbff-44e4085444fc-kube-api-access-bfs4l" (OuterVolumeSpecName: "kube-api-access-bfs4l") pod "2cbdb4d2-c06b-4e03-bbff-44e4085444fc" (UID: "2cbdb4d2-c06b-4e03-bbff-44e4085444fc"). InnerVolumeSpecName "kube-api-access-bfs4l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 23:54:56.995973 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:56.995895    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f0556ab-9fae-434e-ba95-3f3ad3f55d8d-maas-api-tls\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:54:56.995973 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:56.995920    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qj6wd\" (UniqueName: \"kubernetes.io/projected/2f0556ab-9fae-434e-ba95-3f3ad3f55d8d-kube-api-access-qj6wd\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:54:56.995973 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:56.995933    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d600216e-7383-4606-b644-1ef7258899c6-plugins-config-volume\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:54:56.995973 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:56.995944    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bfs4l\" (UniqueName: \"kubernetes.io/projected/2cbdb4d2-c06b-4e03-bbff-44e4085444fc-kube-api-access-bfs4l\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:54:56.995973 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:56.995952    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2cbdb4d2-c06b-4e03-bbff-44e4085444fc-plugins-config-volume\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:54:56.995973 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:56.995962    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mw6qk\" (UniqueName: \"kubernetes.io/projected/d600216e-7383-4606-b644-1ef7258899c6-kube-api-access-mw6qk\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:54:57.244060 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:57.244023    2584 generic.go:356] "Generic (PLEG): container finished" podID="2f0556ab-9fae-434e-ba95-3f3ad3f55d8d" containerID="e24739b8ac6412d8c4277e3b2e13bf3214a6fb1af93af9dc7d0d60649382327d" exitCode=0
Jul 31 23:54:57.244242 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:57.244072    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-99d0e4-7d4d6f455b-p8s4s"
Jul 31 23:54:57.244242 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:57.244105    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-99d0e4-7d4d6f455b-p8s4s" event={"ID":"2f0556ab-9fae-434e-ba95-3f3ad3f55d8d","Type":"ContainerDied","Data":"e24739b8ac6412d8c4277e3b2e13bf3214a6fb1af93af9dc7d0d60649382327d"}
Jul 31 23:54:57.244242 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:57.244162    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-99d0e4-7d4d6f455b-p8s4s" event={"ID":"2f0556ab-9fae-434e-ba95-3f3ad3f55d8d","Type":"ContainerDied","Data":"9dcab5a6abad02d6fb9d0ff47267e5af0293f721d31acfd806049685b4999419"}
Jul 31 23:54:57.244242 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:57.244185    2584 scope.go:117] "RemoveContainer" containerID="e24739b8ac6412d8c4277e3b2e13bf3214a6fb1af93af9dc7d0d60649382327d"
Jul 31 23:54:57.245466 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:57.245446    2584 generic.go:356] "Generic (PLEG): container finished" podID="2cbdb4d2-c06b-4e03-bbff-44e4085444fc" containerID="d577b95a0f1323756ac33b31f2f0c1d979538f45750865bf334cb6c0e9cde1f7" exitCode=0
Jul 31 23:54:57.245541 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:57.245486    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-99d0e4-5ff6d9c7f4-qwzd2"
Jul 31 23:54:57.245593 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:57.245546    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-99d0e4-5ff6d9c7f4-qwzd2" event={"ID":"2cbdb4d2-c06b-4e03-bbff-44e4085444fc","Type":"ContainerDied","Data":"d577b95a0f1323756ac33b31f2f0c1d979538f45750865bf334cb6c0e9cde1f7"}
Jul 31 23:54:57.245642 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:57.245604    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-99d0e4-5ff6d9c7f4-qwzd2" event={"ID":"2cbdb4d2-c06b-4e03-bbff-44e4085444fc","Type":"ContainerDied","Data":"8ef71504b44fe9ef2183dc93167ae8fc5e1911b530ee9ae6f13bff9bd9d0262d"}
Jul 31 23:54:57.246956 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:57.246900    2584 generic.go:356] "Generic (PLEG): container finished" podID="d600216e-7383-4606-b644-1ef7258899c6" containerID="d7ee50b8cf71b01858d27a6770a975b5afa05f581bb38add083c5e47397a7976" exitCode=0
Jul 31 23:54:57.246956 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:57.246950    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-99d0e4-d7d547fbd-j7dfx"
Jul 31 23:54:57.247101 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:57.246950    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-99d0e4-d7d547fbd-j7dfx" event={"ID":"d600216e-7383-4606-b644-1ef7258899c6","Type":"ContainerDied","Data":"d7ee50b8cf71b01858d27a6770a975b5afa05f581bb38add083c5e47397a7976"}
Jul 31 23:54:57.247101 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:57.247072    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-99d0e4-d7d547fbd-j7dfx" event={"ID":"d600216e-7383-4606-b644-1ef7258899c6","Type":"ContainerDied","Data":"9aed5846846503b22ca6d696a14d824e5f59f821b4bc0ccb401addefdacbd811"}
Jul 31 23:54:57.255625 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:57.255607    2584 scope.go:117] "RemoveContainer" containerID="e24739b8ac6412d8c4277e3b2e13bf3214a6fb1af93af9dc7d0d60649382327d"
Jul 31 23:54:57.256357 ip-10-0-128-243 kubenswrapper[2584]: E0731 23:54:57.256330    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e24739b8ac6412d8c4277e3b2e13bf3214a6fb1af93af9dc7d0d60649382327d\": container with ID starting with e24739b8ac6412d8c4277e3b2e13bf3214a6fb1af93af9dc7d0d60649382327d not found: ID does not exist" containerID="e24739b8ac6412d8c4277e3b2e13bf3214a6fb1af93af9dc7d0d60649382327d"
Jul 31 23:54:57.256634 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:57.256608    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e24739b8ac6412d8c4277e3b2e13bf3214a6fb1af93af9dc7d0d60649382327d"} err="failed to get container status \"e24739b8ac6412d8c4277e3b2e13bf3214a6fb1af93af9dc7d0d60649382327d\": rpc error: code = NotFound desc = could not find container \"e24739b8ac6412d8c4277e3b2e13bf3214a6fb1af93af9dc7d0d60649382327d\": container with ID starting with e24739b8ac6412d8c4277e3b2e13bf3214a6fb1af93af9dc7d0d60649382327d not found: ID does not exist"
Jul 31 23:54:57.256767 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:57.256751    2584 scope.go:117] "RemoveContainer" containerID="d577b95a0f1323756ac33b31f2f0c1d979538f45750865bf334cb6c0e9cde1f7"
Jul 31 23:54:57.267723 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:57.267687    2584 scope.go:117] "RemoveContainer" containerID="d577b95a0f1323756ac33b31f2f0c1d979538f45750865bf334cb6c0e9cde1f7"
Jul 31 23:54:57.267986 ip-10-0-128-243 kubenswrapper[2584]: E0731 23:54:57.267966    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d577b95a0f1323756ac33b31f2f0c1d979538f45750865bf334cb6c0e9cde1f7\": container with ID starting with d577b95a0f1323756ac33b31f2f0c1d979538f45750865bf334cb6c0e9cde1f7 not found: ID does not exist" containerID="d577b95a0f1323756ac33b31f2f0c1d979538f45750865bf334cb6c0e9cde1f7"
Jul 31 23:54:57.268040 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:57.267991    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d577b95a0f1323756ac33b31f2f0c1d979538f45750865bf334cb6c0e9cde1f7"} err="failed to get container status \"d577b95a0f1323756ac33b31f2f0c1d979538f45750865bf334cb6c0e9cde1f7\": rpc error: code = NotFound desc = could not find container \"d577b95a0f1323756ac33b31f2f0c1d979538f45750865bf334cb6c0e9cde1f7\": container with ID starting with d577b95a0f1323756ac33b31f2f0c1d979538f45750865bf334cb6c0e9cde1f7 not found: ID does not exist"
Jul 31 23:54:57.268040 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:57.268006    2584 scope.go:117] "RemoveContainer" containerID="d7ee50b8cf71b01858d27a6770a975b5afa05f581bb38add083c5e47397a7976"
Jul 31 23:54:57.270999 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:57.270978    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-99d0e4-7d4d6f455b-p8s4s"]
Jul 31 23:54:57.275481 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:57.275457    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-99d0e4-7d4d6f455b-p8s4s"]
Jul 31 23:54:57.275933 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:57.275918    2584 scope.go:117] "RemoveContainer" containerID="d7ee50b8cf71b01858d27a6770a975b5afa05f581bb38add083c5e47397a7976"
Jul 31 23:54:57.276199 ip-10-0-128-243 kubenswrapper[2584]: E0731 23:54:57.276182    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7ee50b8cf71b01858d27a6770a975b5afa05f581bb38add083c5e47397a7976\": container with ID starting with d7ee50b8cf71b01858d27a6770a975b5afa05f581bb38add083c5e47397a7976 not found: ID does not exist" containerID="d7ee50b8cf71b01858d27a6770a975b5afa05f581bb38add083c5e47397a7976"
Jul 31 23:54:57.276254 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:57.276205    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7ee50b8cf71b01858d27a6770a975b5afa05f581bb38add083c5e47397a7976"} err="failed to get container status \"d7ee50b8cf71b01858d27a6770a975b5afa05f581bb38add083c5e47397a7976\": rpc error: code = NotFound desc = could not find container \"d7ee50b8cf71b01858d27a6770a975b5afa05f581bb38add083c5e47397a7976\": container with ID starting with d7ee50b8cf71b01858d27a6770a975b5afa05f581bb38add083c5e47397a7976 not found: ID does not exist"
Jul 31 23:54:57.284992 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:57.284882    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-99d0e4-5ff6d9c7f4-qwzd2"]
Jul 31 23:54:57.288413 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:57.288402    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-99d0e4-5ff6d9c7f4-qwzd2"]
Jul 31 23:54:57.298161 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:57.298100    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-99d0e4-d7d547fbd-j7dfx"]
Jul 31 23:54:57.304040 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:57.304019    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-99d0e4-d7d547fbd-j7dfx"]
Jul 31 23:54:58.891519 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:58.891490    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2cbdb4d2-c06b-4e03-bbff-44e4085444fc" path="/var/lib/kubelet/pods/2cbdb4d2-c06b-4e03-bbff-44e4085444fc/volumes"
Jul 31 23:54:58.891895 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:58.891878    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f0556ab-9fae-434e-ba95-3f3ad3f55d8d" path="/var/lib/kubelet/pods/2f0556ab-9fae-434e-ba95-3f3ad3f55d8d/volumes"
Jul 31 23:54:58.892195 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:54:58.892183    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d600216e-7383-4606-b644-1ef7258899c6" path="/var/lib/kubelet/pods/d600216e-7383-4606-b644-1ef7258899c6/volumes"
Jul 31 23:55:13.827008 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:13.826970    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-3d60f7-8455c9d5d6-wztsz"]
Jul 31 23:55:13.827589 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:13.827263    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3d60f7-8455c9d5d6-wztsz" podUID="4ca8cdda-d121-4034-9121-c64a7dcf257d" containerName="maas-api" containerID="cri-o://85d329d4a9cc3ba30ac0582153bebc5d6ca85773fb6ac7e687d94feaeae610a5" gracePeriod=30
Jul 31 23:55:13.881376 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:13.881346    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-3d60f7-6dd78b98c8-twrbh"]
Jul 31 23:55:13.881586 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:13.881546    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-3d60f7-6dd78b98c8-twrbh" podUID="c52bc313-e332-4809-bd37-3bdbf62bdc06" containerName="payload-processing" containerID="cri-o://c9b509e2d1161ececee7e923621e1c618d16f717449dfeb55ce4e1934d94f115" gracePeriod=30
Jul 31 23:55:13.887018 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:13.886993    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-3d60f7-bdc4c97d4-8dqv7"]
Jul 31 23:55:13.887223 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:13.887200    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-3d60f7-bdc4c97d4-8dqv7" podUID="a658ca8a-339e-414a-837a-11e522529476" containerName="payload-pre-processing" containerID="cri-o://03da00a7437b8df37cdb65a245e3d848ff5896726386389e3869d86a89290971" gracePeriod=30
Jul 31 23:55:14.159185 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:14.159145    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-3d60f7-6dd78b98c8-twrbh"
Jul 31 23:55:14.187140 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:14.187117    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3d60f7-8455c9d5d6-wztsz"
Jul 31 23:55:14.200634 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:14.200605    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-3d60f7-bdc4c97d4-8dqv7"
Jul 31 23:55:14.246086 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:14.246055    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c52bc313-e332-4809-bd37-3bdbf62bdc06-plugins-config-volume\") pod \"c52bc313-e332-4809-bd37-3bdbf62bdc06\" (UID: \"c52bc313-e332-4809-bd37-3bdbf62bdc06\") "
Jul 31 23:55:14.246257 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:14.246177    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sjgzp\" (UniqueName: \"kubernetes.io/projected/c52bc313-e332-4809-bd37-3bdbf62bdc06-kube-api-access-sjgzp\") pod \"c52bc313-e332-4809-bd37-3bdbf62bdc06\" (UID: \"c52bc313-e332-4809-bd37-3bdbf62bdc06\") "
Jul 31 23:55:14.246490 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:14.246464    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c52bc313-e332-4809-bd37-3bdbf62bdc06-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c52bc313-e332-4809-bd37-3bdbf62bdc06" (UID: "c52bc313-e332-4809-bd37-3bdbf62bdc06"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:55:14.248274 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:14.248253    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c52bc313-e332-4809-bd37-3bdbf62bdc06-kube-api-access-sjgzp" (OuterVolumeSpecName: "kube-api-access-sjgzp") pod "c52bc313-e332-4809-bd37-3bdbf62bdc06" (UID: "c52bc313-e332-4809-bd37-3bdbf62bdc06"). InnerVolumeSpecName "kube-api-access-sjgzp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 23:55:14.333046 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:14.333005    2584 generic.go:356] "Generic (PLEG): container finished" podID="c52bc313-e332-4809-bd37-3bdbf62bdc06" containerID="c9b509e2d1161ececee7e923621e1c618d16f717449dfeb55ce4e1934d94f115" exitCode=0
Jul 31 23:55:14.333046 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:14.333045    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-3d60f7-6dd78b98c8-twrbh"
Jul 31 23:55:14.333296 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:14.333087    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-3d60f7-6dd78b98c8-twrbh" event={"ID":"c52bc313-e332-4809-bd37-3bdbf62bdc06","Type":"ContainerDied","Data":"c9b509e2d1161ececee7e923621e1c618d16f717449dfeb55ce4e1934d94f115"}
Jul 31 23:55:14.333296 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:14.333147    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-3d60f7-6dd78b98c8-twrbh" event={"ID":"c52bc313-e332-4809-bd37-3bdbf62bdc06","Type":"ContainerDied","Data":"94dae7b455befcd268dfdda8b1e29cd88ba6fd6b9c7ca283b7795a33a67d3203"}
Jul 31 23:55:14.333296 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:14.333177    2584 scope.go:117] "RemoveContainer" containerID="c9b509e2d1161ececee7e923621e1c618d16f717449dfeb55ce4e1934d94f115"
Jul 31 23:55:14.334807 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:14.334782    2584 generic.go:356] "Generic (PLEG): container finished" podID="a658ca8a-339e-414a-837a-11e522529476" containerID="03da00a7437b8df37cdb65a245e3d848ff5896726386389e3869d86a89290971" exitCode=0
Jul 31 23:55:14.335043 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:14.334851    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-3d60f7-bdc4c97d4-8dqv7" event={"ID":"a658ca8a-339e-414a-837a-11e522529476","Type":"ContainerDied","Data":"03da00a7437b8df37cdb65a245e3d848ff5896726386389e3869d86a89290971"}
Jul 31 23:55:14.335171 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:14.335070    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-3d60f7-bdc4c97d4-8dqv7" event={"ID":"a658ca8a-339e-414a-837a-11e522529476","Type":"ContainerDied","Data":"c3c2c47055c90da14545d81e95f87bd6716daf0aa626c4f6709bcd409a504df6"}
Jul 31 23:55:14.335171 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:14.334875    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-3d60f7-bdc4c97d4-8dqv7"
Jul 31 23:55:14.336664 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:14.336646    2584 generic.go:356] "Generic (PLEG): container finished" podID="4ca8cdda-d121-4034-9121-c64a7dcf257d" containerID="85d329d4a9cc3ba30ac0582153bebc5d6ca85773fb6ac7e687d94feaeae610a5" exitCode=0
Jul 31 23:55:14.336893 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:14.336759    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3d60f7-8455c9d5d6-wztsz"
Jul 31 23:55:14.337066 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:14.336739    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3d60f7-8455c9d5d6-wztsz" event={"ID":"4ca8cdda-d121-4034-9121-c64a7dcf257d","Type":"ContainerDied","Data":"85d329d4a9cc3ba30ac0582153bebc5d6ca85773fb6ac7e687d94feaeae610a5"}
Jul 31 23:55:14.337199 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:14.337181    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3d60f7-8455c9d5d6-wztsz" event={"ID":"4ca8cdda-d121-4034-9121-c64a7dcf257d","Type":"ContainerDied","Data":"80f22b87322a7ff55daf06c2009d2077968b229ffb1abafb73b3c06ce659491c"}
Jul 31 23:55:14.342627 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:14.342609    2584 scope.go:117] "RemoveContainer" containerID="c9b509e2d1161ececee7e923621e1c618d16f717449dfeb55ce4e1934d94f115"
Jul 31 23:55:14.342878 ip-10-0-128-243 kubenswrapper[2584]: E0731 23:55:14.342861    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c9b509e2d1161ececee7e923621e1c618d16f717449dfeb55ce4e1934d94f115\": container with ID starting with c9b509e2d1161ececee7e923621e1c618d16f717449dfeb55ce4e1934d94f115 not found: ID does not exist" containerID="c9b509e2d1161ececee7e923621e1c618d16f717449dfeb55ce4e1934d94f115"
Jul 31 23:55:14.342940 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:14.342886    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9b509e2d1161ececee7e923621e1c618d16f717449dfeb55ce4e1934d94f115"} err="failed to get container status \"c9b509e2d1161ececee7e923621e1c618d16f717449dfeb55ce4e1934d94f115\": rpc error: code = NotFound desc = could not find container \"c9b509e2d1161ececee7e923621e1c618d16f717449dfeb55ce4e1934d94f115\": container with ID starting with c9b509e2d1161ececee7e923621e1c618d16f717449dfeb55ce4e1934d94f115 not found: ID does not exist"
Jul 31 23:55:14.342940 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:14.342904    2584 scope.go:117] "RemoveContainer" containerID="03da00a7437b8df37cdb65a245e3d848ff5896726386389e3869d86a89290971"
Jul 31 23:55:14.346909 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:14.346893    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fh58p\" (UniqueName: \"kubernetes.io/projected/a658ca8a-339e-414a-837a-11e522529476-kube-api-access-fh58p\") pod \"a658ca8a-339e-414a-837a-11e522529476\" (UID: \"a658ca8a-339e-414a-837a-11e522529476\") "
Jul 31 23:55:14.346992 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:14.346920    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a658ca8a-339e-414a-837a-11e522529476-plugins-config-volume\") pod \"a658ca8a-339e-414a-837a-11e522529476\" (UID: \"a658ca8a-339e-414a-837a-11e522529476\") "
Jul 31 23:55:14.346992 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:14.346966    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dqjd9\" (UniqueName: \"kubernetes.io/projected/4ca8cdda-d121-4034-9121-c64a7dcf257d-kube-api-access-dqjd9\") pod \"4ca8cdda-d121-4034-9121-c64a7dcf257d\" (UID: \"4ca8cdda-d121-4034-9121-c64a7dcf257d\") "
Jul 31 23:55:14.347171 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:14.347149    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4ca8cdda-d121-4034-9121-c64a7dcf257d-maas-api-tls\") pod \"4ca8cdda-d121-4034-9121-c64a7dcf257d\" (UID: \"4ca8cdda-d121-4034-9121-c64a7dcf257d\") "
Jul 31 23:55:14.347270 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:14.347192    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ca8cdda-d121-4034-9121-c64a7dcf257d-openshift-service-ca\") pod \"4ca8cdda-d121-4034-9121-c64a7dcf257d\" (UID: \"4ca8cdda-d121-4034-9121-c64a7dcf257d\") "
Jul 31 23:55:14.347603 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:14.347575    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ca8cdda-d121-4034-9121-c64a7dcf257d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4ca8cdda-d121-4034-9121-c64a7dcf257d" (UID: "4ca8cdda-d121-4034-9121-c64a7dcf257d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:55:14.347691 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:14.347610    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c52bc313-e332-4809-bd37-3bdbf62bdc06-plugins-config-volume\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:55:14.347691 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:14.347632    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sjgzp\" (UniqueName: \"kubernetes.io/projected/c52bc313-e332-4809-bd37-3bdbf62bdc06-kube-api-access-sjgzp\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:55:14.347856 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:14.347747    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a658ca8a-339e-414a-837a-11e522529476-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a658ca8a-339e-414a-837a-11e522529476" (UID: "a658ca8a-339e-414a-837a-11e522529476"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:55:14.349057 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:14.349032    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ca8cdda-d121-4034-9121-c64a7dcf257d-kube-api-access-dqjd9" (OuterVolumeSpecName: "kube-api-access-dqjd9") pod "4ca8cdda-d121-4034-9121-c64a7dcf257d" (UID: "4ca8cdda-d121-4034-9121-c64a7dcf257d"). InnerVolumeSpecName "kube-api-access-dqjd9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 23:55:14.349177 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:14.349157    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a658ca8a-339e-414a-837a-11e522529476-kube-api-access-fh58p" (OuterVolumeSpecName: "kube-api-access-fh58p") pod "a658ca8a-339e-414a-837a-11e522529476" (UID: "a658ca8a-339e-414a-837a-11e522529476"). InnerVolumeSpecName "kube-api-access-fh58p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 23:55:14.349262 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:14.349215    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4ca8cdda-d121-4034-9121-c64a7dcf257d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4ca8cdda-d121-4034-9121-c64a7dcf257d" (UID: "4ca8cdda-d121-4034-9121-c64a7dcf257d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 23:55:14.350861 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:14.350845    2584 scope.go:117] "RemoveContainer" containerID="03da00a7437b8df37cdb65a245e3d848ff5896726386389e3869d86a89290971"
Jul 31 23:55:14.351087 ip-10-0-128-243 kubenswrapper[2584]: E0731 23:55:14.351067    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03da00a7437b8df37cdb65a245e3d848ff5896726386389e3869d86a89290971\": container with ID starting with 03da00a7437b8df37cdb65a245e3d848ff5896726386389e3869d86a89290971 not found: ID does not exist" containerID="03da00a7437b8df37cdb65a245e3d848ff5896726386389e3869d86a89290971"
Jul 31 23:55:14.351142 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:14.351092    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03da00a7437b8df37cdb65a245e3d848ff5896726386389e3869d86a89290971"} err="failed to get container status \"03da00a7437b8df37cdb65a245e3d848ff5896726386389e3869d86a89290971\": rpc error: code = NotFound desc = could not find container \"03da00a7437b8df37cdb65a245e3d848ff5896726386389e3869d86a89290971\": container with ID starting with 03da00a7437b8df37cdb65a245e3d848ff5896726386389e3869d86a89290971 not found: ID does not exist"
Jul 31 23:55:14.351142 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:14.351107    2584 scope.go:117] "RemoveContainer" containerID="85d329d4a9cc3ba30ac0582153bebc5d6ca85773fb6ac7e687d94feaeae610a5"
Jul 31 23:55:14.358298 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:14.358281    2584 scope.go:117] "RemoveContainer" containerID="85d329d4a9cc3ba30ac0582153bebc5d6ca85773fb6ac7e687d94feaeae610a5"
Jul 31 23:55:14.358546 ip-10-0-128-243 kubenswrapper[2584]: E0731 23:55:14.358525    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85d329d4a9cc3ba30ac0582153bebc5d6ca85773fb6ac7e687d94feaeae610a5\": container with ID starting with 85d329d4a9cc3ba30ac0582153bebc5d6ca85773fb6ac7e687d94feaeae610a5 not found: ID does not exist" containerID="85d329d4a9cc3ba30ac0582153bebc5d6ca85773fb6ac7e687d94feaeae610a5"
Jul 31 23:55:14.358628 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:14.358548    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85d329d4a9cc3ba30ac0582153bebc5d6ca85773fb6ac7e687d94feaeae610a5"} err="failed to get container status \"85d329d4a9cc3ba30ac0582153bebc5d6ca85773fb6ac7e687d94feaeae610a5\": rpc error: code = NotFound desc = could not find container \"85d329d4a9cc3ba30ac0582153bebc5d6ca85773fb6ac7e687d94feaeae610a5\": container with ID starting with 85d329d4a9cc3ba30ac0582153bebc5d6ca85773fb6ac7e687d94feaeae610a5 not found: ID does not exist"
Jul 31 23:55:14.393376 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:14.393351    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-3d60f7-6dd78b98c8-twrbh"]
Jul 31 23:55:14.403396 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:14.403370    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-3d60f7-6dd78b98c8-twrbh"]
Jul 31 23:55:14.449009 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:14.448978    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dqjd9\" (UniqueName: \"kubernetes.io/projected/4ca8cdda-d121-4034-9121-c64a7dcf257d-kube-api-access-dqjd9\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:55:14.449009 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:14.449004    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4ca8cdda-d121-4034-9121-c64a7dcf257d-maas-api-tls\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:55:14.449009 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:14.449015    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ca8cdda-d121-4034-9121-c64a7dcf257d-openshift-service-ca\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:55:14.449241 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:14.449026    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fh58p\" (UniqueName: \"kubernetes.io/projected/a658ca8a-339e-414a-837a-11e522529476-kube-api-access-fh58p\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:55:14.449241 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:14.449036    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a658ca8a-339e-414a-837a-11e522529476-plugins-config-volume\") on node \"ip-10-0-128-243.ec2.internal\" DevicePath \"\""
Jul 31 23:55:14.658782 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:14.658756    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-3d60f7-bdc4c97d4-8dqv7"]
Jul 31 23:55:14.662605 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:14.662583    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-3d60f7-bdc4c97d4-8dqv7"]
Jul 31 23:55:14.672140 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:14.672117    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-3d60f7-8455c9d5d6-wztsz"]
Jul 31 23:55:14.677400 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:14.677378    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-3d60f7-8455c9d5d6-wztsz"]
Jul 31 23:55:14.890419 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:14.890380    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ca8cdda-d121-4034-9121-c64a7dcf257d" path="/var/lib/kubelet/pods/4ca8cdda-d121-4034-9121-c64a7dcf257d/volumes"
Jul 31 23:55:14.890877 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:14.890791    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a658ca8a-339e-414a-837a-11e522529476" path="/var/lib/kubelet/pods/a658ca8a-339e-414a-837a-11e522529476/volumes"
Jul 31 23:55:14.891148 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:14.891132    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c52bc313-e332-4809-bd37-3bdbf62bdc06" path="/var/lib/kubelet/pods/c52bc313-e332-4809-bd37-3bdbf62bdc06/volumes"
Jul 31 23:55:26.278058 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:26.277982    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-gsrc4_0f609f8c-bab9-4664-9986-ecfa0943a982/manager/0.log"
Jul 31 23:55:26.390129 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:26.390104    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-74f5bd9766-t8s7d_6e07685d-6508-4ba5-be33-cd0b86504595/manager/0.log"
Jul 31 23:55:26.500710 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:26.500670    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-dr5kw_8dbdab97-abc5-48d8-94ea-1f808b9d772f/server/0.log"
Jul 31 23:55:26.845186 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:26.845157    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8567dd98f6-dqqc2_c31127e6-7ea2-4b22-8bb0-a8ba1fed312b/copy-manifests/0.log"
Jul 31 23:55:26.851001 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:26.850981    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8567dd98f6-dqqc2_c31127e6-7ea2-4b22-8bb0-a8ba1fed312b/manager/0.log"
Jul 31 23:55:27.308637 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:27.308604    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-7c89d6c77-2msn9_353f23a4-5683-4e87-b657-fd0a7a0e73fc/maas-api/0.log"
Jul 31 23:55:27.413407 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:27.413369    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29758995-bnxj2_dbf8b16a-00b6-40c4-bdb4-5b214960658b/cleanup/0.log"
Jul 31 23:55:27.746357 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:27.746250    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-xjm54_282883c3-8777-4c49-9ca0-81581f0d33d5/postgres/0.log"
Jul 31 23:55:28.627554 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:28.627523    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-68d88948d8-trjxn_2346a376-c5a4-4062-a78f-13e2fead3c61/manager/0.log"
Jul 31 23:55:28.734133 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:28.734099    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-5985564776-f4r7v_17cc18ad-0998-4f95-841a-88ec1e5f6fbf/manager/0.log"
Jul 31 23:55:29.628364 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:29.628334    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-hwhbc_6f77d34f-4a8c-4282-9923-35909e938fa6/discovery/0.log"
Jul 31 23:55:29.729016 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:29.728962    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-85c4c7ff5b-mwq54_378ff678-9bd2-42d8-88c8-ae81e219d8d0/kube-auth-proxy/0.log"
Jul 31 23:55:30.186651 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:30.186609    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-29phv_f3023383-4445-4c1e-86a8-446b2e1307fa/payload-processing/0.log"
Jul 31 23:55:30.289589 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:30.289550    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-868556846d-gx64z_24908812-404e-4b8c-958d-335220924aff/router/0.log"
Jul 31 23:55:30.613538 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:30.613508    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-bdwhx_aaaf6395-4024-4492-b68d-77b34efb2204/main/0.log"
Jul 31 23:55:30.844372 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:30.844330    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-srtqk_26a9bdf8-5d25-4d59-a1a4-14ee72ca24a4/main/0.log"
Jul 31 23:55:39.533215 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:39.533179    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-mqwlt_b85a1443-4bc2-48a0-8d29-35f46cbc3d57/global-pull-secret-syncer/0.log"
Jul 31 23:55:39.630600 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:39.630557    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-shjgx_eb4e0ba5-fe8b-4bf7-a86d-15db1a2532b2/konnectivity-agent/0.log"
Jul 31 23:55:39.689101 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:39.689070    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-243.ec2.internal_bf2bb11c3449d46ed43b4ff9c8ef237d/haproxy/0.log"
Jul 31 23:55:44.650997 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:44.650962    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-68d88948d8-trjxn_2346a376-c5a4-4062-a78f-13e2fead3c61/manager/0.log"
Jul 31 23:55:44.678184 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:44.678157    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-5985564776-f4r7v_17cc18ad-0998-4f95-841a-88ec1e5f6fbf/manager/0.log"
Jul 31 23:55:46.703368 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:46.703336    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6f55c4f58d-f6wq6_40b91dc3-a7d9-403a-8208-ca1e019553c5/cluster-monitoring-operator/0.log"
Jul 31 23:55:46.869523 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:46.869496    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fgj4b_b7b2f0b9-22d7-4e2c-a4fa-99d981acc2d2/node-exporter/0.log"
Jul 31 23:55:46.889720 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:46.889674    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fgj4b_b7b2f0b9-22d7-4e2c-a4fa-99d981acc2d2/kube-rbac-proxy/0.log"
Jul 31 23:55:46.914214 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:46.914187    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fgj4b_b7b2f0b9-22d7-4e2c-a4fa-99d981acc2d2/init-textfile/0.log"
Jul 31 23:55:48.824151 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:48.824124    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-f77f48c6d-hxkk5_a1e283c7-af2a-46d7-9712-605cec1c3211/networking-console-plugin/0.log"
Jul 31 23:55:49.301410 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:49.301305    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-6dw7l/perf-node-gather-daemonset-p22rj"]
Jul 31 23:55:49.302098 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:49.302067    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd76d76e-27c6-4fda-b5b2-1253e05ea281" containerName="maas-api"
Jul 31 23:55:49.302098 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:49.302099    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd76d76e-27c6-4fda-b5b2-1253e05ea281" containerName="maas-api"
Jul 31 23:55:49.302266 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:49.302120    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2708980c-46e9-4bfd-b55e-149d9e3267d3" containerName="payload-processing"
Jul 31 23:55:49.302266 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:49.302129    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="2708980c-46e9-4bfd-b55e-149d9e3267d3" containerName="payload-processing"
Jul 31 23:55:49.302266 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:49.302142    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f0556ab-9fae-434e-ba95-3f3ad3f55d8d" containerName="maas-api"
Jul 31 23:55:49.302266 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:49.302150    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f0556ab-9fae-434e-ba95-3f3ad3f55d8d" containerName="maas-api"
Jul 31 23:55:49.302266 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:49.302159    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6be327e-fe3d-470c-986e-76aeef34cee3" containerName="payload-pre-processing"
Jul 31 23:55:49.302266 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:49.302167    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6be327e-fe3d-470c-986e-76aeef34cee3" containerName="payload-pre-processing"
Jul 31 23:55:49.302266 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:49.302186    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d600216e-7383-4606-b644-1ef7258899c6" containerName="payload-pre-processing"
Jul 31 23:55:49.302266 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:49.302194    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="d600216e-7383-4606-b644-1ef7258899c6" containerName="payload-pre-processing"
Jul 31 23:55:49.302266 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:49.302202    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2cbdb4d2-c06b-4e03-bbff-44e4085444fc" containerName="payload-processing"
Jul 31 23:55:49.302266 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:49.302209    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="2cbdb4d2-c06b-4e03-bbff-44e4085444fc" containerName="payload-processing"
Jul 31 23:55:49.302266 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:49.302225    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c52bc313-e332-4809-bd37-3bdbf62bdc06" containerName="payload-processing"
Jul 31 23:55:49.302266 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:49.302232    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="c52bc313-e332-4809-bd37-3bdbf62bdc06" containerName="payload-processing"
Jul 31 23:55:49.302266 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:49.302255    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5da16a9-2cbe-46cc-afdc-76af2b1d878d" containerName="payload-pre-processing"
Jul 31 23:55:49.302266 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:49.302264    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5da16a9-2cbe-46cc-afdc-76af2b1d878d" containerName="payload-pre-processing"
Jul 31 23:55:49.302879 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:49.302297    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ca8cdda-d121-4034-9121-c64a7dcf257d" containerName="maas-api"
Jul 31 23:55:49.302879 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:49.302305    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ca8cdda-d121-4034-9121-c64a7dcf257d" containerName="maas-api"
Jul 31 23:55:49.302879 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:49.302313    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad55314b-fbb0-4edc-a27c-550e3a4f90a1" containerName="maas-api"
Jul 31 23:55:49.302879 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:49.302322    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad55314b-fbb0-4edc-a27c-550e3a4f90a1" containerName="maas-api"
Jul 31 23:55:49.302879 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:49.302346    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a658ca8a-339e-414a-837a-11e522529476" containerName="payload-pre-processing"
Jul 31 23:55:49.302879 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:49.302355    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="a658ca8a-339e-414a-837a-11e522529476" containerName="payload-pre-processing"
Jul 31 23:55:49.302879 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:49.302371    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76714e3b-4bab-476a-ab5b-1bb95995b0e1" containerName="maas-api"
Jul 31 23:55:49.302879 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:49.302379    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="76714e3b-4bab-476a-ab5b-1bb95995b0e1" containerName="maas-api"
Jul 31 23:55:49.302879 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:49.302539    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="2708980c-46e9-4bfd-b55e-149d9e3267d3" containerName="payload-processing"
Jul 31 23:55:49.302879 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:49.302554    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd76d76e-27c6-4fda-b5b2-1253e05ea281" containerName="maas-api"
Jul 31 23:55:49.302879 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:49.302561    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5da16a9-2cbe-46cc-afdc-76af2b1d878d" containerName="payload-pre-processing"
Jul 31 23:55:49.302879 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:49.302577    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="c52bc313-e332-4809-bd37-3bdbf62bdc06" containerName="payload-processing"
Jul 31 23:55:49.302879 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:49.302586    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ca8cdda-d121-4034-9121-c64a7dcf257d" containerName="maas-api"
Jul 31 23:55:49.302879 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:49.302602    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="76714e3b-4bab-476a-ab5b-1bb95995b0e1" containerName="maas-api"
Jul 31 23:55:49.302879 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:49.302617    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f0556ab-9fae-434e-ba95-3f3ad3f55d8d" containerName="maas-api"
Jul 31 23:55:49.302879 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:49.302626    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6be327e-fe3d-470c-986e-76aeef34cee3" containerName="payload-pre-processing"
Jul 31 23:55:49.302879 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:49.302634    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="2cbdb4d2-c06b-4e03-bbff-44e4085444fc" containerName="payload-processing"
Jul 31 23:55:49.302879 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:49.302642    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad55314b-fbb0-4edc-a27c-550e3a4f90a1" containerName="maas-api"
Jul 31 23:55:49.302879 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:49.302656    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="d600216e-7383-4606-b644-1ef7258899c6" containerName="payload-pre-processing"
Jul 31 23:55:49.302879 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:49.302666    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="a658ca8a-339e-414a-837a-11e522529476" containerName="payload-pre-processing"
Jul 31 23:55:49.306452 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:49.306426    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6dw7l/perf-node-gather-daemonset-p22rj"
Jul 31 23:55:49.309175 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:49.309150    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6dw7l\"/\"openshift-service-ca.crt\""
Jul 31 23:55:49.309306 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:49.309172    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-6dw7l\"/\"default-dockercfg-vwvnq\""
Jul 31 23:55:49.310202 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:49.310184    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6dw7l\"/\"kube-root-ca.crt\""
Jul 31 23:55:49.313373 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:49.313351    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6dw7l/perf-node-gather-daemonset-p22rj"]
Jul 31 23:55:49.432267 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:49.432225    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2279aacc-818e-47ac-b3fc-c0c01acccabd-podres\") pod \"perf-node-gather-daemonset-p22rj\" (UID: \"2279aacc-818e-47ac-b3fc-c0c01acccabd\") " pod="openshift-must-gather-6dw7l/perf-node-gather-daemonset-p22rj"
Jul 31 23:55:49.432267 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:49.432266    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kl2dh\" (UniqueName: \"kubernetes.io/projected/2279aacc-818e-47ac-b3fc-c0c01acccabd-kube-api-access-kl2dh\") pod \"perf-node-gather-daemonset-p22rj\" (UID: \"2279aacc-818e-47ac-b3fc-c0c01acccabd\") " pod="openshift-must-gather-6dw7l/perf-node-gather-daemonset-p22rj"
Jul 31 23:55:49.432505 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:49.432372    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2279aacc-818e-47ac-b3fc-c0c01acccabd-proc\") pod \"perf-node-gather-daemonset-p22rj\" (UID: \"2279aacc-818e-47ac-b3fc-c0c01acccabd\") " pod="openshift-must-gather-6dw7l/perf-node-gather-daemonset-p22rj"
Jul 31 23:55:49.432505 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:49.432403    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2279aacc-818e-47ac-b3fc-c0c01acccabd-lib-modules\") pod \"perf-node-gather-daemonset-p22rj\" (UID: \"2279aacc-818e-47ac-b3fc-c0c01acccabd\") " pod="openshift-must-gather-6dw7l/perf-node-gather-daemonset-p22rj"
Jul 31 23:55:49.432505 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:49.432423    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2279aacc-818e-47ac-b3fc-c0c01acccabd-sys\") pod \"perf-node-gather-daemonset-p22rj\" (UID: \"2279aacc-818e-47ac-b3fc-c0c01acccabd\") " pod="openshift-must-gather-6dw7l/perf-node-gather-daemonset-p22rj"
Jul 31 23:55:49.533251 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:49.533218    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2279aacc-818e-47ac-b3fc-c0c01acccabd-podres\") pod \"perf-node-gather-daemonset-p22rj\" (UID: \"2279aacc-818e-47ac-b3fc-c0c01acccabd\") " pod="openshift-must-gather-6dw7l/perf-node-gather-daemonset-p22rj"
Jul 31 23:55:49.533251 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:49.533253    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kl2dh\" (UniqueName: \"kubernetes.io/projected/2279aacc-818e-47ac-b3fc-c0c01acccabd-kube-api-access-kl2dh\") pod \"perf-node-gather-daemonset-p22rj\" (UID: \"2279aacc-818e-47ac-b3fc-c0c01acccabd\") " pod="openshift-must-gather-6dw7l/perf-node-gather-daemonset-p22rj"
Jul 31 23:55:49.533529 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:49.533391    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2279aacc-818e-47ac-b3fc-c0c01acccabd-proc\") pod \"perf-node-gather-daemonset-p22rj\" (UID: \"2279aacc-818e-47ac-b3fc-c0c01acccabd\") " pod="openshift-must-gather-6dw7l/perf-node-gather-daemonset-p22rj"
Jul 31 23:55:49.533529 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:49.533424    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2279aacc-818e-47ac-b3fc-c0c01acccabd-lib-modules\") pod \"perf-node-gather-daemonset-p22rj\" (UID: \"2279aacc-818e-47ac-b3fc-c0c01acccabd\") " pod="openshift-must-gather-6dw7l/perf-node-gather-daemonset-p22rj"
Jul 31 23:55:49.533529 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:49.533434    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2279aacc-818e-47ac-b3fc-c0c01acccabd-podres\") pod \"perf-node-gather-daemonset-p22rj\" (UID: \"2279aacc-818e-47ac-b3fc-c0c01acccabd\") " pod="openshift-must-gather-6dw7l/perf-node-gather-daemonset-p22rj"
Jul 31 23:55:49.533529 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:49.533451    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2279aacc-818e-47ac-b3fc-c0c01acccabd-sys\") pod \"perf-node-gather-daemonset-p22rj\" (UID: \"2279aacc-818e-47ac-b3fc-c0c01acccabd\") " pod="openshift-must-gather-6dw7l/perf-node-gather-daemonset-p22rj"
Jul 31 23:55:49.533529 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:49.533462    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2279aacc-818e-47ac-b3fc-c0c01acccabd-proc\") pod \"perf-node-gather-daemonset-p22rj\" (UID: \"2279aacc-818e-47ac-b3fc-c0c01acccabd\") " pod="openshift-must-gather-6dw7l/perf-node-gather-daemonset-p22rj"
Jul 31 23:55:49.533529 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:49.533509    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2279aacc-818e-47ac-b3fc-c0c01acccabd-sys\") pod \"perf-node-gather-daemonset-p22rj\" (UID: \"2279aacc-818e-47ac-b3fc-c0c01acccabd\") " pod="openshift-must-gather-6dw7l/perf-node-gather-daemonset-p22rj"
Jul 31 23:55:49.533786 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:49.533564    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2279aacc-818e-47ac-b3fc-c0c01acccabd-lib-modules\") pod \"perf-node-gather-daemonset-p22rj\" (UID: \"2279aacc-818e-47ac-b3fc-c0c01acccabd\") " pod="openshift-must-gather-6dw7l/perf-node-gather-daemonset-p22rj"
Jul 31 23:55:49.542124 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:49.542094    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kl2dh\" (UniqueName: \"kubernetes.io/projected/2279aacc-818e-47ac-b3fc-c0c01acccabd-kube-api-access-kl2dh\") pod \"perf-node-gather-daemonset-p22rj\" (UID: \"2279aacc-818e-47ac-b3fc-c0c01acccabd\") " pod="openshift-must-gather-6dw7l/perf-node-gather-daemonset-p22rj"
Jul 31 23:55:49.617537 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:49.617505    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6dw7l/perf-node-gather-daemonset-p22rj"
Jul 31 23:55:49.755083 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:49.755054    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6dw7l/perf-node-gather-daemonset-p22rj"]
Jul 31 23:55:49.757088 ip-10-0-128-243 kubenswrapper[2584]: W0731 23:55:49.757054    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod2279aacc_818e_47ac_b3fc_c0c01acccabd.slice/crio-c4d4a869de7537d467aba970c56a223a8866339569a058cf0d5a081aa2a64ba8 WatchSource:0}: Error finding container c4d4a869de7537d467aba970c56a223a8866339569a058cf0d5a081aa2a64ba8: Status 404 returned error can't find the container with id c4d4a869de7537d467aba970c56a223a8866339569a058cf0d5a081aa2a64ba8
Jul 31 23:55:50.519572 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:50.519533    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6dw7l/perf-node-gather-daemonset-p22rj" event={"ID":"2279aacc-818e-47ac-b3fc-c0c01acccabd","Type":"ContainerStarted","Data":"2d7c2f24d85fe9174f3d5cff6871bf0000cf879d7e1139ce55b94f3cbdf8529c"}
Jul 31 23:55:50.519970 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:50.519586    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6dw7l/perf-node-gather-daemonset-p22rj" event={"ID":"2279aacc-818e-47ac-b3fc-c0c01acccabd","Type":"ContainerStarted","Data":"c4d4a869de7537d467aba970c56a223a8866339569a058cf0d5a081aa2a64ba8"}
Jul 31 23:55:50.519970 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:50.519625    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-6dw7l/perf-node-gather-daemonset-p22rj"
Jul 31 23:55:50.535388 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:50.535338    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-6dw7l/perf-node-gather-daemonset-p22rj" podStartSLOduration=1.535323091 podStartE2EDuration="1.535323091s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 23:55:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:55:50.534047853 +0000 UTC m=+3534.179834183" watchObservedRunningTime="2026-07-31 23:55:50.535323091 +0000 UTC m=+3534.181109441"
Jul 31 23:55:51.820059 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:51.820030    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-fhq7n_735a47e9-8a16-4dd3-93ec-2e3ac70100d4/dns/0.log"
Jul 31 23:55:51.840601 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:51.840574    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-fhq7n_735a47e9-8a16-4dd3-93ec-2e3ac70100d4/kube-rbac-proxy/0.log"
Jul 31 23:55:51.969603 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:51.969572    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-llmj8_148fa8ca-4d3b-4ae1-8be5-30ea87aa412e/dns-node-resolver/0.log"
Jul 31 23:55:52.495429 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:52.495399    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-74d85446d-tlzb9_61cabb50-cf7e-4874-92cb-00e7363aa33b/registry/0.log"
Jul 31 23:55:52.560305 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:52.560276    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-t8qkd_0bb4d7d9-ff22-41eb-8883-b7c1942767fb/node-ca/0.log"
Jul 31 23:55:53.643863 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:53.643829    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-hwhbc_6f77d34f-4a8c-4282-9923-35909e938fa6/discovery/0.log"
Jul 31 23:55:53.662050 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:53.662024    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-85c4c7ff5b-mwq54_378ff678-9bd2-42d8-88c8-ae81e219d8d0/kube-auth-proxy/0.log"
Jul 31 23:55:53.792467 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:53.792436    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-29phv_f3023383-4445-4c1e-86a8-446b2e1307fa/payload-processing/0.log"
Jul 31 23:55:53.812777 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:53.812749    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-868556846d-gx64z_24908812-404e-4b8c-958d-335220924aff/router/0.log"
Jul 31 23:55:54.591733 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:54.591694    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-lbf5k_a0e5b183-2a8a-48c8-b2fa-803bce230632/serve-healthcheck-canary/0.log"
Jul 31 23:55:55.040465 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:55.040376    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-77696685b-k9tqs_715381b4-025e-436c-96d9-8ab63b3750a9/insights-operator/0.log"
Jul 31 23:55:55.042592 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:55.042572    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-77696685b-k9tqs_715381b4-025e-436c-96d9-8ab63b3750a9/insights-operator/1.log"
Jul 31 23:55:55.186947 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:55.186917    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vvnwv_e460911c-90d3-4ebc-baaf-3047df044bc8/kube-rbac-proxy/0.log"
Jul 31 23:55:55.207826 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:55.207804    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vvnwv_e460911c-90d3-4ebc-baaf-3047df044bc8/exporter/0.log"
Jul 31 23:55:55.227030 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:55.227013    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vvnwv_e460911c-90d3-4ebc-baaf-3047df044bc8/extractor/0.log"
Jul 31 23:55:56.533499 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:56.533474    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-6dw7l/perf-node-gather-daemonset-p22rj"
Jul 31 23:55:57.505714 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:57.505669    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-gsrc4_0f609f8c-bab9-4664-9986-ecfa0943a982/manager/0.log"
Jul 31 23:55:57.656434 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:57.656350    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-74f5bd9766-t8s7d_6e07685d-6508-4ba5-be33-cd0b86504595/manager/0.log"
Jul 31 23:55:57.676671 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:57.676651    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-dr5kw_8dbdab97-abc5-48d8-94ea-1f808b9d772f/server/0.log"
Jul 31 23:55:57.863309 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:57.863285    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8567dd98f6-dqqc2_c31127e6-7ea2-4b22-8bb0-a8ba1fed312b/manager/0.log"
Jul 31 23:55:57.889888 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:57.889861    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8567dd98f6-dqqc2_c31127e6-7ea2-4b22-8bb0-a8ba1fed312b/copy-manifests/0.log"
Jul 31 23:55:59.988594 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:59.988557    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-hwhbc_6f77d34f-4a8c-4282-9923-35909e938fa6/discovery/0.log"
Jul 31 23:55:59.998110 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:55:59.998080    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-85c4c7ff5b-mwq54_378ff678-9bd2-42d8-88c8-ae81e219d8d0/kube-auth-proxy/0.log"
Jul 31 23:56:00.093962 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:56:00.093929    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-29phv_f3023383-4445-4c1e-86a8-446b2e1307fa/payload-processing/0.log"
Jul 31 23:56:00.106796 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:56:00.106771    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-868556846d-gx64z_24908812-404e-4b8c-958d-335220924aff/router/0.log"
Jul 31 23:56:01.668215 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:56:01.668179    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-hwhbc_6f77d34f-4a8c-4282-9923-35909e938fa6/discovery/0.log"
Jul 31 23:56:01.682133 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:56:01.682105    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-85c4c7ff5b-mwq54_378ff678-9bd2-42d8-88c8-ae81e219d8d0/kube-auth-proxy/0.log"
Jul 31 23:56:01.781129 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:56:01.781098    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-29phv_f3023383-4445-4c1e-86a8-446b2e1307fa/payload-processing/0.log"
Jul 31 23:56:01.794074 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:56:01.794041    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-868556846d-gx64z_24908812-404e-4b8c-958d-335220924aff/router/0.log"
Jul 31 23:56:04.556197 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:56:04.556165    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7878f4b88c-p58r8_55da6dc5-f4ad-468b-a4c8-497309dad48a/kube-storage-version-migrator-operator/1.log"
Jul 31 23:56:04.556993 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:56:04.556962    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7878f4b88c-p58r8_55da6dc5-f4ad-468b-a4c8-497309dad48a/kube-storage-version-migrator-operator/0.log"
Jul 31 23:56:05.662313 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:56:05.662284    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2trlm_4c7c83f2-713b-4651-b1da-f65c84f8edc9/kube-multus-additional-cni-plugins/0.log"
Jul 31 23:56:05.682916 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:56:05.682890    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2trlm_4c7c83f2-713b-4651-b1da-f65c84f8edc9/egress-router-binary-copy/0.log"
Jul 31 23:56:05.704923 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:56:05.704896    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2trlm_4c7c83f2-713b-4651-b1da-f65c84f8edc9/cni-plugins/0.log"
Jul 31 23:56:05.724771 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:56:05.724749    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2trlm_4c7c83f2-713b-4651-b1da-f65c84f8edc9/bond-cni-plugin/0.log"
Jul 31 23:56:05.745454 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:56:05.745429    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2trlm_4c7c83f2-713b-4651-b1da-f65c84f8edc9/routeoverride-cni/0.log"
Jul 31 23:56:05.765377 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:56:05.765357    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2trlm_4c7c83f2-713b-4651-b1da-f65c84f8edc9/whereabouts-cni-bincopy/0.log"
Jul 31 23:56:05.784911 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:56:05.784890    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2trlm_4c7c83f2-713b-4651-b1da-f65c84f8edc9/whereabouts-cni/0.log"
Jul 31 23:56:06.001571 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:56:06.001491    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-pbslg_639fb18d-e947-41cc-ac53-d4616b7cc85b/kube-multus/0.log"
Jul 31 23:56:06.200138 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:56:06.200101    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-mz5wz_b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945/network-metrics-daemon/0.log"
Jul 31 23:56:06.237899 ip-10-0-128-243 kubenswrapper[2584]: I0731 23:56:06.237878    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-mz5wz_b57a1e5f-f0c7-4ac1-b7d0-7456b22c2945/kube-rbac-proxy/0.log"