Jul 29 12:25:15.447263 ip-10-0-128-25 systemd[1]: Starting Kubernetes Kubelet...
Jul 29 12:25:15.817633 ip-10-0-128-25 kubenswrapper[2567]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 29 12:25:15.817633 ip-10-0-128-25 kubenswrapper[2567]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 29 12:25:15.817633 ip-10-0-128-25 kubenswrapper[2567]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 29 12:25:15.817633 ip-10-0-128-25 kubenswrapper[2567]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 29 12:25:15.817633 ip-10-0-128-25 kubenswrapper[2567]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 29 12:25:15.820311 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.820250    2567 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 29 12:25:15.826738 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.826719    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 12:25:15.826738 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.826735    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 12:25:15.826738 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.826741    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 12:25:15.826919 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.826746    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 12:25:15.826919 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.826750    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 12:25:15.826919 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.826754    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 12:25:15.826919 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.826759    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 12:25:15.826919 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.826762    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 12:25:15.826919 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.826766    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 12:25:15.826919 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.826770    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 12:25:15.826919 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.826774    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 12:25:15.826919 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.826785    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 12:25:15.826919 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.826790    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 12:25:15.826919 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.826794    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 12:25:15.826919 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.826799    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 12:25:15.826919 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.826802    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 12:25:15.826919 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.826806    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 12:25:15.826919 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.826810    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 12:25:15.826919 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.826815    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 12:25:15.826919 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.826818    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 12:25:15.826919 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.826822    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 12:25:15.826919 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.826826    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 12:25:15.827653 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.826830    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 12:25:15.827653 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.826834    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 12:25:15.827653 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.826842    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 12:25:15.827653 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.826847    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 12:25:15.827653 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.826851    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 12:25:15.827653 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.826855    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 12:25:15.827653 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.826859    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 12:25:15.827653 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.826863    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 12:25:15.827653 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.826868    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 12:25:15.827653 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.826872    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 12:25:15.827653 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.826876    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 12:25:15.827653 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.826880    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 12:25:15.827653 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.826885    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 12:25:15.827653 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.826890    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 12:25:15.827653 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.826895    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 12:25:15.827653 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.826904    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 12:25:15.827653 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.826909    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 12:25:15.827653 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.826913    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 12:25:15.827653 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.826916    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 12:25:15.827653 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.826920    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 12:25:15.828140 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.826924    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 12:25:15.828140 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.826928    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 12:25:15.828140 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.826932    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 12:25:15.828140 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.826936    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 12:25:15.828140 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.826940    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 12:25:15.828140 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.826944    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 12:25:15.828140 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.826949    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 12:25:15.828140 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.826958    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 12:25:15.828140 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.826962    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 12:25:15.828140 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.826980    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 12:25:15.828140 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.826985    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 12:25:15.828140 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.826989    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 12:25:15.828140 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.826994    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 12:25:15.828140 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.826999    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 12:25:15.828140 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.827004    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 12:25:15.828140 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.827008    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 12:25:15.828140 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.827012    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 12:25:15.828140 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.827016    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 12:25:15.828140 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.827020    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 12:25:15.828140 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.827029    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 12:25:15.828597 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.827033    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 12:25:15.828597 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.827037    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 12:25:15.828597 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.827041    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 12:25:15.828597 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.827045    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 12:25:15.828597 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.827051    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 12:25:15.828597 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.827056    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 12:25:15.828597 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.827062    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 12:25:15.828597 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.827066    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 12:25:15.828597 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.827071    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 12:25:15.828597 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.827075    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 12:25:15.828597 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.827081    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 12:25:15.828597 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.827087    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 12:25:15.828597 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.827096    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 12:25:15.828597 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.827101    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 12:25:15.828597 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.827105    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 12:25:15.828597 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.827110    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 12:25:15.828597 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.827118    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 12:25:15.828597 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.827124    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 12:25:15.828597 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.827129    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 12:25:15.829039 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.827133    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 12:25:15.829039 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.827138    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 12:25:15.829039 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.827142    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 12:25:15.829039 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.827147    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 12:25:15.829039 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.827156    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 12:25:15.829039 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.827160    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 12:25:15.829039 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.827164    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 12:25:15.829039 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.827168    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 12:25:15.829039 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.827172    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 12:25:15.829039 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.827176    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 12:25:15.829039 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.827181    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 12:25:15.829039 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.827185    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 12:25:15.829039 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.827189    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 12:25:15.829039 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.827193    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 12:25:15.829039 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.827197    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 12:25:15.829039 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.827202    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 12:25:15.829039 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.827205    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 12:25:15.829039 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.827215    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 12:25:15.829039 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828130    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 12:25:15.829516 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828140    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 12:25:15.829516 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828144    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 12:25:15.829516 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828148    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 12:25:15.829516 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828150    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 12:25:15.829516 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828153    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 12:25:15.829516 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828156    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 12:25:15.829516 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828158    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 12:25:15.829516 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828163    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 12:25:15.829516 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828168    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 12:25:15.829516 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828170    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 12:25:15.829516 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828173    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 12:25:15.829516 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828176    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 12:25:15.829516 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828179    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 12:25:15.829516 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828182    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 12:25:15.829516 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828184    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 12:25:15.829516 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828187    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 12:25:15.829516 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828189    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 12:25:15.829516 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828192    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 12:25:15.829516 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828194    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 12:25:15.829991 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828196    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 12:25:15.829991 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828199    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 12:25:15.829991 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828201    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 12:25:15.829991 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828203    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 12:25:15.829991 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828206    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 12:25:15.829991 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828209    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 12:25:15.829991 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828211    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 12:25:15.829991 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828213    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 12:25:15.829991 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828216    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 12:25:15.829991 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828218    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 12:25:15.829991 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828221    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 12:25:15.829991 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828224    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 12:25:15.829991 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828226    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 12:25:15.829991 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828229    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 12:25:15.829991 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828232    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 12:25:15.829991 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828234    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 12:25:15.829991 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828237    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 12:25:15.829991 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828239    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 12:25:15.829991 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828241    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 12:25:15.829991 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828244    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 12:25:15.830479 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828247    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 12:25:15.830479 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828249    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 12:25:15.830479 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828251    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 12:25:15.830479 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828254    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 12:25:15.830479 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828256    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 12:25:15.830479 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828259    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 12:25:15.830479 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828261    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 12:25:15.830479 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828263    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 12:25:15.830479 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828266    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 12:25:15.830479 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828268    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 12:25:15.830479 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828271    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 12:25:15.830479 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828275    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 12:25:15.830479 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828279    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 12:25:15.830479 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828282    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 12:25:15.830479 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828284    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 12:25:15.830479 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828287    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 12:25:15.830479 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828290    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 12:25:15.830479 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828293    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 12:25:15.830479 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828296    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 12:25:15.830479 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828298    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 12:25:15.830941 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828301    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 12:25:15.830941 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828303    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 12:25:15.830941 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828306    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 12:25:15.830941 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828309    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 12:25:15.830941 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828311    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 12:25:15.830941 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828314    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 12:25:15.830941 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828317    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 12:25:15.830941 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828320    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 12:25:15.830941 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828322    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 12:25:15.830941 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828326    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 12:25:15.830941 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828331    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 12:25:15.830941 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828333    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 12:25:15.830941 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828335    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 12:25:15.830941 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828338    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 12:25:15.830941 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828340    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 12:25:15.830941 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828343    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 12:25:15.830941 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828345    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 12:25:15.830941 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828348    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 12:25:15.830941 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828350    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 12:25:15.831394 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828352    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 12:25:15.831394 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828355    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 12:25:15.831394 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828357    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 12:25:15.831394 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828359    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 12:25:15.831394 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828362    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 12:25:15.831394 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828364    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 12:25:15.831394 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828367    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 12:25:15.831394 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828369    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 12:25:15.831394 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828372    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 12:25:15.831394 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828377    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 12:25:15.831394 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828380    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 12:25:15.831394 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828382    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 12:25:15.831394 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828385    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 12:25:15.831394 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828387    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 12:25:15.831394 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828389    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 12:25:15.831394 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828392    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 12:25:15.831394 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828394    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 12:25:15.831394 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828396    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 12:25:15.831394 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828399    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 12:25:15.831394 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828401    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 12:25:15.831851 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828725    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 12:25:15.831851 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828730    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 12:25:15.831851 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828732    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 12:25:15.831851 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828736    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 12:25:15.831851 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828738    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 12:25:15.831851 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828741    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 12:25:15.831851 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828743    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 12:25:15.831851 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828746    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 12:25:15.831851 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828748    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 12:25:15.831851 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828752    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 12:25:15.831851 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828755    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 12:25:15.831851 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828758    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 12:25:15.831851 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828762    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 12:25:15.831851 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828766    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 12:25:15.831851 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828769    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 12:25:15.831851 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828771    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 12:25:15.831851 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828774    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 12:25:15.831851 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828777    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 12:25:15.831851 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828779    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 12:25:15.832316 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828782    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 12:25:15.832316 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828785    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 12:25:15.832316 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828789    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 12:25:15.832316 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828794    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 12:25:15.832316 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828797    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 12:25:15.832316 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828800    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 12:25:15.832316 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828802    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 12:25:15.832316 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828805    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 12:25:15.832316 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828808    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 12:25:15.832316 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828810    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 12:25:15.832316 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828812    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 12:25:15.832316 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828815    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 12:25:15.832316 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828818    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 12:25:15.832316 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828820    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 12:25:15.832316 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828822    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 12:25:15.832316 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828825    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 12:25:15.832316 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828827    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 12:25:15.832316 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828829    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 12:25:15.832316 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828832    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 12:25:15.832316 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828834    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 12:25:15.832759 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828837    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 12:25:15.832759 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828839    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 12:25:15.832759 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828841    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 12:25:15.832759 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828843    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 12:25:15.832759 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828846    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 12:25:15.832759 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828849    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 12:25:15.832759 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828851    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 12:25:15.832759 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828853    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 12:25:15.832759 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828855    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 12:25:15.832759 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828858    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 12:25:15.832759 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828861    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 12:25:15.832759 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828863    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 12:25:15.832759 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828866    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 12:25:15.832759 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828868    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 12:25:15.832759 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828871    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 12:25:15.832759 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828875    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 12:25:15.832759 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828877    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 12:25:15.832759 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828879    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 12:25:15.832759 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828882    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 12:25:15.833269 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828884    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 12:25:15.833269 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828887    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 12:25:15.833269 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828889    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 12:25:15.833269 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828891    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 12:25:15.833269 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828893    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 12:25:15.833269 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828896    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 12:25:15.833269 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828898    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 12:25:15.833269 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828901    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 12:25:15.833269 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828903    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 12:25:15.833269 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828906    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 12:25:15.833269 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828908    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 12:25:15.833269 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828911    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 12:25:15.833269 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828913    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 12:25:15.833269 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828916    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 12:25:15.833269 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828918    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 12:25:15.833269 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828920    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 12:25:15.833269 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828923    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 12:25:15.833269 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828925    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 12:25:15.833269 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828927    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 12:25:15.833269 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828930    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 12:25:15.833773 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828932    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 12:25:15.833773 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828934    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 12:25:15.833773 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828938    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 12:25:15.833773 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828940    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 12:25:15.833773 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828946    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 12:25:15.833773 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828949    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 12:25:15.833773 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828951    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 12:25:15.833773 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828956    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 12:25:15.833773 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828959    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 12:25:15.833773 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828961    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 12:25:15.833773 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828964    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 12:25:15.833773 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828982    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 12:25:15.833773 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828986    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 12:25:15.833773 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828990    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 12:25:15.833773 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.828995    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 12:25:15.833773 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829000    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 12:25:15.833773 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829003    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 12:25:15.833773 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829006    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 12:25:15.833773 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829008    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 12:25:15.834241 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829011    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 12:25:15.834241 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829014    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 12:25:15.834241 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829081    2567 flags.go:64] FLAG: --address="0.0.0.0"
Jul 29 12:25:15.834241 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829087    2567 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 29 12:25:15.834241 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829093    2567 flags.go:64] FLAG: --anonymous-auth="true"
Jul 29 12:25:15.834241 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829097    2567 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 29 12:25:15.834241 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829100    2567 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 29 12:25:15.834241 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829103    2567 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 29 12:25:15.834241 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829108    2567 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 29 12:25:15.834241 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829111    2567 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 29 12:25:15.834241 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829114    2567 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 29 12:25:15.834241 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829117    2567 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 29 12:25:15.834241 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829120    2567 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 29 12:25:15.834241 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829123    2567 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 29 12:25:15.834241 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829126    2567 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 29 12:25:15.834241 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829129    2567 flags.go:64] FLAG: --cgroup-root=""
Jul 29 12:25:15.834241 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829132    2567 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 29 12:25:15.834241 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829134    2567 flags.go:64] FLAG: --client-ca-file=""
Jul 29 12:25:15.834241 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829138    2567 flags.go:64] FLAG: --cloud-provider="external"
Jul 29 12:25:15.834241 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829141    2567 flags.go:64] FLAG: --cluster-dns="[]"
Jul 29 12:25:15.834241 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829147    2567 flags.go:64] FLAG: --cluster-domain=""
Jul 29 12:25:15.834241 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829150    2567 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 29 12:25:15.834241 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829153    2567 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 29 12:25:15.834241 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829156    2567 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 29 12:25:15.834839 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829159    2567 flags.go:64] FLAG: --container-log-max-files="5"
Jul 29 12:25:15.834839 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829162    2567 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 29 12:25:15.834839 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829165    2567 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 29 12:25:15.834839 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829168    2567 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 29 12:25:15.834839 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829171    2567 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 29 12:25:15.834839 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829173    2567 flags.go:64] FLAG: --contention-profiling="false"
Jul 29 12:25:15.834839 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829176    2567 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 29 12:25:15.834839 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829179    2567 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 29 12:25:15.834839 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829182    2567 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 29 12:25:15.834839 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829185    2567 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 29 12:25:15.834839 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829188    2567 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 29 12:25:15.834839 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829191    2567 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 29 12:25:15.834839 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829194    2567 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 29 12:25:15.834839 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829196    2567 flags.go:64] FLAG: --enable-load-reader="false"
Jul 29 12:25:15.834839 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829199    2567 flags.go:64] FLAG: --enable-server="true"
Jul 29 12:25:15.834839 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829202    2567 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 29 12:25:15.834839 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829206    2567 flags.go:64] FLAG: --event-burst="100"
Jul 29 12:25:15.834839 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829208    2567 flags.go:64] FLAG: --event-qps="50"
Jul 29 12:25:15.834839 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829211    2567 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 29 12:25:15.834839 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829214    2567 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 29 12:25:15.834839 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829223    2567 flags.go:64] FLAG: --eviction-hard=""
Jul 29 12:25:15.834839 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829227    2567 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 29 12:25:15.834839 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829229    2567 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 29 12:25:15.834839 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829232    2567 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 29 12:25:15.834839 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829235    2567 flags.go:64] FLAG: --eviction-soft=""
Jul 29 12:25:15.835445 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829238    2567 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 29 12:25:15.835445 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829240    2567 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 29 12:25:15.835445 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829261    2567 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 29 12:25:15.835445 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829265    2567 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 29 12:25:15.835445 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829268    2567 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 29 12:25:15.835445 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829271    2567 flags.go:64] FLAG: --fail-swap-on="true"
Jul 29 12:25:15.835445 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829274    2567 flags.go:64] FLAG: --feature-gates=""
Jul 29 12:25:15.835445 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829278    2567 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 29 12:25:15.835445 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829280    2567 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 29 12:25:15.835445 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829283    2567 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 29 12:25:15.835445 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829286    2567 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 29 12:25:15.835445 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829289    2567 flags.go:64] FLAG: --healthz-port="10248"
Jul 29 12:25:15.835445 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829292    2567 flags.go:64] FLAG: --help="false"
Jul 29 12:25:15.835445 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829295    2567 flags.go:64] FLAG: --hostname-override="ip-10-0-128-25.ec2.internal"
Jul 29 12:25:15.835445 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829298    2567 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 29 12:25:15.835445 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829301    2567 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 29 12:25:15.835445 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829304    2567 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 29 12:25:15.835445 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829307    2567 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 29 12:25:15.835445 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829310    2567 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 29 12:25:15.835445 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829313    2567 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 29 12:25:15.835445 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829316    2567 flags.go:64] FLAG: --image-service-endpoint=""
Jul 29 12:25:15.835445 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829318    2567 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 29 12:25:15.835445 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829321    2567 flags.go:64] FLAG: --kube-api-burst="100"
Jul 29 12:25:15.836003 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829324    2567 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 29 12:25:15.836003 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829327    2567 flags.go:64] FLAG: --kube-api-qps="50"
Jul 29 12:25:15.836003 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829329    2567 flags.go:64] FLAG: --kube-reserved=""
Jul 29 12:25:15.836003 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829332    2567 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 29 12:25:15.836003 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829335    2567 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 29 12:25:15.836003 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829337    2567 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 29 12:25:15.836003 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829340    2567 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 29 12:25:15.836003 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829343    2567 flags.go:64] FLAG: --lock-file=""
Jul 29 12:25:15.836003 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829345    2567 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 29 12:25:15.836003 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829348    2567 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 29 12:25:15.836003 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829351    2567 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 29 12:25:15.836003 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829356    2567 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 29 12:25:15.836003 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829361    2567 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 29 12:25:15.836003 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829364    2567 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 29 12:25:15.836003 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829367    2567 flags.go:64] FLAG: --logging-format="text"
Jul 29 12:25:15.836003 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829370    2567 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 29 12:25:15.836003 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829373    2567 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 29 12:25:15.836003 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829375    2567 flags.go:64] FLAG: --manifest-url=""
Jul 29 12:25:15.836003 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829378    2567 flags.go:64] FLAG: --manifest-url-header=""
Jul 29 12:25:15.836003 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829382    2567 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 29 12:25:15.836003 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829385    2567 flags.go:64] FLAG: --max-open-files="1000000"
Jul 29 12:25:15.836003 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829389    2567 flags.go:64] FLAG: --max-pods="110"
Jul 29 12:25:15.836003 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829392    2567 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 29 12:25:15.836003 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829395    2567 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 29 12:25:15.836003 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829398    2567 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 29 12:25:15.836566 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829401    2567 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 29 12:25:15.836566 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829403    2567 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 29 12:25:15.836566 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829406    2567 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 29 12:25:15.836566 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829409    2567 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 29 12:25:15.836566 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829416    2567 flags.go:64] FLAG: --node-status-max-images="50"
Jul 29 12:25:15.836566 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829418    2567 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 29 12:25:15.836566 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829421    2567 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 29 12:25:15.836566 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829424    2567 flags.go:64] FLAG: --pod-cidr=""
Jul 29 12:25:15.836566 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829427    2567 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 29 12:25:15.836566 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829432    2567 flags.go:64] FLAG: --pod-manifest-path=""
Jul 29 12:25:15.836566 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829434    2567 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 29 12:25:15.836566 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829437    2567 flags.go:64] FLAG: --pods-per-core="0"
Jul 29 12:25:15.836566 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829440    2567 flags.go:64] FLAG: --port="10250"
Jul 29 12:25:15.836566 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829443    2567 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 29 12:25:15.836566 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829446    2567 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-093c63ac5a8e9c847"
Jul 29 12:25:15.836566 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829449    2567 flags.go:64] FLAG: --qos-reserved=""
Jul 29 12:25:15.836566 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829452    2567 flags.go:64] FLAG: --read-only-port="10255"
Jul 29 12:25:15.836566 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829455    2567 flags.go:64] FLAG: --register-node="true"
Jul 29 12:25:15.836566 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829457    2567 flags.go:64] FLAG: --register-with-taints=""
Jul 29 12:25:15.836566 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829461    2567 flags.go:64] FLAG: --registry-burst="10"
Jul 29 12:25:15.836566 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829466    2567 flags.go:64] FLAG: --registry-qps="5"
Jul 29 12:25:15.836566 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829469    2567 flags.go:64] FLAG: --reserved-cpus=""
Jul 29 12:25:15.836566 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829471    2567 flags.go:64] FLAG: --reserved-memory=""
Jul 29 12:25:15.836566 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829475    2567 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 29 12:25:15.836566 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829478    2567 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 29 12:25:15.837150 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829481    2567 flags.go:64] FLAG: --rotate-certificates="false"
Jul 29 12:25:15.837150 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829484    2567 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 29 12:25:15.837150 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829486    2567 flags.go:64] FLAG: --runonce="false"
Jul 29 12:25:15.837150 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829489    2567 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 29 12:25:15.837150 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829492    2567 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 29 12:25:15.837150 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829495    2567 flags.go:64] FLAG: --seccomp-default="false"
Jul 29 12:25:15.837150 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829498    2567 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 29 12:25:15.837150 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829501    2567 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 29 12:25:15.837150 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829504    2567 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 29 12:25:15.837150 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829507    2567 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 29 12:25:15.837150 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829509    2567 flags.go:64] FLAG: --storage-driver-password="root"
Jul 29 12:25:15.837150 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829512    2567 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 29 12:25:15.837150 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829515    2567 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 29 12:25:15.837150 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829518    2567 flags.go:64] FLAG: --storage-driver-user="root"
Jul 29 12:25:15.837150 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829520    2567 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 29 12:25:15.837150 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829523    2567 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 29 12:25:15.837150 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829526    2567 flags.go:64] FLAG: --system-cgroups=""
Jul 29 12:25:15.837150 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829529    2567 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 29 12:25:15.837150 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829535    2567 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 29 12:25:15.837150 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829537    2567 flags.go:64] FLAG: --tls-cert-file=""
Jul 29 12:25:15.837150 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829540    2567 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 29 12:25:15.837150 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829544    2567 flags.go:64] FLAG: --tls-min-version=""
Jul 29 12:25:15.837150 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829547    2567 flags.go:64] FLAG: --tls-private-key-file=""
Jul 29 12:25:15.837150 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829549    2567 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 29 12:25:15.837150 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829552    2567 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 29 12:25:15.837748 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829555    2567 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 29 12:25:15.837748 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829559    2567 flags.go:64] FLAG: --v="2"
Jul 29 12:25:15.837748 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829564    2567 flags.go:64] FLAG: --version="false"
Jul 29 12:25:15.837748 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829569    2567 flags.go:64] FLAG: --vmodule=""
Jul 29 12:25:15.837748 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829572    2567 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 29 12:25:15.837748 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.829575    2567 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 29 12:25:15.837748 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829689    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 12:25:15.837748 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829695    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 12:25:15.837748 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829698    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 12:25:15.837748 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829702    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 12:25:15.837748 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829705    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 12:25:15.837748 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829708    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 12:25:15.837748 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829711    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 12:25:15.837748 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829714    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 12:25:15.837748 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829716    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 12:25:15.837748 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829720    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 12:25:15.837748 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829724    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 12:25:15.837748 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829726    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 12:25:15.837748 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829729    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 12:25:15.837748 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829732    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 12:25:15.837748 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829735    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 12:25:15.838280 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829738    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 12:25:15.838280 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829741    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 12:25:15.838280 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829743    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 12:25:15.838280 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829746    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 12:25:15.838280 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829748    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 12:25:15.838280 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829751    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 12:25:15.838280 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829753    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 12:25:15.838280 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829755    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 12:25:15.838280 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829759    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 12:25:15.838280 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829763    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 12:25:15.838280 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829765    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 12:25:15.838280 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829768    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 12:25:15.838280 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829772    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 12:25:15.838280 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829776    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 12:25:15.838280 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829778    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 12:25:15.838280 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829786    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 12:25:15.838280 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829789    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 12:25:15.838280 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829792    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 12:25:15.838280 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829794    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 12:25:15.838280 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829797    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 12:25:15.838746 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829799    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 12:25:15.838746 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829802    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 12:25:15.838746 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829804    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 12:25:15.838746 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829807    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 12:25:15.838746 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829809    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 12:25:15.838746 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829812    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 12:25:15.838746 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829814    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 12:25:15.838746 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829817    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 12:25:15.838746 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829819    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 12:25:15.838746 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829821    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 12:25:15.838746 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829824    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 12:25:15.838746 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829827    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 12:25:15.838746 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829829    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 12:25:15.838746 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829832    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 12:25:15.838746 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829834    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 12:25:15.838746 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829837    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 12:25:15.838746 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829839    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 12:25:15.838746 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829842    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 12:25:15.838746 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829844    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 12:25:15.839198 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829846    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 12:25:15.839198 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829849    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 12:25:15.839198 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829851    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 12:25:15.839198 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829854    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 12:25:15.839198 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829856    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 12:25:15.839198 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829860    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 12:25:15.839198 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829863    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 12:25:15.839198 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829865    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 12:25:15.839198 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829868    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 12:25:15.839198 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829879    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 12:25:15.839198 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829881    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 12:25:15.839198 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829884    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 12:25:15.839198 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829886    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 12:25:15.839198 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829889    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 12:25:15.839198 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829891    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 12:25:15.839198 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829894    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 12:25:15.839198 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829896    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 12:25:15.839198 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829899    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 12:25:15.839198 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829901    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 12:25:15.839198 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829904    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 12:25:15.839642 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829906    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 12:25:15.839642 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829909    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 12:25:15.839642 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829911    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 12:25:15.839642 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829914    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 12:25:15.839642 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829916    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 12:25:15.839642 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829919    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 12:25:15.839642 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829921    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 12:25:15.839642 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829924    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 12:25:15.839642 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829926    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 12:25:15.839642 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829929    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 12:25:15.839642 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829931    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 12:25:15.839642 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829934    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 12:25:15.839642 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829936    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 12:25:15.839642 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829938    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 12:25:15.839642 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829941    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 12:25:15.839642 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829943    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 12:25:15.839642 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829946    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 12:25:15.839642 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829949    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 12:25:15.839642 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829953    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 12:25:15.839642 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829955    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 12:25:15.840191 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829957    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 12:25:15.840191 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829960    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 12:25:15.840191 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829981    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 12:25:15.840191 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829985    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 12:25:15.840191 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.829989    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 12:25:15.840191 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.830527    2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 29 12:25:15.840191 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.836071    2567 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 29 12:25:15.840191 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.836086    2567 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 29 12:25:15.840191 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.836103    2567 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 29 12:25:15.840191 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.836108    2567 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 29 12:25:15.840191 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836159    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 12:25:15.840191 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836163    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 12:25:15.840191 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836167    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 12:25:15.840191 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836169    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 12:25:15.840525 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836172    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 12:25:15.840525 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836174    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 12:25:15.840525 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836177    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 12:25:15.840525 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836180    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 12:25:15.840525 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836194    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 12:25:15.840525 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836197    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 12:25:15.840525 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836200    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 12:25:15.840525 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836203    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 12:25:15.840525 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836205    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 12:25:15.840525 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836208    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 12:25:15.840525 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836212    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 12:25:15.840525 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836215    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 12:25:15.840525 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836217    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 12:25:15.840525 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836220    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 12:25:15.840525 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836222    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 12:25:15.840525 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836225    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 12:25:15.840525 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836227    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 12:25:15.840525 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836229    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 12:25:15.840525 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836232    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 12:25:15.840525 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836235    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 12:25:15.840991 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836237    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 12:25:15.840991 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836240    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 12:25:15.840991 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836243    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 12:25:15.840991 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836245    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 12:25:15.840991 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836249    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 12:25:15.840991 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836252    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 12:25:15.840991 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836256    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 12:25:15.840991 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836260    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 12:25:15.840991 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836263    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 12:25:15.840991 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836266    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 12:25:15.840991 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836268    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 12:25:15.840991 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836271    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 12:25:15.840991 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836274    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 12:25:15.840991 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836276    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 12:25:15.840991 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836278    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 12:25:15.840991 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836281    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 12:25:15.840991 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836283    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 12:25:15.840991 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836286    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 12:25:15.840991 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836288    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 12:25:15.841452 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836290    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 12:25:15.841452 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836293    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 12:25:15.841452 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836295    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 12:25:15.841452 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836298    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 12:25:15.841452 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836300    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 12:25:15.841452 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836302    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 12:25:15.841452 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836305    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 12:25:15.841452 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836307    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 12:25:15.841452 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836309    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 12:25:15.841452 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836312    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 12:25:15.841452 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836314    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 12:25:15.841452 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836316    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 12:25:15.841452 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836319    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 12:25:15.841452 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836321    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 12:25:15.841452 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836324    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 12:25:15.841452 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836327    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 12:25:15.841452 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836329    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 12:25:15.841452 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836332    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 12:25:15.841452 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836335    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 12:25:15.841452 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836338    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 12:25:15.841910 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836340    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 12:25:15.841910 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836343    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 12:25:15.841910 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836346    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 12:25:15.841910 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836348    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 12:25:15.841910 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836350    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 12:25:15.841910 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836353    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 12:25:15.841910 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836355    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 12:25:15.841910 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836358    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 12:25:15.841910 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836360    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 12:25:15.841910 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836362    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 12:25:15.841910 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836365    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 12:25:15.841910 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836367    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 12:25:15.841910 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836369    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 12:25:15.841910 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836372    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 12:25:15.841910 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836374    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 12:25:15.841910 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836376    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 12:25:15.841910 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836379    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 12:25:15.841910 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836382    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 12:25:15.841910 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836385    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 12:25:15.842514 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836388    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 12:25:15.842514 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836390    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 12:25:15.842514 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836392    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 12:25:15.842514 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836395    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 12:25:15.842514 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836398    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 12:25:15.842514 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836400    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 12:25:15.842514 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836403    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 12:25:15.842514 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836405    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 12:25:15.842514 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836408    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 12:25:15.842514 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836410    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 12:25:15.842514 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836413    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 12:25:15.842514 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836415    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 12:25:15.842514 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836418    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 12:25:15.842514 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836420    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 12:25:15.842514 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836423    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 12:25:15.842514 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836425    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 12:25:15.842514 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836428    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 12:25:15.842892 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.836432    2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 29 12:25:15.842892 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836551    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 12:25:15.842892 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836555    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 12:25:15.842892 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836558    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 12:25:15.842892 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836561    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 12:25:15.842892 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836563    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 12:25:15.842892 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836566    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 12:25:15.842892 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836569    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 12:25:15.842892 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836572    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 12:25:15.842892 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836575    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 12:25:15.842892 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836578    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 12:25:15.842892 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836580    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 12:25:15.842892 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836583    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 12:25:15.842892 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836585    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 12:25:15.842892 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836588    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 12:25:15.843256 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836590    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 12:25:15.843256 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836592    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 12:25:15.843256 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836595    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 12:25:15.843256 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836597    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 12:25:15.843256 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836600    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 12:25:15.843256 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836604    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 12:25:15.843256 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836607    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 12:25:15.843256 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836610    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 12:25:15.843256 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836612    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 12:25:15.843256 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836615    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 12:25:15.843256 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836618    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 12:25:15.843256 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836621    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 12:25:15.843256 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836623    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 12:25:15.843256 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836626    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 12:25:15.843256 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836629    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 12:25:15.843256 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836631    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 12:25:15.843256 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836634    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 12:25:15.843256 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836636    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 12:25:15.843256 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836639    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 12:25:15.843256 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836641    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 12:25:15.843702 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836644    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 12:25:15.843702 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836647    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 12:25:15.843702 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836649    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 12:25:15.843702 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836651    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 12:25:15.843702 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836654    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 12:25:15.843702 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836656    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 12:25:15.843702 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836658    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 12:25:15.843702 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836661    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 12:25:15.843702 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836663    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 12:25:15.843702 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836665    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 12:25:15.843702 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836668    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 12:25:15.843702 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836670    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 12:25:15.843702 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836672    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 12:25:15.843702 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836675    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 12:25:15.843702 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836677    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 12:25:15.843702 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836680    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 12:25:15.843702 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836682    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 12:25:15.843702 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836684    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 12:25:15.843702 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836687    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 12:25:15.844136 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836690    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 12:25:15.844136 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836693    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 12:25:15.844136 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836696    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 12:25:15.844136 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836698    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 12:25:15.844136 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836701    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 12:25:15.844136 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836704    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 12:25:15.844136 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836706    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 12:25:15.844136 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836708    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 12:25:15.844136 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836711    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 12:25:15.844136 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836714    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 12:25:15.844136 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836716    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 12:25:15.844136 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836718    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 12:25:15.844136 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836721    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 12:25:15.844136 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836723    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 12:25:15.844136 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836726    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 12:25:15.844136 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836728    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 12:25:15.844136 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836730    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 12:25:15.844136 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836733    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 12:25:15.844136 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836735    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 12:25:15.844136 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836738    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 12:25:15.844577 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836740    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 12:25:15.844577 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836742    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 12:25:15.844577 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836744    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 12:25:15.844577 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836748    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 12:25:15.844577 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836751    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 12:25:15.844577 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836754    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 12:25:15.844577 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836758    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 12:25:15.844577 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836761    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 12:25:15.844577 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836763    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 12:25:15.844577 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836766    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 12:25:15.844577 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836768    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 12:25:15.844577 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836771    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 12:25:15.844577 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836773    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 12:25:15.844577 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836775    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 12:25:15.844577 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836778    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 12:25:15.844577 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836780    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 12:25:15.844577 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836783    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 12:25:15.844577 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836785    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 12:25:15.844577 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836787    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 12:25:15.845010 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836789    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 12:25:15.845010 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836792    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 12:25:15.845010 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836794    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 12:25:15.845010 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836797    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 12:25:15.845010 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836799    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 12:25:15.845010 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836801    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 12:25:15.845010 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:15.836804    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 12:25:15.845010 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.836808    2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 29 12:25:15.845010 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.836897    2567 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 29 12:25:15.956087 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.956069    2567 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 29 12:25:15.957516 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.957505    2567 server.go:1019] "Starting client certificate rotation"
Jul 29 12:25:15.957626 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.957610    2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 29 12:25:15.958169 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.958158    2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 29 12:25:15.959513 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.959492    2567 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 29 12:25:15.960514 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.960497    2567 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 29 12:25:15.972696 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.972676    2567 log.go:25] "Validated CRI v1 runtime API"
Jul 29 12:25:15.976796 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.976781    2567 log.go:25] "Validated CRI v1 image API"
Jul 29 12:25:15.977543 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.977526    2567 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 29 12:25:15.979851 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.979832    2567 fs.go:135] Filesystem UUIDs: map[24fa3bf4-f74f-434d-8a3c-028a8005fdcf:/dev/nvme0n1p3 716893d2-56c1-42b1-b261-c7f4c03adb8f:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Jul 29 12:25:15.979905 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.979850    2567 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 29 12:25:15.984629 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.984524    2567 manager.go:216] Machine: {Timestamp:2026-07-29 12:25:15.983284601 +0000 UTC m=+0.424403656 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3249498 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec24d9024e4c80a564e5fd29fb837b58 SystemUUID:ec24d902-4e4c-80a5-64e5-fd29fb837b58 BootID:92e71e3a-1344-4a58-af55-f4f0ea9ada61 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:98:1e:d3:75:6b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:98:1e:d3:75:6b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:16:5c:6c:03:32:81 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 29 12:25:15.984629 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.984626    2567 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 29 12:25:15.984727 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.984704    2567 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 29 12:25:15.986206 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.986186    2567 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 29 12:25:15.986341 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.986209    2567 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-25.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 29 12:25:15.987365 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.987355    2567 topology_manager.go:138] "Creating topology manager with none policy"
Jul 29 12:25:15.987396 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.987368    2567 container_manager_linux.go:309] "Creating device plugin manager"
Jul 29 12:25:15.987396 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.987374    2567 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 29 12:25:15.988055 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.988046    2567 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 29 12:25:15.989309 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.989300    2567 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 29 12:25:15.991738 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.991727    2567 state_mem.go:36] "Initialized new in-memory state store"
Jul 29 12:25:15.991843 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.991835    2567 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 29 12:25:15.992582 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.992574    2567 kubelet.go:486] "Attempting to sync node with API server"
Jul 29 12:25:15.992612 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.992585    2567 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 29 12:25:15.992612 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.992601    2567 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 29 12:25:15.992612 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.992610    2567 kubelet.go:398] "Adding apiserver pod source"
Jul 29 12:25:15.992685 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.992618    2567 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 29 12:25:15.992886 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.992872    2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 29 12:25:15.992922 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.992899    2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 29 12:25:15.995022 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.995008    2567 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 29 12:25:15.996203 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.996191    2567 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 29 12:25:15.996236 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.996229    2567 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 29 12:25:15.996792 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.996782    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 29 12:25:15.996819 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.996796    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 29 12:25:15.996819 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.996801    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 29 12:25:15.996819 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.996806    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 29 12:25:15.996819 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.996813    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 29 12:25:15.996819 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.996819    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 29 12:25:15.997039 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.997029    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 29 12:25:15.997072 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.997040    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 29 12:25:15.997072 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.997046    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 29 12:25:15.997072 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.997052    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 29 12:25:15.997072 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.997060    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 29 12:25:15.997072 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.997068    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 29 12:25:15.997839 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.997830    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 29 12:25:15.997871 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.997841    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 29 12:25:15.999139 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:15.999125    2567 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 29 12:25:16.000703 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.000692    2567 server.go:1268] "Started kubelet"
Jul 29 12:25:16.000783 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.000758    2567 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 29 12:25:16.000827 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.000765    2567 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 29 12:25:16.004423 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.004342    2567 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 29 12:25:16.004950 ip-10-0-128-25 systemd[1]: Started Kubernetes Kubelet.
Jul 29 12:25:16.006521 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.005985    2567 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 29 12:25:16.008237 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.008212    2567 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-25.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 29 12:25:16.008347 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:16.008267    2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 29 12:25:16.008347 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:16.008288    2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-25.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 29 12:25:16.008770 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.008757    2567 server.go:310] "Adding debug handlers to kubelet server"
Jul 29 12:25:16.010715 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:16.010697    2567 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 29 12:25:16.012793 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.012778    2567 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 29 12:25:16.012870 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.012824    2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 29 12:25:16.012870 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:16.012844    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-25.ec2.internal\" not found"
Jul 29 12:25:16.013351 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.013333    2567 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 29 12:25:16.013447 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.013432    2567 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 29 12:25:16.013501 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.013451    2567 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 29 12:25:16.013581 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.013567    2567 factory.go:55] Registering systemd factory
Jul 29 12:25:16.013638 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.013587    2567 factory.go:223] Registration of the systemd container factory successfully
Jul 29 12:25:16.013638 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.013590    2567 reconstruct.go:97] "Volume reconstruction finished"
Jul 29 12:25:16.013638 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.013608    2567 reconciler.go:29] "Reconciler: start to sync state"
Jul 29 12:25:16.013638 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:16.012518    2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-25.ec2.internal.18c6c247d0ceec87  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-25.ec2.internal,UID:ip-10-0-128-25.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-25.ec2.internal,},FirstTimestamp:2026-07-29 12:25:16.000668807 +0000 UTC m=+0.441787868,LastTimestamp:2026-07-29 12:25:16.000668807 +0000 UTC m=+0.441787868,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-25.ec2.internal,}"
Jul 29 12:25:16.013822 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.013765    2567 factory.go:153] Registering CRI-O factory
Jul 29 12:25:16.013822 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.013777    2567 factory.go:223] Registration of the crio container factory successfully
Jul 29 12:25:16.013950 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.013842    2567 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 29 12:25:16.013950 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.013874    2567 factory.go:103] Registering Raw factory
Jul 29 12:25:16.013950 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.013885    2567 manager.go:1196] Started watching for new ooms in manager
Jul 29 12:25:16.014249 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.014236    2567 manager.go:349] Starting recovery of all containers
Jul 29 12:25:16.018565 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:16.018381    2567 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-25.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 29 12:25:16.018565 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:16.018486    2567 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 29 12:25:16.020209 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.020179    2567 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 29 12:25:16.023591 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.023577    2567 manager.go:354] Recovery completed
Jul 29 12:25:16.029698 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.029685    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 12:25:16.031324 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.031307    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-25.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 12:25:16.031379 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.031333    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-25.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 12:25:16.031379 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.031343    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-25.ec2.internal" event="NodeHasSufficientPID"
Jul 29 12:25:16.031766 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.031754    2567 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 29 12:25:16.031814 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.031765    2567 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 29 12:25:16.031814 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.031779    2567 state_mem.go:36] "Initialized new in-memory state store"
Jul 29 12:25:16.031814 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.031795    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 12:25:16.032451 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.032438    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-25.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 12:25:16.032529 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.032459    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-25.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 12:25:16.032529 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.032469    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-25.ec2.internal" event="NodeHasSufficientPID"
Jul 29 12:25:16.032942 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.032931    2567 policy_none.go:49] "None policy: Start"
Jul 29 12:25:16.033008 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.032947    2567 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 29 12:25:16.033008 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.032956    2567 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 29 12:25:16.033414 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:16.033346    2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-25.ec2.internal.18c6c247d2a2a962  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-25.ec2.internal,UID:ip-10-0-128-25.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-25.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-25.ec2.internal,},FirstTimestamp:2026-07-29 12:25:16.031322466 +0000 UTC m=+0.472441502,LastTimestamp:2026-07-29 12:25:16.031322466 +0000 UTC m=+0.472441502,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-25.ec2.internal,}"
Jul 29 12:25:16.033846 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.033835    2567 policy_none.go:47] "Start"
Jul 29 12:25:16.042094 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:16.042038    2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-25.ec2.internal.18c6c247d2a2e69e  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-25.ec2.internal,UID:ip-10-0-128-25.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-128-25.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-128-25.ec2.internal,},FirstTimestamp:2026-07-29 12:25:16.031338142 +0000 UTC m=+0.472457178,LastTimestamp:2026-07-29 12:25:16.031338142 +0000 UTC m=+0.472457178,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-25.ec2.internal,}"
Jul 29 12:25:16.052136 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:16.052011    2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-25.ec2.internal.18c6c247d2a3080a  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-25.ec2.internal,UID:ip-10-0-128-25.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-128-25.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-128-25.ec2.internal,},FirstTimestamp:2026-07-29 12:25:16.031346698 +0000 UTC m=+0.472465734,LastTimestamp:2026-07-29 12:25:16.031346698 +0000 UTC m=+0.472465734,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-25.ec2.internal,}"
Jul 29 12:25:16.059293 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.059279    2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-p2vxz"
Jul 29 12:25:16.062015 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:16.061943    2567 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-128-25.ec2.internal.18c6c247d2a2a962\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-25.ec2.internal.18c6c247d2a2a962  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-25.ec2.internal,UID:ip-10-0-128-25.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-25.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-25.ec2.internal,},FirstTimestamp:2026-07-29 12:25:16.031322466 +0000 UTC m=+0.472441502,LastTimestamp:2026-07-29 12:25:16.032449328 +0000 UTC m=+0.473568364,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-25.ec2.internal,}"
Jul 29 12:25:16.069046 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.069029    2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-p2vxz"
Jul 29 12:25:16.071238 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:16.071180    2567 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-128-25.ec2.internal.18c6c247d2a2e69e\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-25.ec2.internal.18c6c247d2a2e69e  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-25.ec2.internal,UID:ip-10-0-128-25.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-128-25.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-128-25.ec2.internal,},FirstTimestamp:2026-07-29 12:25:16.031338142 +0000 UTC m=+0.472457178,LastTimestamp:2026-07-29 12:25:16.032463775 +0000 UTC m=+0.473582811,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-25.ec2.internal,}"
Jul 29 12:25:16.079474 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.079461    2567 manager.go:337] "Starting Device Plugin manager"
Jul 29 12:25:16.079535 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:16.079484    2567 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 29 12:25:16.079535 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.079492    2567 server.go:87] "Starting device plugin registration server"
Jul 29 12:25:16.079715 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.079699    2567 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 29 12:25:16.079790 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.079713    2567 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 29 12:25:16.079900 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.079786    2567 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 29 12:25:16.079900 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.079865    2567 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 29 12:25:16.079900 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.079874    2567 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 29 12:25:16.080340 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:16.080322    2567 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 29 12:25:16.080397 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:16.080374    2567 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-25.ec2.internal\" not found"
Jul 29 12:25:16.167505 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.167483    2567 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 29 12:25:16.168493 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.167516    2567 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 29 12:25:16.168493 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.167532    2567 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 29 12:25:16.168493 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:16.167560    2567 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 29 12:25:16.171130 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.171116    2567 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 12:25:16.180142 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.180129    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 12:25:16.180853 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.180834    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-25.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 12:25:16.180924 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.180858    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-25.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 12:25:16.180924 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.180868    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-25.ec2.internal" event="NodeHasSufficientPID"
Jul 29 12:25:16.180924 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.180885    2567 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-25.ec2.internal"
Jul 29 12:25:16.189800 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.189787    2567 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-25.ec2.internal"
Jul 29 12:25:16.189848 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:16.189805    2567 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-25.ec2.internal\": node \"ip-10-0-128-25.ec2.internal\" not found"
Jul 29 12:25:16.215237 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:16.215217    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-25.ec2.internal\" not found"
Jul 29 12:25:16.268565 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.268533    2567 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-25.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-25.ec2.internal"]
Jul 29 12:25:16.268658 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.268604    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 12:25:16.269312 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.269298    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-25.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 12:25:16.269368 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.269324    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-25.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 12:25:16.269368 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.269334    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-25.ec2.internal" event="NodeHasSufficientPID"
Jul 29 12:25:16.270442 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.270429    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 12:25:16.270559 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.270546    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-25.ec2.internal"
Jul 29 12:25:16.270595 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.270573    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 12:25:16.271113 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.271092    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-25.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 12:25:16.271166 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.271123    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-25.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 12:25:16.271166 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.271132    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-25.ec2.internal" event="NodeHasSufficientPID"
Jul 29 12:25:16.271242 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.271096    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-25.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 12:25:16.271242 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.271200    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-25.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 12:25:16.271242 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.271212    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-25.ec2.internal" event="NodeHasSufficientPID"
Jul 29 12:25:16.272294 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.272282    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-25.ec2.internal"
Jul 29 12:25:16.272334 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.272305    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 12:25:16.272911 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.272895    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-25.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 12:25:16.272984 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.272924    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-25.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 12:25:16.272984 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.272939    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-25.ec2.internal" event="NodeHasSufficientPID"
Jul 29 12:25:16.294473 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:16.294453    2567 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-25.ec2.internal\" not found" node="ip-10-0-128-25.ec2.internal"
Jul 29 12:25:16.297806 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:16.297791    2567 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-25.ec2.internal\" not found" node="ip-10-0-128-25.ec2.internal"
Jul 29 12:25:16.315497 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:16.315479    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-25.ec2.internal\" not found"
Jul 29 12:25:16.315569 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.315531    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/875ad84743ea631b485f1bbadf831c17-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-25.ec2.internal\" (UID: \"875ad84743ea631b485f1bbadf831c17\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-25.ec2.internal"
Jul 29 12:25:16.315569 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.315549    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/875ad84743ea631b485f1bbadf831c17-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-25.ec2.internal\" (UID: \"875ad84743ea631b485f1bbadf831c17\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-25.ec2.internal"
Jul 29 12:25:16.315569 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.315564    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/eee88d5337a9ab304c1e57b0fd78f1d9-config\") pod \"kube-apiserver-proxy-ip-10-0-128-25.ec2.internal\" (UID: \"eee88d5337a9ab304c1e57b0fd78f1d9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-25.ec2.internal"
Jul 29 12:25:16.416036 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:16.415982    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-25.ec2.internal\" not found"
Jul 29 12:25:16.416036 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.416029    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/875ad84743ea631b485f1bbadf831c17-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-25.ec2.internal\" (UID: \"875ad84743ea631b485f1bbadf831c17\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-25.ec2.internal"
Jul 29 12:25:16.416216 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.416048    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/875ad84743ea631b485f1bbadf831c17-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-25.ec2.internal\" (UID: \"875ad84743ea631b485f1bbadf831c17\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-25.ec2.internal"
Jul 29 12:25:16.416216 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.416063    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/eee88d5337a9ab304c1e57b0fd78f1d9-config\") pod \"kube-apiserver-proxy-ip-10-0-128-25.ec2.internal\" (UID: \"eee88d5337a9ab304c1e57b0fd78f1d9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-25.ec2.internal"
Jul 29 12:25:16.416216 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.416097    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/eee88d5337a9ab304c1e57b0fd78f1d9-config\") pod \"kube-apiserver-proxy-ip-10-0-128-25.ec2.internal\" (UID: \"eee88d5337a9ab304c1e57b0fd78f1d9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-25.ec2.internal"
Jul 29 12:25:16.416216 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.416148    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/875ad84743ea631b485f1bbadf831c17-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-25.ec2.internal\" (UID: \"875ad84743ea631b485f1bbadf831c17\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-25.ec2.internal"
Jul 29 12:25:16.416216 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:16.416192    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/875ad84743ea631b485f1bbadf831c17-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-25.ec2.internal\" (UID: \"875ad84743ea631b485f1bbadf831c17\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-25.ec2.internal"
Jul 29 12:25:16.516618 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:16.516587    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-25.ec2.internal\" not found"
Jul 29 12:25:18.344246 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:18.344219    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-25.ec2.internal"
Jul 29 12:25:18.348477 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:18.348179    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-25.ec2.internal"
Jul 29 12:25:18.364239 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:18.364215    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-25.ec2.internal\" not found"
Jul 29 12:25:18.464661 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:18.464602    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-25.ec2.internal\" not found"
Jul 29 12:25:18.564975 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:18.564948    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-25.ec2.internal\" not found"
Jul 29 12:25:18.665360 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:18.665328    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-25.ec2.internal\" not found"
Jul 29 12:25:18.704519 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:18.704497    2567 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 29 12:25:18.704640 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:18.704621    2567 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 29 12:25:18.766063 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:18.766006    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-25.ec2.internal\" not found"
Jul 29 12:25:18.793483 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:18.793462    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 12:25:18.794228 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:18.794213    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-25.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 12:25:18.794322 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:18.794243    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-25.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 12:25:18.794322 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:18.794257    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-25.ec2.internal" event="NodeHasSufficientPID"
Jul 29 12:25:18.819552 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:18.819524    2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-28 12:20:17 +0000 UTC" deadline="2028-01-25 03:35:02.949196254 +0000 UTC"
Jul 29 12:25:18.819552 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:18.819550    2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13071h9m44.129649322s"
Jul 29 12:25:18.823603 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:18.823588    2567 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 12:25:18.860897 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:18.860877    2567 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-25.ec2.internal"
Jul 29 12:25:18.879804 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:18.879785    2567 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 29 12:25:18.881465 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:18.881451    2567 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-25.ec2.internal"
Jul 29 12:25:18.893838 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:18.892234    2567 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 29 12:25:18.930357 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:18.930340    2567 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 12:25:18.960285 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:18.960256    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeee88d5337a9ab304c1e57b0fd78f1d9.slice/crio-9aab815f999dc9029b5748c672bb636f569a51bf2330d5bf480694e301567e36 WatchSource:0}: Error finding container 9aab815f999dc9029b5748c672bb636f569a51bf2330d5bf480694e301567e36: Status 404 returned error can't find the container with id 9aab815f999dc9029b5748c672bb636f569a51bf2330d5bf480694e301567e36
Jul 29 12:25:18.966269 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:18.966256    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 12:25:19.145878 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:19.145855    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod875ad84743ea631b485f1bbadf831c17.slice/crio-8acc388b3e1b871337a203fea8fb20b3deab333db39517ba33dc800b4f00d32c WatchSource:0}: Error finding container 8acc388b3e1b871337a203fea8fb20b3deab333db39517ba33dc800b4f00d32c: Status 404 returned error can't find the container with id 8acc388b3e1b871337a203fea8fb20b3deab333db39517ba33dc800b4f00d32c
Jul 29 12:25:19.285032 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.285007    2567 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 12:25:19.662795 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.662770    2567 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 12:25:19.741469 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.741447    2567 apiserver.go:52] "Watching apiserver"
Jul 29 12:25:19.747851 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.747830    2567 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 29 12:25:19.750368 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.750041    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-w97vv","kube-system/konnectivity-agent-znk2m","kube-system/kube-apiserver-proxy-ip-10-0-128-25.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzmrx","openshift-image-registry/node-ca-clmgr","openshift-multus/multus-additional-cni-plugins-h2qrx","openshift-multus/multus-ph59r","openshift-multus/network-metrics-daemon-qkdl7","openshift-cluster-node-tuning-operator/tuned-z99ld","openshift-dns/node-resolver-8rddq","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-25.ec2.internal","openshift-network-diagnostics/network-check-target-r4hjk","openshift-network-operator/iptables-alerter-nkfzw","openshift-ovn-kubernetes/ovnkube-node-kj82f"]
Jul 29 12:25:19.751506 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.751411    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-clmgr"
Jul 29 12:25:19.752606 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.752586    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qkdl7"
Jul 29 12:25:19.752742 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:19.752654    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qkdl7" podUID="68d50be9-1461-43d4-811e-74ff3fed1a75"
Jul 29 12:25:19.753860 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.753788    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-h2qrx"
Jul 29 12:25:19.754700 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.754681    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 29 12:25:19.754821 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.754799    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 29 12:25:19.754900 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.754868    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-fpkns\""
Jul 29 12:25:19.755012 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.754990    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-ph59r"
Jul 29 12:25:19.755485 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.755465    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 29 12:25:19.756586 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.756149    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzmrx"
Jul 29 12:25:19.758080 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.758064    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w97vv"
Jul 29 12:25:19.758167 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:19.758117    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w97vv" podUID="1635829e-9a40-4521-8430-88adbb07a97a"
Jul 29 12:25:19.760139 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.759546    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 29 12:25:19.760139 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.759611    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 29 12:25:19.760139 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.759928    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 29 12:25:19.760139 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.759988    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 29 12:25:19.760139 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.760006    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 29 12:25:19.760378 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.760159    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-znk2m"
Jul 29 12:25:19.760378 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.760237    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-xkk2l\""
Jul 29 12:25:19.760378 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.760248    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 29 12:25:19.761229 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.760816    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 29 12:25:19.761229 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.761147    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-7b4rm\""
Jul 29 12:25:19.761669 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.761392    2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 29 12:25:19.762687 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.762323    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 29 12:25:19.764500 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.762809    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-kw8fz\""
Jul 29 12:25:19.764500 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.762887    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 29 12:25:19.764500 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.763184    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-z99ld"
Jul 29 12:25:19.764500 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.764105    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 29 12:25:19.764500 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.764154    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-7snzf\""
Jul 29 12:25:19.764770 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.764502    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 29 12:25:19.765362 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.765333    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8rddq"
Jul 29 12:25:19.766169 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.766046    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 29 12:25:19.766288 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.766243    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-m9b7r\""
Jul 29 12:25:19.766288 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.766253    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 29 12:25:19.767275 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.767133    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r4hjk"
Jul 29 12:25:19.767275 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:19.767204    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r4hjk" podUID="8e7279c1-35b7-4f54-8b02-fdac281ef757"
Jul 29 12:25:19.769264 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.768845    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nkfzw"
Jul 29 12:25:19.769951 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.769929    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-kj82f"
Jul 29 12:25:19.770542 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.770521    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1635829e-9a40-4521-8430-88adbb07a97a-original-pull-secret\") pod \"global-pull-secret-syncer-w97vv\" (UID: \"1635829e-9a40-4521-8430-88adbb07a97a\") " pod="kube-system/global-pull-secret-syncer-w97vv"
Jul 29 12:25:19.770643 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.770546    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/192debc3-6fd5-40d8-bca9-b58b50a97fa5-var-lib-kubelet\") pod \"tuned-z99ld\" (UID: \"192debc3-6fd5-40d8-bca9-b58b50a97fa5\") " pod="openshift-cluster-node-tuning-operator/tuned-z99ld"
Jul 29 12:25:19.770643 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.770527    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-5mkfx\""
Jul 29 12:25:19.770643 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.770561    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0f03f162-cb82-442c-8e40-b8877aa1996f-cnibin\") pod \"multus-ph59r\" (UID: \"0f03f162-cb82-442c-8e40-b8877aa1996f\") " pod="openshift-multus/multus-ph59r"
Jul 29 12:25:19.770643 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.770577    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0f03f162-cb82-442c-8e40-b8877aa1996f-multus-conf-dir\") pod \"multus-ph59r\" (UID: \"0f03f162-cb82-442c-8e40-b8877aa1996f\") " pod="openshift-multus/multus-ph59r"
Jul 29 12:25:19.770643 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.770586    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 29 12:25:19.770643 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.770629    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bz88c\" (UniqueName: \"kubernetes.io/projected/0f03f162-cb82-442c-8e40-b8877aa1996f-kube-api-access-bz88c\") pod \"multus-ph59r\" (UID: \"0f03f162-cb82-442c-8e40-b8877aa1996f\") " pod="openshift-multus/multus-ph59r"
Jul 29 12:25:19.770926 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.770646    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 29 12:25:19.770926 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.770690    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a09948e8-234f-41ef-866d-10ddb32e33fd-os-release\") pod \"multus-additional-cni-plugins-h2qrx\" (UID: \"a09948e8-234f-41ef-866d-10ddb32e33fd\") " pod="openshift-multus/multus-additional-cni-plugins-h2qrx"
Jul 29 12:25:19.770926 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.770718    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4sr5\" (UniqueName: \"kubernetes.io/projected/a09948e8-234f-41ef-866d-10ddb32e33fd-kube-api-access-v4sr5\") pod \"multus-additional-cni-plugins-h2qrx\" (UID: \"a09948e8-234f-41ef-866d-10ddb32e33fd\") " pod="openshift-multus/multus-additional-cni-plugins-h2qrx"
Jul 29 12:25:19.770926 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.770742    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/192debc3-6fd5-40d8-bca9-b58b50a97fa5-etc-sysctl-d\") pod \"tuned-z99ld\" (UID: \"192debc3-6fd5-40d8-bca9-b58b50a97fa5\") " pod="openshift-cluster-node-tuning-operator/tuned-z99ld"
Jul 29 12:25:19.770926 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.770804    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/192debc3-6fd5-40d8-bca9-b58b50a97fa5-etc-tuned\") pod \"tuned-z99ld\" (UID: \"192debc3-6fd5-40d8-bca9-b58b50a97fa5\") " pod="openshift-cluster-node-tuning-operator/tuned-z99ld"
Jul 29 12:25:19.770926 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.770833    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0f03f162-cb82-442c-8e40-b8877aa1996f-hostroot\") pod \"multus-ph59r\" (UID: \"0f03f162-cb82-442c-8e40-b8877aa1996f\") " pod="openshift-multus/multus-ph59r"
Jul 29 12:25:19.770926 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.770856    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1840c2b3-2714-4f4e-9255-2cb612abb53e-etc-selinux\") pod \"aws-ebs-csi-driver-node-lzmrx\" (UID: \"1840c2b3-2714-4f4e-9255-2cb612abb53e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzmrx"
Jul 29 12:25:19.770926 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.770877    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a09948e8-234f-41ef-866d-10ddb32e33fd-cnibin\") pod \"multus-additional-cni-plugins-h2qrx\" (UID: \"a09948e8-234f-41ef-866d-10ddb32e33fd\") " pod="openshift-multus/multus-additional-cni-plugins-h2qrx"
Jul 29 12:25:19.770926 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.770891    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1635829e-9a40-4521-8430-88adbb07a97a-kubelet-config\") pod \"global-pull-secret-syncer-w97vv\" (UID: \"1635829e-9a40-4521-8430-88adbb07a97a\") " pod="kube-system/global-pull-secret-syncer-w97vv"
Jul 29 12:25:19.770926 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.770894    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 29 12:25:19.770926 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.770909    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7be0cf2c-db65-486e-8545-eabd24ad8428-agent-certs\") pod \"konnectivity-agent-znk2m\" (UID: \"7be0cf2c-db65-486e-8545-eabd24ad8428\") " pod="kube-system/konnectivity-agent-znk2m"
Jul 29 12:25:19.771355 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.770930    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/192debc3-6fd5-40d8-bca9-b58b50a97fa5-sys\") pod \"tuned-z99ld\" (UID: \"192debc3-6fd5-40d8-bca9-b58b50a97fa5\") " pod="openshift-cluster-node-tuning-operator/tuned-z99ld"
Jul 29 12:25:19.771355 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.770963    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0f03f162-cb82-442c-8e40-b8877aa1996f-host-run-multus-certs\") pod \"multus-ph59r\" (UID: \"0f03f162-cb82-442c-8e40-b8877aa1996f\") " pod="openshift-multus/multus-ph59r"
Jul 29 12:25:19.771355 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.770981    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 29 12:25:19.771355 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.771022    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/192debc3-6fd5-40d8-bca9-b58b50a97fa5-etc-sysctl-conf\") pod \"tuned-z99ld\" (UID: \"192debc3-6fd5-40d8-bca9-b58b50a97fa5\") " pod="openshift-cluster-node-tuning-operator/tuned-z99ld"
Jul 29 12:25:19.771355 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.771051    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0f03f162-cb82-442c-8e40-b8877aa1996f-host-var-lib-cni-multus\") pod \"multus-ph59r\" (UID: \"0f03f162-cb82-442c-8e40-b8877aa1996f\") " pod="openshift-multus/multus-ph59r"
Jul 29 12:25:19.771355 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.771074    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1840c2b3-2714-4f4e-9255-2cb612abb53e-device-dir\") pod \"aws-ebs-csi-driver-node-lzmrx\" (UID: \"1840c2b3-2714-4f4e-9255-2cb612abb53e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzmrx"
Jul 29 12:25:19.771355 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.771101    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 29 12:25:19.771355 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.771133    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a09948e8-234f-41ef-866d-10ddb32e33fd-system-cni-dir\") pod \"multus-additional-cni-plugins-h2qrx\" (UID: \"a09948e8-234f-41ef-866d-10ddb32e33fd\") " pod="openshift-multus/multus-additional-cni-plugins-h2qrx"
Jul 29 12:25:19.771355 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.771162    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a09948e8-234f-41ef-866d-10ddb32e33fd-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-h2qrx\" (UID: \"a09948e8-234f-41ef-866d-10ddb32e33fd\") " pod="openshift-multus/multus-additional-cni-plugins-h2qrx"
Jul 29 12:25:19.771355 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.771196    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0f03f162-cb82-442c-8e40-b8877aa1996f-host-run-netns\") pod \"multus-ph59r\" (UID: \"0f03f162-cb82-442c-8e40-b8877aa1996f\") " pod="openshift-multus/multus-ph59r"
Jul 29 12:25:19.771355 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.771253    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0f03f162-cb82-442c-8e40-b8877aa1996f-host-var-lib-kubelet\") pod \"multus-ph59r\" (UID: \"0f03f162-cb82-442c-8e40-b8877aa1996f\") " pod="openshift-multus/multus-ph59r"
Jul 29 12:25:19.771355 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.771281    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/192debc3-6fd5-40d8-bca9-b58b50a97fa5-etc-modprobe-d\") pod \"tuned-z99ld\" (UID: \"192debc3-6fd5-40d8-bca9-b58b50a97fa5\") " pod="openshift-cluster-node-tuning-operator/tuned-z99ld"
Jul 29 12:25:19.771355 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.771307    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/192debc3-6fd5-40d8-bca9-b58b50a97fa5-run\") pod \"tuned-z99ld\" (UID: \"192debc3-6fd5-40d8-bca9-b58b50a97fa5\") " pod="openshift-cluster-node-tuning-operator/tuned-z99ld"
Jul 29 12:25:19.771355 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.771330    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1840c2b3-2714-4f4e-9255-2cb612abb53e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lzmrx\" (UID: \"1840c2b3-2714-4f4e-9255-2cb612abb53e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzmrx"
Jul 29 12:25:19.771355 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.771352    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4bdec0f6-396e-4392-9e8d-e2bad0289b8f-hosts-file\") pod \"node-resolver-8rddq\" (UID: \"4bdec0f6-396e-4392-9e8d-e2bad0289b8f\") " pod="openshift-dns/node-resolver-8rddq"
Jul 29 12:25:19.772135 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.771373    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68d50be9-1461-43d4-811e-74ff3fed1a75-metrics-certs\") pod \"network-metrics-daemon-qkdl7\" (UID: \"68d50be9-1461-43d4-811e-74ff3fed1a75\") " pod="openshift-multus/network-metrics-daemon-qkdl7"
Jul 29 12:25:19.772135 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.771391    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/192debc3-6fd5-40d8-bca9-b58b50a97fa5-tmp\") pod \"tuned-z99ld\" (UID: \"192debc3-6fd5-40d8-bca9-b58b50a97fa5\") " pod="openshift-cluster-node-tuning-operator/tuned-z99ld"
Jul 29 12:25:19.772135 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.771425    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7be0cf2c-db65-486e-8545-eabd24ad8428-konnectivity-ca\") pod \"konnectivity-agent-znk2m\" (UID: \"7be0cf2c-db65-486e-8545-eabd24ad8428\") " pod="kube-system/konnectivity-agent-znk2m"
Jul 29 12:25:19.772135 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.771389    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-l9k6k\""
Jul 29 12:25:19.772135 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.771479    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0f03f162-cb82-442c-8e40-b8877aa1996f-cni-binary-copy\") pod \"multus-ph59r\" (UID: \"0f03f162-cb82-442c-8e40-b8877aa1996f\") " pod="openshift-multus/multus-ph59r"
Jul 29 12:25:19.772135 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.771512    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjjwl\" (UniqueName: \"kubernetes.io/projected/4bdec0f6-396e-4392-9e8d-e2bad0289b8f-kube-api-access-bjjwl\") pod \"node-resolver-8rddq\" (UID: \"4bdec0f6-396e-4392-9e8d-e2bad0289b8f\") " pod="openshift-dns/node-resolver-8rddq"
Jul 29 12:25:19.772135 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.771544    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fr22\" (UniqueName: \"kubernetes.io/projected/68d50be9-1461-43d4-811e-74ff3fed1a75-kube-api-access-9fr22\") pod \"network-metrics-daemon-qkdl7\" (UID: \"68d50be9-1461-43d4-811e-74ff3fed1a75\") " pod="openshift-multus/network-metrics-daemon-qkdl7"
Jul 29 12:25:19.772135 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.771567    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a09948e8-234f-41ef-866d-10ddb32e33fd-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-h2qrx\" (UID: \"a09948e8-234f-41ef-866d-10ddb32e33fd\") " pod="openshift-multus/multus-additional-cni-plugins-h2qrx"
Jul 29 12:25:19.772135 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.771588    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/192debc3-6fd5-40d8-bca9-b58b50a97fa5-etc-kubernetes\") pod \"tuned-z99ld\" (UID: \"192debc3-6fd5-40d8-bca9-b58b50a97fa5\") " pod="openshift-cluster-node-tuning-operator/tuned-z99ld"
Jul 29 12:25:19.772135 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.771610    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/192debc3-6fd5-40d8-bca9-b58b50a97fa5-host\") pod \"tuned-z99ld\" (UID: \"192debc3-6fd5-40d8-bca9-b58b50a97fa5\") " pod="openshift-cluster-node-tuning-operator/tuned-z99ld"
Jul 29 12:25:19.772135 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.771641    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6xrh\" (UniqueName: \"kubernetes.io/projected/192debc3-6fd5-40d8-bca9-b58b50a97fa5-kube-api-access-v6xrh\") pod \"tuned-z99ld\" (UID: \"192debc3-6fd5-40d8-bca9-b58b50a97fa5\") " pod="openshift-cluster-node-tuning-operator/tuned-z99ld"
Jul 29 12:25:19.772135 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.771667    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glc5q\" (UniqueName: \"kubernetes.io/projected/caf950ed-dbb9-4b7d-a802-062882ff4899-kube-api-access-glc5q\") pod \"node-ca-clmgr\" (UID: \"caf950ed-dbb9-4b7d-a802-062882ff4899\") " pod="openshift-image-registry/node-ca-clmgr"
Jul 29 12:25:19.772135 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.771690    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0f03f162-cb82-442c-8e40-b8877aa1996f-host-run-k8s-cni-cncf-io\") pod \"multus-ph59r\" (UID: \"0f03f162-cb82-442c-8e40-b8877aa1996f\") " pod="openshift-multus/multus-ph59r"
Jul 29 12:25:19.772135 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.771723    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1840c2b3-2714-4f4e-9255-2cb612abb53e-socket-dir\") pod \"aws-ebs-csi-driver-node-lzmrx\" (UID: \"1840c2b3-2714-4f4e-9255-2cb612abb53e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzmrx"
Jul 29 12:25:19.772135 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.771746    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/192debc3-6fd5-40d8-bca9-b58b50a97fa5-etc-sysconfig\") pod \"tuned-z99ld\" (UID: \"192debc3-6fd5-40d8-bca9-b58b50a97fa5\") " pod="openshift-cluster-node-tuning-operator/tuned-z99ld"
Jul 29 12:25:19.772135 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.771765    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/192debc3-6fd5-40d8-bca9-b58b50a97fa5-etc-systemd\") pod \"tuned-z99ld\" (UID: \"192debc3-6fd5-40d8-bca9-b58b50a97fa5\") " pod="openshift-cluster-node-tuning-operator/tuned-z99ld"
Jul 29 12:25:19.772135 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.771801    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0f03f162-cb82-442c-8e40-b8877aa1996f-etc-kubernetes\") pod \"multus-ph59r\" (UID: \"0f03f162-cb82-442c-8e40-b8877aa1996f\") " pod="openshift-multus/multus-ph59r"
Jul 29 12:25:19.772847 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.771839    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0f03f162-cb82-442c-8e40-b8877aa1996f-system-cni-dir\") pod \"multus-ph59r\" (UID: \"0f03f162-cb82-442c-8e40-b8877aa1996f\") " pod="openshift-multus/multus-ph59r"
Jul 29 12:25:19.772847 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.771865    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0f03f162-cb82-442c-8e40-b8877aa1996f-os-release\") pod \"multus-ph59r\" (UID: \"0f03f162-cb82-442c-8e40-b8877aa1996f\") " pod="openshift-multus/multus-ph59r"
Jul 29 12:25:19.772847 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.771888    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdd82\" (UniqueName: \"kubernetes.io/projected/1840c2b3-2714-4f4e-9255-2cb612abb53e-kube-api-access-qdd82\") pod \"aws-ebs-csi-driver-node-lzmrx\" (UID: \"1840c2b3-2714-4f4e-9255-2cb612abb53e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzmrx"
Jul 29 12:25:19.772847 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.771926    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a09948e8-234f-41ef-866d-10ddb32e33fd-cni-binary-copy\") pod \"multus-additional-cni-plugins-h2qrx\" (UID: \"a09948e8-234f-41ef-866d-10ddb32e33fd\") " pod="openshift-multus/multus-additional-cni-plugins-h2qrx"
Jul 29 12:25:19.772847 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.771961    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1635829e-9a40-4521-8430-88adbb07a97a-dbus\") pod \"global-pull-secret-syncer-w97vv\" (UID: \"1635829e-9a40-4521-8430-88adbb07a97a\") " pod="kube-system/global-pull-secret-syncer-w97vv"
Jul 29 12:25:19.772847 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.771982    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1840c2b3-2714-4f4e-9255-2cb612abb53e-sys-fs\") pod \"aws-ebs-csi-driver-node-lzmrx\" (UID: \"1840c2b3-2714-4f4e-9255-2cb612abb53e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzmrx"
Jul 29 12:25:19.772847 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.772004    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4bdec0f6-396e-4392-9e8d-e2bad0289b8f-tmp-dir\") pod \"node-resolver-8rddq\" (UID: \"4bdec0f6-396e-4392-9e8d-e2bad0289b8f\") " pod="openshift-dns/node-resolver-8rddq"
Jul 29 12:25:19.772847 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.772026    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a09948e8-234f-41ef-866d-10ddb32e33fd-tuning-conf-dir\") pod \"multus-additional-cni-plugins-h2qrx\" (UID: \"a09948e8-234f-41ef-866d-10ddb32e33fd\") " pod="openshift-multus/multus-additional-cni-plugins-h2qrx"
Jul 29 12:25:19.772847 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.772052    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1840c2b3-2714-4f4e-9255-2cb612abb53e-registration-dir\") pod \"aws-ebs-csi-driver-node-lzmrx\" (UID: \"1840c2b3-2714-4f4e-9255-2cb612abb53e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzmrx"
Jul 29 12:25:19.772847 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.772087    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0f03f162-cb82-442c-8e40-b8877aa1996f-multus-daemon-config\") pod \"multus-ph59r\" (UID: \"0f03f162-cb82-442c-8e40-b8877aa1996f\") " pod="openshift-multus/multus-ph59r"
Jul 29 12:25:19.772847 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.772116    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/192debc3-6fd5-40d8-bca9-b58b50a97fa5-lib-modules\") pod \"tuned-z99ld\" (UID: \"192debc3-6fd5-40d8-bca9-b58b50a97fa5\") " pod="openshift-cluster-node-tuning-operator/tuned-z99ld"
Jul 29 12:25:19.772847 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.772137    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/caf950ed-dbb9-4b7d-a802-062882ff4899-host\") pod \"node-ca-clmgr\" (UID: \"caf950ed-dbb9-4b7d-a802-062882ff4899\") " pod="openshift-image-registry/node-ca-clmgr"
Jul 29 12:25:19.772847 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.772159    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/caf950ed-dbb9-4b7d-a802-062882ff4899-serviceca\") pod \"node-ca-clmgr\" (UID: \"caf950ed-dbb9-4b7d-a802-062882ff4899\") " pod="openshift-image-registry/node-ca-clmgr"
Jul 29 12:25:19.772847 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.772181    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0f03f162-cb82-442c-8e40-b8877aa1996f-multus-cni-dir\") pod \"multus-ph59r\" (UID: \"0f03f162-cb82-442c-8e40-b8877aa1996f\") " pod="openshift-multus/multus-ph59r"
Jul 29 12:25:19.772847 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.772202    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0f03f162-cb82-442c-8e40-b8877aa1996f-multus-socket-dir-parent\") pod \"multus-ph59r\" (UID: \"0f03f162-cb82-442c-8e40-b8877aa1996f\") " pod="openshift-multus/multus-ph59r"
Jul 29 12:25:19.772847 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.772225    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0f03f162-cb82-442c-8e40-b8877aa1996f-host-var-lib-cni-bin\") pod \"multus-ph59r\" (UID: \"0f03f162-cb82-442c-8e40-b8877aa1996f\") " pod="openshift-multus/multus-ph59r"
Jul 29 12:25:19.772847 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.772593    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 29 12:25:19.772847 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.772616    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 29 12:25:19.773560 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.772666    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 29 12:25:19.773560 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.772861    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 29 12:25:19.773560 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.773306    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-pc2w9\""
Jul 29 12:25:19.773560 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.773310    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 29 12:25:19.773560 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.773470    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 29 12:25:19.775873 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.775654    2567 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 29 12:25:19.808035 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.808019    2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-rc28n"
Jul 29 12:25:19.816094 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.816078    2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-rc28n"
Jul 29 12:25:19.861541 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.861519    2567 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 29 12:25:19.873243 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.873222    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/192debc3-6fd5-40d8-bca9-b58b50a97fa5-etc-sysconfig\") pod \"tuned-z99ld\" (UID: \"192debc3-6fd5-40d8-bca9-b58b50a97fa5\") " pod="openshift-cluster-node-tuning-operator/tuned-z99ld"
Jul 29 12:25:19.873342 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.873280    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/192debc3-6fd5-40d8-bca9-b58b50a97fa5-etc-systemd\") pod \"tuned-z99ld\" (UID: \"192debc3-6fd5-40d8-bca9-b58b50a97fa5\") " pod="openshift-cluster-node-tuning-operator/tuned-z99ld"
Jul 29 12:25:19.873342 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.873231    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/192debc3-6fd5-40d8-bca9-b58b50a97fa5-etc-sysconfig\") pod \"tuned-z99ld\" (UID: \"192debc3-6fd5-40d8-bca9-b58b50a97fa5\") " pod="openshift-cluster-node-tuning-operator/tuned-z99ld"
Jul 29 12:25:19.873342 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.873306    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0f03f162-cb82-442c-8e40-b8877aa1996f-etc-kubernetes\") pod \"multus-ph59r\" (UID: \"0f03f162-cb82-442c-8e40-b8877aa1996f\") " pod="openshift-multus/multus-ph59r"
Jul 29 12:25:19.873524 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.873371    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a6d5f3a2-26cc-40a7-9841-2fbfc247661a-run-systemd\") pod \"ovnkube-node-kj82f\" (UID: \"a6d5f3a2-26cc-40a7-9841-2fbfc247661a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kj82f"
Jul 29 12:25:19.873524 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.873384    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/192debc3-6fd5-40d8-bca9-b58b50a97fa5-etc-systemd\") pod \"tuned-z99ld\" (UID: \"192debc3-6fd5-40d8-bca9-b58b50a97fa5\") " pod="openshift-cluster-node-tuning-operator/tuned-z99ld"
Jul 29 12:25:19.873524 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.873426    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0f03f162-cb82-442c-8e40-b8877aa1996f-system-cni-dir\") pod \"multus-ph59r\" (UID: \"0f03f162-cb82-442c-8e40-b8877aa1996f\") " pod="openshift-multus/multus-ph59r"
Jul 29 12:25:19.873524 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.873456    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0f03f162-cb82-442c-8e40-b8877aa1996f-etc-kubernetes\") pod \"multus-ph59r\" (UID: \"0f03f162-cb82-442c-8e40-b8877aa1996f\") " pod="openshift-multus/multus-ph59r"
Jul 29 12:25:19.873524 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.873479    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0f03f162-cb82-442c-8e40-b8877aa1996f-system-cni-dir\") pod \"multus-ph59r\" (UID: \"0f03f162-cb82-442c-8e40-b8877aa1996f\") " pod="openshift-multus/multus-ph59r"
Jul 29 12:25:19.873524 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.873497    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0f03f162-cb82-442c-8e40-b8877aa1996f-os-release\") pod \"multus-ph59r\" (UID: \"0f03f162-cb82-442c-8e40-b8877aa1996f\") " pod="openshift-multus/multus-ph59r"
Jul 29 12:25:19.873792 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.873529    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0f03f162-cb82-442c-8e40-b8877aa1996f-os-release\") pod \"multus-ph59r\" (UID: \"0f03f162-cb82-442c-8e40-b8877aa1996f\") " pod="openshift-multus/multus-ph59r"
Jul 29 12:25:19.873792 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.873535    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdd82\" (UniqueName: \"kubernetes.io/projected/1840c2b3-2714-4f4e-9255-2cb612abb53e-kube-api-access-qdd82\") pod \"aws-ebs-csi-driver-node-lzmrx\" (UID: \"1840c2b3-2714-4f4e-9255-2cb612abb53e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzmrx"
Jul 29 12:25:19.873792 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.873571    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a6d5f3a2-26cc-40a7-9841-2fbfc247661a-etc-openvswitch\") pod \"ovnkube-node-kj82f\" (UID: \"a6d5f3a2-26cc-40a7-9841-2fbfc247661a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kj82f"
Jul 29 12:25:19.873792 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.873598    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a6d5f3a2-26cc-40a7-9841-2fbfc247661a-ovn-node-metrics-cert\") pod \"ovnkube-node-kj82f\" (UID: \"a6d5f3a2-26cc-40a7-9841-2fbfc247661a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kj82f"
Jul 29 12:25:19.873792 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.873625    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a09948e8-234f-41ef-866d-10ddb32e33fd-cni-binary-copy\") pod \"multus-additional-cni-plugins-h2qrx\" (UID: \"a09948e8-234f-41ef-866d-10ddb32e33fd\") " pod="openshift-multus/multus-additional-cni-plugins-h2qrx"
Jul 29 12:25:19.873792 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.873659    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1635829e-9a40-4521-8430-88adbb07a97a-dbus\") pod \"global-pull-secret-syncer-w97vv\" (UID: \"1635829e-9a40-4521-8430-88adbb07a97a\") " pod="kube-system/global-pull-secret-syncer-w97vv"
Jul 29 12:25:19.873792 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.873682    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1840c2b3-2714-4f4e-9255-2cb612abb53e-sys-fs\") pod \"aws-ebs-csi-driver-node-lzmrx\" (UID: \"1840c2b3-2714-4f4e-9255-2cb612abb53e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzmrx"
Jul 29 12:25:19.873792 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.873707    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a6d5f3a2-26cc-40a7-9841-2fbfc247661a-host-cni-netd\") pod \"ovnkube-node-kj82f\" (UID: \"a6d5f3a2-26cc-40a7-9841-2fbfc247661a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kj82f"
Jul 29 12:25:19.873792 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.873729    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4bdec0f6-396e-4392-9e8d-e2bad0289b8f-tmp-dir\") pod \"node-resolver-8rddq\" (UID: \"4bdec0f6-396e-4392-9e8d-e2bad0289b8f\") " pod="openshift-dns/node-resolver-8rddq"
Jul 29 12:25:19.873792 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.873766    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a09948e8-234f-41ef-866d-10ddb32e33fd-tuning-conf-dir\") pod \"multus-additional-cni-plugins-h2qrx\" (UID: \"a09948e8-234f-41ef-866d-10ddb32e33fd\") " pod="openshift-multus/multus-additional-cni-plugins-h2qrx"
Jul 29 12:25:19.873792 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.873788    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1840c2b3-2714-4f4e-9255-2cb612abb53e-registration-dir\") pod \"aws-ebs-csi-driver-node-lzmrx\" (UID: \"1840c2b3-2714-4f4e-9255-2cb612abb53e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzmrx"
Jul 29 12:25:19.874280 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.873812    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a6d5f3a2-26cc-40a7-9841-2fbfc247661a-run-openvswitch\") pod \"ovnkube-node-kj82f\" (UID: \"a6d5f3a2-26cc-40a7-9841-2fbfc247661a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kj82f"
Jul 29 12:25:19.874280 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.873846    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a6d5f3a2-26cc-40a7-9841-2fbfc247661a-run-ovn\") pod \"ovnkube-node-kj82f\" (UID: \"a6d5f3a2-26cc-40a7-9841-2fbfc247661a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kj82f"
Jul 29 12:25:19.874280 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.873878    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4038ffd5-00ac-4173-bd2a-4a8b3bb0ad5a-host-slash\") pod \"iptables-alerter-nkfzw\" (UID: \"4038ffd5-00ac-4173-bd2a-4a8b3bb0ad5a\") " pod="openshift-network-operator/iptables-alerter-nkfzw"
Jul 29 12:25:19.874280 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.873918    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0f03f162-cb82-442c-8e40-b8877aa1996f-multus-daemon-config\") pod \"multus-ph59r\" (UID: \"0f03f162-cb82-442c-8e40-b8877aa1996f\") " pod="openshift-multus/multus-ph59r"
Jul 29 12:25:19.874280 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.873933    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/192debc3-6fd5-40d8-bca9-b58b50a97fa5-lib-modules\") pod \"tuned-z99ld\" (UID: \"192debc3-6fd5-40d8-bca9-b58b50a97fa5\") " pod="openshift-cluster-node-tuning-operator/tuned-z99ld"
Jul 29 12:25:19.874280 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.873949    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/caf950ed-dbb9-4b7d-a802-062882ff4899-host\") pod \"node-ca-clmgr\" (UID: \"caf950ed-dbb9-4b7d-a802-062882ff4899\") " pod="openshift-image-registry/node-ca-clmgr"
Jul 29 12:25:19.874280 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.873962    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/caf950ed-dbb9-4b7d-a802-062882ff4899-serviceca\") pod \"node-ca-clmgr\" (UID: \"caf950ed-dbb9-4b7d-a802-062882ff4899\") " pod="openshift-image-registry/node-ca-clmgr"
Jul 29 12:25:19.874280 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.873982    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0f03f162-cb82-442c-8e40-b8877aa1996f-multus-cni-dir\") pod \"multus-ph59r\" (UID: \"0f03f162-cb82-442c-8e40-b8877aa1996f\") " pod="openshift-multus/multus-ph59r"
Jul 29 12:25:19.874280 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.874002    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0f03f162-cb82-442c-8e40-b8877aa1996f-multus-socket-dir-parent\") pod \"multus-ph59r\" (UID: \"0f03f162-cb82-442c-8e40-b8877aa1996f\") " pod="openshift-multus/multus-ph59r"
Jul 29 12:25:19.874280 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.874018    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0f03f162-cb82-442c-8e40-b8877aa1996f-host-var-lib-cni-bin\") pod \"multus-ph59r\" (UID: \"0f03f162-cb82-442c-8e40-b8877aa1996f\") " pod="openshift-multus/multus-ph59r"
Jul 29 12:25:19.874280 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.874032    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a6d5f3a2-26cc-40a7-9841-2fbfc247661a-systemd-units\") pod \"ovnkube-node-kj82f\" (UID: \"a6d5f3a2-26cc-40a7-9841-2fbfc247661a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kj82f"
Jul 29 12:25:19.874280 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.874051    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4bdec0f6-396e-4392-9e8d-e2bad0289b8f-tmp-dir\") pod \"node-resolver-8rddq\" (UID: \"4bdec0f6-396e-4392-9e8d-e2bad0289b8f\") " pod="openshift-dns/node-resolver-8rddq"
Jul 29 12:25:19.874280 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.874051    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1635829e-9a40-4521-8430-88adbb07a97a-original-pull-secret\") pod \"global-pull-secret-syncer-w97vv\" (UID: \"1635829e-9a40-4521-8430-88adbb07a97a\") " pod="kube-system/global-pull-secret-syncer-w97vv"
Jul 29 12:25:19.874280 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.874090    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/192debc3-6fd5-40d8-bca9-b58b50a97fa5-var-lib-kubelet\") pod \"tuned-z99ld\" (UID: \"192debc3-6fd5-40d8-bca9-b58b50a97fa5\") " pod="openshift-cluster-node-tuning-operator/tuned-z99ld"
Jul 29 12:25:19.874280 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.874110    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0f03f162-cb82-442c-8e40-b8877aa1996f-cnibin\") pod \"multus-ph59r\" (UID: \"0f03f162-cb82-442c-8e40-b8877aa1996f\") " pod="openshift-multus/multus-ph59r"
Jul 29 12:25:19.874280 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.874112    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a09948e8-234f-41ef-866d-10ddb32e33fd-tuning-conf-dir\") pod \"multus-additional-cni-plugins-h2qrx\" (UID: \"a09948e8-234f-41ef-866d-10ddb32e33fd\") " pod="openshift-multus/multus-additional-cni-plugins-h2qrx"
Jul 29 12:25:19.874280 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:19.874121    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 12:25:19.874280 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.874132    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0f03f162-cb82-442c-8e40-b8877aa1996f-multus-conf-dir\") pod \"multus-ph59r\" (UID: \"0f03f162-cb82-442c-8e40-b8877aa1996f\") " pod="openshift-multus/multus-ph59r"
Jul 29 12:25:19.875069 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.874155    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bz88c\" (UniqueName: \"kubernetes.io/projected/0f03f162-cb82-442c-8e40-b8877aa1996f-kube-api-access-bz88c\") pod \"multus-ph59r\" (UID: \"0f03f162-cb82-442c-8e40-b8877aa1996f\") " pod="openshift-multus/multus-ph59r"
Jul 29 12:25:19.875069 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:19.874181    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1635829e-9a40-4521-8430-88adbb07a97a-original-pull-secret podName:1635829e-9a40-4521-8430-88adbb07a97a nodeName:}" failed. No retries permitted until 2026-07-29 12:25:20.374162327 +0000 UTC m=+3.067850632 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1635829e-9a40-4521-8430-88adbb07a97a-original-pull-secret") pod "global-pull-secret-syncer-w97vv" (UID: "1635829e-9a40-4521-8430-88adbb07a97a") : object "kube-system"/"original-pull-secret" not registered
Jul 29 12:25:19.875069 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.874184    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a09948e8-234f-41ef-866d-10ddb32e33fd-cni-binary-copy\") pod \"multus-additional-cni-plugins-h2qrx\" (UID: \"a09948e8-234f-41ef-866d-10ddb32e33fd\") " pod="openshift-multus/multus-additional-cni-plugins-h2qrx"
Jul 29 12:25:19.875069 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.874201    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/192debc3-6fd5-40d8-bca9-b58b50a97fa5-var-lib-kubelet\") pod \"tuned-z99ld\" (UID: \"192debc3-6fd5-40d8-bca9-b58b50a97fa5\") " pod="openshift-cluster-node-tuning-operator/tuned-z99ld"
Jul 29 12:25:19.875069 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.874228    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a6d5f3a2-26cc-40a7-9841-2fbfc247661a-var-lib-openvswitch\") pod \"ovnkube-node-kj82f\" (UID: \"a6d5f3a2-26cc-40a7-9841-2fbfc247661a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kj82f"
Jul 29 12:25:19.875069 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.874256    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a6d5f3a2-26cc-40a7-9841-2fbfc247661a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kj82f\" (UID: \"a6d5f3a2-26cc-40a7-9841-2fbfc247661a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kj82f"
Jul 29 12:25:19.875069 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.874261    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1840c2b3-2714-4f4e-9255-2cb612abb53e-registration-dir\") pod \"aws-ebs-csi-driver-node-lzmrx\" (UID: \"1840c2b3-2714-4f4e-9255-2cb612abb53e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzmrx"
Jul 29 12:25:19.875069 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.874292    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1635829e-9a40-4521-8430-88adbb07a97a-dbus\") pod \"global-pull-secret-syncer-w97vv\" (UID: \"1635829e-9a40-4521-8430-88adbb07a97a\") " pod="kube-system/global-pull-secret-syncer-w97vv"
Jul 29 12:25:19.875069 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.874312    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/caf950ed-dbb9-4b7d-a802-062882ff4899-host\") pod \"node-ca-clmgr\" (UID: \"caf950ed-dbb9-4b7d-a802-062882ff4899\") " pod="openshift-image-registry/node-ca-clmgr"
Jul 29 12:25:19.875069 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.874296    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a6d5f3a2-26cc-40a7-9841-2fbfc247661a-ovnkube-script-lib\") pod \"ovnkube-node-kj82f\" (UID: \"a6d5f3a2-26cc-40a7-9841-2fbfc247661a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kj82f"
Jul 29 12:25:19.875069 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.874355    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0f03f162-cb82-442c-8e40-b8877aa1996f-host-var-lib-cni-bin\") pod \"multus-ph59r\" (UID: \"0f03f162-cb82-442c-8e40-b8877aa1996f\") " pod="openshift-multus/multus-ph59r"
Jul 29 12:25:19.875069 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.874365    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0f03f162-cb82-442c-8e40-b8877aa1996f-multus-cni-dir\") pod \"multus-ph59r\" (UID: \"0f03f162-cb82-442c-8e40-b8877aa1996f\") " pod="openshift-multus/multus-ph59r"
Jul 29 12:25:19.875069 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.874369    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1840c2b3-2714-4f4e-9255-2cb612abb53e-sys-fs\") pod \"aws-ebs-csi-driver-node-lzmrx\" (UID: \"1840c2b3-2714-4f4e-9255-2cb612abb53e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzmrx"
Jul 29 12:25:19.875069 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.874387    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a09948e8-234f-41ef-866d-10ddb32e33fd-os-release\") pod \"multus-additional-cni-plugins-h2qrx\" (UID: \"a09948e8-234f-41ef-866d-10ddb32e33fd\") " pod="openshift-multus/multus-additional-cni-plugins-h2qrx"
Jul 29 12:25:19.875069 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.874428    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4sr5\" (UniqueName: \"kubernetes.io/projected/a09948e8-234f-41ef-866d-10ddb32e33fd-kube-api-access-v4sr5\") pod \"multus-additional-cni-plugins-h2qrx\" (UID: \"a09948e8-234f-41ef-866d-10ddb32e33fd\") " pod="openshift-multus/multus-additional-cni-plugins-h2qrx"
Jul 29 12:25:19.875069 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.874450    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0f03f162-cb82-442c-8e40-b8877aa1996f-cnibin\") pod \"multus-ph59r\" (UID: \"0f03f162-cb82-442c-8e40-b8877aa1996f\") " pod="openshift-multus/multus-ph59r"
Jul 29 12:25:19.875800 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.874452    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/192debc3-6fd5-40d8-bca9-b58b50a97fa5-etc-sysctl-d\") pod \"tuned-z99ld\" (UID: \"192debc3-6fd5-40d8-bca9-b58b50a97fa5\") " pod="openshift-cluster-node-tuning-operator/tuned-z99ld"
Jul 29 12:25:19.875800 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.874489    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/192debc3-6fd5-40d8-bca9-b58b50a97fa5-etc-tuned\") pod \"tuned-z99ld\" (UID: \"192debc3-6fd5-40d8-bca9-b58b50a97fa5\") " pod="openshift-cluster-node-tuning-operator/tuned-z99ld"
Jul 29 12:25:19.875800 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.874512    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0f03f162-cb82-442c-8e40-b8877aa1996f-hostroot\") pod \"multus-ph59r\" (UID: \"0f03f162-cb82-442c-8e40-b8877aa1996f\") " pod="openshift-multus/multus-ph59r"
Jul 29 12:25:19.875800 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.874535    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1840c2b3-2714-4f4e-9255-2cb612abb53e-etc-selinux\") pod \"aws-ebs-csi-driver-node-lzmrx\" (UID: \"1840c2b3-2714-4f4e-9255-2cb612abb53e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzmrx"
Jul 29 12:25:19.875800 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.874543    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/192debc3-6fd5-40d8-bca9-b58b50a97fa5-etc-sysctl-d\") pod \"tuned-z99ld\" (UID: \"192debc3-6fd5-40d8-bca9-b58b50a97fa5\") " pod="openshift-cluster-node-tuning-operator/tuned-z99ld"
Jul 29 12:25:19.875800 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.874560    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnsg6\" (UniqueName: \"kubernetes.io/projected/8e7279c1-35b7-4f54-8b02-fdac281ef757-kube-api-access-fnsg6\") pod \"network-check-target-r4hjk\" (UID: \"8e7279c1-35b7-4f54-8b02-fdac281ef757\") " pod="openshift-network-diagnostics/network-check-target-r4hjk"
Jul 29 12:25:19.875800 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.874585    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfvj2\" (UniqueName: \"kubernetes.io/projected/4038ffd5-00ac-4173-bd2a-4a8b3bb0ad5a-kube-api-access-lfvj2\") pod \"iptables-alerter-nkfzw\" (UID: \"4038ffd5-00ac-4173-bd2a-4a8b3bb0ad5a\") " pod="openshift-network-operator/iptables-alerter-nkfzw"
Jul 29 12:25:19.875800 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.874604    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a09948e8-234f-41ef-866d-10ddb32e33fd-os-release\") pod \"multus-additional-cni-plugins-h2qrx\" (UID: \"a09948e8-234f-41ef-866d-10ddb32e33fd\") " pod="openshift-multus/multus-additional-cni-plugins-h2qrx"
Jul 29 12:25:19.875800 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.874611    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a09948e8-234f-41ef-866d-10ddb32e33fd-cnibin\") pod \"multus-additional-cni-plugins-h2qrx\" (UID: \"a09948e8-234f-41ef-866d-10ddb32e33fd\") " pod="openshift-multus/multus-additional-cni-plugins-h2qrx"
Jul 29 12:25:19.875800 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.874636    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1635829e-9a40-4521-8430-88adbb07a97a-kubelet-config\") pod \"global-pull-secret-syncer-w97vv\" (UID: \"1635829e-9a40-4521-8430-88adbb07a97a\") " pod="kube-system/global-pull-secret-syncer-w97vv"
Jul 29 12:25:19.875800 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.874660    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7be0cf2c-db65-486e-8545-eabd24ad8428-agent-certs\") pod \"konnectivity-agent-znk2m\" (UID: \"7be0cf2c-db65-486e-8545-eabd24ad8428\") " pod="kube-system/konnectivity-agent-znk2m"
Jul 29 12:25:19.875800 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.874681    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/192debc3-6fd5-40d8-bca9-b58b50a97fa5-sys\") pod \"tuned-z99ld\" (UID: \"192debc3-6fd5-40d8-bca9-b58b50a97fa5\") " pod="openshift-cluster-node-tuning-operator/tuned-z99ld"
Jul 29 12:25:19.875800 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.874704    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0f03f162-cb82-442c-8e40-b8877aa1996f-host-run-multus-certs\") pod \"multus-ph59r\" (UID: \"0f03f162-cb82-442c-8e40-b8877aa1996f\") " pod="openshift-multus/multus-ph59r"
Jul 29 12:25:19.875800 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.874705    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/caf950ed-dbb9-4b7d-a802-062882ff4899-serviceca\") pod \"node-ca-clmgr\" (UID: \"caf950ed-dbb9-4b7d-a802-062882ff4899\") " pod="openshift-image-registry/node-ca-clmgr"
Jul 29 12:25:19.875800 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.874742    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a6d5f3a2-26cc-40a7-9841-2fbfc247661a-host-kubelet\") pod \"ovnkube-node-kj82f\" (UID: \"a6d5f3a2-26cc-40a7-9841-2fbfc247661a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kj82f"
Jul 29 12:25:19.875800 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.874754    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a09948e8-234f-41ef-866d-10ddb32e33fd-cnibin\") pod \"multus-additional-cni-plugins-h2qrx\" (UID: \"a09948e8-234f-41ef-866d-10ddb32e33fd\") " pod="openshift-multus/multus-additional-cni-plugins-h2qrx"
Jul 29 12:25:19.875800 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.874772    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a6d5f3a2-26cc-40a7-9841-2fbfc247661a-host-run-ovn-kubernetes\") pod \"ovnkube-node-kj82f\" (UID: \"a6d5f3a2-26cc-40a7-9841-2fbfc247661a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kj82f"
Jul 29 12:25:19.876537 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.874796    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a6d5f3a2-26cc-40a7-9841-2fbfc247661a-ovnkube-config\") pod \"ovnkube-node-kj82f\" (UID: \"a6d5f3a2-26cc-40a7-9841-2fbfc247661a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kj82f"
Jul 29 12:25:19.876537 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.874798    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1635829e-9a40-4521-8430-88adbb07a97a-kubelet-config\") pod \"global-pull-secret-syncer-w97vv\" (UID: \"1635829e-9a40-4521-8430-88adbb07a97a\") " pod="kube-system/global-pull-secret-syncer-w97vv"
Jul 29 12:25:19.876537 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.874848    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/192debc3-6fd5-40d8-bca9-b58b50a97fa5-sys\") pod \"tuned-z99ld\" (UID: \"192debc3-6fd5-40d8-bca9-b58b50a97fa5\") " pod="openshift-cluster-node-tuning-operator/tuned-z99ld"
Jul 29 12:25:19.876537 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.874897    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0f03f162-cb82-442c-8e40-b8877aa1996f-multus-socket-dir-parent\") pod \"multus-ph59r\" (UID: \"0f03f162-cb82-442c-8e40-b8877aa1996f\") " pod="openshift-multus/multus-ph59r"
Jul 29 12:25:19.876537 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.874928    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0f03f162-cb82-442c-8e40-b8877aa1996f-multus-conf-dir\") pod \"multus-ph59r\" (UID: \"0f03f162-cb82-442c-8e40-b8877aa1996f\") " pod="openshift-multus/multus-ph59r"
Jul 29 12:25:19.876537 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.874939    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/192debc3-6fd5-40d8-bca9-b58b50a97fa5-lib-modules\") pod \"tuned-z99ld\" (UID: \"192debc3-6fd5-40d8-bca9-b58b50a97fa5\") " pod="openshift-cluster-node-tuning-operator/tuned-z99ld"
Jul 29 12:25:19.876537 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.874943    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0f03f162-cb82-442c-8e40-b8877aa1996f-host-run-multus-certs\") pod \"multus-ph59r\" (UID: \"0f03f162-cb82-442c-8e40-b8877aa1996f\") " pod="openshift-multus/multus-ph59r"
Jul 29 12:25:19.876537 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.874850    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0f03f162-cb82-442c-8e40-b8877aa1996f-multus-daemon-config\") pod \"multus-ph59r\" (UID: \"0f03f162-cb82-442c-8e40-b8877aa1996f\") " pod="openshift-multus/multus-ph59r"
Jul 29 12:25:19.876537 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.874972    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0f03f162-cb82-442c-8e40-b8877aa1996f-hostroot\") pod \"multus-ph59r\" (UID: \"0f03f162-cb82-442c-8e40-b8877aa1996f\") " pod="openshift-multus/multus-ph59r"
Jul 29 12:25:19.876537 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.874992    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/192debc3-6fd5-40d8-bca9-b58b50a97fa5-etc-sysctl-conf\") pod \"tuned-z99ld\" (UID: \"192debc3-6fd5-40d8-bca9-b58b50a97fa5\") " pod="openshift-cluster-node-tuning-operator/tuned-z99ld"
Jul 29 12:25:19.876537 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.875016    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0f03f162-cb82-442c-8e40-b8877aa1996f-host-var-lib-cni-multus\") pod \"multus-ph59r\" (UID: \"0f03f162-cb82-442c-8e40-b8877aa1996f\") " pod="openshift-multus/multus-ph59r"
Jul 29 12:25:19.876537 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.875032    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1840c2b3-2714-4f4e-9255-2cb612abb53e-device-dir\") pod \"aws-ebs-csi-driver-node-lzmrx\" (UID: \"1840c2b3-2714-4f4e-9255-2cb612abb53e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzmrx"
Jul 29 12:25:19.876537 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.875036    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1840c2b3-2714-4f4e-9255-2cb612abb53e-etc-selinux\") pod \"aws-ebs-csi-driver-node-lzmrx\" (UID: \"1840c2b3-2714-4f4e-9255-2cb612abb53e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzmrx"
Jul 29 12:25:19.876537 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.875047    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a6d5f3a2-26cc-40a7-9841-2fbfc247661a-log-socket\") pod \"ovnkube-node-kj82f\" (UID: \"a6d5f3a2-26cc-40a7-9841-2fbfc247661a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kj82f"
Jul 29 12:25:19.876537 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.875069    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a09948e8-234f-41ef-866d-10ddb32e33fd-system-cni-dir\") pod \"multus-additional-cni-plugins-h2qrx\" (UID: \"a09948e8-234f-41ef-866d-10ddb32e33fd\") " pod="openshift-multus/multus-additional-cni-plugins-h2qrx"
Jul 29 12:25:19.876537 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.875072    2567 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 29 12:25:19.876537 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.875091    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0f03f162-cb82-442c-8e40-b8877aa1996f-host-var-lib-cni-multus\") pod \"multus-ph59r\" (UID: \"0f03f162-cb82-442c-8e40-b8877aa1996f\") " pod="openshift-multus/multus-ph59r"
Jul 29 12:25:19.876537 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.875100    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1840c2b3-2714-4f4e-9255-2cb612abb53e-device-dir\") pod \"aws-ebs-csi-driver-node-lzmrx\" (UID: \"1840c2b3-2714-4f4e-9255-2cb612abb53e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzmrx"
Jul 29 12:25:19.877189 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.875093    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a09948e8-234f-41ef-866d-10ddb32e33fd-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-h2qrx\" (UID: \"a09948e8-234f-41ef-866d-10ddb32e33fd\") " pod="openshift-multus/multus-additional-cni-plugins-h2qrx"
Jul 29 12:25:19.877189 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.875140    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/192debc3-6fd5-40d8-bca9-b58b50a97fa5-etc-sysctl-conf\") pod \"tuned-z99ld\" (UID: \"192debc3-6fd5-40d8-bca9-b58b50a97fa5\") " pod="openshift-cluster-node-tuning-operator/tuned-z99ld"
Jul 29 12:25:19.877189 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.875141    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a09948e8-234f-41ef-866d-10ddb32e33fd-system-cni-dir\") pod \"multus-additional-cni-plugins-h2qrx\" (UID: \"a09948e8-234f-41ef-866d-10ddb32e33fd\") " pod="openshift-multus/multus-additional-cni-plugins-h2qrx"
Jul 29 12:25:19.877189 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.875141    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0f03f162-cb82-442c-8e40-b8877aa1996f-host-run-netns\") pod \"multus-ph59r\" (UID: \"0f03f162-cb82-442c-8e40-b8877aa1996f\") " pod="openshift-multus/multus-ph59r"
Jul 29 12:25:19.877189 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.875168    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0f03f162-cb82-442c-8e40-b8877aa1996f-host-run-netns\") pod \"multus-ph59r\" (UID: \"0f03f162-cb82-442c-8e40-b8877aa1996f\") " pod="openshift-multus/multus-ph59r"
Jul 29 12:25:19.877189 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.875184    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0f03f162-cb82-442c-8e40-b8877aa1996f-host-var-lib-kubelet\") pod \"multus-ph59r\" (UID: \"0f03f162-cb82-442c-8e40-b8877aa1996f\") " pod="openshift-multus/multus-ph59r"
Jul 29 12:25:19.877189 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.875208    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a6d5f3a2-26cc-40a7-9841-2fbfc247661a-host-slash\") pod \"ovnkube-node-kj82f\" (UID: \"a6d5f3a2-26cc-40a7-9841-2fbfc247661a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kj82f"
Jul 29 12:25:19.877189 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.875230    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a6d5f3a2-26cc-40a7-9841-2fbfc247661a-host-cni-bin\") pod \"ovnkube-node-kj82f\" (UID: \"a6d5f3a2-26cc-40a7-9841-2fbfc247661a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kj82f"
Jul 29 12:25:19.877189 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.875253    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4038ffd5-00ac-4173-bd2a-4a8b3bb0ad5a-iptables-alerter-script\") pod \"iptables-alerter-nkfzw\" (UID: \"4038ffd5-00ac-4173-bd2a-4a8b3bb0ad5a\") " pod="openshift-network-operator/iptables-alerter-nkfzw"
Jul 29 12:25:19.877189 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.875274    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0f03f162-cb82-442c-8e40-b8877aa1996f-host-var-lib-kubelet\") pod \"multus-ph59r\" (UID: \"0f03f162-cb82-442c-8e40-b8877aa1996f\") " pod="openshift-multus/multus-ph59r"
Jul 29 12:25:19.877189 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.875276    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/192debc3-6fd5-40d8-bca9-b58b50a97fa5-etc-modprobe-d\") pod \"tuned-z99ld\" (UID: \"192debc3-6fd5-40d8-bca9-b58b50a97fa5\") " pod="openshift-cluster-node-tuning-operator/tuned-z99ld"
Jul 29 12:25:19.877189 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.875327    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/192debc3-6fd5-40d8-bca9-b58b50a97fa5-run\") pod \"tuned-z99ld\" (UID: \"192debc3-6fd5-40d8-bca9-b58b50a97fa5\") " pod="openshift-cluster-node-tuning-operator/tuned-z99ld"
Jul 29 12:25:19.877189 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.875341    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/192debc3-6fd5-40d8-bca9-b58b50a97fa5-etc-modprobe-d\") pod \"tuned-z99ld\" (UID: \"192debc3-6fd5-40d8-bca9-b58b50a97fa5\") " pod="openshift-cluster-node-tuning-operator/tuned-z99ld"
Jul 29 12:25:19.877189 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.875351    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbhws\" (UniqueName: \"kubernetes.io/projected/a6d5f3a2-26cc-40a7-9841-2fbfc247661a-kube-api-access-fbhws\") pod \"ovnkube-node-kj82f\" (UID: \"a6d5f3a2-26cc-40a7-9841-2fbfc247661a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kj82f"
Jul 29 12:25:19.877189 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.875373    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1840c2b3-2714-4f4e-9255-2cb612abb53e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lzmrx\" (UID: \"1840c2b3-2714-4f4e-9255-2cb612abb53e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzmrx"
Jul 29 12:25:19.877189 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.875388    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/192debc3-6fd5-40d8-bca9-b58b50a97fa5-run\") pod \"tuned-z99ld\" (UID: \"192debc3-6fd5-40d8-bca9-b58b50a97fa5\") " pod="openshift-cluster-node-tuning-operator/tuned-z99ld"
Jul 29 12:25:19.877189 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.875392    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4bdec0f6-396e-4392-9e8d-e2bad0289b8f-hosts-file\") pod \"node-resolver-8rddq\" (UID: \"4bdec0f6-396e-4392-9e8d-e2bad0289b8f\") " pod="openshift-dns/node-resolver-8rddq"
Jul 29 12:25:19.877777 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.875440    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68d50be9-1461-43d4-811e-74ff3fed1a75-metrics-certs\") pod \"network-metrics-daemon-qkdl7\" (UID: \"68d50be9-1461-43d4-811e-74ff3fed1a75\") " pod="openshift-multus/network-metrics-daemon-qkdl7"
Jul 29 12:25:19.877777 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.875461    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/192debc3-6fd5-40d8-bca9-b58b50a97fa5-tmp\") pod \"tuned-z99ld\" (UID: \"192debc3-6fd5-40d8-bca9-b58b50a97fa5\") " pod="openshift-cluster-node-tuning-operator/tuned-z99ld"
Jul 29 12:25:19.877777 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.875469    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1840c2b3-2714-4f4e-9255-2cb612abb53e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lzmrx\" (UID: \"1840c2b3-2714-4f4e-9255-2cb612abb53e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzmrx"
Jul 29 12:25:19.877777 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.875441    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4bdec0f6-396e-4392-9e8d-e2bad0289b8f-hosts-file\") pod \"node-resolver-8rddq\" (UID: \"4bdec0f6-396e-4392-9e8d-e2bad0289b8f\") " pod="openshift-dns/node-resolver-8rddq"
Jul 29 12:25:19.877777 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.875480    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7be0cf2c-db65-486e-8545-eabd24ad8428-konnectivity-ca\") pod \"konnectivity-agent-znk2m\" (UID: \"7be0cf2c-db65-486e-8545-eabd24ad8428\") " pod="kube-system/konnectivity-agent-znk2m"
Jul 29 12:25:19.877777 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.875502    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0f03f162-cb82-442c-8e40-b8877aa1996f-cni-binary-copy\") pod \"multus-ph59r\" (UID: \"0f03f162-cb82-442c-8e40-b8877aa1996f\") " pod="openshift-multus/multus-ph59r"
Jul 29 12:25:19.877777 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.875522    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a6d5f3a2-26cc-40a7-9841-2fbfc247661a-node-log\") pod \"ovnkube-node-kj82f\" (UID: \"a6d5f3a2-26cc-40a7-9841-2fbfc247661a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kj82f"
Jul 29 12:25:19.877777 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.875540    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a6d5f3a2-26cc-40a7-9841-2fbfc247661a-env-overrides\") pod \"ovnkube-node-kj82f\" (UID: \"a6d5f3a2-26cc-40a7-9841-2fbfc247661a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kj82f"
Jul 29 12:25:19.877777 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.875545    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a09948e8-234f-41ef-866d-10ddb32e33fd-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-h2qrx\" (UID: \"a09948e8-234f-41ef-866d-10ddb32e33fd\") " pod="openshift-multus/multus-additional-cni-plugins-h2qrx"
Jul 29 12:25:19.877777 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.875559    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjjwl\" (UniqueName: \"kubernetes.io/projected/4bdec0f6-396e-4392-9e8d-e2bad0289b8f-kube-api-access-bjjwl\") pod \"node-resolver-8rddq\" (UID: \"4bdec0f6-396e-4392-9e8d-e2bad0289b8f\") " pod="openshift-dns/node-resolver-8rddq"
Jul 29 12:25:19.877777 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.875592    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fr22\" (UniqueName: \"kubernetes.io/projected/68d50be9-1461-43d4-811e-74ff3fed1a75-kube-api-access-9fr22\") pod \"network-metrics-daemon-qkdl7\" (UID: \"68d50be9-1461-43d4-811e-74ff3fed1a75\") " pod="openshift-multus/network-metrics-daemon-qkdl7"
Jul 29 12:25:19.877777 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.875620    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a09948e8-234f-41ef-866d-10ddb32e33fd-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-h2qrx\" (UID: \"a09948e8-234f-41ef-866d-10ddb32e33fd\") " pod="openshift-multus/multus-additional-cni-plugins-h2qrx"
Jul 29 12:25:19.877777 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.875643    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/192debc3-6fd5-40d8-bca9-b58b50a97fa5-etc-kubernetes\") pod \"tuned-z99ld\" (UID: \"192debc3-6fd5-40d8-bca9-b58b50a97fa5\") " pod="openshift-cluster-node-tuning-operator/tuned-z99ld"
Jul 29 12:25:19.877777 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.875667    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/192debc3-6fd5-40d8-bca9-b58b50a97fa5-host\") pod \"tuned-z99ld\" (UID: \"192debc3-6fd5-40d8-bca9-b58b50a97fa5\") " pod="openshift-cluster-node-tuning-operator/tuned-z99ld"
Jul 29 12:25:19.877777 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.875689    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6xrh\" (UniqueName: \"kubernetes.io/projected/192debc3-6fd5-40d8-bca9-b58b50a97fa5-kube-api-access-v6xrh\") pod \"tuned-z99ld\" (UID: \"192debc3-6fd5-40d8-bca9-b58b50a97fa5\") " pod="openshift-cluster-node-tuning-operator/tuned-z99ld"
Jul 29 12:25:19.877777 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.875712    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glc5q\" (UniqueName: \"kubernetes.io/projected/caf950ed-dbb9-4b7d-a802-062882ff4899-kube-api-access-glc5q\") pod \"node-ca-clmgr\" (UID: \"caf950ed-dbb9-4b7d-a802-062882ff4899\") " pod="openshift-image-registry/node-ca-clmgr"
Jul 29 12:25:19.877777 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.875736    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0f03f162-cb82-442c-8e40-b8877aa1996f-host-run-k8s-cni-cncf-io\") pod \"multus-ph59r\" (UID: \"0f03f162-cb82-442c-8e40-b8877aa1996f\") " pod="openshift-multus/multus-ph59r"
Jul 29 12:25:19.877777 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:19.875757    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 12:25:19.878373 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.875761    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1840c2b3-2714-4f4e-9255-2cb612abb53e-socket-dir\") pod \"aws-ebs-csi-driver-node-lzmrx\" (UID: \"1840c2b3-2714-4f4e-9255-2cb612abb53e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzmrx"
Jul 29 12:25:19.878373 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.875786    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a6d5f3a2-26cc-40a7-9841-2fbfc247661a-host-run-netns\") pod \"ovnkube-node-kj82f\" (UID: \"a6d5f3a2-26cc-40a7-9841-2fbfc247661a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kj82f"
Jul 29 12:25:19.878373 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:19.875795    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68d50be9-1461-43d4-811e-74ff3fed1a75-metrics-certs podName:68d50be9-1461-43d4-811e-74ff3fed1a75 nodeName:}" failed. No retries permitted until 2026-07-29 12:25:20.375781893 +0000 UTC m=+3.069470182 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/68d50be9-1461-43d4-811e-74ff3fed1a75-metrics-certs") pod "network-metrics-daemon-qkdl7" (UID: "68d50be9-1461-43d4-811e-74ff3fed1a75") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 12:25:19.878373 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.875867    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7be0cf2c-db65-486e-8545-eabd24ad8428-konnectivity-ca\") pod \"konnectivity-agent-znk2m\" (UID: \"7be0cf2c-db65-486e-8545-eabd24ad8428\") " pod="kube-system/konnectivity-agent-znk2m"
Jul 29 12:25:19.878373 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.876201    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0f03f162-cb82-442c-8e40-b8877aa1996f-cni-binary-copy\") pod \"multus-ph59r\" (UID: \"0f03f162-cb82-442c-8e40-b8877aa1996f\") " pod="openshift-multus/multus-ph59r"
Jul 29 12:25:19.878373 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.876268    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0f03f162-cb82-442c-8e40-b8877aa1996f-host-run-k8s-cni-cncf-io\") pod \"multus-ph59r\" (UID: \"0f03f162-cb82-442c-8e40-b8877aa1996f\") " pod="openshift-multus/multus-ph59r"
Jul 29 12:25:19.878373 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.876269    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/192debc3-6fd5-40d8-bca9-b58b50a97fa5-host\") pod \"tuned-z99ld\" (UID: \"192debc3-6fd5-40d8-bca9-b58b50a97fa5\") " pod="openshift-cluster-node-tuning-operator/tuned-z99ld"
Jul 29 12:25:19.878373 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.876308    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/192debc3-6fd5-40d8-bca9-b58b50a97fa5-etc-kubernetes\") pod \"tuned-z99ld\" (UID: \"192debc3-6fd5-40d8-bca9-b58b50a97fa5\") " pod="openshift-cluster-node-tuning-operator/tuned-z99ld"
Jul 29 12:25:19.878373 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.876434    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1840c2b3-2714-4f4e-9255-2cb612abb53e-socket-dir\") pod \"aws-ebs-csi-driver-node-lzmrx\" (UID: \"1840c2b3-2714-4f4e-9255-2cb612abb53e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzmrx"
Jul 29 12:25:19.878373 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.876459    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a09948e8-234f-41ef-866d-10ddb32e33fd-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-h2qrx\" (UID: \"a09948e8-234f-41ef-866d-10ddb32e33fd\") " pod="openshift-multus/multus-additional-cni-plugins-h2qrx"
Jul 29 12:25:19.878743 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.878498    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7be0cf2c-db65-486e-8545-eabd24ad8428-agent-certs\") pod \"konnectivity-agent-znk2m\" (UID: \"7be0cf2c-db65-486e-8545-eabd24ad8428\") " pod="kube-system/konnectivity-agent-znk2m"
Jul 29 12:25:19.878795 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.878737    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/192debc3-6fd5-40d8-bca9-b58b50a97fa5-etc-tuned\") pod \"tuned-z99ld\" (UID: \"192debc3-6fd5-40d8-bca9-b58b50a97fa5\") " pod="openshift-cluster-node-tuning-operator/tuned-z99ld"
Jul 29 12:25:19.878795 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.878769    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/192debc3-6fd5-40d8-bca9-b58b50a97fa5-tmp\") pod \"tuned-z99ld\" (UID: \"192debc3-6fd5-40d8-bca9-b58b50a97fa5\") " pod="openshift-cluster-node-tuning-operator/tuned-z99ld"
Jul 29 12:25:19.887034 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.887011    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdd82\" (UniqueName: \"kubernetes.io/projected/1840c2b3-2714-4f4e-9255-2cb612abb53e-kube-api-access-qdd82\") pod \"aws-ebs-csi-driver-node-lzmrx\" (UID: \"1840c2b3-2714-4f4e-9255-2cb612abb53e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzmrx"
Jul 29 12:25:19.890155 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.890131    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4sr5\" (UniqueName: \"kubernetes.io/projected/a09948e8-234f-41ef-866d-10ddb32e33fd-kube-api-access-v4sr5\") pod \"multus-additional-cni-plugins-h2qrx\" (UID: \"a09948e8-234f-41ef-866d-10ddb32e33fd\") " pod="openshift-multus/multus-additional-cni-plugins-h2qrx"
Jul 29 12:25:19.890714 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.890690    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6xrh\" (UniqueName: \"kubernetes.io/projected/192debc3-6fd5-40d8-bca9-b58b50a97fa5-kube-api-access-v6xrh\") pod \"tuned-z99ld\" (UID: \"192debc3-6fd5-40d8-bca9-b58b50a97fa5\") " pod="openshift-cluster-node-tuning-operator/tuned-z99ld"
Jul 29 12:25:19.891286 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.891230    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bz88c\" (UniqueName: \"kubernetes.io/projected/0f03f162-cb82-442c-8e40-b8877aa1996f-kube-api-access-bz88c\") pod \"multus-ph59r\" (UID: \"0f03f162-cb82-442c-8e40-b8877aa1996f\") " pod="openshift-multus/multus-ph59r"
Jul 29 12:25:19.891391 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.891362    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-glc5q\" (UniqueName: \"kubernetes.io/projected/caf950ed-dbb9-4b7d-a802-062882ff4899-kube-api-access-glc5q\") pod \"node-ca-clmgr\" (UID: \"caf950ed-dbb9-4b7d-a802-062882ff4899\") " pod="openshift-image-registry/node-ca-clmgr"
Jul 29 12:25:19.891756 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.891729    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fr22\" (UniqueName: \"kubernetes.io/projected/68d50be9-1461-43d4-811e-74ff3fed1a75-kube-api-access-9fr22\") pod \"network-metrics-daemon-qkdl7\" (UID: \"68d50be9-1461-43d4-811e-74ff3fed1a75\") " pod="openshift-multus/network-metrics-daemon-qkdl7"
Jul 29 12:25:19.892700 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.892657    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjjwl\" (UniqueName: \"kubernetes.io/projected/4bdec0f6-396e-4392-9e8d-e2bad0289b8f-kube-api-access-bjjwl\") pod \"node-resolver-8rddq\" (UID: \"4bdec0f6-396e-4392-9e8d-e2bad0289b8f\") " pod="openshift-dns/node-resolver-8rddq"
Jul 29 12:25:19.918447 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.918378    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-25.ec2.internal" event={"ID":"875ad84743ea631b485f1bbadf831c17","Type":"ContainerStarted","Data":"8acc388b3e1b871337a203fea8fb20b3deab333db39517ba33dc800b4f00d32c"}
Jul 29 12:25:19.919215 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.919198    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-25.ec2.internal" event={"ID":"eee88d5337a9ab304c1e57b0fd78f1d9","Type":"ContainerStarted","Data":"9aab815f999dc9029b5748c672bb636f569a51bf2330d5bf480694e301567e36"}
Jul 29 12:25:19.976165 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.976141    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a6d5f3a2-26cc-40a7-9841-2fbfc247661a-host-slash\") pod \"ovnkube-node-kj82f\" (UID: \"a6d5f3a2-26cc-40a7-9841-2fbfc247661a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kj82f"
Jul 29 12:25:19.976280 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.976169    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a6d5f3a2-26cc-40a7-9841-2fbfc247661a-host-cni-bin\") pod \"ovnkube-node-kj82f\" (UID: \"a6d5f3a2-26cc-40a7-9841-2fbfc247661a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kj82f"
Jul 29 12:25:19.976280 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.976185    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4038ffd5-00ac-4173-bd2a-4a8b3bb0ad5a-iptables-alerter-script\") pod \"iptables-alerter-nkfzw\" (UID: \"4038ffd5-00ac-4173-bd2a-4a8b3bb0ad5a\") " pod="openshift-network-operator/iptables-alerter-nkfzw"
Jul 29 12:25:19.976280 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.976206    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbhws\" (UniqueName: \"kubernetes.io/projected/a6d5f3a2-26cc-40a7-9841-2fbfc247661a-kube-api-access-fbhws\") pod \"ovnkube-node-kj82f\" (UID: \"a6d5f3a2-26cc-40a7-9841-2fbfc247661a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kj82f"
Jul 29 12:25:19.976280 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.976242    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a6d5f3a2-26cc-40a7-9841-2fbfc247661a-node-log\") pod \"ovnkube-node-kj82f\" (UID: \"a6d5f3a2-26cc-40a7-9841-2fbfc247661a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kj82f"
Jul 29 12:25:19.976280 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.976259    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a6d5f3a2-26cc-40a7-9841-2fbfc247661a-host-cni-bin\") pod \"ovnkube-node-kj82f\" (UID: \"a6d5f3a2-26cc-40a7-9841-2fbfc247661a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kj82f"
Jul 29 12:25:19.976495 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.976264    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a6d5f3a2-26cc-40a7-9841-2fbfc247661a-env-overrides\") pod \"ovnkube-node-kj82f\" (UID: \"a6d5f3a2-26cc-40a7-9841-2fbfc247661a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kj82f"
Jul 29 12:25:19.976495 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.976294    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a6d5f3a2-26cc-40a7-9841-2fbfc247661a-node-log\") pod \"ovnkube-node-kj82f\" (UID: \"a6d5f3a2-26cc-40a7-9841-2fbfc247661a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kj82f"
Jul 29 12:25:19.976495 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.976242    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a6d5f3a2-26cc-40a7-9841-2fbfc247661a-host-slash\") pod \"ovnkube-node-kj82f\" (UID: \"a6d5f3a2-26cc-40a7-9841-2fbfc247661a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kj82f"
Jul 29 12:25:19.976495 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.976325    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a6d5f3a2-26cc-40a7-9841-2fbfc247661a-host-run-netns\") pod \"ovnkube-node-kj82f\" (UID: \"a6d5f3a2-26cc-40a7-9841-2fbfc247661a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kj82f"
Jul 29 12:25:19.976495 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.976351    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a6d5f3a2-26cc-40a7-9841-2fbfc247661a-run-systemd\") pod \"ovnkube-node-kj82f\" (UID: \"a6d5f3a2-26cc-40a7-9841-2fbfc247661a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kj82f"
Jul 29 12:25:19.976495 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.976377    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a6d5f3a2-26cc-40a7-9841-2fbfc247661a-etc-openvswitch\") pod \"ovnkube-node-kj82f\" (UID: \"a6d5f3a2-26cc-40a7-9841-2fbfc247661a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kj82f"
Jul 29 12:25:19.976495 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.976416    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a6d5f3a2-26cc-40a7-9841-2fbfc247661a-ovn-node-metrics-cert\") pod \"ovnkube-node-kj82f\" (UID: \"a6d5f3a2-26cc-40a7-9841-2fbfc247661a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kj82f"
Jul 29 12:25:19.976495 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.976432    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a6d5f3a2-26cc-40a7-9841-2fbfc247661a-host-run-netns\") pod \"ovnkube-node-kj82f\" (UID: \"a6d5f3a2-26cc-40a7-9841-2fbfc247661a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kj82f"
Jul 29 12:25:19.976495 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.976443    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a6d5f3a2-26cc-40a7-9841-2fbfc247661a-host-cni-netd\") pod \"ovnkube-node-kj82f\" (UID: \"a6d5f3a2-26cc-40a7-9841-2fbfc247661a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kj82f"
Jul 29 12:25:19.976495 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.976470    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a6d5f3a2-26cc-40a7-9841-2fbfc247661a-run-openvswitch\") pod \"ovnkube-node-kj82f\" (UID: \"a6d5f3a2-26cc-40a7-9841-2fbfc247661a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kj82f"
Jul 29 12:25:19.976495 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.976475    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a6d5f3a2-26cc-40a7-9841-2fbfc247661a-etc-openvswitch\") pod \"ovnkube-node-kj82f\" (UID: \"a6d5f3a2-26cc-40a7-9841-2fbfc247661a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kj82f"
Jul 29 12:25:19.976900 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.976512    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a6d5f3a2-26cc-40a7-9841-2fbfc247661a-run-openvswitch\") pod \"ovnkube-node-kj82f\" (UID: \"a6d5f3a2-26cc-40a7-9841-2fbfc247661a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kj82f"
Jul 29 12:25:19.976900 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.976545    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a6d5f3a2-26cc-40a7-9841-2fbfc247661a-run-systemd\") pod \"ovnkube-node-kj82f\" (UID: \"a6d5f3a2-26cc-40a7-9841-2fbfc247661a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kj82f"
Jul 29 12:25:19.976900 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.976598    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a6d5f3a2-26cc-40a7-9841-2fbfc247661a-run-ovn\") pod \"ovnkube-node-kj82f\" (UID: \"a6d5f3a2-26cc-40a7-9841-2fbfc247661a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kj82f"
Jul 29 12:25:19.976900 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.976625    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4038ffd5-00ac-4173-bd2a-4a8b3bb0ad5a-host-slash\") pod \"iptables-alerter-nkfzw\" (UID: \"4038ffd5-00ac-4173-bd2a-4a8b3bb0ad5a\") " pod="openshift-network-operator/iptables-alerter-nkfzw"
Jul 29 12:25:19.976900 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.976661    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a6d5f3a2-26cc-40a7-9841-2fbfc247661a-systemd-units\") pod \"ovnkube-node-kj82f\" (UID: \"a6d5f3a2-26cc-40a7-9841-2fbfc247661a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kj82f"
Jul 29 12:25:19.976900 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.976717    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a6d5f3a2-26cc-40a7-9841-2fbfc247661a-var-lib-openvswitch\") pod \"ovnkube-node-kj82f\" (UID: \"a6d5f3a2-26cc-40a7-9841-2fbfc247661a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kj82f"
Jul 29 12:25:19.976900 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.976742    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a6d5f3a2-26cc-40a7-9841-2fbfc247661a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kj82f\" (UID: \"a6d5f3a2-26cc-40a7-9841-2fbfc247661a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kj82f"
Jul 29 12:25:19.976900 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.976767    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a6d5f3a2-26cc-40a7-9841-2fbfc247661a-ovnkube-script-lib\") pod \"ovnkube-node-kj82f\" (UID: \"a6d5f3a2-26cc-40a7-9841-2fbfc247661a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kj82f"
Jul 29 12:25:19.976900 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.976798    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnsg6\" (UniqueName: \"kubernetes.io/projected/8e7279c1-35b7-4f54-8b02-fdac281ef757-kube-api-access-fnsg6\") pod \"network-check-target-r4hjk\" (UID: \"8e7279c1-35b7-4f54-8b02-fdac281ef757\") " pod="openshift-network-diagnostics/network-check-target-r4hjk"
Jul 29 12:25:19.976900 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.976822    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfvj2\" (UniqueName: \"kubernetes.io/projected/4038ffd5-00ac-4173-bd2a-4a8b3bb0ad5a-kube-api-access-lfvj2\") pod \"iptables-alerter-nkfzw\" (UID: \"4038ffd5-00ac-4173-bd2a-4a8b3bb0ad5a\") " pod="openshift-network-operator/iptables-alerter-nkfzw"
Jul 29 12:25:19.976900 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.976820    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4038ffd5-00ac-4173-bd2a-4a8b3bb0ad5a-iptables-alerter-script\") pod \"iptables-alerter-nkfzw\" (UID: \"4038ffd5-00ac-4173-bd2a-4a8b3bb0ad5a\") " pod="openshift-network-operator/iptables-alerter-nkfzw"
Jul 29 12:25:19.976900 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.976841    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a6d5f3a2-26cc-40a7-9841-2fbfc247661a-host-cni-netd\") pod \"ovnkube-node-kj82f\" (UID: \"a6d5f3a2-26cc-40a7-9841-2fbfc247661a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kj82f"
Jul 29 12:25:19.976900 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.976873    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a6d5f3a2-26cc-40a7-9841-2fbfc247661a-var-lib-openvswitch\") pod \"ovnkube-node-kj82f\" (UID: \"a6d5f3a2-26cc-40a7-9841-2fbfc247661a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kj82f"
Jul 29 12:25:19.976900 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.976880    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a6d5f3a2-26cc-40a7-9841-2fbfc247661a-env-overrides\") pod \"ovnkube-node-kj82f\" (UID: \"a6d5f3a2-26cc-40a7-9841-2fbfc247661a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kj82f"
Jul 29 12:25:19.976900 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.976903    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4038ffd5-00ac-4173-bd2a-4a8b3bb0ad5a-host-slash\") pod \"iptables-alerter-nkfzw\" (UID: \"4038ffd5-00ac-4173-bd2a-4a8b3bb0ad5a\") " pod="openshift-network-operator/iptables-alerter-nkfzw"
Jul 29 12:25:19.977420 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.976902    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a6d5f3a2-26cc-40a7-9841-2fbfc247661a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kj82f\" (UID: \"a6d5f3a2-26cc-40a7-9841-2fbfc247661a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kj82f"
Jul 29 12:25:19.977420 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.976923    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a6d5f3a2-26cc-40a7-9841-2fbfc247661a-run-ovn\") pod \"ovnkube-node-kj82f\" (UID: \"a6d5f3a2-26cc-40a7-9841-2fbfc247661a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kj82f"
Jul 29 12:25:19.977420 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.976945    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a6d5f3a2-26cc-40a7-9841-2fbfc247661a-systemd-units\") pod \"ovnkube-node-kj82f\" (UID: \"a6d5f3a2-26cc-40a7-9841-2fbfc247661a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kj82f"
Jul 29 12:25:19.977420 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.976953    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a6d5f3a2-26cc-40a7-9841-2fbfc247661a-host-kubelet\") pod \"ovnkube-node-kj82f\" (UID: \"a6d5f3a2-26cc-40a7-9841-2fbfc247661a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kj82f"
Jul 29 12:25:19.977420 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.976970    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a6d5f3a2-26cc-40a7-9841-2fbfc247661a-host-run-ovn-kubernetes\") pod \"ovnkube-node-kj82f\" (UID: \"a6d5f3a2-26cc-40a7-9841-2fbfc247661a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kj82f"
Jul 29 12:25:19.977420 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.976985    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a6d5f3a2-26cc-40a7-9841-2fbfc247661a-ovnkube-config\") pod \"ovnkube-node-kj82f\" (UID: \"a6d5f3a2-26cc-40a7-9841-2fbfc247661a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kj82f"
Jul 29 12:25:19.977420 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.977028    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a6d5f3a2-26cc-40a7-9841-2fbfc247661a-host-run-ovn-kubernetes\") pod \"ovnkube-node-kj82f\" (UID: \"a6d5f3a2-26cc-40a7-9841-2fbfc247661a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kj82f"
Jul 29 12:25:19.977420 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.977088    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a6d5f3a2-26cc-40a7-9841-2fbfc247661a-host-kubelet\") pod \"ovnkube-node-kj82f\" (UID: \"a6d5f3a2-26cc-40a7-9841-2fbfc247661a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kj82f"
Jul 29 12:25:19.977420 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.977118    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a6d5f3a2-26cc-40a7-9841-2fbfc247661a-log-socket\") pod \"ovnkube-node-kj82f\" (UID: \"a6d5f3a2-26cc-40a7-9841-2fbfc247661a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kj82f"
Jul 29 12:25:19.977420 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.977172    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a6d5f3a2-26cc-40a7-9841-2fbfc247661a-log-socket\") pod \"ovnkube-node-kj82f\" (UID: \"a6d5f3a2-26cc-40a7-9841-2fbfc247661a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kj82f"
Jul 29 12:25:19.977420 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.977353    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a6d5f3a2-26cc-40a7-9841-2fbfc247661a-ovnkube-script-lib\") pod \"ovnkube-node-kj82f\" (UID: \"a6d5f3a2-26cc-40a7-9841-2fbfc247661a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kj82f"
Jul 29 12:25:19.977420 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.977395    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a6d5f3a2-26cc-40a7-9841-2fbfc247661a-ovnkube-config\") pod \"ovnkube-node-kj82f\" (UID: \"a6d5f3a2-26cc-40a7-9841-2fbfc247661a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kj82f"
Jul 29 12:25:19.978625 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.978606    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a6d5f3a2-26cc-40a7-9841-2fbfc247661a-ovn-node-metrics-cert\") pod \"ovnkube-node-kj82f\" (UID: \"a6d5f3a2-26cc-40a7-9841-2fbfc247661a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kj82f"
Jul 29 12:25:19.983067 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:19.983050    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 12:25:19.983158 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:19.983073    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 12:25:19.983158 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:19.983084    2567 projected.go:196] Error preparing data for projected volume kube-api-access-fnsg6 for pod openshift-network-diagnostics/network-check-target-r4hjk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 12:25:19.983158 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:19.983133    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8e7279c1-35b7-4f54-8b02-fdac281ef757-kube-api-access-fnsg6 podName:8e7279c1-35b7-4f54-8b02-fdac281ef757 nodeName:}" failed. No retries permitted until 2026-07-29 12:25:20.483119982 +0000 UTC m=+3.176808270 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-fnsg6" (UniqueName: "kubernetes.io/projected/8e7279c1-35b7-4f54-8b02-fdac281ef757-kube-api-access-fnsg6") pod "network-check-target-r4hjk" (UID: "8e7279c1-35b7-4f54-8b02-fdac281ef757") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 12:25:19.985161 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.985142    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfvj2\" (UniqueName: \"kubernetes.io/projected/4038ffd5-00ac-4173-bd2a-4a8b3bb0ad5a-kube-api-access-lfvj2\") pod \"iptables-alerter-nkfzw\" (UID: \"4038ffd5-00ac-4173-bd2a-4a8b3bb0ad5a\") " pod="openshift-network-operator/iptables-alerter-nkfzw"
Jul 29 12:25:19.985348 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:19.985332    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbhws\" (UniqueName: \"kubernetes.io/projected/a6d5f3a2-26cc-40a7-9841-2fbfc247661a-kube-api-access-fbhws\") pod \"ovnkube-node-kj82f\" (UID: \"a6d5f3a2-26cc-40a7-9841-2fbfc247661a\") " pod="openshift-ovn-kubernetes/ovnkube-node-kj82f"
Jul 29 12:25:20.066153 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:20.066123    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-clmgr"
Jul 29 12:25:20.075766 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:20.075745    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-h2qrx"
Jul 29 12:25:20.084410 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:20.084383    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-ph59r"
Jul 29 12:25:20.089051 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:20.089032    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzmrx"
Jul 29 12:25:20.095637 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:20.095616    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-znk2m"
Jul 29 12:25:20.103177 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:20.103160    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-z99ld"
Jul 29 12:25:20.110453 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:20.110435    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8rddq"
Jul 29 12:25:20.116869 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:20.116852    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nkfzw"
Jul 29 12:25:20.122432 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:20.122413    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-kj82f"
Jul 29 12:25:20.379613 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:20.379583    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68d50be9-1461-43d4-811e-74ff3fed1a75-metrics-certs\") pod \"network-metrics-daemon-qkdl7\" (UID: \"68d50be9-1461-43d4-811e-74ff3fed1a75\") " pod="openshift-multus/network-metrics-daemon-qkdl7"
Jul 29 12:25:20.379768 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:20.379633    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1635829e-9a40-4521-8430-88adbb07a97a-original-pull-secret\") pod \"global-pull-secret-syncer-w97vv\" (UID: \"1635829e-9a40-4521-8430-88adbb07a97a\") " pod="kube-system/global-pull-secret-syncer-w97vv"
Jul 29 12:25:20.379768 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:20.379732    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 12:25:20.379868 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:20.379778    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1635829e-9a40-4521-8430-88adbb07a97a-original-pull-secret podName:1635829e-9a40-4521-8430-88adbb07a97a nodeName:}" failed. No retries permitted until 2026-07-29 12:25:21.379766132 +0000 UTC m=+4.073454421 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1635829e-9a40-4521-8430-88adbb07a97a-original-pull-secret") pod "global-pull-secret-syncer-w97vv" (UID: "1635829e-9a40-4521-8430-88adbb07a97a") : object "kube-system"/"original-pull-secret" not registered
Jul 29 12:25:20.379868 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:20.379732    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 12:25:20.379868 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:20.379841    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68d50be9-1461-43d4-811e-74ff3fed1a75-metrics-certs podName:68d50be9-1461-43d4-811e-74ff3fed1a75 nodeName:}" failed. No retries permitted until 2026-07-29 12:25:21.379830361 +0000 UTC m=+4.073518657 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/68d50be9-1461-43d4-811e-74ff3fed1a75-metrics-certs") pod "network-metrics-daemon-qkdl7" (UID: "68d50be9-1461-43d4-811e-74ff3fed1a75") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 12:25:20.532012 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:20.531986    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod192debc3_6fd5_40d8_bca9_b58b50a97fa5.slice/crio-89fbd2ba3ea1abee0218227e55124db3b191568012974e15d6c36b38d11afe53 WatchSource:0}: Error finding container 89fbd2ba3ea1abee0218227e55124db3b191568012974e15d6c36b38d11afe53: Status 404 returned error can't find the container with id 89fbd2ba3ea1abee0218227e55124db3b191568012974e15d6c36b38d11afe53
Jul 29 12:25:20.542285 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:20.542213    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7be0cf2c_db65_486e_8545_eabd24ad8428.slice/crio-082b14f30161e9b11179d58022a6cf2142fafc0b01b6ad90786342f9462726a8 WatchSource:0}: Error finding container 082b14f30161e9b11179d58022a6cf2142fafc0b01b6ad90786342f9462726a8: Status 404 returned error can't find the container with id 082b14f30161e9b11179d58022a6cf2142fafc0b01b6ad90786342f9462726a8
Jul 29 12:25:20.545451 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:20.545420    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcaf950ed_dbb9_4b7d_a802_062882ff4899.slice/crio-717250d9132e74010368eb01a62ccd9de984e731f8789b0a5c0269954bf5d725 WatchSource:0}: Error finding container 717250d9132e74010368eb01a62ccd9de984e731f8789b0a5c0269954bf5d725: Status 404 returned error can't find the container with id 717250d9132e74010368eb01a62ccd9de984e731f8789b0a5c0269954bf5d725
Jul 29 12:25:20.548032 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:20.548001    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f03f162_cb82_442c_8e40_b8877aa1996f.slice/crio-896aa42d2b89c4cedfd5dcf07a4a34141415228c7813067118bc5c47d7a69b1f WatchSource:0}: Error finding container 896aa42d2b89c4cedfd5dcf07a4a34141415228c7813067118bc5c47d7a69b1f: Status 404 returned error can't find the container with id 896aa42d2b89c4cedfd5dcf07a4a34141415228c7813067118bc5c47d7a69b1f
Jul 29 12:25:20.549427 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:20.549310    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6d5f3a2_26cc_40a7_9841_2fbfc247661a.slice/crio-f976ee8fd4f93aad92472bc4bb11abc756f1479b10af2563b62dcaccae5dfdc2 WatchSource:0}: Error finding container f976ee8fd4f93aad92472bc4bb11abc756f1479b10af2563b62dcaccae5dfdc2: Status 404 returned error can't find the container with id f976ee8fd4f93aad92472bc4bb11abc756f1479b10af2563b62dcaccae5dfdc2
Jul 29 12:25:20.550706 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:20.550688    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda09948e8_234f_41ef_866d_10ddb32e33fd.slice/crio-94d58c08868a95b52fcba172906964ce5c6de202c3dbbfd7be83c4b0496d0ac8 WatchSource:0}: Error finding container 94d58c08868a95b52fcba172906964ce5c6de202c3dbbfd7be83c4b0496d0ac8: Status 404 returned error can't find the container with id 94d58c08868a95b52fcba172906964ce5c6de202c3dbbfd7be83c4b0496d0ac8
Jul 29 12:25:20.551024 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:20.551005    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bdec0f6_396e_4392_9e8d_e2bad0289b8f.slice/crio-c7be55670a78daac46593ad22c77428c7f6decb31ebfe4303c39bbd989d16ad1 WatchSource:0}: Error finding container c7be55670a78daac46593ad22c77428c7f6decb31ebfe4303c39bbd989d16ad1: Status 404 returned error can't find the container with id c7be55670a78daac46593ad22c77428c7f6decb31ebfe4303c39bbd989d16ad1
Jul 29 12:25:20.552171 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:20.552154    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1840c2b3_2714_4f4e_9255_2cb612abb53e.slice/crio-f74e36cd62398dc568d78976a9c6cd71ba754b4776b8b83d941e4b2430696e00 WatchSource:0}: Error finding container f74e36cd62398dc568d78976a9c6cd71ba754b4776b8b83d941e4b2430696e00: Status 404 returned error can't find the container with id f74e36cd62398dc568d78976a9c6cd71ba754b4776b8b83d941e4b2430696e00
Jul 29 12:25:20.553095 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:20.553070    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4038ffd5_00ac_4173_bd2a_4a8b3bb0ad5a.slice/crio-64687504b16e9602fdcdc731b868073eb43344b4ee11d9c6c4739dbc676acc06 WatchSource:0}: Error finding container 64687504b16e9602fdcdc731b868073eb43344b4ee11d9c6c4739dbc676acc06: Status 404 returned error can't find the container with id 64687504b16e9602fdcdc731b868073eb43344b4ee11d9c6c4739dbc676acc06
Jul 29 12:25:20.581269 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:20.581245    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnsg6\" (UniqueName: \"kubernetes.io/projected/8e7279c1-35b7-4f54-8b02-fdac281ef757-kube-api-access-fnsg6\") pod \"network-check-target-r4hjk\" (UID: \"8e7279c1-35b7-4f54-8b02-fdac281ef757\") " pod="openshift-network-diagnostics/network-check-target-r4hjk"
Jul 29 12:25:20.581429 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:20.581414    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 12:25:20.581468 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:20.581439    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 12:25:20.581468 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:20.581451    2567 projected.go:196] Error preparing data for projected volume kube-api-access-fnsg6 for pod openshift-network-diagnostics/network-check-target-r4hjk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 12:25:20.581526 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:20.581502    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8e7279c1-35b7-4f54-8b02-fdac281ef757-kube-api-access-fnsg6 podName:8e7279c1-35b7-4f54-8b02-fdac281ef757 nodeName:}" failed. No retries permitted until 2026-07-29 12:25:21.581484768 +0000 UTC m=+4.275173070 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-fnsg6" (UniqueName: "kubernetes.io/projected/8e7279c1-35b7-4f54-8b02-fdac281ef757-kube-api-access-fnsg6") pod "network-check-target-r4hjk" (UID: "8e7279c1-35b7-4f54-8b02-fdac281ef757") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 12:25:20.817742 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:20.817691    2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-28 12:20:19 +0000 UTC" deadline="2028-04-10 15:53:14.786907006 +0000 UTC"
Jul 29 12:25:20.817742 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:20.817737    2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14907h27m53.969173639s"
Jul 29 12:25:20.924001 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:20.923574    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-25.ec2.internal" event={"ID":"eee88d5337a9ab304c1e57b0fd78f1d9","Type":"ContainerStarted","Data":"087cd57085a695fade6f450ad426c1b605c4e5506ee95ba2718902173be396e9"}
Jul 29 12:25:20.925739 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:20.925666    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h2qrx" event={"ID":"a09948e8-234f-41ef-866d-10ddb32e33fd","Type":"ContainerStarted","Data":"94d58c08868a95b52fcba172906964ce5c6de202c3dbbfd7be83c4b0496d0ac8"}
Jul 29 12:25:20.927420 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:20.927379    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-clmgr" event={"ID":"caf950ed-dbb9-4b7d-a802-062882ff4899","Type":"ContainerStarted","Data":"717250d9132e74010368eb01a62ccd9de984e731f8789b0a5c0269954bf5d725"}
Jul 29 12:25:20.929828 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:20.929783    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-z99ld" event={"ID":"192debc3-6fd5-40d8-bca9-b58b50a97fa5","Type":"ContainerStarted","Data":"89fbd2ba3ea1abee0218227e55124db3b191568012974e15d6c36b38d11afe53"}
Jul 29 12:25:20.931138 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:20.931088    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nkfzw" event={"ID":"4038ffd5-00ac-4173-bd2a-4a8b3bb0ad5a","Type":"ContainerStarted","Data":"64687504b16e9602fdcdc731b868073eb43344b4ee11d9c6c4739dbc676acc06"}
Jul 29 12:25:20.932498 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:20.932422    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzmrx" event={"ID":"1840c2b3-2714-4f4e-9255-2cb612abb53e","Type":"ContainerStarted","Data":"f74e36cd62398dc568d78976a9c6cd71ba754b4776b8b83d941e4b2430696e00"}
Jul 29 12:25:20.933742 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:20.933716    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8rddq" event={"ID":"4bdec0f6-396e-4392-9e8d-e2bad0289b8f","Type":"ContainerStarted","Data":"c7be55670a78daac46593ad22c77428c7f6decb31ebfe4303c39bbd989d16ad1"}
Jul 29 12:25:20.935310 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:20.935288    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kj82f" event={"ID":"a6d5f3a2-26cc-40a7-9841-2fbfc247661a","Type":"ContainerStarted","Data":"f976ee8fd4f93aad92472bc4bb11abc756f1479b10af2563b62dcaccae5dfdc2"}
Jul 29 12:25:20.937269 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:20.937246    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-ph59r" event={"ID":"0f03f162-cb82-442c-8e40-b8877aa1996f","Type":"ContainerStarted","Data":"896aa42d2b89c4cedfd5dcf07a4a34141415228c7813067118bc5c47d7a69b1f"}
Jul 29 12:25:20.938921 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:20.938897    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-znk2m" event={"ID":"7be0cf2c-db65-486e-8545-eabd24ad8428","Type":"ContainerStarted","Data":"082b14f30161e9b11179d58022a6cf2142fafc0b01b6ad90786342f9462726a8"}
Jul 29 12:25:21.388326 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:21.388291    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1635829e-9a40-4521-8430-88adbb07a97a-original-pull-secret\") pod \"global-pull-secret-syncer-w97vv\" (UID: \"1635829e-9a40-4521-8430-88adbb07a97a\") " pod="kube-system/global-pull-secret-syncer-w97vv"
Jul 29 12:25:21.388532 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:21.388364    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68d50be9-1461-43d4-811e-74ff3fed1a75-metrics-certs\") pod \"network-metrics-daemon-qkdl7\" (UID: \"68d50be9-1461-43d4-811e-74ff3fed1a75\") " pod="openshift-multus/network-metrics-daemon-qkdl7"
Jul 29 12:25:21.388532 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:21.388500    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 12:25:21.388659 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:21.388557    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68d50be9-1461-43d4-811e-74ff3fed1a75-metrics-certs podName:68d50be9-1461-43d4-811e-74ff3fed1a75 nodeName:}" failed. No retries permitted until 2026-07-29 12:25:23.388540064 +0000 UTC m=+6.082228353 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/68d50be9-1461-43d4-811e-74ff3fed1a75-metrics-certs") pod "network-metrics-daemon-qkdl7" (UID: "68d50be9-1461-43d4-811e-74ff3fed1a75") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 12:25:21.389047 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:21.389027    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 12:25:21.389152 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:21.389087    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1635829e-9a40-4521-8430-88adbb07a97a-original-pull-secret podName:1635829e-9a40-4521-8430-88adbb07a97a nodeName:}" failed. No retries permitted until 2026-07-29 12:25:23.389071068 +0000 UTC m=+6.082759360 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1635829e-9a40-4521-8430-88adbb07a97a-original-pull-secret") pod "global-pull-secret-syncer-w97vv" (UID: "1635829e-9a40-4521-8430-88adbb07a97a") : object "kube-system"/"original-pull-secret" not registered
Jul 29 12:25:21.590378 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:21.590344    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnsg6\" (UniqueName: \"kubernetes.io/projected/8e7279c1-35b7-4f54-8b02-fdac281ef757-kube-api-access-fnsg6\") pod \"network-check-target-r4hjk\" (UID: \"8e7279c1-35b7-4f54-8b02-fdac281ef757\") " pod="openshift-network-diagnostics/network-check-target-r4hjk"
Jul 29 12:25:21.590552 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:21.590538    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 12:25:21.590680 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:21.590558    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 12:25:21.590680 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:21.590570    2567 projected.go:196] Error preparing data for projected volume kube-api-access-fnsg6 for pod openshift-network-diagnostics/network-check-target-r4hjk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 12:25:21.590680 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:21.590624    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8e7279c1-35b7-4f54-8b02-fdac281ef757-kube-api-access-fnsg6 podName:8e7279c1-35b7-4f54-8b02-fdac281ef757 nodeName:}" failed. No retries permitted until 2026-07-29 12:25:23.590606928 +0000 UTC m=+6.284295231 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-fnsg6" (UniqueName: "kubernetes.io/projected/8e7279c1-35b7-4f54-8b02-fdac281ef757-kube-api-access-fnsg6") pod "network-check-target-r4hjk" (UID: "8e7279c1-35b7-4f54-8b02-fdac281ef757") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 12:25:21.819036 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:21.818787    2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-28 12:20:19 +0000 UTC" deadline="2028-02-27 02:52:18.691370032 +0000 UTC"
Jul 29 12:25:21.819036 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:21.819038    2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13862h26m56.872338223s"
Jul 29 12:25:21.915435 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:21.915388    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qkdl7"
Jul 29 12:25:21.915625 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:21.915600    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qkdl7" podUID="68d50be9-1461-43d4-811e-74ff3fed1a75"
Jul 29 12:25:21.916000 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:21.915982    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w97vv"
Jul 29 12:25:21.916097 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:21.916078    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w97vv" podUID="1635829e-9a40-4521-8430-88adbb07a97a"
Jul 29 12:25:21.916232 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:21.916212    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r4hjk"
Jul 29 12:25:21.916312 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:21.916294    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r4hjk" podUID="8e7279c1-35b7-4f54-8b02-fdac281ef757"
Jul 29 12:25:22.948445 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:22.948348    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-25.ec2.internal" event={"ID":"875ad84743ea631b485f1bbadf831c17","Type":"ContainerStarted","Data":"7f6844999bdaf28ae2240ccd6de2843a0cd85dd194c0bb6e8fad127f39453a14"}
Jul 29 12:25:22.964756 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:22.964703    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-25.ec2.internal" podStartSLOduration=4.964685312 podStartE2EDuration="4.964685312s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 12:25:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:25:20.941432186 +0000 UTC m=+3.635120500" watchObservedRunningTime="2026-07-29 12:25:22.964685312 +0000 UTC m=+5.658373625"
Jul 29 12:25:23.408573 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:23.408539    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68d50be9-1461-43d4-811e-74ff3fed1a75-metrics-certs\") pod \"network-metrics-daemon-qkdl7\" (UID: \"68d50be9-1461-43d4-811e-74ff3fed1a75\") " pod="openshift-multus/network-metrics-daemon-qkdl7"
Jul 29 12:25:23.408763 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:23.408607    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1635829e-9a40-4521-8430-88adbb07a97a-original-pull-secret\") pod \"global-pull-secret-syncer-w97vv\" (UID: \"1635829e-9a40-4521-8430-88adbb07a97a\") " pod="kube-system/global-pull-secret-syncer-w97vv"
Jul 29 12:25:23.408763 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:23.408757    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 12:25:23.408857 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:23.408795    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 12:25:23.408857 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:23.408809    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1635829e-9a40-4521-8430-88adbb07a97a-original-pull-secret podName:1635829e-9a40-4521-8430-88adbb07a97a nodeName:}" failed. No retries permitted until 2026-07-29 12:25:27.408792461 +0000 UTC m=+10.102480764 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1635829e-9a40-4521-8430-88adbb07a97a-original-pull-secret") pod "global-pull-secret-syncer-w97vv" (UID: "1635829e-9a40-4521-8430-88adbb07a97a") : object "kube-system"/"original-pull-secret" not registered
Jul 29 12:25:23.408857 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:23.408855    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68d50be9-1461-43d4-811e-74ff3fed1a75-metrics-certs podName:68d50be9-1461-43d4-811e-74ff3fed1a75 nodeName:}" failed. No retries permitted until 2026-07-29 12:25:27.40883745 +0000 UTC m=+10.102525746 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/68d50be9-1461-43d4-811e-74ff3fed1a75-metrics-certs") pod "network-metrics-daemon-qkdl7" (UID: "68d50be9-1461-43d4-811e-74ff3fed1a75") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 12:25:23.610282 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:23.610215    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnsg6\" (UniqueName: \"kubernetes.io/projected/8e7279c1-35b7-4f54-8b02-fdac281ef757-kube-api-access-fnsg6\") pod \"network-check-target-r4hjk\" (UID: \"8e7279c1-35b7-4f54-8b02-fdac281ef757\") " pod="openshift-network-diagnostics/network-check-target-r4hjk"
Jul 29 12:25:23.610458 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:23.610357    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 12:25:23.610458 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:23.610380    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 12:25:23.610458 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:23.610391    2567 projected.go:196] Error preparing data for projected volume kube-api-access-fnsg6 for pod openshift-network-diagnostics/network-check-target-r4hjk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 12:25:23.610458 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:23.610463    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8e7279c1-35b7-4f54-8b02-fdac281ef757-kube-api-access-fnsg6 podName:8e7279c1-35b7-4f54-8b02-fdac281ef757 nodeName:}" failed. No retries permitted until 2026-07-29 12:25:27.610444009 +0000 UTC m=+10.304132300 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-fnsg6" (UniqueName: "kubernetes.io/projected/8e7279c1-35b7-4f54-8b02-fdac281ef757-kube-api-access-fnsg6") pod "network-check-target-r4hjk" (UID: "8e7279c1-35b7-4f54-8b02-fdac281ef757") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 12:25:23.917287 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:23.916509    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w97vv"
Jul 29 12:25:23.917287 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:23.916639    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w97vv" podUID="1635829e-9a40-4521-8430-88adbb07a97a"
Jul 29 12:25:23.917287 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:23.916979    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qkdl7"
Jul 29 12:25:23.917287 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:23.917091    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qkdl7" podUID="68d50be9-1461-43d4-811e-74ff3fed1a75"
Jul 29 12:25:23.917287 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:23.917138    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r4hjk"
Jul 29 12:25:23.917287 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:23.917204    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r4hjk" podUID="8e7279c1-35b7-4f54-8b02-fdac281ef757"
Jul 29 12:25:25.915728 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:25.915699    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r4hjk"
Jul 29 12:25:25.916184 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:25.915822    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r4hjk" podUID="8e7279c1-35b7-4f54-8b02-fdac281ef757"
Jul 29 12:25:25.916184 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:25.916131    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qkdl7"
Jul 29 12:25:25.916281 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:25.916214    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qkdl7" podUID="68d50be9-1461-43d4-811e-74ff3fed1a75"
Jul 29 12:25:25.916417 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:25.916288    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w97vv"
Jul 29 12:25:25.916417 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:25.916381    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w97vv" podUID="1635829e-9a40-4521-8430-88adbb07a97a"
Jul 29 12:25:25.956646 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:25.956514    2567 generic.go:356] "Generic (PLEG): container finished" podID="875ad84743ea631b485f1bbadf831c17" containerID="7f6844999bdaf28ae2240ccd6de2843a0cd85dd194c0bb6e8fad127f39453a14" exitCode=0
Jul 29 12:25:25.956646 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:25.956561    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-25.ec2.internal" event={"ID":"875ad84743ea631b485f1bbadf831c17","Type":"ContainerDied","Data":"7f6844999bdaf28ae2240ccd6de2843a0cd85dd194c0bb6e8fad127f39453a14"}
Jul 29 12:25:27.440580 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:27.440372    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68d50be9-1461-43d4-811e-74ff3fed1a75-metrics-certs\") pod \"network-metrics-daemon-qkdl7\" (UID: \"68d50be9-1461-43d4-811e-74ff3fed1a75\") " pod="openshift-multus/network-metrics-daemon-qkdl7"
Jul 29 12:25:27.440580 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:27.440450    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1635829e-9a40-4521-8430-88adbb07a97a-original-pull-secret\") pod \"global-pull-secret-syncer-w97vv\" (UID: \"1635829e-9a40-4521-8430-88adbb07a97a\") " pod="kube-system/global-pull-secret-syncer-w97vv"
Jul 29 12:25:27.440580 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:27.440509    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 12:25:27.440580 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:27.440541    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 12:25:27.440580 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:27.440569    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68d50be9-1461-43d4-811e-74ff3fed1a75-metrics-certs podName:68d50be9-1461-43d4-811e-74ff3fed1a75 nodeName:}" failed. No retries permitted until 2026-07-29 12:25:35.440551411 +0000 UTC m=+18.134239715 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/68d50be9-1461-43d4-811e-74ff3fed1a75-metrics-certs") pod "network-metrics-daemon-qkdl7" (UID: "68d50be9-1461-43d4-811e-74ff3fed1a75") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 12:25:27.440580 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:27.440585    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1635829e-9a40-4521-8430-88adbb07a97a-original-pull-secret podName:1635829e-9a40-4521-8430-88adbb07a97a nodeName:}" failed. No retries permitted until 2026-07-29 12:25:35.440578091 +0000 UTC m=+18.134266380 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1635829e-9a40-4521-8430-88adbb07a97a-original-pull-secret") pod "global-pull-secret-syncer-w97vv" (UID: "1635829e-9a40-4521-8430-88adbb07a97a") : object "kube-system"/"original-pull-secret" not registered
Jul 29 12:25:27.642521 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:27.642488    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnsg6\" (UniqueName: \"kubernetes.io/projected/8e7279c1-35b7-4f54-8b02-fdac281ef757-kube-api-access-fnsg6\") pod \"network-check-target-r4hjk\" (UID: \"8e7279c1-35b7-4f54-8b02-fdac281ef757\") " pod="openshift-network-diagnostics/network-check-target-r4hjk"
Jul 29 12:25:27.642737 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:27.642694    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 12:25:27.642737 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:27.642720    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 12:25:27.642737 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:27.642733    2567 projected.go:196] Error preparing data for projected volume kube-api-access-fnsg6 for pod openshift-network-diagnostics/network-check-target-r4hjk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 12:25:27.642884 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:27.642797    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8e7279c1-35b7-4f54-8b02-fdac281ef757-kube-api-access-fnsg6 podName:8e7279c1-35b7-4f54-8b02-fdac281ef757 nodeName:}" failed. No retries permitted until 2026-07-29 12:25:35.642779017 +0000 UTC m=+18.336467309 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-fnsg6" (UniqueName: "kubernetes.io/projected/8e7279c1-35b7-4f54-8b02-fdac281ef757-kube-api-access-fnsg6") pod "network-check-target-r4hjk" (UID: "8e7279c1-35b7-4f54-8b02-fdac281ef757") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 12:25:27.917058 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:27.916316    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w97vv"
Jul 29 12:25:27.917058 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:27.916427    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w97vv" podUID="1635829e-9a40-4521-8430-88adbb07a97a"
Jul 29 12:25:27.917058 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:27.916801    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r4hjk"
Jul 29 12:25:27.917058 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:27.916883    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r4hjk" podUID="8e7279c1-35b7-4f54-8b02-fdac281ef757"
Jul 29 12:25:27.917058 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:27.916938    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qkdl7"
Jul 29 12:25:27.917058 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:27.917012    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qkdl7" podUID="68d50be9-1461-43d4-811e-74ff3fed1a75"
Jul 29 12:25:29.915735 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:29.915706    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qkdl7"
Jul 29 12:25:29.916155 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:29.915700    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r4hjk"
Jul 29 12:25:29.916155 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:29.915829    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qkdl7" podUID="68d50be9-1461-43d4-811e-74ff3fed1a75"
Jul 29 12:25:29.916155 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:29.915861    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w97vv"
Jul 29 12:25:29.916155 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:29.915962    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r4hjk" podUID="8e7279c1-35b7-4f54-8b02-fdac281ef757"
Jul 29 12:25:29.916155 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:29.915979    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w97vv" podUID="1635829e-9a40-4521-8430-88adbb07a97a"
Jul 29 12:25:31.915827 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:31.915793    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r4hjk"
Jul 29 12:25:31.916250 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:31.915802    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qkdl7"
Jul 29 12:25:31.916250 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:31.915913    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r4hjk" podUID="8e7279c1-35b7-4f54-8b02-fdac281ef757"
Jul 29 12:25:31.916250 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:31.915925    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w97vv"
Jul 29 12:25:31.916250 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:31.916014    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qkdl7" podUID="68d50be9-1461-43d4-811e-74ff3fed1a75"
Jul 29 12:25:31.916250 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:31.916118    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w97vv" podUID="1635829e-9a40-4521-8430-88adbb07a97a"
Jul 29 12:25:33.915718 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:33.915661    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qkdl7"
Jul 29 12:25:33.916180 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:33.915661    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r4hjk"
Jul 29 12:25:33.916180 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:33.915793    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qkdl7" podUID="68d50be9-1461-43d4-811e-74ff3fed1a75"
Jul 29 12:25:33.916180 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:33.915813    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w97vv"
Jul 29 12:25:33.916180 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:33.915911    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r4hjk" podUID="8e7279c1-35b7-4f54-8b02-fdac281ef757"
Jul 29 12:25:33.916180 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:33.916003    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w97vv" podUID="1635829e-9a40-4521-8430-88adbb07a97a"
Jul 29 12:25:35.501320 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:35.501288    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68d50be9-1461-43d4-811e-74ff3fed1a75-metrics-certs\") pod \"network-metrics-daemon-qkdl7\" (UID: \"68d50be9-1461-43d4-811e-74ff3fed1a75\") " pod="openshift-multus/network-metrics-daemon-qkdl7"
Jul 29 12:25:35.501886 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:35.501336    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1635829e-9a40-4521-8430-88adbb07a97a-original-pull-secret\") pod \"global-pull-secret-syncer-w97vv\" (UID: \"1635829e-9a40-4521-8430-88adbb07a97a\") " pod="kube-system/global-pull-secret-syncer-w97vv"
Jul 29 12:25:35.501886 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:35.501447    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 12:25:35.501886 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:35.501465    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 12:25:35.501886 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:35.501519    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68d50be9-1461-43d4-811e-74ff3fed1a75-metrics-certs podName:68d50be9-1461-43d4-811e-74ff3fed1a75 nodeName:}" failed. No retries permitted until 2026-07-29 12:25:51.501497304 +0000 UTC m=+34.195185610 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/68d50be9-1461-43d4-811e-74ff3fed1a75-metrics-certs") pod "network-metrics-daemon-qkdl7" (UID: "68d50be9-1461-43d4-811e-74ff3fed1a75") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 12:25:35.501886 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:35.501540    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1635829e-9a40-4521-8430-88adbb07a97a-original-pull-secret podName:1635829e-9a40-4521-8430-88adbb07a97a nodeName:}" failed. No retries permitted until 2026-07-29 12:25:51.501532895 +0000 UTC m=+34.195221184 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1635829e-9a40-4521-8430-88adbb07a97a-original-pull-secret") pod "global-pull-secret-syncer-w97vv" (UID: "1635829e-9a40-4521-8430-88adbb07a97a") : object "kube-system"/"original-pull-secret" not registered
Jul 29 12:25:35.702940 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:35.702908    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnsg6\" (UniqueName: \"kubernetes.io/projected/8e7279c1-35b7-4f54-8b02-fdac281ef757-kube-api-access-fnsg6\") pod \"network-check-target-r4hjk\" (UID: \"8e7279c1-35b7-4f54-8b02-fdac281ef757\") " pod="openshift-network-diagnostics/network-check-target-r4hjk"
Jul 29 12:25:35.703097 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:35.703053    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 12:25:35.703097 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:35.703071    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 12:25:35.703097 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:35.703082    2567 projected.go:196] Error preparing data for projected volume kube-api-access-fnsg6 for pod openshift-network-diagnostics/network-check-target-r4hjk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 12:25:35.703203 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:35.703141    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8e7279c1-35b7-4f54-8b02-fdac281ef757-kube-api-access-fnsg6 podName:8e7279c1-35b7-4f54-8b02-fdac281ef757 nodeName:}" failed. No retries permitted until 2026-07-29 12:25:51.703122837 +0000 UTC m=+34.396811131 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-fnsg6" (UniqueName: "kubernetes.io/projected/8e7279c1-35b7-4f54-8b02-fdac281ef757-kube-api-access-fnsg6") pod "network-check-target-r4hjk" (UID: "8e7279c1-35b7-4f54-8b02-fdac281ef757") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 12:25:35.915926 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:35.915897    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w97vv"
Jul 29 12:25:35.915926 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:35.915924    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qkdl7"
Jul 29 12:25:35.916106 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:35.915989    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w97vv" podUID="1635829e-9a40-4521-8430-88adbb07a97a"
Jul 29 12:25:35.916106 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:35.915897    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r4hjk"
Jul 29 12:25:35.916106 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:35.916092    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qkdl7" podUID="68d50be9-1461-43d4-811e-74ff3fed1a75"
Jul 29 12:25:35.916218 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:35.916141    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r4hjk" podUID="8e7279c1-35b7-4f54-8b02-fdac281ef757"
Jul 29 12:25:37.916745 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:37.916715    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qkdl7"
Jul 29 12:25:37.917235 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:37.916811    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w97vv"
Jul 29 12:25:37.917235 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:37.916851    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r4hjk"
Jul 29 12:25:37.917235 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:37.916890    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qkdl7" podUID="68d50be9-1461-43d4-811e-74ff3fed1a75"
Jul 29 12:25:37.917235 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:37.916920    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r4hjk" podUID="8e7279c1-35b7-4f54-8b02-fdac281ef757"
Jul 29 12:25:37.917235 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:37.916994    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w97vv" podUID="1635829e-9a40-4521-8430-88adbb07a97a"
Jul 29 12:25:38.979461 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:38.979211    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-z99ld" event={"ID":"192debc3-6fd5-40d8-bca9-b58b50a97fa5","Type":"ContainerStarted","Data":"483fc896b6f594f48bbd322f919f55889b6cfee049c8120c361478256169b0c0"}
Jul 29 12:25:38.981133 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:38.981109    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-25.ec2.internal" event={"ID":"875ad84743ea631b485f1bbadf831c17","Type":"ContainerStarted","Data":"befda767f18d2e25de1061834d49bb57360b059c811a186c27302330553f838d"}
Jul 29 12:25:38.982482 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:38.982458    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzmrx" event={"ID":"1840c2b3-2714-4f4e-9255-2cb612abb53e","Type":"ContainerStarted","Data":"3be2f14e251c492f117c508863e15969717f4df857553f182fa5fe8743f9536e"}
Jul 29 12:25:38.983767 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:38.983749    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8rddq" event={"ID":"4bdec0f6-396e-4392-9e8d-e2bad0289b8f","Type":"ContainerStarted","Data":"d6e92cfcf110a708acbafee54f62bdef75d6aabc62118a9e690e98ae93cc0190"}
Jul 29 12:25:38.985106 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:38.985076    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-ph59r" event={"ID":"0f03f162-cb82-442c-8e40-b8877aa1996f","Type":"ContainerStarted","Data":"db56773358e6bbad5938fd82505024cbcebde8684138bafb029a3936bc84e091"}
Jul 29 12:25:38.986249 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:38.986225    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-znk2m" event={"ID":"7be0cf2c-db65-486e-8545-eabd24ad8428","Type":"ContainerStarted","Data":"b46946bc0d7690ff43ab3aab43761728b27f519116dbe9c1bfa5315440807c7b"}
Jul 29 12:25:38.987531 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:38.987512    2567 generic.go:356] "Generic (PLEG): container finished" podID="a09948e8-234f-41ef-866d-10ddb32e33fd" containerID="67c93e8e9f59bb18da79028480177ad2fd88ec3158b833ab8ab6da7e3ed340c8" exitCode=0
Jul 29 12:25:38.987614 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:38.987572    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h2qrx" event={"ID":"a09948e8-234f-41ef-866d-10ddb32e33fd","Type":"ContainerDied","Data":"67c93e8e9f59bb18da79028480177ad2fd88ec3158b833ab8ab6da7e3ed340c8"}
Jul 29 12:25:38.988968 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:38.988894    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-clmgr" event={"ID":"caf950ed-dbb9-4b7d-a802-062882ff4899","Type":"ContainerStarted","Data":"ba50e3965a8b12e95d76c4691c9b942b46f20c2bf32f8051584bd4305a8213f5"}
Jul 29 12:25:38.999396 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:38.999362    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-z99ld" podStartSLOduration=3.255142872 podStartE2EDuration="20.999349933s" totalImagesPullingTime="17.744207061s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 12:25:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:25:38.999089791 +0000 UTC m=+21.692778100" watchObservedRunningTime="2026-07-29 12:25:38.999349933 +0000 UTC m=+21.693038244"
Jul 29 12:25:39.037895 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:39.037861    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-8rddq" podStartSLOduration=3.310657644 podStartE2EDuration="21.037837159s" totalImagesPullingTime="17.727179515s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 12:25:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:25:39.019366645 +0000 UTC m=+21.713054956" watchObservedRunningTime="2026-07-29 12:25:39.037837159 +0000 UTC m=+21.731525469"
Jul 29 12:25:39.037967 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:39.037924    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-25.ec2.internal" podStartSLOduration=21.037914075 podStartE2EDuration="21.037914075s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 12:25:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:25:39.037816832 +0000 UTC m=+21.731505142" watchObservedRunningTime="2026-07-29 12:25:39.037914075 +0000 UTC m=+21.731602384"
Jul 29 12:25:39.065627 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:39.065596    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-ph59r" podStartSLOduration=3.297944839 podStartE2EDuration="21.065583988s" totalImagesPullingTime="17.767639149s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 12:25:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:25:39.065416851 +0000 UTC m=+21.759105172" watchObservedRunningTime="2026-07-29 12:25:39.065583988 +0000 UTC m=+21.759272298"
Jul 29 12:25:39.083316 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:39.083281    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-clmgr" podStartSLOduration=4.6141613889999995 podStartE2EDuration="22.083269299s" totalImagesPullingTime="17.46910791s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 12:25:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:25:39.082964637 +0000 UTC m=+21.776652974" watchObservedRunningTime="2026-07-29 12:25:39.083269299 +0000 UTC m=+21.776957608"
Jul 29 12:25:39.915921 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:39.915858    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w97vv"
Jul 29 12:25:39.916042 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:39.915858    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qkdl7"
Jul 29 12:25:39.916042 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:39.916030    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r4hjk"
Jul 29 12:25:39.916146 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:39.915979    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w97vv" podUID="1635829e-9a40-4521-8430-88adbb07a97a"
Jul 29 12:25:39.916146 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:39.916097    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qkdl7" podUID="68d50be9-1461-43d4-811e-74ff3fed1a75"
Jul 29 12:25:39.916247 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:39.916155    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r4hjk" podUID="8e7279c1-35b7-4f54-8b02-fdac281ef757"
Jul 29 12:25:39.993097 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:39.993067    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nkfzw" event={"ID":"4038ffd5-00ac-4173-bd2a-4a8b3bb0ad5a","Type":"ContainerStarted","Data":"3f64a7fba27794470ef7712c0b24aafb88976a6b0a293ae0c1515e3f4b35f761"}
Jul 29 12:25:39.996589 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:39.996558    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kj82f_a6d5f3a2-26cc-40a7-9841-2fbfc247661a/ovn-acl-logging/0.log"
Jul 29 12:25:39.997047 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:39.997018    2567 generic.go:356] "Generic (PLEG): container finished" podID="a6d5f3a2-26cc-40a7-9841-2fbfc247661a" containerID="323ae5f7045509f6020108a93662f5cbb5ef6e4ce881203a13c7e081b1938b6e" exitCode=1
Jul 29 12:25:39.997161 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:39.997140    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kj82f" event={"ID":"a6d5f3a2-26cc-40a7-9841-2fbfc247661a","Type":"ContainerStarted","Data":"1fed5fed1b8637cb49155b5e914553cbcdbade6fa4e2210e53c87410bc3f46ce"}
Jul 29 12:25:39.997228 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:39.997214    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kj82f" event={"ID":"a6d5f3a2-26cc-40a7-9841-2fbfc247661a","Type":"ContainerStarted","Data":"5b5a6aac9f1b0700df40b16ddd57983c8dc5e476d171a0380485d92d538ce2aa"}
Jul 29 12:25:39.997281 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:39.997264    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kj82f" event={"ID":"a6d5f3a2-26cc-40a7-9841-2fbfc247661a","Type":"ContainerStarted","Data":"3baa9fa36642d79a396989cc5792b26b12c3b6c3707bfdf1c27faad7424c2a10"}
Jul 29 12:25:39.997330 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:39.997313    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kj82f" event={"ID":"a6d5f3a2-26cc-40a7-9841-2fbfc247661a","Type":"ContainerStarted","Data":"488935492eff67a7554b5322f8058016c9244b653c76d12718ba82755ed44483"}
Jul 29 12:25:39.997378 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:39.997358    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kj82f" event={"ID":"a6d5f3a2-26cc-40a7-9841-2fbfc247661a","Type":"ContainerDied","Data":"323ae5f7045509f6020108a93662f5cbb5ef6e4ce881203a13c7e081b1938b6e"}
Jul 29 12:25:39.997454 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:39.997420    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kj82f" event={"ID":"a6d5f3a2-26cc-40a7-9841-2fbfc247661a","Type":"ContainerStarted","Data":"9dab7ae13c42f0203fdc82228653007007803841320d47f9ebcb94e54b506993"}
Jul 29 12:25:40.011378 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:40.010520    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-znk2m" podStartSLOduration=4.538880391 podStartE2EDuration="22.010504075s" totalImagesPullingTime="17.471623684s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:25:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:25:39.133733955 +0000 UTC m=+21.827422265" watchObservedRunningTime="2026-07-29 12:25:40.010504075 +0000 UTC m=+22.704192386"
Jul 29 12:25:40.011378 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:40.011250    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-nkfzw" podStartSLOduration=4.280771371 podStartE2EDuration="22.011236472s" totalImagesPullingTime="17.730465101s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 12:25:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:25:40.010741898 +0000 UTC m=+22.704430227" watchObservedRunningTime="2026-07-29 12:25:40.011236472 +0000 UTC m=+22.704924785"
Jul 29 12:25:40.088317 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:40.088295    2567 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 29 12:25:40.840829 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:40.840719    2567 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-29T12:25:40.088312573Z","UUID":"d409c4e9-f54f-4dfc-b1b5-a7b6712f0f85","Handler":null,"Name":"","Endpoint":""}
Jul 29 12:25:40.842184 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:40.842159    2567 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 29 12:25:40.842304 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:40.842194    2567 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 29 12:25:41.000700 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:41.000672    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzmrx" event={"ID":"1840c2b3-2714-4f4e-9255-2cb612abb53e","Type":"ContainerStarted","Data":"3470860e3fc15cf1302b5833571912356fab47e2db81875bb1b7fcd886c7aa6e"}
Jul 29 12:25:41.915499 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:41.915471    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qkdl7"
Jul 29 12:25:41.915499 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:41.915495    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w97vv"
Jul 29 12:25:41.915734 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:41.915472    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r4hjk"
Jul 29 12:25:41.915734 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:41.915594    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qkdl7" podUID="68d50be9-1461-43d4-811e-74ff3fed1a75"
Jul 29 12:25:41.915734 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:41.915662    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w97vv" podUID="1635829e-9a40-4521-8430-88adbb07a97a"
Jul 29 12:25:41.915883 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:41.915740    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r4hjk" podUID="8e7279c1-35b7-4f54-8b02-fdac281ef757"
Jul 29 12:25:42.007034 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:42.007001    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzmrx" event={"ID":"1840c2b3-2714-4f4e-9255-2cb612abb53e","Type":"ContainerStarted","Data":"fe1da5cb996fb53c3a462eb0e7a41762a89e37277ec4a02ecc1d8aaf05c376ab"}
Jul 29 12:25:42.025739 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:42.025690    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzmrx" podStartSLOduration=3.950967646 podStartE2EDuration="24.025675063s" totalImagesPullingTime="20.074707417s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 12:25:18 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:25:42.02535176 +0000 UTC m=+24.719040069" watchObservedRunningTime="2026-07-29 12:25:42.025675063 +0000 UTC m=+24.719363373"
Jul 29 12:25:43.793555 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:43.793348    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-znk2m"
Jul 29 12:25:43.794133 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:43.793961    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-znk2m"
Jul 29 12:25:43.916168 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:43.916142    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r4hjk"
Jul 29 12:25:43.916168 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:43.916155    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qkdl7"
Jul 29 12:25:43.916298 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:43.916148    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w97vv"
Jul 29 12:25:43.916298 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:43.916236    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r4hjk" podUID="8e7279c1-35b7-4f54-8b02-fdac281ef757"
Jul 29 12:25:43.916360 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:43.916331    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w97vv" podUID="1635829e-9a40-4521-8430-88adbb07a97a"
Jul 29 12:25:43.916468 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:43.916444    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qkdl7" podUID="68d50be9-1461-43d4-811e-74ff3fed1a75"
Jul 29 12:25:44.015232 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:44.015204    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kj82f_a6d5f3a2-26cc-40a7-9841-2fbfc247661a/ovn-acl-logging/0.log"
Jul 29 12:25:44.015642 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:44.015617    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kj82f" event={"ID":"a6d5f3a2-26cc-40a7-9841-2fbfc247661a","Type":"ContainerStarted","Data":"1428f35b1c6d9757e8610c274ad0a9f8a52fe2b16f1c01b4056b4bd817065057"}
Jul 29 12:25:44.017523 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:44.017496    2567 generic.go:356] "Generic (PLEG): container finished" podID="a09948e8-234f-41ef-866d-10ddb32e33fd" containerID="255f37c7a6adac42d3b748e51f48a6e9901f46b24bc1b9c74b5019daa223043a" exitCode=0
Jul 29 12:25:44.017640 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:44.017585    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h2qrx" event={"ID":"a09948e8-234f-41ef-866d-10ddb32e33fd","Type":"ContainerDied","Data":"255f37c7a6adac42d3b748e51f48a6e9901f46b24bc1b9c74b5019daa223043a"}
Jul 29 12:25:44.017698 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:44.017645    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-znk2m"
Jul 29 12:25:44.018182 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:44.018159    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-znk2m"
Jul 29 12:25:45.915834 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:45.915617    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qkdl7"
Jul 29 12:25:45.916315 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:45.915627    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w97vv"
Jul 29 12:25:45.916315 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:45.915947    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qkdl7" podUID="68d50be9-1461-43d4-811e-74ff3fed1a75"
Jul 29 12:25:45.916315 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:45.915691    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r4hjk"
Jul 29 12:25:45.916315 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:45.915999    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w97vv" podUID="1635829e-9a40-4521-8430-88adbb07a97a"
Jul 29 12:25:45.916315 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:45.916067    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r4hjk" podUID="8e7279c1-35b7-4f54-8b02-fdac281ef757"
Jul 29 12:25:46.025319 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:46.025303    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kj82f_a6d5f3a2-26cc-40a7-9841-2fbfc247661a/ovn-acl-logging/0.log"
Jul 29 12:25:46.025696 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:46.025679    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kj82f" event={"ID":"a6d5f3a2-26cc-40a7-9841-2fbfc247661a","Type":"ContainerStarted","Data":"9be84fea4cec0af28d3528d4952edceab70ffb214be24039594f1f4d6218db7f"}
Jul 29 12:25:46.025896 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:46.025878    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kj82f"
Jul 29 12:25:46.026079 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:46.026057    2567 scope.go:117] "RemoveContainer" containerID="323ae5f7045509f6020108a93662f5cbb5ef6e4ce881203a13c7e081b1938b6e"
Jul 29 12:25:46.028034 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:46.028010    2567 generic.go:356] "Generic (PLEG): container finished" podID="a09948e8-234f-41ef-866d-10ddb32e33fd" containerID="1f23d8caac05dbfd4850356e93ba2d1635ed249b3e166a6946d4d85248113976" exitCode=0
Jul 29 12:25:46.028111 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:46.028095    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h2qrx" event={"ID":"a09948e8-234f-41ef-866d-10ddb32e33fd","Type":"ContainerDied","Data":"1f23d8caac05dbfd4850356e93ba2d1635ed249b3e166a6946d4d85248113976"}
Jul 29 12:25:46.042665 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:46.042649    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kj82f"
Jul 29 12:25:47.034859 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:47.034833    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kj82f_a6d5f3a2-26cc-40a7-9841-2fbfc247661a/ovn-acl-logging/0.log"
Jul 29 12:25:47.035341 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:47.035214    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kj82f" event={"ID":"a6d5f3a2-26cc-40a7-9841-2fbfc247661a","Type":"ContainerStarted","Data":"4efddfa769dfa54c5ef41d8b5ad89e3e90c06719b9047da771a536c7b73b863f"}
Jul 29 12:25:47.035705 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:47.035596    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kj82f"
Jul 29 12:25:47.035705 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:47.035692    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kj82f"
Jul 29 12:25:47.050725 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:47.050703    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kj82f"
Jul 29 12:25:47.078611 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:47.078570    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-kj82f" podStartSLOduration=10.634633182 podStartE2EDuration="29.078557485s" totalImagesPullingTime="18.443924303s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 12:25:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:25:47.07616968 +0000 UTC m=+29.769858000" watchObservedRunningTime="2026-07-29 12:25:47.078557485 +0000 UTC m=+29.772245794"
Jul 29 12:25:47.768710 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:47.768606    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-w97vv"]
Jul 29 12:25:47.768876 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:47.768754    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w97vv"
Jul 29 12:25:47.768938 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:47.768880    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w97vv" podUID="1635829e-9a40-4521-8430-88adbb07a97a"
Jul 29 12:25:47.772229 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:47.772196    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-qkdl7"]
Jul 29 12:25:47.772326 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:47.772298    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qkdl7"
Jul 29 12:25:47.772391 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:47.772375    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qkdl7" podUID="68d50be9-1461-43d4-811e-74ff3fed1a75"
Jul 29 12:25:47.774258 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:47.774230    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-r4hjk"]
Jul 29 12:25:47.774360 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:47.774339    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r4hjk"
Jul 29 12:25:47.774528 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:47.774492    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r4hjk" podUID="8e7279c1-35b7-4f54-8b02-fdac281ef757"
Jul 29 12:25:48.040181 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:48.040154    2567 generic.go:356] "Generic (PLEG): container finished" podID="a09948e8-234f-41ef-866d-10ddb32e33fd" containerID="72fa33b48508c2b348327a57cda22cb92edea893dc66dec9b9f125cd776a6443" exitCode=0
Jul 29 12:25:48.040546 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:48.040230    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h2qrx" event={"ID":"a09948e8-234f-41ef-866d-10ddb32e33fd","Type":"ContainerDied","Data":"72fa33b48508c2b348327a57cda22cb92edea893dc66dec9b9f125cd776a6443"}
Jul 29 12:25:48.915981 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:48.915838    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w97vv"
Jul 29 12:25:48.916136 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:48.916061    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w97vv" podUID="1635829e-9a40-4521-8430-88adbb07a97a"
Jul 29 12:25:49.915693 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:49.915619    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r4hjk"
Jul 29 12:25:49.916237 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:49.915721    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r4hjk" podUID="8e7279c1-35b7-4f54-8b02-fdac281ef757"
Jul 29 12:25:49.916237 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:49.915780    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qkdl7"
Jul 29 12:25:49.916237 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:49.915891    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qkdl7" podUID="68d50be9-1461-43d4-811e-74ff3fed1a75"
Jul 29 12:25:50.915662 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:50.915628    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w97vv"
Jul 29 12:25:50.915793 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:50.915755    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w97vv" podUID="1635829e-9a40-4521-8430-88adbb07a97a"
Jul 29 12:25:51.102507 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.102445    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-25.ec2.internal" event="NodeReady"
Jul 29 12:25:51.102643 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.102571    2567 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 29 12:25:51.139108 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.139076    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7f46c77f5c-tnln6"]
Jul 29 12:25:51.144482 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.144463    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f46c77f5c-tnln6"
Jul 29 12:25:51.148462 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.148442    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 29 12:25:51.148565 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.148476    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-bggs6\""
Jul 29 12:25:51.149450 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.149420    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 29 12:25:51.149796 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.149777    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 29 12:25:51.158542 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.158518    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7f46c77f5c-tnln6"]
Jul 29 12:25:51.159181 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.159160    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 29 12:25:51.160814 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.160794    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-sdj8z"]
Jul 29 12:25:51.164821 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.164804    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-w8d7p"]
Jul 29 12:25:51.164944 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.164925    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-sdj8z"
Jul 29 12:25:51.167631 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.167611    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 29 12:25:51.167721 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.167712    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 29 12:25:51.167942 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.167925    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-zrh5t\""
Jul 29 12:25:51.168591 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.168573    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-w8d7p"
Jul 29 12:25:51.171082 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.171062    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 29 12:25:51.171569 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.171553    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 29 12:25:51.171671 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.171559    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 29 12:25:51.171671 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.171608    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-2fqhh\""
Jul 29 12:25:51.177417 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.177385    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-sdj8z"]
Jul 29 12:25:51.179485 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.179467    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-w8d7p"]
Jul 29 12:25:51.227087 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.227061    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/dff975a8-8807-4dbe-9124-7c96c00a1949-image-registry-private-configuration\") pod \"image-registry-7f46c77f5c-tnln6\" (UID: \"dff975a8-8807-4dbe-9124-7c96c00a1949\") " pod="openshift-image-registry/image-registry-7f46c77f5c-tnln6"
Jul 29 12:25:51.227187 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.227142    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/dff975a8-8807-4dbe-9124-7c96c00a1949-registry-certificates\") pod \"image-registry-7f46c77f5c-tnln6\" (UID: \"dff975a8-8807-4dbe-9124-7c96c00a1949\") " pod="openshift-image-registry/image-registry-7f46c77f5c-tnln6"
Jul 29 12:25:51.227187 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.227162    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dff975a8-8807-4dbe-9124-7c96c00a1949-trusted-ca\") pod \"image-registry-7f46c77f5c-tnln6\" (UID: \"dff975a8-8807-4dbe-9124-7c96c00a1949\") " pod="openshift-image-registry/image-registry-7f46c77f5c-tnln6"
Jul 29 12:25:51.227290 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.227197    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/dff975a8-8807-4dbe-9124-7c96c00a1949-ca-trust-extracted\") pod \"image-registry-7f46c77f5c-tnln6\" (UID: \"dff975a8-8807-4dbe-9124-7c96c00a1949\") " pod="openshift-image-registry/image-registry-7f46c77f5c-tnln6"
Jul 29 12:25:51.227290 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.227281    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dff975a8-8807-4dbe-9124-7c96c00a1949-registry-tls\") pod \"image-registry-7f46c77f5c-tnln6\" (UID: \"dff975a8-8807-4dbe-9124-7c96c00a1949\") " pod="openshift-image-registry/image-registry-7f46c77f5c-tnln6"
Jul 29 12:25:51.227386 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.227303    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/dff975a8-8807-4dbe-9124-7c96c00a1949-installation-pull-secrets\") pod \"image-registry-7f46c77f5c-tnln6\" (UID: \"dff975a8-8807-4dbe-9124-7c96c00a1949\") " pod="openshift-image-registry/image-registry-7f46c77f5c-tnln6"
Jul 29 12:25:51.227386 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.227341    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dff975a8-8807-4dbe-9124-7c96c00a1949-bound-sa-token\") pod \"image-registry-7f46c77f5c-tnln6\" (UID: \"dff975a8-8807-4dbe-9124-7c96c00a1949\") " pod="openshift-image-registry/image-registry-7f46c77f5c-tnln6"
Jul 29 12:25:51.227386 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.227365    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bfs7\" (UniqueName: \"kubernetes.io/projected/dff975a8-8807-4dbe-9124-7c96c00a1949-kube-api-access-6bfs7\") pod \"image-registry-7f46c77f5c-tnln6\" (UID: \"dff975a8-8807-4dbe-9124-7c96c00a1949\") " pod="openshift-image-registry/image-registry-7f46c77f5c-tnln6"
Jul 29 12:25:51.328514 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.328487    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/dff975a8-8807-4dbe-9124-7c96c00a1949-installation-pull-secrets\") pod \"image-registry-7f46c77f5c-tnln6\" (UID: \"dff975a8-8807-4dbe-9124-7c96c00a1949\") " pod="openshift-image-registry/image-registry-7f46c77f5c-tnln6"
Jul 29 12:25:51.328639 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.328521    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dff975a8-8807-4dbe-9124-7c96c00a1949-bound-sa-token\") pod \"image-registry-7f46c77f5c-tnln6\" (UID: \"dff975a8-8807-4dbe-9124-7c96c00a1949\") " pod="openshift-image-registry/image-registry-7f46c77f5c-tnln6"
Jul 29 12:25:51.328639 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.328540    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bfs7\" (UniqueName: \"kubernetes.io/projected/dff975a8-8807-4dbe-9124-7c96c00a1949-kube-api-access-6bfs7\") pod \"image-registry-7f46c77f5c-tnln6\" (UID: \"dff975a8-8807-4dbe-9124-7c96c00a1949\") " pod="openshift-image-registry/image-registry-7f46c77f5c-tnln6"
Jul 29 12:25:51.328762 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.328685    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/dff975a8-8807-4dbe-9124-7c96c00a1949-image-registry-private-configuration\") pod \"image-registry-7f46c77f5c-tnln6\" (UID: \"dff975a8-8807-4dbe-9124-7c96c00a1949\") " pod="openshift-image-registry/image-registry-7f46c77f5c-tnln6"
Jul 29 12:25:51.328762 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.328730    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e9572260-1f9a-4f9d-93db-1cc7f72dcc7a-metrics-tls\") pod \"dns-default-sdj8z\" (UID: \"e9572260-1f9a-4f9d-93db-1cc7f72dcc7a\") " pod="openshift-dns/dns-default-sdj8z"
Jul 29 12:25:51.328854 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.328770    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5e7bd908-6123-4f1e-a5ef-109f1b3b0069-cert\") pod \"ingress-canary-w8d7p\" (UID: \"5e7bd908-6123-4f1e-a5ef-109f1b3b0069\") " pod="openshift-ingress-canary/ingress-canary-w8d7p"
Jul 29 12:25:51.328854 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.328799    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgrkm\" (UniqueName: \"kubernetes.io/projected/5e7bd908-6123-4f1e-a5ef-109f1b3b0069-kube-api-access-jgrkm\") pod \"ingress-canary-w8d7p\" (UID: \"5e7bd908-6123-4f1e-a5ef-109f1b3b0069\") " pod="openshift-ingress-canary/ingress-canary-w8d7p"
Jul 29 12:25:51.328952 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.328852    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/dff975a8-8807-4dbe-9124-7c96c00a1949-registry-certificates\") pod \"image-registry-7f46c77f5c-tnln6\" (UID: \"dff975a8-8807-4dbe-9124-7c96c00a1949\") " pod="openshift-image-registry/image-registry-7f46c77f5c-tnln6"
Jul 29 12:25:51.328952 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.328877    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dff975a8-8807-4dbe-9124-7c96c00a1949-trusted-ca\") pod \"image-registry-7f46c77f5c-tnln6\" (UID: \"dff975a8-8807-4dbe-9124-7c96c00a1949\") " pod="openshift-image-registry/image-registry-7f46c77f5c-tnln6"
Jul 29 12:25:51.328952 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.328906    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9572260-1f9a-4f9d-93db-1cc7f72dcc7a-tmp-dir\") pod \"dns-default-sdj8z\" (UID: \"e9572260-1f9a-4f9d-93db-1cc7f72dcc7a\") " pod="openshift-dns/dns-default-sdj8z"
Jul 29 12:25:51.328952 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.328941    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrp6x\" (UniqueName: \"kubernetes.io/projected/e9572260-1f9a-4f9d-93db-1cc7f72dcc7a-kube-api-access-wrp6x\") pod \"dns-default-sdj8z\" (UID: \"e9572260-1f9a-4f9d-93db-1cc7f72dcc7a\") " pod="openshift-dns/dns-default-sdj8z"
Jul 29 12:25:51.329134 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.328971    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e9572260-1f9a-4f9d-93db-1cc7f72dcc7a-config-volume\") pod \"dns-default-sdj8z\" (UID: \"e9572260-1f9a-4f9d-93db-1cc7f72dcc7a\") " pod="openshift-dns/dns-default-sdj8z"
Jul 29 12:25:51.329134 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.329005    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/dff975a8-8807-4dbe-9124-7c96c00a1949-ca-trust-extracted\") pod \"image-registry-7f46c77f5c-tnln6\" (UID: \"dff975a8-8807-4dbe-9124-7c96c00a1949\") " pod="openshift-image-registry/image-registry-7f46c77f5c-tnln6"
Jul 29 12:25:51.329229 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.329176    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dff975a8-8807-4dbe-9124-7c96c00a1949-registry-tls\") pod \"image-registry-7f46c77f5c-tnln6\" (UID: \"dff975a8-8807-4dbe-9124-7c96c00a1949\") " pod="openshift-image-registry/image-registry-7f46c77f5c-tnln6"
Jul 29 12:25:51.329304 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:51.329284    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 12:25:51.329304 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:51.329304    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f46c77f5c-tnln6: secret "image-registry-tls" not found
Jul 29 12:25:51.329469 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.329305    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/dff975a8-8807-4dbe-9124-7c96c00a1949-ca-trust-extracted\") pod \"image-registry-7f46c77f5c-tnln6\" (UID: \"dff975a8-8807-4dbe-9124-7c96c00a1949\") " pod="openshift-image-registry/image-registry-7f46c77f5c-tnln6"
Jul 29 12:25:51.329469 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:51.329352    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dff975a8-8807-4dbe-9124-7c96c00a1949-registry-tls podName:dff975a8-8807-4dbe-9124-7c96c00a1949 nodeName:}" failed. No retries permitted until 2026-07-29 12:25:51.829335165 +0000 UTC m=+34.523023456 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/dff975a8-8807-4dbe-9124-7c96c00a1949-registry-tls") pod "image-registry-7f46c77f5c-tnln6" (UID: "dff975a8-8807-4dbe-9124-7c96c00a1949") : secret "image-registry-tls" not found
Jul 29 12:25:51.329469 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.329385    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/dff975a8-8807-4dbe-9124-7c96c00a1949-registry-certificates\") pod \"image-registry-7f46c77f5c-tnln6\" (UID: \"dff975a8-8807-4dbe-9124-7c96c00a1949\") " pod="openshift-image-registry/image-registry-7f46c77f5c-tnln6"
Jul 29 12:25:51.329795 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.329742    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dff975a8-8807-4dbe-9124-7c96c00a1949-trusted-ca\") pod \"image-registry-7f46c77f5c-tnln6\" (UID: \"dff975a8-8807-4dbe-9124-7c96c00a1949\") " pod="openshift-image-registry/image-registry-7f46c77f5c-tnln6"
Jul 29 12:25:51.332929 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.332907    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/dff975a8-8807-4dbe-9124-7c96c00a1949-image-registry-private-configuration\") pod \"image-registry-7f46c77f5c-tnln6\" (UID: \"dff975a8-8807-4dbe-9124-7c96c00a1949\") " pod="openshift-image-registry/image-registry-7f46c77f5c-tnln6"
Jul 29 12:25:51.333023 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.332907    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/dff975a8-8807-4dbe-9124-7c96c00a1949-installation-pull-secrets\") pod \"image-registry-7f46c77f5c-tnln6\" (UID: \"dff975a8-8807-4dbe-9124-7c96c00a1949\") " pod="openshift-image-registry/image-registry-7f46c77f5c-tnln6"
Jul 29 12:25:51.340678 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.340642    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bfs7\" (UniqueName: \"kubernetes.io/projected/dff975a8-8807-4dbe-9124-7c96c00a1949-kube-api-access-6bfs7\") pod \"image-registry-7f46c77f5c-tnln6\" (UID: \"dff975a8-8807-4dbe-9124-7c96c00a1949\") " pod="openshift-image-registry/image-registry-7f46c77f5c-tnln6"
Jul 29 12:25:51.341370 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.341354    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dff975a8-8807-4dbe-9124-7c96c00a1949-bound-sa-token\") pod \"image-registry-7f46c77f5c-tnln6\" (UID: \"dff975a8-8807-4dbe-9124-7c96c00a1949\") " pod="openshift-image-registry/image-registry-7f46c77f5c-tnln6"
Jul 29 12:25:51.429624 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.429567    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e9572260-1f9a-4f9d-93db-1cc7f72dcc7a-metrics-tls\") pod \"dns-default-sdj8z\" (UID: \"e9572260-1f9a-4f9d-93db-1cc7f72dcc7a\") " pod="openshift-dns/dns-default-sdj8z"
Jul 29 12:25:51.429624 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.429609    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5e7bd908-6123-4f1e-a5ef-109f1b3b0069-cert\") pod \"ingress-canary-w8d7p\" (UID: \"5e7bd908-6123-4f1e-a5ef-109f1b3b0069\") " pod="openshift-ingress-canary/ingress-canary-w8d7p"
Jul 29 12:25:51.429785 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.429633    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgrkm\" (UniqueName: \"kubernetes.io/projected/5e7bd908-6123-4f1e-a5ef-109f1b3b0069-kube-api-access-jgrkm\") pod \"ingress-canary-w8d7p\" (UID: \"5e7bd908-6123-4f1e-a5ef-109f1b3b0069\") " pod="openshift-ingress-canary/ingress-canary-w8d7p"
Jul 29 12:25:51.429785 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.429667    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9572260-1f9a-4f9d-93db-1cc7f72dcc7a-tmp-dir\") pod \"dns-default-sdj8z\" (UID: \"e9572260-1f9a-4f9d-93db-1cc7f72dcc7a\") " pod="openshift-dns/dns-default-sdj8z"
Jul 29 12:25:51.429785 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.429697    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrp6x\" (UniqueName: \"kubernetes.io/projected/e9572260-1f9a-4f9d-93db-1cc7f72dcc7a-kube-api-access-wrp6x\") pod \"dns-default-sdj8z\" (UID: \"e9572260-1f9a-4f9d-93db-1cc7f72dcc7a\") " pod="openshift-dns/dns-default-sdj8z"
Jul 29 12:25:51.429785 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:51.429714    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 12:25:51.429785 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.429721    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e9572260-1f9a-4f9d-93db-1cc7f72dcc7a-config-volume\") pod \"dns-default-sdj8z\" (UID: \"e9572260-1f9a-4f9d-93db-1cc7f72dcc7a\") " pod="openshift-dns/dns-default-sdj8z"
Jul 29 12:25:51.429785 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:51.429780    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e9572260-1f9a-4f9d-93db-1cc7f72dcc7a-metrics-tls podName:e9572260-1f9a-4f9d-93db-1cc7f72dcc7a nodeName:}" failed. No retries permitted until 2026-07-29 12:25:51.929760074 +0000 UTC m=+34.623448370 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e9572260-1f9a-4f9d-93db-1cc7f72dcc7a-metrics-tls") pod "dns-default-sdj8z" (UID: "e9572260-1f9a-4f9d-93db-1cc7f72dcc7a") : secret "dns-default-metrics-tls" not found
Jul 29 12:25:51.430086 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.430025    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9572260-1f9a-4f9d-93db-1cc7f72dcc7a-tmp-dir\") pod \"dns-default-sdj8z\" (UID: \"e9572260-1f9a-4f9d-93db-1cc7f72dcc7a\") " pod="openshift-dns/dns-default-sdj8z"
Jul 29 12:25:51.430320 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.430301    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e9572260-1f9a-4f9d-93db-1cc7f72dcc7a-config-volume\") pod \"dns-default-sdj8z\" (UID: \"e9572260-1f9a-4f9d-93db-1cc7f72dcc7a\") " pod="openshift-dns/dns-default-sdj8z"
Jul 29 12:25:51.431736 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.431719    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5e7bd908-6123-4f1e-a5ef-109f1b3b0069-cert\") pod \"ingress-canary-w8d7p\" (UID: \"5e7bd908-6123-4f1e-a5ef-109f1b3b0069\") " pod="openshift-ingress-canary/ingress-canary-w8d7p"
Jul 29 12:25:51.439881 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.439864    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrp6x\" (UniqueName: \"kubernetes.io/projected/e9572260-1f9a-4f9d-93db-1cc7f72dcc7a-kube-api-access-wrp6x\") pod \"dns-default-sdj8z\" (UID: \"e9572260-1f9a-4f9d-93db-1cc7f72dcc7a\") " pod="openshift-dns/dns-default-sdj8z"
Jul 29 12:25:51.440314 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.440296    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgrkm\" (UniqueName: \"kubernetes.io/projected/5e7bd908-6123-4f1e-a5ef-109f1b3b0069-kube-api-access-jgrkm\") pod \"ingress-canary-w8d7p\" (UID: \"5e7bd908-6123-4f1e-a5ef-109f1b3b0069\") " pod="openshift-ingress-canary/ingress-canary-w8d7p"
Jul 29 12:25:51.482442 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.482421    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-w8d7p"
Jul 29 12:25:51.531129 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.531107    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68d50be9-1461-43d4-811e-74ff3fed1a75-metrics-certs\") pod \"network-metrics-daemon-qkdl7\" (UID: \"68d50be9-1461-43d4-811e-74ff3fed1a75\") " pod="openshift-multus/network-metrics-daemon-qkdl7"
Jul 29 12:25:51.531226 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.531171    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1635829e-9a40-4521-8430-88adbb07a97a-original-pull-secret\") pod \"global-pull-secret-syncer-w97vv\" (UID: \"1635829e-9a40-4521-8430-88adbb07a97a\") " pod="kube-system/global-pull-secret-syncer-w97vv"
Jul 29 12:25:51.531285 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:51.531240    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 12:25:51.531285 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:51.531254    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 12:25:51.531357 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:51.531300    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68d50be9-1461-43d4-811e-74ff3fed1a75-metrics-certs podName:68d50be9-1461-43d4-811e-74ff3fed1a75 nodeName:}" failed. No retries permitted until 2026-07-29 12:26:23.531283243 +0000 UTC m=+66.224971534 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/68d50be9-1461-43d4-811e-74ff3fed1a75-metrics-certs") pod "network-metrics-daemon-qkdl7" (UID: "68d50be9-1461-43d4-811e-74ff3fed1a75") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 12:25:51.531357 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:51.531316    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1635829e-9a40-4521-8430-88adbb07a97a-original-pull-secret podName:1635829e-9a40-4521-8430-88adbb07a97a nodeName:}" failed. No retries permitted until 2026-07-29 12:26:23.531310459 +0000 UTC m=+66.224998748 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1635829e-9a40-4521-8430-88adbb07a97a-original-pull-secret") pod "global-pull-secret-syncer-w97vv" (UID: "1635829e-9a40-4521-8430-88adbb07a97a") : object "kube-system"/"original-pull-secret" not registered
Jul 29 12:25:51.733397 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.733329    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnsg6\" (UniqueName: \"kubernetes.io/projected/8e7279c1-35b7-4f54-8b02-fdac281ef757-kube-api-access-fnsg6\") pod \"network-check-target-r4hjk\" (UID: \"8e7279c1-35b7-4f54-8b02-fdac281ef757\") " pod="openshift-network-diagnostics/network-check-target-r4hjk"
Jul 29 12:25:51.733541 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:51.733480    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 12:25:51.733541 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:51.733504    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 12:25:51.733541 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:51.733516    2567 projected.go:196] Error preparing data for projected volume kube-api-access-fnsg6 for pod openshift-network-diagnostics/network-check-target-r4hjk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 12:25:51.733644 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:51.733568    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8e7279c1-35b7-4f54-8b02-fdac281ef757-kube-api-access-fnsg6 podName:8e7279c1-35b7-4f54-8b02-fdac281ef757 nodeName:}" failed. No retries permitted until 2026-07-29 12:26:23.733554167 +0000 UTC m=+66.427242456 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-fnsg6" (UniqueName: "kubernetes.io/projected/8e7279c1-35b7-4f54-8b02-fdac281ef757-kube-api-access-fnsg6") pod "network-check-target-r4hjk" (UID: "8e7279c1-35b7-4f54-8b02-fdac281ef757") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 12:25:51.834300 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.834266    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dff975a8-8807-4dbe-9124-7c96c00a1949-registry-tls\") pod \"image-registry-7f46c77f5c-tnln6\" (UID: \"dff975a8-8807-4dbe-9124-7c96c00a1949\") " pod="openshift-image-registry/image-registry-7f46c77f5c-tnln6"
Jul 29 12:25:51.834461 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:51.834391    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 12:25:51.834461 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:51.834423    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f46c77f5c-tnln6: secret "image-registry-tls" not found
Jul 29 12:25:51.834554 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:51.834479    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dff975a8-8807-4dbe-9124-7c96c00a1949-registry-tls podName:dff975a8-8807-4dbe-9124-7c96c00a1949 nodeName:}" failed. No retries permitted until 2026-07-29 12:25:52.834461965 +0000 UTC m=+35.528150270 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/dff975a8-8807-4dbe-9124-7c96c00a1949-registry-tls") pod "image-registry-7f46c77f5c-tnln6" (UID: "dff975a8-8807-4dbe-9124-7c96c00a1949") : secret "image-registry-tls" not found
Jul 29 12:25:51.915367 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.915339    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r4hjk"
Jul 29 12:25:51.915538 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.915373    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qkdl7"
Jul 29 12:25:51.917948 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.917914    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 29 12:25:51.918486 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.917945    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 29 12:25:51.918486 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.918000    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 29 12:25:51.918486 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.918202    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hdxp9\""
Jul 29 12:25:51.918486 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.918211    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-p4f75\""
Jul 29 12:25:51.935329 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:51.935311    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e9572260-1f9a-4f9d-93db-1cc7f72dcc7a-metrics-tls\") pod \"dns-default-sdj8z\" (UID: \"e9572260-1f9a-4f9d-93db-1cc7f72dcc7a\") " pod="openshift-dns/dns-default-sdj8z"
Jul 29 12:25:51.935456 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:51.935444    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 12:25:51.935515 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:51.935506    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e9572260-1f9a-4f9d-93db-1cc7f72dcc7a-metrics-tls podName:e9572260-1f9a-4f9d-93db-1cc7f72dcc7a nodeName:}" failed. No retries permitted until 2026-07-29 12:25:52.93549047 +0000 UTC m=+35.629178765 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e9572260-1f9a-4f9d-93db-1cc7f72dcc7a-metrics-tls") pod "dns-default-sdj8z" (UID: "e9572260-1f9a-4f9d-93db-1cc7f72dcc7a") : secret "dns-default-metrics-tls" not found
Jul 29 12:25:52.294434 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:52.294274    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-w8d7p"]
Jul 29 12:25:52.297104 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:52.297083    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e7bd908_6123_4f1e_a5ef_109f1b3b0069.slice/crio-5af88e7c44b334da382a64fcd838e90a6b95d0aa41c4a470a6e036ae1c1ae603 WatchSource:0}: Error finding container 5af88e7c44b334da382a64fcd838e90a6b95d0aa41c4a470a6e036ae1c1ae603: Status 404 returned error can't find the container with id 5af88e7c44b334da382a64fcd838e90a6b95d0aa41c4a470a6e036ae1c1ae603
Jul 29 12:25:52.844033 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:52.841079    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dff975a8-8807-4dbe-9124-7c96c00a1949-registry-tls\") pod \"image-registry-7f46c77f5c-tnln6\" (UID: \"dff975a8-8807-4dbe-9124-7c96c00a1949\") " pod="openshift-image-registry/image-registry-7f46c77f5c-tnln6"
Jul 29 12:25:52.844033 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:52.841446    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 12:25:52.844033 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:52.841468    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f46c77f5c-tnln6: secret "image-registry-tls" not found
Jul 29 12:25:52.844033 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:52.841556    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dff975a8-8807-4dbe-9124-7c96c00a1949-registry-tls podName:dff975a8-8807-4dbe-9124-7c96c00a1949 nodeName:}" failed. No retries permitted until 2026-07-29 12:25:54.841529319 +0000 UTC m=+37.535217613 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/dff975a8-8807-4dbe-9124-7c96c00a1949-registry-tls") pod "image-registry-7f46c77f5c-tnln6" (UID: "dff975a8-8807-4dbe-9124-7c96c00a1949") : secret "image-registry-tls" not found
Jul 29 12:25:52.915524 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:52.915494    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w97vv"
Jul 29 12:25:52.919039 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:52.919016    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 29 12:25:52.941617 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:52.941595    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e9572260-1f9a-4f9d-93db-1cc7f72dcc7a-metrics-tls\") pod \"dns-default-sdj8z\" (UID: \"e9572260-1f9a-4f9d-93db-1cc7f72dcc7a\") " pod="openshift-dns/dns-default-sdj8z"
Jul 29 12:25:52.941735 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:52.941726    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 12:25:52.941795 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:52.941774    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e9572260-1f9a-4f9d-93db-1cc7f72dcc7a-metrics-tls podName:e9572260-1f9a-4f9d-93db-1cc7f72dcc7a nodeName:}" failed. No retries permitted until 2026-07-29 12:25:54.941761467 +0000 UTC m=+37.635449756 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e9572260-1f9a-4f9d-93db-1cc7f72dcc7a-metrics-tls") pod "dns-default-sdj8z" (UID: "e9572260-1f9a-4f9d-93db-1cc7f72dcc7a") : secret "dns-default-metrics-tls" not found
Jul 29 12:25:53.052377 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:53.052344    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-w8d7p" event={"ID":"5e7bd908-6123-4f1e-a5ef-109f1b3b0069","Type":"ContainerStarted","Data":"5af88e7c44b334da382a64fcd838e90a6b95d0aa41c4a470a6e036ae1c1ae603"}
Jul 29 12:25:53.055500 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:53.055477    2567 generic.go:356] "Generic (PLEG): container finished" podID="a09948e8-234f-41ef-866d-10ddb32e33fd" containerID="1b574aa3d3047145c55ce507608a3305c00f18fa0ab0df892febcb43f9b5a1ef" exitCode=0
Jul 29 12:25:53.055603 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:53.055517    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h2qrx" event={"ID":"a09948e8-234f-41ef-866d-10ddb32e33fd","Type":"ContainerDied","Data":"1b574aa3d3047145c55ce507608a3305c00f18fa0ab0df892febcb43f9b5a1ef"}
Jul 29 12:25:54.061296 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:54.061261    2567 generic.go:356] "Generic (PLEG): container finished" podID="a09948e8-234f-41ef-866d-10ddb32e33fd" containerID="56067005c40cf1b9741cf4820af6d5d0bff5ab463e05507d86aa58eb0fdbdda0" exitCode=0
Jul 29 12:25:54.061738 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:54.061311    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h2qrx" event={"ID":"a09948e8-234f-41ef-866d-10ddb32e33fd","Type":"ContainerDied","Data":"56067005c40cf1b9741cf4820af6d5d0bff5ab463e05507d86aa58eb0fdbdda0"}
Jul 29 12:25:54.858444 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:54.858394    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dff975a8-8807-4dbe-9124-7c96c00a1949-registry-tls\") pod \"image-registry-7f46c77f5c-tnln6\" (UID: \"dff975a8-8807-4dbe-9124-7c96c00a1949\") " pod="openshift-image-registry/image-registry-7f46c77f5c-tnln6"
Jul 29 12:25:54.858621 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:54.858564    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 12:25:54.858621 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:54.858585    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f46c77f5c-tnln6: secret "image-registry-tls" not found
Jul 29 12:25:54.858781 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:54.858650    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dff975a8-8807-4dbe-9124-7c96c00a1949-registry-tls podName:dff975a8-8807-4dbe-9124-7c96c00a1949 nodeName:}" failed. No retries permitted until 2026-07-29 12:25:58.8586314 +0000 UTC m=+41.552319693 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/dff975a8-8807-4dbe-9124-7c96c00a1949-registry-tls") pod "image-registry-7f46c77f5c-tnln6" (UID: "dff975a8-8807-4dbe-9124-7c96c00a1949") : secret "image-registry-tls" not found
Jul 29 12:25:54.958875 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:54.958849    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e9572260-1f9a-4f9d-93db-1cc7f72dcc7a-metrics-tls\") pod \"dns-default-sdj8z\" (UID: \"e9572260-1f9a-4f9d-93db-1cc7f72dcc7a\") " pod="openshift-dns/dns-default-sdj8z"
Jul 29 12:25:54.958997 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:54.958981    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 12:25:54.959040 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:54.959032    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e9572260-1f9a-4f9d-93db-1cc7f72dcc7a-metrics-tls podName:e9572260-1f9a-4f9d-93db-1cc7f72dcc7a nodeName:}" failed. No retries permitted until 2026-07-29 12:25:58.959019336 +0000 UTC m=+41.652707625 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e9572260-1f9a-4f9d-93db-1cc7f72dcc7a-metrics-tls") pod "dns-default-sdj8z" (UID: "e9572260-1f9a-4f9d-93db-1cc7f72dcc7a") : secret "dns-default-metrics-tls" not found
Jul 29 12:25:55.066849 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:55.066718    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h2qrx" event={"ID":"a09948e8-234f-41ef-866d-10ddb32e33fd","Type":"ContainerStarted","Data":"32254b13dd46f4e47689b0c794ad5ef829c161c61b958563cabf5578b7c077e8"}
Jul 29 12:25:55.091185 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:55.091139    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-h2qrx" podStartSLOduration=9.212464808 podStartE2EDuration="38.091126693s" totalImagesPullingTime="28.878661885s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 12:25:17 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:25:55.089594322 +0000 UTC m=+37.783282632" watchObservedRunningTime="2026-07-29 12:25:55.091126693 +0000 UTC m=+37.784815002"
Jul 29 12:25:56.070738 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:56.070694    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-w8d7p" event={"ID":"5e7bd908-6123-4f1e-a5ef-109f1b3b0069","Type":"ContainerStarted","Data":"a94c498cef299df2a675a9ad58320d15c6920d6db8f3c8c0f2ede836e5b1d7b0"}
Jul 29 12:25:56.087338 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:56.087267    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-w8d7p" podStartSLOduration=2.359297177 podStartE2EDuration="5.087250646s" totalImagesPullingTime="2.727953469s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:25:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:25:56.086660904 +0000 UTC m=+38.780349248" watchObservedRunningTime="2026-07-29 12:25:56.087250646 +0000 UTC m=+38.780938955"
Jul 29 12:25:57.300464 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.300433    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-rmcgr"]
Jul 29 12:25:57.323967 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.323947    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-q7w6t"]
Jul 29 12:25:57.324073 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.324057    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rmcgr"
Jul 29 12:25:57.327356 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.327334    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 29 12:25:57.328505 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.328459    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 29 12:25:57.328505 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.328478    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 29 12:25:57.328667 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.328508    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 29 12:25:57.328667 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.328554    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-lwp2d\""
Jul 29 12:25:57.338014 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.337995    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-rmcgr"]
Jul 29 12:25:57.338097 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.338025    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-q7w6t"]
Jul 29 12:25:57.338136 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.338095    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-q7w6t"
Jul 29 12:25:57.340582 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.340560    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 29 12:25:57.340669 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.340559    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-7jh6p\""
Jul 29 12:25:57.340669 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.340588    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 29 12:25:57.340669 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.340625    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 29 12:25:57.340815 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.340801    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 29 12:25:57.399734 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.399716    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-vwjkc"]
Jul 29 12:25:57.421727 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.421685    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-d48j6"]
Jul 29 12:25:57.421802 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.421786    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-vwjkc"
Jul 29 12:25:57.423906 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.423892    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-dkx9n\""
Jul 29 12:25:57.441763 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.441746    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-g8xkd"]
Jul 29 12:25:57.441872 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.441855    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-d48j6"
Jul 29 12:25:57.444527 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.444510    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 29 12:25:57.444629 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.444527    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 29 12:25:57.444794 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.444780    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-dkqd9\""
Jul 29 12:25:57.467890 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.467871    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-vwjkc"]
Jul 29 12:25:57.467967 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.467895    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-d48j6"]
Jul 29 12:25:57.467967 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.467908    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-g8xkd"]
Jul 29 12:25:57.468029 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.467972    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-g8xkd"
Jul 29 12:25:57.470435 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.470417    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 29 12:25:57.470517 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.470436    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 29 12:25:57.470517 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.470476    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 29 12:25:57.470517 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.470495    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 29 12:25:57.470677 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.470581    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-ndxgj\""
Jul 29 12:25:57.474803 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.474784    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/2188f74f-2cb0-4a90-a994-ceb43417c819-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-rmcgr\" (UID: \"2188f74f-2cb0-4a90-a994-ceb43417c819\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rmcgr"
Jul 29 12:25:57.474873 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.474813    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xdp9\" (UniqueName: \"kubernetes.io/projected/74488661-38b3-4b24-b276-ae97f76f66fa-kube-api-access-2xdp9\") pod \"service-ca-operator-5d6c75f84f-q7w6t\" (UID: \"74488661-38b3-4b24-b276-ae97f76f66fa\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-q7w6t"
Jul 29 12:25:57.474873 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.474855    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2188f74f-2cb0-4a90-a994-ceb43417c819-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-rmcgr\" (UID: \"2188f74f-2cb0-4a90-a994-ceb43417c819\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rmcgr"
Jul 29 12:25:57.474949 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.474914    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/74488661-38b3-4b24-b276-ae97f76f66fa-config\") pod \"service-ca-operator-5d6c75f84f-q7w6t\" (UID: \"74488661-38b3-4b24-b276-ae97f76f66fa\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-q7w6t"
Jul 29 12:25:57.474949 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.474931    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86cdg\" (UniqueName: \"kubernetes.io/projected/2188f74f-2cb0-4a90-a994-ceb43417c819-kube-api-access-86cdg\") pod \"cluster-monitoring-operator-548b489cb8-rmcgr\" (UID: \"2188f74f-2cb0-4a90-a994-ceb43417c819\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rmcgr"
Jul 29 12:25:57.475016 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.474966    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/74488661-38b3-4b24-b276-ae97f76f66fa-serving-cert\") pod \"service-ca-operator-5d6c75f84f-q7w6t\" (UID: \"74488661-38b3-4b24-b276-ae97f76f66fa\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-q7w6t"
Jul 29 12:25:57.575449 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.575421    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/2188f74f-2cb0-4a90-a994-ceb43417c819-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-rmcgr\" (UID: \"2188f74f-2cb0-4a90-a994-ceb43417c819\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rmcgr"
Jul 29 12:25:57.575529 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.575455    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/271b0e50-2034-4105-b7d0-49e85d78c021-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-d48j6\" (UID: \"271b0e50-2034-4105-b7d0-49e85d78c021\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-d48j6"
Jul 29 12:25:57.575529 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.575475    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xdp9\" (UniqueName: \"kubernetes.io/projected/74488661-38b3-4b24-b276-ae97f76f66fa-kube-api-access-2xdp9\") pod \"service-ca-operator-5d6c75f84f-q7w6t\" (UID: \"74488661-38b3-4b24-b276-ae97f76f66fa\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-q7w6t"
Jul 29 12:25:57.575529 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.575492    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2188f74f-2cb0-4a90-a994-ceb43417c819-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-rmcgr\" (UID: \"2188f74f-2cb0-4a90-a994-ceb43417c819\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rmcgr"
Jul 29 12:25:57.575656 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:57.575626    2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 29 12:25:57.575656 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.575636    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c223418a-dbfe-4991-bc94-feebbfd40bfe-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-g8xkd\" (UID: \"c223418a-dbfe-4991-bc94-feebbfd40bfe\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-g8xkd"
Jul 29 12:25:57.575752 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.575724    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/74488661-38b3-4b24-b276-ae97f76f66fa-config\") pod \"service-ca-operator-5d6c75f84f-q7w6t\" (UID: \"74488661-38b3-4b24-b276-ae97f76f66fa\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-q7w6t"
Jul 29 12:25:57.575802 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:57.575757    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2188f74f-2cb0-4a90-a994-ceb43417c819-cluster-monitoring-operator-tls podName:2188f74f-2cb0-4a90-a994-ceb43417c819 nodeName:}" failed. No retries permitted until 2026-07-29 12:25:58.075738448 +0000 UTC m=+40.769426737 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2188f74f-2cb0-4a90-a994-ceb43417c819-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-rmcgr" (UID: "2188f74f-2cb0-4a90-a994-ceb43417c819") : secret "cluster-monitoring-operator-tls" not found
Jul 29 12:25:57.575802 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.575788    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86cdg\" (UniqueName: \"kubernetes.io/projected/2188f74f-2cb0-4a90-a994-ceb43417c819-kube-api-access-86cdg\") pod \"cluster-monitoring-operator-548b489cb8-rmcgr\" (UID: \"2188f74f-2cb0-4a90-a994-ceb43417c819\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rmcgr"
Jul 29 12:25:57.575908 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.575807    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/271b0e50-2034-4105-b7d0-49e85d78c021-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-d48j6\" (UID: \"271b0e50-2034-4105-b7d0-49e85d78c021\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-d48j6"
Jul 29 12:25:57.575908 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.575832    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c223418a-dbfe-4991-bc94-feebbfd40bfe-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-g8xkd\" (UID: \"c223418a-dbfe-4991-bc94-feebbfd40bfe\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-g8xkd"
Jul 29 12:25:57.575908 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.575851    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fwg5\" (UniqueName: \"kubernetes.io/projected/91d82511-f915-4220-a7e7-f476c1f22307-kube-api-access-8fwg5\") pod \"network-check-source-7d88cb55d8-vwjkc\" (UID: \"91d82511-f915-4220-a7e7-f476c1f22307\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-vwjkc"
Jul 29 12:25:57.576154 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.575906    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/74488661-38b3-4b24-b276-ae97f76f66fa-serving-cert\") pod \"service-ca-operator-5d6c75f84f-q7w6t\" (UID: \"74488661-38b3-4b24-b276-ae97f76f66fa\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-q7w6t"
Jul 29 12:25:57.576154 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.575937    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jp87\" (UniqueName: \"kubernetes.io/projected/c223418a-dbfe-4991-bc94-feebbfd40bfe-kube-api-access-4jp87\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-g8xkd\" (UID: \"c223418a-dbfe-4991-bc94-feebbfd40bfe\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-g8xkd"
Jul 29 12:25:57.576269 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.576174    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/2188f74f-2cb0-4a90-a994-ceb43417c819-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-rmcgr\" (UID: \"2188f74f-2cb0-4a90-a994-ceb43417c819\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rmcgr"
Jul 29 12:25:57.576315 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.576283    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/74488661-38b3-4b24-b276-ae97f76f66fa-config\") pod \"service-ca-operator-5d6c75f84f-q7w6t\" (UID: \"74488661-38b3-4b24-b276-ae97f76f66fa\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-q7w6t"
Jul 29 12:25:57.579341 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.579325    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/74488661-38b3-4b24-b276-ae97f76f66fa-serving-cert\") pod \"service-ca-operator-5d6c75f84f-q7w6t\" (UID: \"74488661-38b3-4b24-b276-ae97f76f66fa\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-q7w6t"
Jul 29 12:25:57.598077 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.598056    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-86cdg\" (UniqueName: \"kubernetes.io/projected/2188f74f-2cb0-4a90-a994-ceb43417c819-kube-api-access-86cdg\") pod \"cluster-monitoring-operator-548b489cb8-rmcgr\" (UID: \"2188f74f-2cb0-4a90-a994-ceb43417c819\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rmcgr"
Jul 29 12:25:57.602650 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.602623    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xdp9\" (UniqueName: \"kubernetes.io/projected/74488661-38b3-4b24-b276-ae97f76f66fa-kube-api-access-2xdp9\") pod \"service-ca-operator-5d6c75f84f-q7w6t\" (UID: \"74488661-38b3-4b24-b276-ae97f76f66fa\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-q7w6t"
Jul 29 12:25:57.645753 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.645732    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-q7w6t"
Jul 29 12:25:57.677008 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.676968    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/271b0e50-2034-4105-b7d0-49e85d78c021-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-d48j6\" (UID: \"271b0e50-2034-4105-b7d0-49e85d78c021\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-d48j6"
Jul 29 12:25:57.677071 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.677008    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c223418a-dbfe-4991-bc94-feebbfd40bfe-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-g8xkd\" (UID: \"c223418a-dbfe-4991-bc94-feebbfd40bfe\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-g8xkd"
Jul 29 12:25:57.677109 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.677073    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/271b0e50-2034-4105-b7d0-49e85d78c021-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-d48j6\" (UID: \"271b0e50-2034-4105-b7d0-49e85d78c021\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-d48j6"
Jul 29 12:25:57.677109 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.677102    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c223418a-dbfe-4991-bc94-feebbfd40bfe-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-g8xkd\" (UID: \"c223418a-dbfe-4991-bc94-feebbfd40bfe\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-g8xkd"
Jul 29 12:25:57.677182 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.677126    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fwg5\" (UniqueName: \"kubernetes.io/projected/91d82511-f915-4220-a7e7-f476c1f22307-kube-api-access-8fwg5\") pod \"network-check-source-7d88cb55d8-vwjkc\" (UID: \"91d82511-f915-4220-a7e7-f476c1f22307\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-vwjkc"
Jul 29 12:25:57.677214 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:57.677189    2567 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 29 12:25:57.677278 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:57.677256    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/271b0e50-2034-4105-b7d0-49e85d78c021-networking-console-plugin-cert podName:271b0e50-2034-4105-b7d0-49e85d78c021 nodeName:}" failed. No retries permitted until 2026-07-29 12:25:58.177224216 +0000 UTC m=+40.870912505 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/271b0e50-2034-4105-b7d0-49e85d78c021-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-d48j6" (UID: "271b0e50-2034-4105-b7d0-49e85d78c021") : secret "networking-console-plugin-cert" not found
Jul 29 12:25:57.677328 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.677304    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jp87\" (UniqueName: \"kubernetes.io/projected/c223418a-dbfe-4991-bc94-feebbfd40bfe-kube-api-access-4jp87\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-g8xkd\" (UID: \"c223418a-dbfe-4991-bc94-feebbfd40bfe\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-g8xkd"
Jul 29 12:25:57.677650 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.677635    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/271b0e50-2034-4105-b7d0-49e85d78c021-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-d48j6\" (UID: \"271b0e50-2034-4105-b7d0-49e85d78c021\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-d48j6"
Jul 29 12:25:57.677702 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.677652    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c223418a-dbfe-4991-bc94-feebbfd40bfe-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-g8xkd\" (UID: \"c223418a-dbfe-4991-bc94-feebbfd40bfe\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-g8xkd"
Jul 29 12:25:57.678811 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.678795    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c223418a-dbfe-4991-bc94-feebbfd40bfe-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-g8xkd\" (UID: \"c223418a-dbfe-4991-bc94-feebbfd40bfe\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-g8xkd"
Jul 29 12:25:57.698913 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.698890    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jp87\" (UniqueName: \"kubernetes.io/projected/c223418a-dbfe-4991-bc94-feebbfd40bfe-kube-api-access-4jp87\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-g8xkd\" (UID: \"c223418a-dbfe-4991-bc94-feebbfd40bfe\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-g8xkd"
Jul 29 12:25:57.699005 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.698989    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fwg5\" (UniqueName: \"kubernetes.io/projected/91d82511-f915-4220-a7e7-f476c1f22307-kube-api-access-8fwg5\") pod \"network-check-source-7d88cb55d8-vwjkc\" (UID: \"91d82511-f915-4220-a7e7-f476c1f22307\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-vwjkc"
Jul 29 12:25:57.730229 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.730206    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-vwjkc"
Jul 29 12:25:57.775695 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.775671    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-g8xkd"
Jul 29 12:25:57.794713 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.794654    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-q7w6t"]
Jul 29 12:25:57.797475 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:57.797450    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74488661_38b3_4b24_b276_ae97f76f66fa.slice/crio-05123caba97d02bcef4f6f6e65d9772b6e4b8a842c3a249b6f19b4591dee3416 WatchSource:0}: Error finding container 05123caba97d02bcef4f6f6e65d9772b6e4b8a842c3a249b6f19b4591dee3416: Status 404 returned error can't find the container with id 05123caba97d02bcef4f6f6e65d9772b6e4b8a842c3a249b6f19b4591dee3416
Jul 29 12:25:57.890804 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.890780    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-vwjkc"]
Jul 29 12:25:57.892903 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:57.892862    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91d82511_f915_4220_a7e7_f476c1f22307.slice/crio-222afc5ed0d29f2a92f0de9f401b18b8a085a48101ebe231d09fdf3b31ac5a77 WatchSource:0}: Error finding container 222afc5ed0d29f2a92f0de9f401b18b8a085a48101ebe231d09fdf3b31ac5a77: Status 404 returned error can't find the container with id 222afc5ed0d29f2a92f0de9f401b18b8a085a48101ebe231d09fdf3b31ac5a77
Jul 29 12:25:57.933423 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:57.933351    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-g8xkd"]
Jul 29 12:25:57.936293 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:25:57.936270    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc223418a_dbfe_4991_bc94_feebbfd40bfe.slice/crio-fb8459fb898d33ddd6588e5d226e3976aa25f151626c32326129a8e92ab3897e WatchSource:0}: Error finding container fb8459fb898d33ddd6588e5d226e3976aa25f151626c32326129a8e92ab3897e: Status 404 returned error can't find the container with id fb8459fb898d33ddd6588e5d226e3976aa25f151626c32326129a8e92ab3897e
Jul 29 12:25:58.076305 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:58.076280    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-g8xkd" event={"ID":"c223418a-dbfe-4991-bc94-feebbfd40bfe","Type":"ContainerStarted","Data":"fb8459fb898d33ddd6588e5d226e3976aa25f151626c32326129a8e92ab3897e"}
Jul 29 12:25:58.077265 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:58.077241    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-vwjkc" event={"ID":"91d82511-f915-4220-a7e7-f476c1f22307","Type":"ContainerStarted","Data":"222afc5ed0d29f2a92f0de9f401b18b8a085a48101ebe231d09fdf3b31ac5a77"}
Jul 29 12:25:58.078153 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:58.078134    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-q7w6t" event={"ID":"74488661-38b3-4b24-b276-ae97f76f66fa","Type":"ContainerStarted","Data":"05123caba97d02bcef4f6f6e65d9772b6e4b8a842c3a249b6f19b4591dee3416"}
Jul 29 12:25:58.080395 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:58.080378    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2188f74f-2cb0-4a90-a994-ceb43417c819-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-rmcgr\" (UID: \"2188f74f-2cb0-4a90-a994-ceb43417c819\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rmcgr"
Jul 29 12:25:58.080516 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:58.080504    2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 29 12:25:58.080571 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:58.080562    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2188f74f-2cb0-4a90-a994-ceb43417c819-cluster-monitoring-operator-tls podName:2188f74f-2cb0-4a90-a994-ceb43417c819 nodeName:}" failed. No retries permitted until 2026-07-29 12:25:59.080548011 +0000 UTC m=+41.774236300 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2188f74f-2cb0-4a90-a994-ceb43417c819-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-rmcgr" (UID: "2188f74f-2cb0-4a90-a994-ceb43417c819") : secret "cluster-monitoring-operator-tls" not found
Jul 29 12:25:58.181346 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:58.181327    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/271b0e50-2034-4105-b7d0-49e85d78c021-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-d48j6\" (UID: \"271b0e50-2034-4105-b7d0-49e85d78c021\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-d48j6"
Jul 29 12:25:58.181482 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:58.181467    2567 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 29 12:25:58.181534 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:58.181524    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/271b0e50-2034-4105-b7d0-49e85d78c021-networking-console-plugin-cert podName:271b0e50-2034-4105-b7d0-49e85d78c021 nodeName:}" failed. No retries permitted until 2026-07-29 12:25:59.181511936 +0000 UTC m=+41.875200227 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/271b0e50-2034-4105-b7d0-49e85d78c021-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-d48j6" (UID: "271b0e50-2034-4105-b7d0-49e85d78c021") : secret "networking-console-plugin-cert" not found
Jul 29 12:25:58.887877 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:58.887732    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dff975a8-8807-4dbe-9124-7c96c00a1949-registry-tls\") pod \"image-registry-7f46c77f5c-tnln6\" (UID: \"dff975a8-8807-4dbe-9124-7c96c00a1949\") " pod="openshift-image-registry/image-registry-7f46c77f5c-tnln6"
Jul 29 12:25:58.888231 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:58.887894    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 12:25:58.888231 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:58.887915    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f46c77f5c-tnln6: secret "image-registry-tls" not found
Jul 29 12:25:58.888231 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:58.887987    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dff975a8-8807-4dbe-9124-7c96c00a1949-registry-tls podName:dff975a8-8807-4dbe-9124-7c96c00a1949 nodeName:}" failed. No retries permitted until 2026-07-29 12:26:06.887969067 +0000 UTC m=+49.581657359 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/dff975a8-8807-4dbe-9124-7c96c00a1949-registry-tls") pod "image-registry-7f46c77f5c-tnln6" (UID: "dff975a8-8807-4dbe-9124-7c96c00a1949") : secret "image-registry-tls" not found
Jul 29 12:25:58.988137 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:58.988106    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e9572260-1f9a-4f9d-93db-1cc7f72dcc7a-metrics-tls\") pod \"dns-default-sdj8z\" (UID: \"e9572260-1f9a-4f9d-93db-1cc7f72dcc7a\") " pod="openshift-dns/dns-default-sdj8z"
Jul 29 12:25:58.988292 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:58.988279    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 12:25:58.988358 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:58.988346    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e9572260-1f9a-4f9d-93db-1cc7f72dcc7a-metrics-tls podName:e9572260-1f9a-4f9d-93db-1cc7f72dcc7a nodeName:}" failed. No retries permitted until 2026-07-29 12:26:06.988325372 +0000 UTC m=+49.682013670 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e9572260-1f9a-4f9d-93db-1cc7f72dcc7a-metrics-tls") pod "dns-default-sdj8z" (UID: "e9572260-1f9a-4f9d-93db-1cc7f72dcc7a") : secret "dns-default-metrics-tls" not found
Jul 29 12:25:59.088924 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:59.088841    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2188f74f-2cb0-4a90-a994-ceb43417c819-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-rmcgr\" (UID: \"2188f74f-2cb0-4a90-a994-ceb43417c819\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rmcgr"
Jul 29 12:25:59.089074 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:59.089045    2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 29 12:25:59.089178 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:59.089166    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2188f74f-2cb0-4a90-a994-ceb43417c819-cluster-monitoring-operator-tls podName:2188f74f-2cb0-4a90-a994-ceb43417c819 nodeName:}" failed. No retries permitted until 2026-07-29 12:26:01.089147207 +0000 UTC m=+43.782835510 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2188f74f-2cb0-4a90-a994-ceb43417c819-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-rmcgr" (UID: "2188f74f-2cb0-4a90-a994-ceb43417c819") : secret "cluster-monitoring-operator-tls" not found
Jul 29 12:25:59.190149 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:25:59.190071    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/271b0e50-2034-4105-b7d0-49e85d78c021-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-d48j6\" (UID: \"271b0e50-2034-4105-b7d0-49e85d78c021\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-d48j6"
Jul 29 12:25:59.190291 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:59.190253    2567 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 29 12:25:59.190344 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:25:59.190315    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/271b0e50-2034-4105-b7d0-49e85d78c021-networking-console-plugin-cert podName:271b0e50-2034-4105-b7d0-49e85d78c021 nodeName:}" failed. No retries permitted until 2026-07-29 12:26:01.190298223 +0000 UTC m=+43.883986515 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/271b0e50-2034-4105-b7d0-49e85d78c021-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-d48j6" (UID: "271b0e50-2034-4105-b7d0-49e85d78c021") : secret "networking-console-plugin-cert" not found
Jul 29 12:26:01.105319 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:01.105291    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2188f74f-2cb0-4a90-a994-ceb43417c819-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-rmcgr\" (UID: \"2188f74f-2cb0-4a90-a994-ceb43417c819\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rmcgr"
Jul 29 12:26:01.105701 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:26:01.105394    2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 29 12:26:01.105701 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:26:01.105459    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2188f74f-2cb0-4a90-a994-ceb43417c819-cluster-monitoring-operator-tls podName:2188f74f-2cb0-4a90-a994-ceb43417c819 nodeName:}" failed. No retries permitted until 2026-07-29 12:26:05.105445726 +0000 UTC m=+47.799134016 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2188f74f-2cb0-4a90-a994-ceb43417c819-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-rmcgr" (UID: "2188f74f-2cb0-4a90-a994-ceb43417c819") : secret "cluster-monitoring-operator-tls" not found
Jul 29 12:26:01.205626 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:01.205594    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/271b0e50-2034-4105-b7d0-49e85d78c021-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-d48j6\" (UID: \"271b0e50-2034-4105-b7d0-49e85d78c021\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-d48j6"
Jul 29 12:26:01.205775 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:26:01.205742    2567 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 29 12:26:01.205814 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:26:01.205806    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/271b0e50-2034-4105-b7d0-49e85d78c021-networking-console-plugin-cert podName:271b0e50-2034-4105-b7d0-49e85d78c021 nodeName:}" failed. No retries permitted until 2026-07-29 12:26:05.205791402 +0000 UTC m=+47.899479692 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/271b0e50-2034-4105-b7d0-49e85d78c021-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-d48j6" (UID: "271b0e50-2034-4105-b7d0-49e85d78c021") : secret "networking-console-plugin-cert" not found
Jul 29 12:26:03.093930 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:03.093892    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-g8xkd" event={"ID":"c223418a-dbfe-4991-bc94-feebbfd40bfe","Type":"ContainerStarted","Data":"12c7ccd3a046f76ff06e8d212664b2d46a0900d9a6ae18df7f808af7a0b2d192"}
Jul 29 12:26:03.095513 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:03.095487    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-vwjkc" event={"ID":"91d82511-f915-4220-a7e7-f476c1f22307","Type":"ContainerStarted","Data":"7d32bd2e69e0d5b90480d7e36aba648724cd05027a9c3cc17019cc7cfa7009c0"}
Jul 29 12:26:03.096978 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:03.096958    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-q7w6t" event={"ID":"74488661-38b3-4b24-b276-ae97f76f66fa","Type":"ContainerStarted","Data":"36f89f1f0fdf5afd8b4119a63c6eb915e66c18c28bcf22bacc999b4ed227228d"}
Jul 29 12:26:03.110573 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:03.110525    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-g8xkd" podStartSLOduration=1.7371930359999999 podStartE2EDuration="6.110506683s" totalImagesPullingTime="4.373313647s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:25:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:26:03.110017227 +0000 UTC m=+45.803705540" watchObservedRunningTime="2026-07-29 12:26:03.110506683 +0000 UTC m=+45.804194994"
Jul 29 12:26:03.125373 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:03.125329    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-vwjkc" podStartSLOduration=1.7137399169999998 podStartE2EDuration="6.125312143s" totalImagesPullingTime="4.411572226s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:25:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:26:03.124298436 +0000 UTC m=+45.817986747" watchObservedRunningTime="2026-07-29 12:26:03.125312143 +0000 UTC m=+45.819000455"
Jul 29 12:26:03.141849 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:03.141815    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-q7w6t" podStartSLOduration=1.635069002 podStartE2EDuration="6.141803621s" totalImagesPullingTime="4.506734619s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:25:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:26:03.14076203 +0000 UTC m=+45.834450367" watchObservedRunningTime="2026-07-29 12:26:03.141803621 +0000 UTC m=+45.835491977"
Jul 29 12:26:04.036049 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:04.036015    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-2q2mn"]
Jul 29 12:26:04.072731 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:04.072700    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2q2mn"]
Jul 29 12:26:04.072864 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:04.072819    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2q2mn"
Jul 29 12:26:04.076218 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:04.076187    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 29 12:26:04.076357 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:04.076227    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 29 12:26:04.076357 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:04.076232    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 29 12:26:04.077197 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:04.077181    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 29 12:26:04.077256 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:04.077196    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-lp4bb\""
Jul 29 12:26:04.130663 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:04.130633    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5edc73cd-3f85-4c3e-be4a-f311e46492a1-data-volume\") pod \"insights-runtime-extractor-2q2mn\" (UID: \"5edc73cd-3f85-4c3e-be4a-f311e46492a1\") " pod="openshift-insights/insights-runtime-extractor-2q2mn"
Jul 29 12:26:04.131007 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:04.130725    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5edc73cd-3f85-4c3e-be4a-f311e46492a1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2q2mn\" (UID: \"5edc73cd-3f85-4c3e-be4a-f311e46492a1\") " pod="openshift-insights/insights-runtime-extractor-2q2mn"
Jul 29 12:26:04.131007 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:04.130784    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5edc73cd-3f85-4c3e-be4a-f311e46492a1-crio-socket\") pod \"insights-runtime-extractor-2q2mn\" (UID: \"5edc73cd-3f85-4c3e-be4a-f311e46492a1\") " pod="openshift-insights/insights-runtime-extractor-2q2mn"
Jul 29 12:26:04.131007 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:04.130810    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5edc73cd-3f85-4c3e-be4a-f311e46492a1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2q2mn\" (UID: \"5edc73cd-3f85-4c3e-be4a-f311e46492a1\") " pod="openshift-insights/insights-runtime-extractor-2q2mn"
Jul 29 12:26:04.131118 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:04.131002    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tsnt\" (UniqueName: \"kubernetes.io/projected/5edc73cd-3f85-4c3e-be4a-f311e46492a1-kube-api-access-8tsnt\") pod \"insights-runtime-extractor-2q2mn\" (UID: \"5edc73cd-3f85-4c3e-be4a-f311e46492a1\") " pod="openshift-insights/insights-runtime-extractor-2q2mn"
Jul 29 12:26:04.231515 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:04.231484    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5edc73cd-3f85-4c3e-be4a-f311e46492a1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2q2mn\" (UID: \"5edc73cd-3f85-4c3e-be4a-f311e46492a1\") " pod="openshift-insights/insights-runtime-extractor-2q2mn"
Jul 29 12:26:04.231669 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:04.231519    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5edc73cd-3f85-4c3e-be4a-f311e46492a1-crio-socket\") pod \"insights-runtime-extractor-2q2mn\" (UID: \"5edc73cd-3f85-4c3e-be4a-f311e46492a1\") " pod="openshift-insights/insights-runtime-extractor-2q2mn"
Jul 29 12:26:04.231669 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:04.231545    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5edc73cd-3f85-4c3e-be4a-f311e46492a1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2q2mn\" (UID: \"5edc73cd-3f85-4c3e-be4a-f311e46492a1\") " pod="openshift-insights/insights-runtime-extractor-2q2mn"
Jul 29 12:26:04.231669 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:26:04.231603    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 12:26:04.231669 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:26:04.231615    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 12:26:04.231880 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:26:04.231683    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5edc73cd-3f85-4c3e-be4a-f311e46492a1-kube-rbac-proxy-cm podName:5edc73cd-3f85-4c3e-be4a-f311e46492a1 nodeName:}" failed. No retries permitted until 2026-07-29 12:26:04.731665792 +0000 UTC m=+47.425354081 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5edc73cd-3f85-4c3e-be4a-f311e46492a1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2q2mn" (UID: "5edc73cd-3f85-4c3e-be4a-f311e46492a1") : configmap "kube-rbac-proxy" not found
Jul 29 12:26:04.231880 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:04.231686    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5edc73cd-3f85-4c3e-be4a-f311e46492a1-crio-socket\") pod \"insights-runtime-extractor-2q2mn\" (UID: \"5edc73cd-3f85-4c3e-be4a-f311e46492a1\") " pod="openshift-insights/insights-runtime-extractor-2q2mn"
Jul 29 12:26:04.231880 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:26:04.231700    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5edc73cd-3f85-4c3e-be4a-f311e46492a1-insights-runtime-extractor-tls podName:5edc73cd-3f85-4c3e-be4a-f311e46492a1 nodeName:}" failed. No retries permitted until 2026-07-29 12:26:04.731691419 +0000 UTC m=+47.425379717 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5edc73cd-3f85-4c3e-be4a-f311e46492a1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2q2mn" (UID: "5edc73cd-3f85-4c3e-be4a-f311e46492a1") : secret "insights-runtime-extractor-tls" not found
Jul 29 12:26:04.231880 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:04.231734    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8tsnt\" (UniqueName: \"kubernetes.io/projected/5edc73cd-3f85-4c3e-be4a-f311e46492a1-kube-api-access-8tsnt\") pod \"insights-runtime-extractor-2q2mn\" (UID: \"5edc73cd-3f85-4c3e-be4a-f311e46492a1\") " pod="openshift-insights/insights-runtime-extractor-2q2mn"
Jul 29 12:26:04.231880 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:04.231793    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5edc73cd-3f85-4c3e-be4a-f311e46492a1-data-volume\") pod \"insights-runtime-extractor-2q2mn\" (UID: \"5edc73cd-3f85-4c3e-be4a-f311e46492a1\") " pod="openshift-insights/insights-runtime-extractor-2q2mn"
Jul 29 12:26:04.232074 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:04.232058    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5edc73cd-3f85-4c3e-be4a-f311e46492a1-data-volume\") pod \"insights-runtime-extractor-2q2mn\" (UID: \"5edc73cd-3f85-4c3e-be4a-f311e46492a1\") " pod="openshift-insights/insights-runtime-extractor-2q2mn"
Jul 29 12:26:04.245667 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:04.245621    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tsnt\" (UniqueName: \"kubernetes.io/projected/5edc73cd-3f85-4c3e-be4a-f311e46492a1-kube-api-access-8tsnt\") pod \"insights-runtime-extractor-2q2mn\" (UID: \"5edc73cd-3f85-4c3e-be4a-f311e46492a1\") " pod="openshift-insights/insights-runtime-extractor-2q2mn"
Jul 29 12:26:04.735966 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:04.735933    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5edc73cd-3f85-4c3e-be4a-f311e46492a1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2q2mn\" (UID: \"5edc73cd-3f85-4c3e-be4a-f311e46492a1\") " pod="openshift-insights/insights-runtime-extractor-2q2mn"
Jul 29 12:26:04.736147 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:04.735974    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5edc73cd-3f85-4c3e-be4a-f311e46492a1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2q2mn\" (UID: \"5edc73cd-3f85-4c3e-be4a-f311e46492a1\") " pod="openshift-insights/insights-runtime-extractor-2q2mn"
Jul 29 12:26:04.736147 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:26:04.736059    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 12:26:04.736147 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:26:04.736090    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 12:26:04.736311 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:26:04.736156    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5edc73cd-3f85-4c3e-be4a-f311e46492a1-insights-runtime-extractor-tls podName:5edc73cd-3f85-4c3e-be4a-f311e46492a1 nodeName:}" failed. No retries permitted until 2026-07-29 12:26:05.73614173 +0000 UTC m=+48.429830020 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5edc73cd-3f85-4c3e-be4a-f311e46492a1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2q2mn" (UID: "5edc73cd-3f85-4c3e-be4a-f311e46492a1") : secret "insights-runtime-extractor-tls" not found
Jul 29 12:26:04.736311 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:26:04.736179    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5edc73cd-3f85-4c3e-be4a-f311e46492a1-kube-rbac-proxy-cm podName:5edc73cd-3f85-4c3e-be4a-f311e46492a1 nodeName:}" failed. No retries permitted until 2026-07-29 12:26:05.736171938 +0000 UTC m=+48.429860230 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5edc73cd-3f85-4c3e-be4a-f311e46492a1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2q2mn" (UID: "5edc73cd-3f85-4c3e-be4a-f311e46492a1") : configmap "kube-rbac-proxy" not found
Jul 29 12:26:05.139587 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:05.139555    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2188f74f-2cb0-4a90-a994-ceb43417c819-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-rmcgr\" (UID: \"2188f74f-2cb0-4a90-a994-ceb43417c819\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rmcgr"
Jul 29 12:26:05.139952 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:26:05.139690    2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 29 12:26:05.139952 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:26:05.139749    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2188f74f-2cb0-4a90-a994-ceb43417c819-cluster-monitoring-operator-tls podName:2188f74f-2cb0-4a90-a994-ceb43417c819 nodeName:}" failed. No retries permitted until 2026-07-29 12:26:13.139735049 +0000 UTC m=+55.833423339 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2188f74f-2cb0-4a90-a994-ceb43417c819-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-rmcgr" (UID: "2188f74f-2cb0-4a90-a994-ceb43417c819") : secret "cluster-monitoring-operator-tls" not found
Jul 29 12:26:05.240432 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:05.240384    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/271b0e50-2034-4105-b7d0-49e85d78c021-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-d48j6\" (UID: \"271b0e50-2034-4105-b7d0-49e85d78c021\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-d48j6"
Jul 29 12:26:05.240582 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:26:05.240525    2567 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 29 12:26:05.240620 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:26:05.240587    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/271b0e50-2034-4105-b7d0-49e85d78c021-networking-console-plugin-cert podName:271b0e50-2034-4105-b7d0-49e85d78c021 nodeName:}" failed. No retries permitted until 2026-07-29 12:26:13.240573334 +0000 UTC m=+55.934261628 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/271b0e50-2034-4105-b7d0-49e85d78c021-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-d48j6" (UID: "271b0e50-2034-4105-b7d0-49e85d78c021") : secret "networking-console-plugin-cert" not found
Jul 29 12:26:05.745124 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:05.745088    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5edc73cd-3f85-4c3e-be4a-f311e46492a1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2q2mn\" (UID: \"5edc73cd-3f85-4c3e-be4a-f311e46492a1\") " pod="openshift-insights/insights-runtime-extractor-2q2mn"
Jul 29 12:26:05.745295 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:05.745129    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5edc73cd-3f85-4c3e-be4a-f311e46492a1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2q2mn\" (UID: \"5edc73cd-3f85-4c3e-be4a-f311e46492a1\") " pod="openshift-insights/insights-runtime-extractor-2q2mn"
Jul 29 12:26:05.745295 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:26:05.745233    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 12:26:05.745428 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:26:05.745298    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5edc73cd-3f85-4c3e-be4a-f311e46492a1-insights-runtime-extractor-tls podName:5edc73cd-3f85-4c3e-be4a-f311e46492a1 nodeName:}" failed. No retries permitted until 2026-07-29 12:26:07.74528406 +0000 UTC m=+50.438972353 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5edc73cd-3f85-4c3e-be4a-f311e46492a1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2q2mn" (UID: "5edc73cd-3f85-4c3e-be4a-f311e46492a1") : secret "insights-runtime-extractor-tls" not found
Jul 29 12:26:05.745428 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:26:05.745238    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 12:26:05.745519 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:26:05.745445    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5edc73cd-3f85-4c3e-be4a-f311e46492a1-kube-rbac-proxy-cm podName:5edc73cd-3f85-4c3e-be4a-f311e46492a1 nodeName:}" failed. No retries permitted until 2026-07-29 12:26:07.74542347 +0000 UTC m=+50.439111764 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5edc73cd-3f85-4c3e-be4a-f311e46492a1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2q2mn" (UID: "5edc73cd-3f85-4c3e-be4a-f311e46492a1") : configmap "kube-rbac-proxy" not found
Jul 29 12:26:05.810186 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:05.810155    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8rddq_4bdec0f6-396e-4392-9e8d-e2bad0289b8f/dns-node-resolver/0.log"
Jul 29 12:26:06.955340 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:06.955309    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dff975a8-8807-4dbe-9124-7c96c00a1949-registry-tls\") pod \"image-registry-7f46c77f5c-tnln6\" (UID: \"dff975a8-8807-4dbe-9124-7c96c00a1949\") " pod="openshift-image-registry/image-registry-7f46c77f5c-tnln6"
Jul 29 12:26:06.955752 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:26:06.955494    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 12:26:06.955752 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:26:06.955512    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f46c77f5c-tnln6: secret "image-registry-tls" not found
Jul 29 12:26:06.955752 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:26:06.955573    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dff975a8-8807-4dbe-9124-7c96c00a1949-registry-tls podName:dff975a8-8807-4dbe-9124-7c96c00a1949 nodeName:}" failed. No retries permitted until 2026-07-29 12:26:22.955558441 +0000 UTC m=+65.649246730 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/dff975a8-8807-4dbe-9124-7c96c00a1949-registry-tls") pod "image-registry-7f46c77f5c-tnln6" (UID: "dff975a8-8807-4dbe-9124-7c96c00a1949") : secret "image-registry-tls" not found
Jul 29 12:26:07.010525 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:07.010503    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-clmgr_caf950ed-dbb9-4b7d-a802-062882ff4899/node-ca/0.log"
Jul 29 12:26:07.056472 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:07.056443    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e9572260-1f9a-4f9d-93db-1cc7f72dcc7a-metrics-tls\") pod \"dns-default-sdj8z\" (UID: \"e9572260-1f9a-4f9d-93db-1cc7f72dcc7a\") " pod="openshift-dns/dns-default-sdj8z"
Jul 29 12:26:07.056610 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:26:07.056558    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 12:26:07.056678 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:26:07.056620    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e9572260-1f9a-4f9d-93db-1cc7f72dcc7a-metrics-tls podName:e9572260-1f9a-4f9d-93db-1cc7f72dcc7a nodeName:}" failed. No retries permitted until 2026-07-29 12:26:23.056602282 +0000 UTC m=+65.750290584 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e9572260-1f9a-4f9d-93db-1cc7f72dcc7a-metrics-tls") pod "dns-default-sdj8z" (UID: "e9572260-1f9a-4f9d-93db-1cc7f72dcc7a") : secret "dns-default-metrics-tls" not found
Jul 29 12:26:07.763265 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:07.763236    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5edc73cd-3f85-4c3e-be4a-f311e46492a1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2q2mn\" (UID: \"5edc73cd-3f85-4c3e-be4a-f311e46492a1\") " pod="openshift-insights/insights-runtime-extractor-2q2mn"
Jul 29 12:26:07.763265 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:07.763270    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5edc73cd-3f85-4c3e-be4a-f311e46492a1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2q2mn\" (UID: \"5edc73cd-3f85-4c3e-be4a-f311e46492a1\") " pod="openshift-insights/insights-runtime-extractor-2q2mn"
Jul 29 12:26:07.763491 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:26:07.763387    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 12:26:07.763491 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:26:07.763476    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5edc73cd-3f85-4c3e-be4a-f311e46492a1-insights-runtime-extractor-tls podName:5edc73cd-3f85-4c3e-be4a-f311e46492a1 nodeName:}" failed. No retries permitted until 2026-07-29 12:26:11.763461608 +0000 UTC m=+54.457149898 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5edc73cd-3f85-4c3e-be4a-f311e46492a1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2q2mn" (UID: "5edc73cd-3f85-4c3e-be4a-f311e46492a1") : secret "insights-runtime-extractor-tls" not found
Jul 29 12:26:07.763731 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:07.763715    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5edc73cd-3f85-4c3e-be4a-f311e46492a1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2q2mn\" (UID: \"5edc73cd-3f85-4c3e-be4a-f311e46492a1\") " pod="openshift-insights/insights-runtime-extractor-2q2mn"
Jul 29 12:26:08.012796 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:08.012765    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-w8d7p_5e7bd908-6123-4f1e-a5ef-109f1b3b0069/serve-healthcheck-canary/0.log"
Jul 29 12:26:11.796688 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:11.796647    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5edc73cd-3f85-4c3e-be4a-f311e46492a1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2q2mn\" (UID: \"5edc73cd-3f85-4c3e-be4a-f311e46492a1\") " pod="openshift-insights/insights-runtime-extractor-2q2mn"
Jul 29 12:26:11.797146 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:26:11.796798    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 12:26:11.797146 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:26:11.796857    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5edc73cd-3f85-4c3e-be4a-f311e46492a1-insights-runtime-extractor-tls podName:5edc73cd-3f85-4c3e-be4a-f311e46492a1 nodeName:}" failed. No retries permitted until 2026-07-29 12:26:19.796843736 +0000 UTC m=+62.490532025 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5edc73cd-3f85-4c3e-be4a-f311e46492a1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2q2mn" (UID: "5edc73cd-3f85-4c3e-be4a-f311e46492a1") : secret "insights-runtime-extractor-tls" not found
Jul 29 12:26:13.208122 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:13.208077    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2188f74f-2cb0-4a90-a994-ceb43417c819-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-rmcgr\" (UID: \"2188f74f-2cb0-4a90-a994-ceb43417c819\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rmcgr"
Jul 29 12:26:13.208550 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:26:13.208227    2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 29 12:26:13.208550 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:26:13.208293    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2188f74f-2cb0-4a90-a994-ceb43417c819-cluster-monitoring-operator-tls podName:2188f74f-2cb0-4a90-a994-ceb43417c819 nodeName:}" failed. No retries permitted until 2026-07-29 12:26:29.208274959 +0000 UTC m=+71.901963283 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2188f74f-2cb0-4a90-a994-ceb43417c819-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-rmcgr" (UID: "2188f74f-2cb0-4a90-a994-ceb43417c819") : secret "cluster-monitoring-operator-tls" not found
Jul 29 12:26:13.308879 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:13.308844    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/271b0e50-2034-4105-b7d0-49e85d78c021-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-d48j6\" (UID: \"271b0e50-2034-4105-b7d0-49e85d78c021\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-d48j6"
Jul 29 12:26:13.309033 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:26:13.308964    2567 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 29 12:26:13.309098 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:26:13.309036    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/271b0e50-2034-4105-b7d0-49e85d78c021-networking-console-plugin-cert podName:271b0e50-2034-4105-b7d0-49e85d78c021 nodeName:}" failed. No retries permitted until 2026-07-29 12:26:29.309016589 +0000 UTC m=+72.002704879 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/271b0e50-2034-4105-b7d0-49e85d78c021-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-d48j6" (UID: "271b0e50-2034-4105-b7d0-49e85d78c021") : secret "networking-console-plugin-cert" not found
Jul 29 12:26:19.052284 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:19.052256    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kj82f"
Jul 29 12:26:19.859659 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:19.859618    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5edc73cd-3f85-4c3e-be4a-f311e46492a1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2q2mn\" (UID: \"5edc73cd-3f85-4c3e-be4a-f311e46492a1\") " pod="openshift-insights/insights-runtime-extractor-2q2mn"
Jul 29 12:26:19.861841 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:19.861809    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5edc73cd-3f85-4c3e-be4a-f311e46492a1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2q2mn\" (UID: \"5edc73cd-3f85-4c3e-be4a-f311e46492a1\") " pod="openshift-insights/insights-runtime-extractor-2q2mn"
Jul 29 12:26:19.985014 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:19.984986    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-lp4bb\""
Jul 29 12:26:19.992851 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:19.992828    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2q2mn"
Jul 29 12:26:20.124365 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:20.124294    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2q2mn"]
Jul 29 12:26:20.128204 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:26:20.128178    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5edc73cd_3f85_4c3e_be4a_f311e46492a1.slice/crio-ca6a863027540bc8bab0b1252dea27d7ca5ff1d70ed27dfee78fb2aecbe51cdc WatchSource:0}: Error finding container ca6a863027540bc8bab0b1252dea27d7ca5ff1d70ed27dfee78fb2aecbe51cdc: Status 404 returned error can't find the container with id ca6a863027540bc8bab0b1252dea27d7ca5ff1d70ed27dfee78fb2aecbe51cdc
Jul 29 12:26:20.139451 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:20.139427    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2q2mn" event={"ID":"5edc73cd-3f85-4c3e-be4a-f311e46492a1","Type":"ContainerStarted","Data":"ca6a863027540bc8bab0b1252dea27d7ca5ff1d70ed27dfee78fb2aecbe51cdc"}
Jul 29 12:26:22.992028 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:22.991989    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dff975a8-8807-4dbe-9124-7c96c00a1949-registry-tls\") pod \"image-registry-7f46c77f5c-tnln6\" (UID: \"dff975a8-8807-4dbe-9124-7c96c00a1949\") " pod="openshift-image-registry/image-registry-7f46c77f5c-tnln6"
Jul 29 12:26:22.994564 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:22.994540    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dff975a8-8807-4dbe-9124-7c96c00a1949-registry-tls\") pod \"image-registry-7f46c77f5c-tnln6\" (UID: \"dff975a8-8807-4dbe-9124-7c96c00a1949\") " pod="openshift-image-registry/image-registry-7f46c77f5c-tnln6"
Jul 29 12:26:23.092777 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:23.092743    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e9572260-1f9a-4f9d-93db-1cc7f72dcc7a-metrics-tls\") pod \"dns-default-sdj8z\" (UID: \"e9572260-1f9a-4f9d-93db-1cc7f72dcc7a\") " pod="openshift-dns/dns-default-sdj8z"
Jul 29 12:26:23.095385 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:23.095358    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e9572260-1f9a-4f9d-93db-1cc7f72dcc7a-metrics-tls\") pod \"dns-default-sdj8z\" (UID: \"e9572260-1f9a-4f9d-93db-1cc7f72dcc7a\") " pod="openshift-dns/dns-default-sdj8z"
Jul 29 12:26:23.258123 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:23.258053    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-bggs6\""
Jul 29 12:26:23.266210 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:23.266183    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f46c77f5c-tnln6"
Jul 29 12:26:23.279879 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:23.279852    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-zrh5t\""
Jul 29 12:26:23.287755 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:23.287732    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-sdj8z"
Jul 29 12:26:23.597780 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:23.597738    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1635829e-9a40-4521-8430-88adbb07a97a-original-pull-secret\") pod \"global-pull-secret-syncer-w97vv\" (UID: \"1635829e-9a40-4521-8430-88adbb07a97a\") " pod="kube-system/global-pull-secret-syncer-w97vv"
Jul 29 12:26:23.597967 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:23.597806    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68d50be9-1461-43d4-811e-74ff3fed1a75-metrics-certs\") pod \"network-metrics-daemon-qkdl7\" (UID: \"68d50be9-1461-43d4-811e-74ff3fed1a75\") " pod="openshift-multus/network-metrics-daemon-qkdl7"
Jul 29 12:26:23.600552 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:23.600526    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 29 12:26:23.600700 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:23.600585    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 29 12:26:23.611106 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:23.611085    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68d50be9-1461-43d4-811e-74ff3fed1a75-metrics-certs\") pod \"network-metrics-daemon-qkdl7\" (UID: \"68d50be9-1461-43d4-811e-74ff3fed1a75\") " pod="openshift-multus/network-metrics-daemon-qkdl7"
Jul 29 12:26:23.611225 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:23.611127    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1635829e-9a40-4521-8430-88adbb07a97a-original-pull-secret\") pod \"global-pull-secret-syncer-w97vv\" (UID: \"1635829e-9a40-4521-8430-88adbb07a97a\") " pod="kube-system/global-pull-secret-syncer-w97vv"
Jul 29 12:26:23.734548 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:23.734517    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-p4f75\""
Jul 29 12:26:23.742866 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:23.742839    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qkdl7"
Jul 29 12:26:23.799728 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:23.799691    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnsg6\" (UniqueName: \"kubernetes.io/projected/8e7279c1-35b7-4f54-8b02-fdac281ef757-kube-api-access-fnsg6\") pod \"network-check-target-r4hjk\" (UID: \"8e7279c1-35b7-4f54-8b02-fdac281ef757\") " pod="openshift-network-diagnostics/network-check-target-r4hjk"
Jul 29 12:26:23.802353 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:23.802329    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnsg6\" (UniqueName: \"kubernetes.io/projected/8e7279c1-35b7-4f54-8b02-fdac281ef757-kube-api-access-fnsg6\") pod \"network-check-target-r4hjk\" (UID: \"8e7279c1-35b7-4f54-8b02-fdac281ef757\") " pod="openshift-network-diagnostics/network-check-target-r4hjk"
Jul 29 12:26:23.826044 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:23.826019    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w97vv"
Jul 29 12:26:24.029505 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:24.029478    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hdxp9\""
Jul 29 12:26:24.037754 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:24.037735    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r4hjk"
Jul 29 12:26:25.175586 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:25.175556    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-qkdl7"]
Jul 29 12:26:25.339741 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:26:25.339708    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68d50be9_1461_43d4_811e_74ff3fed1a75.slice/crio-35ccbb4fbab0be0aac572718fd08143dd8cd734d3237a8f2224ab9927bc83110 WatchSource:0}: Error finding container 35ccbb4fbab0be0aac572718fd08143dd8cd734d3237a8f2224ab9927bc83110: Status 404 returned error can't find the container with id 35ccbb4fbab0be0aac572718fd08143dd8cd734d3237a8f2224ab9927bc83110
Jul 29 12:26:25.403612 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:25.403466    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-r4hjk"]
Jul 29 12:26:25.404893 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:25.404810    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-w97vv"]
Jul 29 12:26:25.405957 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:25.405933    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-sdj8z"]
Jul 29 12:26:25.406746 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:26:25.406482    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1635829e_9a40_4521_8430_88adbb07a97a.slice/crio-56b57ea36928ca10e73c3aa05f342cc05f898bf471ad643010c0511c29cedbda WatchSource:0}: Error finding container 56b57ea36928ca10e73c3aa05f342cc05f898bf471ad643010c0511c29cedbda: Status 404 returned error can't find the container with id 56b57ea36928ca10e73c3aa05f342cc05f898bf471ad643010c0511c29cedbda
Jul 29 12:26:25.407047 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:25.407017    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7f46c77f5c-tnln6"]
Jul 29 12:26:25.407104 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:26:25.407055    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e7279c1_35b7_4f54_8b02_fdac281ef757.slice/crio-b26205dee7fe853be2d67bed6db62064da9135590334c2766038f263db07e705 WatchSource:0}: Error finding container b26205dee7fe853be2d67bed6db62064da9135590334c2766038f263db07e705: Status 404 returned error can't find the container with id b26205dee7fe853be2d67bed6db62064da9135590334c2766038f263db07e705
Jul 29 12:26:25.407864 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:26:25.407808    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9572260_1f9a_4f9d_93db_1cc7f72dcc7a.slice/crio-4ae84af3ca1401c5bd300db44fa02e51cccf2891dbf0c2c66c414edac30bd359 WatchSource:0}: Error finding container 4ae84af3ca1401c5bd300db44fa02e51cccf2891dbf0c2c66c414edac30bd359: Status 404 returned error can't find the container with id 4ae84af3ca1401c5bd300db44fa02e51cccf2891dbf0c2c66c414edac30bd359
Jul 29 12:26:25.408611 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:26:25.408578    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddff975a8_8807_4dbe_9124_7c96c00a1949.slice/crio-4e7f66dab42d4865a658cfc2fc77aa67486d5bb205266a92ffe72c7ae5ce537b WatchSource:0}: Error finding container 4e7f66dab42d4865a658cfc2fc77aa67486d5bb205266a92ffe72c7ae5ce537b: Status 404 returned error can't find the container with id 4e7f66dab42d4865a658cfc2fc77aa67486d5bb205266a92ffe72c7ae5ce537b
Jul 29 12:26:26.166340 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:26.166297    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7f46c77f5c-tnln6" event={"ID":"dff975a8-8807-4dbe-9124-7c96c00a1949","Type":"ContainerStarted","Data":"9467c35e85784519cf3bde26dda4711714434113d15345d5500dbe5d83644197"}
Jul 29 12:26:26.166532 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:26.166361    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7f46c77f5c-tnln6" event={"ID":"dff975a8-8807-4dbe-9124-7c96c00a1949","Type":"ContainerStarted","Data":"4e7f66dab42d4865a658cfc2fc77aa67486d5bb205266a92ffe72c7ae5ce537b"}
Jul 29 12:26:26.166687 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:26.166665    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7f46c77f5c-tnln6"
Jul 29 12:26:26.169295 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:26.169234    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-w97vv" event={"ID":"1635829e-9a40-4521-8430-88adbb07a97a","Type":"ContainerStarted","Data":"56b57ea36928ca10e73c3aa05f342cc05f898bf471ad643010c0511c29cedbda"}
Jul 29 12:26:26.173878 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:26.173109    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-r4hjk" event={"ID":"8e7279c1-35b7-4f54-8b02-fdac281ef757","Type":"ContainerStarted","Data":"3c51c111b81d74dac8c9d92db84dc8fe5489e1ce9d8e2145d7df3a7f9573b640"}
Jul 29 12:26:26.173878 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:26.173184    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-r4hjk" event={"ID":"8e7279c1-35b7-4f54-8b02-fdac281ef757","Type":"ContainerStarted","Data":"b26205dee7fe853be2d67bed6db62064da9135590334c2766038f263db07e705"}
Jul 29 12:26:26.173878 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:26.173830    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-r4hjk"
Jul 29 12:26:26.175386 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:26.175361    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-sdj8z" event={"ID":"e9572260-1f9a-4f9d-93db-1cc7f72dcc7a","Type":"ContainerStarted","Data":"4ae84af3ca1401c5bd300db44fa02e51cccf2891dbf0c2c66c414edac30bd359"}
Jul 29 12:26:26.177093 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:26.177017    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qkdl7" event={"ID":"68d50be9-1461-43d4-811e-74ff3fed1a75","Type":"ContainerStarted","Data":"35ccbb4fbab0be0aac572718fd08143dd8cd734d3237a8f2224ab9927bc83110"}
Jul 29 12:26:26.179647 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:26.179626    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2q2mn" event={"ID":"5edc73cd-3f85-4c3e-be4a-f311e46492a1","Type":"ContainerStarted","Data":"8e8105b8dfb99978dd7cf2db01ceaf9704d85aeb3e903994c968688c8689f8a4"}
Jul 29 12:26:26.203593 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:26.202732    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7f46c77f5c-tnln6" podStartSLOduration=68.202714371 podStartE2EDuration="1m8.202714371s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:25:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:26:26.201333407 +0000 UTC m=+68.895021719" watchObservedRunningTime="2026-07-29 12:26:26.202714371 +0000 UTC m=+68.896402683"
Jul 29 12:26:26.239334 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:26.238529    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-r4hjk" podStartSLOduration=68.238510788 podStartE2EDuration="1m8.238510788s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:25:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:26:26.235981783 +0000 UTC m=+68.929670108" watchObservedRunningTime="2026-07-29 12:26:26.238510788 +0000 UTC m=+68.932199300"
Jul 29 12:26:27.186308 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:27.185939    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qkdl7" event={"ID":"68d50be9-1461-43d4-811e-74ff3fed1a75","Type":"ContainerStarted","Data":"f632ff3b0ba6f1a73435dab6d787b1ed98a026dd18b351a21bba12ab9f04c9a1"}
Jul 29 12:26:27.186308 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:27.186001    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qkdl7" event={"ID":"68d50be9-1461-43d4-811e-74ff3fed1a75","Type":"ContainerStarted","Data":"f6e6b873a1c3537387ae2ea5d14ae951b4d7e853e1fc20fee0d899e8f856f47c"}
Jul 29 12:26:27.206354 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:27.206193    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-qkdl7" podStartSLOduration=68.796408971 podStartE2EDuration="1m10.206172367s" totalImagesPullingTime="1.409763396s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:25:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:26:27.204576703 +0000 UTC m=+69.898265048" watchObservedRunningTime="2026-07-29 12:26:27.206172367 +0000 UTC m=+69.899860680"
Jul 29 12:26:28.994347 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:28.994309    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-68cfb779b6-94kd7"]
Jul 29 12:26:28.996701 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:28.996681    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68cfb779b6-94kd7"
Jul 29 12:26:29.001543 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:29.001524    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 29 12:26:29.014730 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:29.013494    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-68cfb779b6-94kd7"]
Jul 29 12:26:29.014730 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:29.013679    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 29 12:26:29.014730 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:29.013909    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 29 12:26:29.014730 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:29.014110    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 29 12:26:29.041704 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:29.041668    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/b283c8d3-3932-4e72-9f6d-939b49996766-klusterlet-config\") pod \"klusterlet-addon-workmgr-68cfb779b6-94kd7\" (UID: \"b283c8d3-3932-4e72-9f6d-939b49996766\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68cfb779b6-94kd7"
Jul 29 12:26:29.041832 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:29.041793    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b283c8d3-3932-4e72-9f6d-939b49996766-tmp\") pod \"klusterlet-addon-workmgr-68cfb779b6-94kd7\" (UID: \"b283c8d3-3932-4e72-9f6d-939b49996766\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68cfb779b6-94kd7"
Jul 29 12:26:29.041879 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:29.041831    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xq9dr\" (UniqueName: \"kubernetes.io/projected/b283c8d3-3932-4e72-9f6d-939b49996766-kube-api-access-xq9dr\") pod \"klusterlet-addon-workmgr-68cfb779b6-94kd7\" (UID: \"b283c8d3-3932-4e72-9f6d-939b49996766\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68cfb779b6-94kd7"
Jul 29 12:26:29.080287 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:29.080260    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-7c97dc4555-lljkn"]
Jul 29 12:26:29.083961 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:29.083943    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-lljkn"
Jul 29 12:26:29.087718 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:29.087697    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-lxj2v\""
Jul 29 12:26:29.088105 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:29.088086    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 29 12:26:29.088205 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:29.088091    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 29 12:26:29.094246 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:29.094194    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-lljkn"]
Jul 29 12:26:29.142427 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:29.142387    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/b283c8d3-3932-4e72-9f6d-939b49996766-klusterlet-config\") pod \"klusterlet-addon-workmgr-68cfb779b6-94kd7\" (UID: \"b283c8d3-3932-4e72-9f6d-939b49996766\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68cfb779b6-94kd7"
Jul 29 12:26:29.142568 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:29.142481    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpm42\" (UniqueName: \"kubernetes.io/projected/8ebccfd1-f963-40fb-a3f4-ddcb9b3cf8d6-kube-api-access-dpm42\") pod \"downloads-7c97dc4555-lljkn\" (UID: \"8ebccfd1-f963-40fb-a3f4-ddcb9b3cf8d6\") " pod="openshift-console/downloads-7c97dc4555-lljkn"
Jul 29 12:26:29.142568 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:29.142506    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b283c8d3-3932-4e72-9f6d-939b49996766-tmp\") pod \"klusterlet-addon-workmgr-68cfb779b6-94kd7\" (UID: \"b283c8d3-3932-4e72-9f6d-939b49996766\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68cfb779b6-94kd7"
Jul 29 12:26:29.142568 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:29.142539    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xq9dr\" (UniqueName: \"kubernetes.io/projected/b283c8d3-3932-4e72-9f6d-939b49996766-kube-api-access-xq9dr\") pod \"klusterlet-addon-workmgr-68cfb779b6-94kd7\" (UID: \"b283c8d3-3932-4e72-9f6d-939b49996766\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68cfb779b6-94kd7"
Jul 29 12:26:29.142989 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:29.142967    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b283c8d3-3932-4e72-9f6d-939b49996766-tmp\") pod \"klusterlet-addon-workmgr-68cfb779b6-94kd7\" (UID: \"b283c8d3-3932-4e72-9f6d-939b49996766\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68cfb779b6-94kd7"
Jul 29 12:26:29.145259 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:29.145240    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/b283c8d3-3932-4e72-9f6d-939b49996766-klusterlet-config\") pod \"klusterlet-addon-workmgr-68cfb779b6-94kd7\" (UID: \"b283c8d3-3932-4e72-9f6d-939b49996766\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68cfb779b6-94kd7"
Jul 29 12:26:29.155987 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:29.155958    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xq9dr\" (UniqueName: \"kubernetes.io/projected/b283c8d3-3932-4e72-9f6d-939b49996766-kube-api-access-xq9dr\") pod \"klusterlet-addon-workmgr-68cfb779b6-94kd7\" (UID: \"b283c8d3-3932-4e72-9f6d-939b49996766\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68cfb779b6-94kd7"
Jul 29 12:26:29.243687 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:29.243658    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2188f74f-2cb0-4a90-a994-ceb43417c819-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-rmcgr\" (UID: \"2188f74f-2cb0-4a90-a994-ceb43417c819\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rmcgr"
Jul 29 12:26:29.243817 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:29.243712    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpm42\" (UniqueName: \"kubernetes.io/projected/8ebccfd1-f963-40fb-a3f4-ddcb9b3cf8d6-kube-api-access-dpm42\") pod \"downloads-7c97dc4555-lljkn\" (UID: \"8ebccfd1-f963-40fb-a3f4-ddcb9b3cf8d6\") " pod="openshift-console/downloads-7c97dc4555-lljkn"
Jul 29 12:26:29.246194 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:29.246132    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2188f74f-2cb0-4a90-a994-ceb43417c819-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-rmcgr\" (UID: \"2188f74f-2cb0-4a90-a994-ceb43417c819\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rmcgr"
Jul 29 12:26:29.263095 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:29.263066    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpm42\" (UniqueName: \"kubernetes.io/projected/8ebccfd1-f963-40fb-a3f4-ddcb9b3cf8d6-kube-api-access-dpm42\") pod \"downloads-7c97dc4555-lljkn\" (UID: \"8ebccfd1-f963-40fb-a3f4-ddcb9b3cf8d6\") " pod="openshift-console/downloads-7c97dc4555-lljkn"
Jul 29 12:26:29.312059 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:29.312036    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68cfb779b6-94kd7"
Jul 29 12:26:29.345003 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:29.344976    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/271b0e50-2034-4105-b7d0-49e85d78c021-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-d48j6\" (UID: \"271b0e50-2034-4105-b7d0-49e85d78c021\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-d48j6"
Jul 29 12:26:29.347381 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:29.347359    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/271b0e50-2034-4105-b7d0-49e85d78c021-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-d48j6\" (UID: \"271b0e50-2034-4105-b7d0-49e85d78c021\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-d48j6"
Jul 29 12:26:29.394798 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:29.394778    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-lljkn"
Jul 29 12:26:29.434869 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:29.434848    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-lwp2d\""
Jul 29 12:26:29.442946 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:29.442929    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rmcgr"
Jul 29 12:26:29.552034 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:29.552005    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-dkqd9\""
Jul 29 12:26:29.560215 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:29.560198    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-d48j6"
Jul 29 12:26:31.197592 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:31.197513    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-lljkn"]
Jul 29 12:26:31.213150 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:31.213126    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-68cfb779b6-94kd7"]
Jul 29 12:26:31.215044 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:26:31.215020    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb283c8d3_3932_4e72_9f6d_939b49996766.slice/crio-8a1688c447428fce7bcae5578aa570fc6605ff92698948c0af0af5021d6363d1 WatchSource:0}: Error finding container 8a1688c447428fce7bcae5578aa570fc6605ff92698948c0af0af5021d6363d1: Status 404 returned error can't find the container with id 8a1688c447428fce7bcae5578aa570fc6605ff92698948c0af0af5021d6363d1
Jul 29 12:26:31.234697 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:31.234063    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-d48j6"]
Jul 29 12:26:31.237740 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:26:31.236649    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod271b0e50_2034_4105_b7d0_49e85d78c021.slice/crio-f876e8ccb5f4520d5f2eca1d7a1881eba5bcdee1c9507e115e32b8e3a2c3db3e WatchSource:0}: Error finding container f876e8ccb5f4520d5f2eca1d7a1881eba5bcdee1c9507e115e32b8e3a2c3db3e: Status 404 returned error can't find the container with id f876e8ccb5f4520d5f2eca1d7a1881eba5bcdee1c9507e115e32b8e3a2c3db3e
Jul 29 12:26:31.403874 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:31.403840    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-rmcgr"]
Jul 29 12:26:31.406393 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:26:31.406363    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2188f74f_2cb0_4a90_a994_ceb43417c819.slice/crio-75c0eb2b0bb213275c47ad17433007161f940efb73dabe8584e447631da39802 WatchSource:0}: Error finding container 75c0eb2b0bb213275c47ad17433007161f940efb73dabe8584e447631da39802: Status 404 returned error can't find the container with id 75c0eb2b0bb213275c47ad17433007161f940efb73dabe8584e447631da39802
Jul 29 12:26:32.215806 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:32.215623    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-w97vv" event={"ID":"1635829e-9a40-4521-8430-88adbb07a97a","Type":"ContainerStarted","Data":"287395c8dcae972875591647dcbeaf9babf58c642d2c9d2f0de302e78dc1a9b7"}
Jul 29 12:26:32.218940 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:32.218909    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-d48j6" event={"ID":"271b0e50-2034-4105-b7d0-49e85d78c021","Type":"ContainerStarted","Data":"f876e8ccb5f4520d5f2eca1d7a1881eba5bcdee1c9507e115e32b8e3a2c3db3e"}
Jul 29 12:26:32.224598 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:32.224530    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-sdj8z" event={"ID":"e9572260-1f9a-4f9d-93db-1cc7f72dcc7a","Type":"ContainerStarted","Data":"d182fbe28c69b843b588ba44cbc6cdc38ca7e7f8d887331f4e274bff12dc5a09"}
Jul 29 12:26:32.224598 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:32.224576    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-sdj8z" event={"ID":"e9572260-1f9a-4f9d-93db-1cc7f72dcc7a","Type":"ContainerStarted","Data":"3ed135e5921c65a96835a9ad48a93c9c36c62f56560d8af189c26642f3f4cacf"}
Jul 29 12:26:32.225133 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:32.225065    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-sdj8z"
Jul 29 12:26:32.226916 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:32.226792    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68cfb779b6-94kd7" event={"ID":"b283c8d3-3932-4e72-9f6d-939b49996766","Type":"ContainerStarted","Data":"8a1688c447428fce7bcae5578aa570fc6605ff92698948c0af0af5021d6363d1"}
Jul 29 12:26:32.235196 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:32.234715    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-w97vv" podStartSLOduration=68.635370614 podStartE2EDuration="1m14.234697337s" totalImagesPullingTime="5.599326723s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 12:25:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:26:32.23275614 +0000 UTC m=+74.926444452" watchObservedRunningTime="2026-07-29 12:26:32.234697337 +0000 UTC m=+74.928385648"
Jul 29 12:26:32.235196 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:32.235133    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2q2mn" event={"ID":"5edc73cd-3f85-4c3e-be4a-f311e46492a1","Type":"ContainerStarted","Data":"905ba0d0a6d6a22d092f5a7830fdd61311ee65b38ad3b1ede750af07691a6d6e"}
Jul 29 12:26:32.240262 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:32.240237    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rmcgr" event={"ID":"2188f74f-2cb0-4a90-a994-ceb43417c819","Type":"ContainerStarted","Data":"75c0eb2b0bb213275c47ad17433007161f940efb73dabe8584e447631da39802"}
Jul 29 12:26:32.242986 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:32.242944    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-lljkn" event={"ID":"8ebccfd1-f963-40fb-a3f4-ddcb9b3cf8d6","Type":"ContainerStarted","Data":"148903876e42d3fc1eb1d09ef24396b64054e9c3de9e58fcd66a5cc525779a6c"}
Jul 29 12:26:32.254479 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:32.253337    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-sdj8z" podStartSLOduration=35.658239192 podStartE2EDuration="41.253319438s" totalImagesPullingTime="5.595080246s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:25:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:26:32.251663258 +0000 UTC m=+74.945351571" watchObservedRunningTime="2026-07-29 12:26:32.253319438 +0000 UTC m=+74.947007750"
Jul 29 12:26:38.039317 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:38.039289    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-66c6bd9f9c-fnvkh"]
Jul 29 12:26:38.045348 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:38.045325    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66c6bd9f9c-fnvkh"
Jul 29 12:26:38.048727 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:38.048701    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 29 12:26:38.048861 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:38.048701    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 29 12:26:38.048861 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:38.048810    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 29 12:26:38.049693 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:38.049675    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 29 12:26:38.049801 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:38.049788    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 29 12:26:38.049862 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:38.049801    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-gwdkx\""
Jul 29 12:26:38.055199 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:38.054682    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66c6bd9f9c-fnvkh"]
Jul 29 12:26:38.055199 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:38.054787    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 29 12:26:38.116056 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:38.116027    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f88daa17-e943-49ad-b56a-f80ccabcebc0-console-config\") pod \"console-66c6bd9f9c-fnvkh\" (UID: \"f88daa17-e943-49ad-b56a-f80ccabcebc0\") " pod="openshift-console/console-66c6bd9f9c-fnvkh"
Jul 29 12:26:38.116199 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:38.116089    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f88daa17-e943-49ad-b56a-f80ccabcebc0-service-ca\") pod \"console-66c6bd9f9c-fnvkh\" (UID: \"f88daa17-e943-49ad-b56a-f80ccabcebc0\") " pod="openshift-console/console-66c6bd9f9c-fnvkh"
Jul 29 12:26:38.116199 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:38.116146    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64sg8\" (UniqueName: \"kubernetes.io/projected/f88daa17-e943-49ad-b56a-f80ccabcebc0-kube-api-access-64sg8\") pod \"console-66c6bd9f9c-fnvkh\" (UID: \"f88daa17-e943-49ad-b56a-f80ccabcebc0\") " pod="openshift-console/console-66c6bd9f9c-fnvkh"
Jul 29 12:26:38.116321 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:38.116211    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f88daa17-e943-49ad-b56a-f80ccabcebc0-console-serving-cert\") pod \"console-66c6bd9f9c-fnvkh\" (UID: \"f88daa17-e943-49ad-b56a-f80ccabcebc0\") " pod="openshift-console/console-66c6bd9f9c-fnvkh"
Jul 29 12:26:38.116321 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:38.116244    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f88daa17-e943-49ad-b56a-f80ccabcebc0-trusted-ca-bundle\") pod \"console-66c6bd9f9c-fnvkh\" (UID: \"f88daa17-e943-49ad-b56a-f80ccabcebc0\") " pod="openshift-console/console-66c6bd9f9c-fnvkh"
Jul 29 12:26:38.116321 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:38.116294    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f88daa17-e943-49ad-b56a-f80ccabcebc0-oauth-serving-cert\") pod \"console-66c6bd9f9c-fnvkh\" (UID: \"f88daa17-e943-49ad-b56a-f80ccabcebc0\") " pod="openshift-console/console-66c6bd9f9c-fnvkh"
Jul 29 12:26:38.116555 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:38.116340    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f88daa17-e943-49ad-b56a-f80ccabcebc0-console-oauth-config\") pod \"console-66c6bd9f9c-fnvkh\" (UID: \"f88daa17-e943-49ad-b56a-f80ccabcebc0\") " pod="openshift-console/console-66c6bd9f9c-fnvkh"
Jul 29 12:26:38.217726 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:38.217690    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64sg8\" (UniqueName: \"kubernetes.io/projected/f88daa17-e943-49ad-b56a-f80ccabcebc0-kube-api-access-64sg8\") pod \"console-66c6bd9f9c-fnvkh\" (UID: \"f88daa17-e943-49ad-b56a-f80ccabcebc0\") " pod="openshift-console/console-66c6bd9f9c-fnvkh"
Jul 29 12:26:38.217883 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:38.217745    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f88daa17-e943-49ad-b56a-f80ccabcebc0-console-serving-cert\") pod \"console-66c6bd9f9c-fnvkh\" (UID: \"f88daa17-e943-49ad-b56a-f80ccabcebc0\") " pod="openshift-console/console-66c6bd9f9c-fnvkh"
Jul 29 12:26:38.217883 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:38.217777    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f88daa17-e943-49ad-b56a-f80ccabcebc0-trusted-ca-bundle\") pod \"console-66c6bd9f9c-fnvkh\" (UID: \"f88daa17-e943-49ad-b56a-f80ccabcebc0\") " pod="openshift-console/console-66c6bd9f9c-fnvkh"
Jul 29 12:26:38.218054 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:38.218035    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f88daa17-e943-49ad-b56a-f80ccabcebc0-oauth-serving-cert\") pod \"console-66c6bd9f9c-fnvkh\" (UID: \"f88daa17-e943-49ad-b56a-f80ccabcebc0\") " pod="openshift-console/console-66c6bd9f9c-fnvkh"
Jul 29 12:26:38.218092 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:38.218084    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f88daa17-e943-49ad-b56a-f80ccabcebc0-console-oauth-config\") pod \"console-66c6bd9f9c-fnvkh\" (UID: \"f88daa17-e943-49ad-b56a-f80ccabcebc0\") " pod="openshift-console/console-66c6bd9f9c-fnvkh"
Jul 29 12:26:38.218134 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:38.218104    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f88daa17-e943-49ad-b56a-f80ccabcebc0-console-config\") pod \"console-66c6bd9f9c-fnvkh\" (UID: \"f88daa17-e943-49ad-b56a-f80ccabcebc0\") " pod="openshift-console/console-66c6bd9f9c-fnvkh"
Jul 29 12:26:38.218169 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:38.218150    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f88daa17-e943-49ad-b56a-f80ccabcebc0-service-ca\") pod \"console-66c6bd9f9c-fnvkh\" (UID: \"f88daa17-e943-49ad-b56a-f80ccabcebc0\") " pod="openshift-console/console-66c6bd9f9c-fnvkh"
Jul 29 12:26:38.218653 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:38.218622    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f88daa17-e943-49ad-b56a-f80ccabcebc0-trusted-ca-bundle\") pod \"console-66c6bd9f9c-fnvkh\" (UID: \"f88daa17-e943-49ad-b56a-f80ccabcebc0\") " pod="openshift-console/console-66c6bd9f9c-fnvkh"
Jul 29 12:26:38.218952 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:38.218929    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f88daa17-e943-49ad-b56a-f80ccabcebc0-service-ca\") pod \"console-66c6bd9f9c-fnvkh\" (UID: \"f88daa17-e943-49ad-b56a-f80ccabcebc0\") " pod="openshift-console/console-66c6bd9f9c-fnvkh"
Jul 29 12:26:38.219024 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:38.218988    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f88daa17-e943-49ad-b56a-f80ccabcebc0-console-config\") pod \"console-66c6bd9f9c-fnvkh\" (UID: \"f88daa17-e943-49ad-b56a-f80ccabcebc0\") " pod="openshift-console/console-66c6bd9f9c-fnvkh"
Jul 29 12:26:38.219084 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:38.219005    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f88daa17-e943-49ad-b56a-f80ccabcebc0-oauth-serving-cert\") pod \"console-66c6bd9f9c-fnvkh\" (UID: \"f88daa17-e943-49ad-b56a-f80ccabcebc0\") " pod="openshift-console/console-66c6bd9f9c-fnvkh"
Jul 29 12:26:38.220638 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:38.220613    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f88daa17-e943-49ad-b56a-f80ccabcebc0-console-oauth-config\") pod \"console-66c6bd9f9c-fnvkh\" (UID: \"f88daa17-e943-49ad-b56a-f80ccabcebc0\") " pod="openshift-console/console-66c6bd9f9c-fnvkh"
Jul 29 12:26:38.220801 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:38.220762    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f88daa17-e943-49ad-b56a-f80ccabcebc0-console-serving-cert\") pod \"console-66c6bd9f9c-fnvkh\" (UID: \"f88daa17-e943-49ad-b56a-f80ccabcebc0\") " pod="openshift-console/console-66c6bd9f9c-fnvkh"
Jul 29 12:26:38.226497 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:38.226475    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-64sg8\" (UniqueName: \"kubernetes.io/projected/f88daa17-e943-49ad-b56a-f80ccabcebc0-kube-api-access-64sg8\") pod \"console-66c6bd9f9c-fnvkh\" (UID: \"f88daa17-e943-49ad-b56a-f80ccabcebc0\") " pod="openshift-console/console-66c6bd9f9c-fnvkh"
Jul 29 12:26:38.359040 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:38.358963    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66c6bd9f9c-fnvkh"
Jul 29 12:26:38.559172 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:38.559080    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66c6bd9f9c-fnvkh"]
Jul 29 12:26:38.561699 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:26:38.561667    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf88daa17_e943_49ad_b56a_f80ccabcebc0.slice/crio-c9ff5d27f2ce4022c6f1998ed0795f9beb24bc4e09a7dfb01fefbee848b22be0 WatchSource:0}: Error finding container c9ff5d27f2ce4022c6f1998ed0795f9beb24bc4e09a7dfb01fefbee848b22be0: Status 404 returned error can't find the container with id c9ff5d27f2ce4022c6f1998ed0795f9beb24bc4e09a7dfb01fefbee848b22be0
Jul 29 12:26:39.034423 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:39.034368    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-ljpg7"]
Jul 29 12:26:39.038965 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:39.038942    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-ljpg7"
Jul 29 12:26:39.041745 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:39.041723    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 29 12:26:39.042096 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:39.041991    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-z22dn\""
Jul 29 12:26:39.051613 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:39.051574    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-ljpg7"]
Jul 29 12:26:39.124607 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:39.124569    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/3ba5769c-b445-4466-b7b3-376de6bf5fe2-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-ljpg7\" (UID: \"3ba5769c-b445-4466-b7b3-376de6bf5fe2\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-ljpg7"
Jul 29 12:26:39.224968 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:39.224935    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/3ba5769c-b445-4466-b7b3-376de6bf5fe2-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-ljpg7\" (UID: \"3ba5769c-b445-4466-b7b3-376de6bf5fe2\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-ljpg7"
Jul 29 12:26:39.225150 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:26:39.225134    2567 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Jul 29 12:26:39.225222 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:26:39.225209    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ba5769c-b445-4466-b7b3-376de6bf5fe2-tls-certificates podName:3ba5769c-b445-4466-b7b3-376de6bf5fe2 nodeName:}" failed. No retries permitted until 2026-07-29 12:26:39.72518887 +0000 UTC m=+82.418877161 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/3ba5769c-b445-4466-b7b3-376de6bf5fe2-tls-certificates") pod "prometheus-operator-admission-webhook-7f8d78f54b-ljpg7" (UID: "3ba5769c-b445-4466-b7b3-376de6bf5fe2") : secret "prometheus-operator-admission-webhook-tls" not found
Jul 29 12:26:39.285698 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:39.285614    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2q2mn" event={"ID":"5edc73cd-3f85-4c3e-be4a-f311e46492a1","Type":"ContainerStarted","Data":"28a17a4bf0c8dbb36dd4265ba6627474ba3f5ce8346837ca3ce06d1dd808759a"}
Jul 29 12:26:39.288975 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:39.288909    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rmcgr" event={"ID":"2188f74f-2cb0-4a90-a994-ceb43417c819","Type":"ContainerStarted","Data":"7fde13785f3c9d32cbc1e6980212ded2bbade42f28d7dcd7725da2395d2bfd66"}
Jul 29 12:26:39.297470 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:39.297435    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-d48j6" event={"ID":"271b0e50-2034-4105-b7d0-49e85d78c021","Type":"ContainerStarted","Data":"ca0935135b3f91c00189eeb019c06543cb043b43e4dfc186555eefe48dd55770"}
Jul 29 12:26:39.299451 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:39.299427    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66c6bd9f9c-fnvkh" event={"ID":"f88daa17-e943-49ad-b56a-f80ccabcebc0","Type":"ContainerStarted","Data":"c9ff5d27f2ce4022c6f1998ed0795f9beb24bc4e09a7dfb01fefbee848b22be0"}
Jul 29 12:26:39.301550 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:39.301423    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68cfb779b6-94kd7" event={"ID":"b283c8d3-3932-4e72-9f6d-939b49996766","Type":"ContainerStarted","Data":"f1294b86d1547d2031b5b1b47488cac11135d7f7e523ca3c4ec4cd68dbc95cba"}
Jul 29 12:26:39.301853 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:39.301833    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68cfb779b6-94kd7"
Jul 29 12:26:39.303617 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:39.303600    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68cfb779b6-94kd7"
Jul 29 12:26:39.308544 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:39.308496    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-2q2mn" podStartSLOduration=17.271414997 podStartE2EDuration="35.30847865s" totalImagesPullingTime="18.037063653s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 12:26:04 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:26:39.307392954 +0000 UTC m=+82.001081280" watchObservedRunningTime="2026-07-29 12:26:39.30847865 +0000 UTC m=+82.002166962"
Jul 29 12:26:39.325687 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:39.325632    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rmcgr" podStartSLOduration=35.327046894 podStartE2EDuration="42.325613902s" totalImagesPullingTime="6.998567008s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:25:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:26:39.325103296 +0000 UTC m=+82.018791608" watchObservedRunningTime="2026-07-29 12:26:39.325613902 +0000 UTC m=+82.019302215"
Jul 29 12:26:39.345360 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:39.345311    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-74c9f884b7-d48j6" podStartSLOduration=35.187261298 podStartE2EDuration="42.34529369s" totalImagesPullingTime="7.158032392s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:25:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:26:39.343697565 +0000 UTC m=+82.037385898" watchObservedRunningTime="2026-07-29 12:26:39.34529369 +0000 UTC m=+82.038982001"
Jul 29 12:26:39.360780 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:39.360347    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68cfb779b6-94kd7" podStartSLOduration=4.149898875 podStartE2EDuration="11.360330047s" totalImagesPullingTime="7.210431172s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:26:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:26:39.358928346 +0000 UTC m=+82.052616660" watchObservedRunningTime="2026-07-29 12:26:39.360330047 +0000 UTC m=+82.054018357"
Jul 29 12:26:39.728554 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:39.728472    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/3ba5769c-b445-4466-b7b3-376de6bf5fe2-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-ljpg7\" (UID: \"3ba5769c-b445-4466-b7b3-376de6bf5fe2\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-ljpg7"
Jul 29 12:26:39.731666 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:39.731637    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/3ba5769c-b445-4466-b7b3-376de6bf5fe2-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-ljpg7\" (UID: \"3ba5769c-b445-4466-b7b3-376de6bf5fe2\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-ljpg7"
Jul 29 12:26:39.950853 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:39.950493    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-ljpg7"
Jul 29 12:26:40.109970 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:40.109933    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-ljpg7"]
Jul 29 12:26:40.114158 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:26:40.114120    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ba5769c_b445_4466_b7b3_376de6bf5fe2.slice/crio-c534a7c84fad4f8ba5de274ead42c575bdbf09e41b76010368d4ecda5992b5c4 WatchSource:0}: Error finding container c534a7c84fad4f8ba5de274ead42c575bdbf09e41b76010368d4ecda5992b5c4: Status 404 returned error can't find the container with id c534a7c84fad4f8ba5de274ead42c575bdbf09e41b76010368d4ecda5992b5c4
Jul 29 12:26:40.307416 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:40.307372    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-ljpg7" event={"ID":"3ba5769c-b445-4466-b7b3-376de6bf5fe2","Type":"ContainerStarted","Data":"c534a7c84fad4f8ba5de274ead42c575bdbf09e41b76010368d4ecda5992b5c4"}
Jul 29 12:26:42.319934 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:42.319687    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66c6bd9f9c-fnvkh" event={"ID":"f88daa17-e943-49ad-b56a-f80ccabcebc0","Type":"ContainerStarted","Data":"635324da685dcdba4e80404433f8e80f354073c60f367fb65c5a24b95743ac83"}
Jul 29 12:26:43.261836 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:43.261805    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-sdj8z"
Jul 29 12:26:43.287418 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:43.287352    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-66c6bd9f9c-fnvkh" podStartSLOduration=2.150183267 podStartE2EDuration="5.28733139s" totalImagesPullingTime="3.137148123s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:26:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:26:42.336976524 +0000 UTC m=+85.030664860" watchObservedRunningTime="2026-07-29 12:26:43.28733139 +0000 UTC m=+85.981019702"
Jul 29 12:26:43.325837 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:43.325803    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-ljpg7" event={"ID":"3ba5769c-b445-4466-b7b3-376de6bf5fe2","Type":"ContainerStarted","Data":"364bae8e7f05a1f981c2e1fba7d4ab596e2758287325d53299a98c6c107206f8"}
Jul 29 12:26:43.326341 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:43.326316    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-ljpg7"
Jul 29 12:26:43.332083 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:43.332040    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-ljpg7"
Jul 29 12:26:43.349829 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:43.349784    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-ljpg7" podStartSLOduration=2.029471692 podStartE2EDuration="4.34976923s" totalImagesPullingTime="2.320297538s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:26:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:26:43.348875303 +0000 UTC m=+86.042563615" watchObservedRunningTime="2026-07-29 12:26:43.34976923 +0000 UTC m=+86.043457540"
Jul 29 12:26:47.191518 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:47.191491    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7f46c77f5c-tnln6"
Jul 29 12:26:48.359869 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:48.359832    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-66c6bd9f9c-fnvkh"
Jul 29 12:26:48.360419 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:48.359900    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-66c6bd9f9c-fnvkh"
Jul 29 12:26:48.365191 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:48.365168    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-66c6bd9f9c-fnvkh"
Jul 29 12:26:49.353544 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:49.353513    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-66c6bd9f9c-fnvkh"
Jul 29 12:26:49.674417 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:49.674322    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-rzxkb"]
Jul 29 12:26:50.007877 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:50.007806    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rzxkb"
Jul 29 12:26:50.011172 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:50.011127    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 29 12:26:50.012415 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:50.012377    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 29 12:26:50.014009 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:50.013972    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-wzq9d\""
Jul 29 12:26:50.014193 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:50.014115    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 29 12:26:50.014314 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:50.014284    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 29 12:26:50.111189 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:50.111150    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7gst6\" (UniqueName: \"kubernetes.io/projected/ab16f311-c741-4ce9-92b2-bd2e46b049f7-kube-api-access-7gst6\") pod \"node-exporter-rzxkb\" (UID: \"ab16f311-c741-4ce9-92b2-bd2e46b049f7\") " pod="openshift-monitoring/node-exporter-rzxkb"
Jul 29 12:26:50.111423 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:50.111207    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ab16f311-c741-4ce9-92b2-bd2e46b049f7-sys\") pod \"node-exporter-rzxkb\" (UID: \"ab16f311-c741-4ce9-92b2-bd2e46b049f7\") " pod="openshift-monitoring/node-exporter-rzxkb"
Jul 29 12:26:50.111423 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:50.111264    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ab16f311-c741-4ce9-92b2-bd2e46b049f7-root\") pod \"node-exporter-rzxkb\" (UID: \"ab16f311-c741-4ce9-92b2-bd2e46b049f7\") " pod="openshift-monitoring/node-exporter-rzxkb"
Jul 29 12:26:50.111423 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:50.111314    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ab16f311-c741-4ce9-92b2-bd2e46b049f7-node-exporter-accelerators-collector-config\") pod \"node-exporter-rzxkb\" (UID: \"ab16f311-c741-4ce9-92b2-bd2e46b049f7\") " pod="openshift-monitoring/node-exporter-rzxkb"
Jul 29 12:26:50.111423 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:50.111344    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ab16f311-c741-4ce9-92b2-bd2e46b049f7-node-exporter-textfile\") pod \"node-exporter-rzxkb\" (UID: \"ab16f311-c741-4ce9-92b2-bd2e46b049f7\") " pod="openshift-monitoring/node-exporter-rzxkb"
Jul 29 12:26:50.111423 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:50.111373    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ab16f311-c741-4ce9-92b2-bd2e46b049f7-node-exporter-wtmp\") pod \"node-exporter-rzxkb\" (UID: \"ab16f311-c741-4ce9-92b2-bd2e46b049f7\") " pod="openshift-monitoring/node-exporter-rzxkb"
Jul 29 12:26:50.111423 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:50.111389    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ab16f311-c741-4ce9-92b2-bd2e46b049f7-metrics-client-ca\") pod \"node-exporter-rzxkb\" (UID: \"ab16f311-c741-4ce9-92b2-bd2e46b049f7\") " pod="openshift-monitoring/node-exporter-rzxkb"
Jul 29 12:26:50.111637 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:50.111445    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ab16f311-c741-4ce9-92b2-bd2e46b049f7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rzxkb\" (UID: \"ab16f311-c741-4ce9-92b2-bd2e46b049f7\") " pod="openshift-monitoring/node-exporter-rzxkb"
Jul 29 12:26:50.111637 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:50.111463    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ab16f311-c741-4ce9-92b2-bd2e46b049f7-node-exporter-tls\") pod \"node-exporter-rzxkb\" (UID: \"ab16f311-c741-4ce9-92b2-bd2e46b049f7\") " pod="openshift-monitoring/node-exporter-rzxkb"
Jul 29 12:26:50.212667 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:50.212637    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ab16f311-c741-4ce9-92b2-bd2e46b049f7-node-exporter-textfile\") pod \"node-exporter-rzxkb\" (UID: \"ab16f311-c741-4ce9-92b2-bd2e46b049f7\") " pod="openshift-monitoring/node-exporter-rzxkb"
Jul 29 12:26:50.212906 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:50.212672    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ab16f311-c741-4ce9-92b2-bd2e46b049f7-node-exporter-wtmp\") pod \"node-exporter-rzxkb\" (UID: \"ab16f311-c741-4ce9-92b2-bd2e46b049f7\") " pod="openshift-monitoring/node-exporter-rzxkb"
Jul 29 12:26:50.212906 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:50.212690    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ab16f311-c741-4ce9-92b2-bd2e46b049f7-metrics-client-ca\") pod \"node-exporter-rzxkb\" (UID: \"ab16f311-c741-4ce9-92b2-bd2e46b049f7\") " pod="openshift-monitoring/node-exporter-rzxkb"
Jul 29 12:26:50.212906 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:50.212718    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ab16f311-c741-4ce9-92b2-bd2e46b049f7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rzxkb\" (UID: \"ab16f311-c741-4ce9-92b2-bd2e46b049f7\") " pod="openshift-monitoring/node-exporter-rzxkb"
Jul 29 12:26:50.212906 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:50.212736    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ab16f311-c741-4ce9-92b2-bd2e46b049f7-node-exporter-tls\") pod \"node-exporter-rzxkb\" (UID: \"ab16f311-c741-4ce9-92b2-bd2e46b049f7\") " pod="openshift-monitoring/node-exporter-rzxkb"
Jul 29 12:26:50.212906 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:50.212772    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7gst6\" (UniqueName: \"kubernetes.io/projected/ab16f311-c741-4ce9-92b2-bd2e46b049f7-kube-api-access-7gst6\") pod \"node-exporter-rzxkb\" (UID: \"ab16f311-c741-4ce9-92b2-bd2e46b049f7\") " pod="openshift-monitoring/node-exporter-rzxkb"
Jul 29 12:26:50.212906 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:50.212840    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ab16f311-c741-4ce9-92b2-bd2e46b049f7-node-exporter-wtmp\") pod \"node-exporter-rzxkb\" (UID: \"ab16f311-c741-4ce9-92b2-bd2e46b049f7\") " pod="openshift-monitoring/node-exporter-rzxkb"
Jul 29 12:26:50.212906 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:50.212893    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ab16f311-c741-4ce9-92b2-bd2e46b049f7-sys\") pod \"node-exporter-rzxkb\" (UID: \"ab16f311-c741-4ce9-92b2-bd2e46b049f7\") " pod="openshift-monitoring/node-exporter-rzxkb"
Jul 29 12:26:50.213242 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:50.212937    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ab16f311-c741-4ce9-92b2-bd2e46b049f7-root\") pod \"node-exporter-rzxkb\" (UID: \"ab16f311-c741-4ce9-92b2-bd2e46b049f7\") " pod="openshift-monitoring/node-exporter-rzxkb"
Jul 29 12:26:50.213242 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:50.212955    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ab16f311-c741-4ce9-92b2-bd2e46b049f7-node-exporter-accelerators-collector-config\") pod \"node-exporter-rzxkb\" (UID: \"ab16f311-c741-4ce9-92b2-bd2e46b049f7\") " pod="openshift-monitoring/node-exporter-rzxkb"
Jul 29 12:26:50.213242 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:50.212994    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ab16f311-c741-4ce9-92b2-bd2e46b049f7-node-exporter-textfile\") pod \"node-exporter-rzxkb\" (UID: \"ab16f311-c741-4ce9-92b2-bd2e46b049f7\") " pod="openshift-monitoring/node-exporter-rzxkb"
Jul 29 12:26:50.213242 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:50.213032    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ab16f311-c741-4ce9-92b2-bd2e46b049f7-sys\") pod \"node-exporter-rzxkb\" (UID: \"ab16f311-c741-4ce9-92b2-bd2e46b049f7\") " pod="openshift-monitoring/node-exporter-rzxkb"
Jul 29 12:26:50.213242 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:50.213057    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ab16f311-c741-4ce9-92b2-bd2e46b049f7-root\") pod \"node-exporter-rzxkb\" (UID: \"ab16f311-c741-4ce9-92b2-bd2e46b049f7\") " pod="openshift-monitoring/node-exporter-rzxkb"
Jul 29 12:26:50.213472 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:50.213378    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ab16f311-c741-4ce9-92b2-bd2e46b049f7-metrics-client-ca\") pod \"node-exporter-rzxkb\" (UID: \"ab16f311-c741-4ce9-92b2-bd2e46b049f7\") " pod="openshift-monitoring/node-exporter-rzxkb"
Jul 29 12:26:50.213472 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:50.213389    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ab16f311-c741-4ce9-92b2-bd2e46b049f7-node-exporter-accelerators-collector-config\") pod \"node-exporter-rzxkb\" (UID: \"ab16f311-c741-4ce9-92b2-bd2e46b049f7\") " pod="openshift-monitoring/node-exporter-rzxkb"
Jul 29 12:26:50.214983 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:50.214962    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ab16f311-c741-4ce9-92b2-bd2e46b049f7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rzxkb\" (UID: \"ab16f311-c741-4ce9-92b2-bd2e46b049f7\") " pod="openshift-monitoring/node-exporter-rzxkb"
Jul 29 12:26:50.215086 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:50.215065    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ab16f311-c741-4ce9-92b2-bd2e46b049f7-node-exporter-tls\") pod \"node-exporter-rzxkb\" (UID: \"ab16f311-c741-4ce9-92b2-bd2e46b049f7\") " pod="openshift-monitoring/node-exporter-rzxkb"
Jul 29 12:26:50.225335 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:50.225313    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7gst6\" (UniqueName: \"kubernetes.io/projected/ab16f311-c741-4ce9-92b2-bd2e46b049f7-kube-api-access-7gst6\") pod \"node-exporter-rzxkb\" (UID: \"ab16f311-c741-4ce9-92b2-bd2e46b049f7\") " pod="openshift-monitoring/node-exporter-rzxkb"
Jul 29 12:26:50.318265 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:50.318243    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rzxkb"
Jul 29 12:26:50.371479 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:26:50.371453    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab16f311_c741_4ce9_92b2_bd2e46b049f7.slice/crio-a2bd4ed1bf85dbfa95144a01d76f61b1474375f644b29485a49aaf1787777ec6 WatchSource:0}: Error finding container a2bd4ed1bf85dbfa95144a01d76f61b1474375f644b29485a49aaf1787777ec6: Status 404 returned error can't find the container with id a2bd4ed1bf85dbfa95144a01d76f61b1474375f644b29485a49aaf1787777ec6
Jul 29 12:26:50.755617 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:50.755541    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 12:26:50.775273 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:50.775241    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 29 12:26:50.779421 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:50.779180    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 29 12:26:50.779421 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:50.779231    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 29 12:26:50.779421 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:50.779181    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 29 12:26:50.779681 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:50.779537    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 29 12:26:50.779681 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:50.779625    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 29 12:26:50.779681 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:50.779539    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 29 12:26:50.779903 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:50.779877    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 29 12:26:50.779991 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:50.779938    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 29 12:26:50.779991 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:50.779883    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 29 12:26:50.780095 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:50.779881    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-qc5k7\""
Jul 29 12:26:50.794815 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:50.794741    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 12:26:50.919839 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:50.919793    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2118756c-36d8-4f5c-894a-d55d27deb59f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2118756c-36d8-4f5c-894a-d55d27deb59f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 12:26:50.920127 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:50.920103    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2118756c-36d8-4f5c-894a-d55d27deb59f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2118756c-36d8-4f5c-894a-d55d27deb59f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 12:26:50.920318 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:50.920287    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2118756c-36d8-4f5c-894a-d55d27deb59f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2118756c-36d8-4f5c-894a-d55d27deb59f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 12:26:50.920509 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:50.920487    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2118756c-36d8-4f5c-894a-d55d27deb59f-config-volume\") pod \"alertmanager-main-0\" (UID: \"2118756c-36d8-4f5c-894a-d55d27deb59f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 12:26:50.920651 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:50.920624    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2118756c-36d8-4f5c-894a-d55d27deb59f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2118756c-36d8-4f5c-894a-d55d27deb59f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 12:26:50.920755 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:50.920682    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2118756c-36d8-4f5c-894a-d55d27deb59f-web-config\") pod \"alertmanager-main-0\" (UID: \"2118756c-36d8-4f5c-894a-d55d27deb59f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 12:26:50.920755 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:50.920722    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzdk9\" (UniqueName: \"kubernetes.io/projected/2118756c-36d8-4f5c-894a-d55d27deb59f-kube-api-access-zzdk9\") pod \"alertmanager-main-0\" (UID: \"2118756c-36d8-4f5c-894a-d55d27deb59f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 12:26:50.920870 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:50.920778    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2118756c-36d8-4f5c-894a-d55d27deb59f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2118756c-36d8-4f5c-894a-d55d27deb59f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 12:26:50.920870 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:50.920815    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2118756c-36d8-4f5c-894a-d55d27deb59f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2118756c-36d8-4f5c-894a-d55d27deb59f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 12:26:50.920870 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:50.920862    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2118756c-36d8-4f5c-894a-d55d27deb59f-config-out\") pod \"alertmanager-main-0\" (UID: \"2118756c-36d8-4f5c-894a-d55d27deb59f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 12:26:50.921016 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:50.920891    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2118756c-36d8-4f5c-894a-d55d27deb59f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2118756c-36d8-4f5c-894a-d55d27deb59f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 12:26:50.921016 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:50.920944    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2118756c-36d8-4f5c-894a-d55d27deb59f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2118756c-36d8-4f5c-894a-d55d27deb59f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 12:26:50.921016 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:50.920968    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2118756c-36d8-4f5c-894a-d55d27deb59f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2118756c-36d8-4f5c-894a-d55d27deb59f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 12:26:51.022126 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:51.022031    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2118756c-36d8-4f5c-894a-d55d27deb59f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2118756c-36d8-4f5c-894a-d55d27deb59f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 12:26:51.022126 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:51.022082    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2118756c-36d8-4f5c-894a-d55d27deb59f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2118756c-36d8-4f5c-894a-d55d27deb59f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 12:26:51.022388 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:51.022219    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2118756c-36d8-4f5c-894a-d55d27deb59f-config-volume\") pod \"alertmanager-main-0\" (UID: \"2118756c-36d8-4f5c-894a-d55d27deb59f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 12:26:51.022388 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:51.022264    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2118756c-36d8-4f5c-894a-d55d27deb59f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2118756c-36d8-4f5c-894a-d55d27deb59f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 12:26:51.022388 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:51.022311    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2118756c-36d8-4f5c-894a-d55d27deb59f-web-config\") pod \"alertmanager-main-0\" (UID: \"2118756c-36d8-4f5c-894a-d55d27deb59f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 12:26:51.022388 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:51.022345    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzdk9\" (UniqueName: \"kubernetes.io/projected/2118756c-36d8-4f5c-894a-d55d27deb59f-kube-api-access-zzdk9\") pod \"alertmanager-main-0\" (UID: \"2118756c-36d8-4f5c-894a-d55d27deb59f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 12:26:51.022388 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:51.022378    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2118756c-36d8-4f5c-894a-d55d27deb59f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2118756c-36d8-4f5c-894a-d55d27deb59f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 12:26:51.022657 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:51.022430    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2118756c-36d8-4f5c-894a-d55d27deb59f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2118756c-36d8-4f5c-894a-d55d27deb59f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 12:26:51.022657 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:51.022483    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2118756c-36d8-4f5c-894a-d55d27deb59f-config-out\") pod \"alertmanager-main-0\" (UID: \"2118756c-36d8-4f5c-894a-d55d27deb59f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 12:26:51.022657 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:51.022517    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2118756c-36d8-4f5c-894a-d55d27deb59f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2118756c-36d8-4f5c-894a-d55d27deb59f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 12:26:51.022657 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:51.022542    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2118756c-36d8-4f5c-894a-d55d27deb59f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2118756c-36d8-4f5c-894a-d55d27deb59f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 12:26:51.022657 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:51.022566    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2118756c-36d8-4f5c-894a-d55d27deb59f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2118756c-36d8-4f5c-894a-d55d27deb59f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 12:26:51.022657 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:51.022600    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2118756c-36d8-4f5c-894a-d55d27deb59f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2118756c-36d8-4f5c-894a-d55d27deb59f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 12:26:51.022932 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:51.022867    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2118756c-36d8-4f5c-894a-d55d27deb59f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2118756c-36d8-4f5c-894a-d55d27deb59f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 12:26:51.022932 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:51.022908    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2118756c-36d8-4f5c-894a-d55d27deb59f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2118756c-36d8-4f5c-894a-d55d27deb59f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 12:26:51.024858 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:51.024824    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2118756c-36d8-4f5c-894a-d55d27deb59f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2118756c-36d8-4f5c-894a-d55d27deb59f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 12:26:51.025804 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:51.025512    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2118756c-36d8-4f5c-894a-d55d27deb59f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2118756c-36d8-4f5c-894a-d55d27deb59f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 12:26:51.025909 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:51.025813    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2118756c-36d8-4f5c-894a-d55d27deb59f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2118756c-36d8-4f5c-894a-d55d27deb59f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 12:26:51.026596 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:51.026558    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2118756c-36d8-4f5c-894a-d55d27deb59f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2118756c-36d8-4f5c-894a-d55d27deb59f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 12:26:51.027431 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:51.026971    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2118756c-36d8-4f5c-894a-d55d27deb59f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2118756c-36d8-4f5c-894a-d55d27deb59f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 12:26:51.030543 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:51.027431    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2118756c-36d8-4f5c-894a-d55d27deb59f-config-out\") pod \"alertmanager-main-0\" (UID: \"2118756c-36d8-4f5c-894a-d55d27deb59f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 12:26:51.030543 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:51.027994    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2118756c-36d8-4f5c-894a-d55d27deb59f-web-config\") pod \"alertmanager-main-0\" (UID: \"2118756c-36d8-4f5c-894a-d55d27deb59f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 12:26:51.030543 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:51.029008    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2118756c-36d8-4f5c-894a-d55d27deb59f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2118756c-36d8-4f5c-894a-d55d27deb59f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 12:26:51.030543 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:51.029119    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2118756c-36d8-4f5c-894a-d55d27deb59f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2118756c-36d8-4f5c-894a-d55d27deb59f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 12:26:51.030765 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:51.030640    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2118756c-36d8-4f5c-894a-d55d27deb59f-config-volume\") pod \"alertmanager-main-0\" (UID: \"2118756c-36d8-4f5c-894a-d55d27deb59f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 12:26:51.035817 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:51.035793    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzdk9\" (UniqueName: \"kubernetes.io/projected/2118756c-36d8-4f5c-894a-d55d27deb59f-kube-api-access-zzdk9\") pod \"alertmanager-main-0\" (UID: \"2118756c-36d8-4f5c-894a-d55d27deb59f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 12:26:51.086763 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:51.086721    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 29 12:26:51.091490 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:51.091457    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7f46c77f5c-tnln6"]
Jul 29 12:26:51.300366 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:51.300336    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 12:26:51.303343 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:26:51.303308    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2118756c_36d8_4f5c_894a_d55d27deb59f.slice/crio-b97db17f02b32867f7da0eb6a3b3623e057c24844e2a5f4283daa9af9fb64c03 WatchSource:0}: Error finding container b97db17f02b32867f7da0eb6a3b3623e057c24844e2a5f4283daa9af9fb64c03: Status 404 returned error can't find the container with id b97db17f02b32867f7da0eb6a3b3623e057c24844e2a5f4283daa9af9fb64c03
Jul 29 12:26:51.366190 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:51.366156    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-lljkn" event={"ID":"8ebccfd1-f963-40fb-a3f4-ddcb9b3cf8d6","Type":"ContainerStarted","Data":"d925feb81a038b3831f3f9e8604e181856617250b4424ee2033269ea81321cfa"}
Jul 29 12:26:51.366783 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:51.366761    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-7c97dc4555-lljkn"
Jul 29 12:26:51.369668 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:51.369644    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rzxkb" event={"ID":"ab16f311-c741-4ce9-92b2-bd2e46b049f7","Type":"ContainerStarted","Data":"a2bd4ed1bf85dbfa95144a01d76f61b1474375f644b29485a49aaf1787777ec6"}
Jul 29 12:26:51.372284 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:51.372260    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2118756c-36d8-4f5c-894a-d55d27deb59f","Type":"ContainerStarted","Data":"b97db17f02b32867f7da0eb6a3b3623e057c24844e2a5f4283daa9af9fb64c03"}
Jul 29 12:26:51.377322 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:51.377296    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-7c97dc4555-lljkn"
Jul 29 12:26:51.385456 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:51.385376    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-7c97dc4555-lljkn" podStartSLOduration=3.277499093 podStartE2EDuration="22.385360672s" totalImagesPullingTime="19.107861579s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:26:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:26:51.384247737 +0000 UTC m=+94.077936063" watchObservedRunningTime="2026-07-29 12:26:51.385360672 +0000 UTC m=+94.079048985"
Jul 29 12:26:53.386920 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:53.386887    2567 generic.go:356] "Generic (PLEG): container finished" podID="ab16f311-c741-4ce9-92b2-bd2e46b049f7" containerID="a1af17603a1e61b7670fc6bd4a9d6280b75c9b9cd2da897b00dae2db59e68a04" exitCode=0
Jul 29 12:26:53.387360 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:53.386978    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rzxkb" event={"ID":"ab16f311-c741-4ce9-92b2-bd2e46b049f7","Type":"ContainerDied","Data":"a1af17603a1e61b7670fc6bd4a9d6280b75c9b9cd2da897b00dae2db59e68a04"}
Jul 29 12:26:54.337188 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.337154    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-579695d779-p47b4"]
Jul 29 12:26:54.363527 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.363497    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5479989fc7-qqwlm"]
Jul 29 12:26:54.363685 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.363602    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-579695d779-p47b4"
Jul 29 12:26:54.383732 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.383702    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-579695d779-p47b4"]
Jul 29 12:26:54.383847 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.383747    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5479989fc7-qqwlm"]
Jul 29 12:26:54.383920 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.383838    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5479989fc7-qqwlm"
Jul 29 12:26:54.383969 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.383844    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-f79zm"]
Jul 29 12:26:54.386454 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.386432    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 29 12:26:54.386771 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.386747    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 29 12:26:54.386861 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.386753    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 29 12:26:54.386921 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.386876    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-tqkmt\""
Jul 29 12:26:54.387239 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.387200    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 29 12:26:54.387751 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.387730    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-3fnfs7gbgmt3j\""
Jul 29 12:26:54.393633 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.393606    2567 generic.go:356] "Generic (PLEG): container finished" podID="2118756c-36d8-4f5c-894a-d55d27deb59f" containerID="3d9d16bad77e9feafa2c5f2e7e2dceca43202479f5a605e50d14b1dfce846da1" exitCode=0
Jul 29 12:26:54.405014 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.404987    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2118756c-36d8-4f5c-894a-d55d27deb59f","Type":"ContainerDied","Data":"3d9d16bad77e9feafa2c5f2e7e2dceca43202479f5a605e50d14b1dfce846da1"}
Jul 29 12:26:54.405115 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.405046    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rzxkb" event={"ID":"ab16f311-c741-4ce9-92b2-bd2e46b049f7","Type":"ContainerStarted","Data":"e7cbb42490c179e669395f8f7a3334f0c6e97a300bd01f5da72c6d04d3e0978f"}
Jul 29 12:26:54.405115 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.405064    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rzxkb" event={"ID":"ab16f311-c741-4ce9-92b2-bd2e46b049f7","Type":"ContainerStarted","Data":"8b9a427c549a971a8e5ca879e7e44a83a27b177e25079fb856c792af158fda97"}
Jul 29 12:26:54.405115 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.405087    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-f79zm"]
Jul 29 12:26:54.405251 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.405156    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-f79zm"
Jul 29 12:26:54.407666 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.407647    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-8q5fr\""
Jul 29 12:26:54.407749 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.407706    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 29 12:26:54.452191 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.452168    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d9e1e62f-87ee-4c09-b3f5-2393aede7247-client-ca-bundle\") pod \"metrics-server-5479989fc7-qqwlm\" (UID: \"d9e1e62f-87ee-4c09-b3f5-2393aede7247\") " pod="openshift-monitoring/metrics-server-5479989fc7-qqwlm"
Jul 29 12:26:54.452302 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.452207    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxvrj\" (UniqueName: \"kubernetes.io/projected/d9e1e62f-87ee-4c09-b3f5-2393aede7247-kube-api-access-nxvrj\") pod \"metrics-server-5479989fc7-qqwlm\" (UID: \"d9e1e62f-87ee-4c09-b3f5-2393aede7247\") " pod="openshift-monitoring/metrics-server-5479989fc7-qqwlm"
Jul 29 12:26:54.452302 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.452286    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/d9e1e62f-87ee-4c09-b3f5-2393aede7247-audit-log\") pod \"metrics-server-5479989fc7-qqwlm\" (UID: \"d9e1e62f-87ee-4c09-b3f5-2393aede7247\") " pod="openshift-monitoring/metrics-server-5479989fc7-qqwlm"
Jul 29 12:26:54.452388 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.452343    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3e2a11eb-61da-4dd1-a03c-cb8810e610a5-console-config\") pod \"console-579695d779-p47b4\" (UID: \"3e2a11eb-61da-4dd1-a03c-cb8810e610a5\") " pod="openshift-console/console-579695d779-p47b4"
Jul 29 12:26:54.452469 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.452384    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3e2a11eb-61da-4dd1-a03c-cb8810e610a5-service-ca\") pod \"console-579695d779-p47b4\" (UID: \"3e2a11eb-61da-4dd1-a03c-cb8810e610a5\") " pod="openshift-console/console-579695d779-p47b4"
Jul 29 12:26:54.452469 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.452422    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3e2a11eb-61da-4dd1-a03c-cb8810e610a5-console-serving-cert\") pod \"console-579695d779-p47b4\" (UID: \"3e2a11eb-61da-4dd1-a03c-cb8810e610a5\") " pod="openshift-console/console-579695d779-p47b4"
Jul 29 12:26:54.452566 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.452503    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3e2a11eb-61da-4dd1-a03c-cb8810e610a5-oauth-serving-cert\") pod \"console-579695d779-p47b4\" (UID: \"3e2a11eb-61da-4dd1-a03c-cb8810e610a5\") " pod="openshift-console/console-579695d779-p47b4"
Jul 29 12:26:54.452566 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.452543    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/d9e1e62f-87ee-4c09-b3f5-2393aede7247-secret-metrics-server-client-certs\") pod \"metrics-server-5479989fc7-qqwlm\" (UID: \"d9e1e62f-87ee-4c09-b3f5-2393aede7247\") " pod="openshift-monitoring/metrics-server-5479989fc7-qqwlm"
Jul 29 12:26:54.452650 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.452575    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9lfm\" (UniqueName: \"kubernetes.io/projected/3e2a11eb-61da-4dd1-a03c-cb8810e610a5-kube-api-access-w9lfm\") pod \"console-579695d779-p47b4\" (UID: \"3e2a11eb-61da-4dd1-a03c-cb8810e610a5\") " pod="openshift-console/console-579695d779-p47b4"
Jul 29 12:26:54.452650 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.452605    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d9e1e62f-87ee-4c09-b3f5-2393aede7247-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5479989fc7-qqwlm\" (UID: \"d9e1e62f-87ee-4c09-b3f5-2393aede7247\") " pod="openshift-monitoring/metrics-server-5479989fc7-qqwlm"
Jul 29 12:26:54.452650 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.452623    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/d9e1e62f-87ee-4c09-b3f5-2393aede7247-metrics-server-audit-profiles\") pod \"metrics-server-5479989fc7-qqwlm\" (UID: \"d9e1e62f-87ee-4c09-b3f5-2393aede7247\") " pod="openshift-monitoring/metrics-server-5479989fc7-qqwlm"
Jul 29 12:26:54.452650 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.452641    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e2a11eb-61da-4dd1-a03c-cb8810e610a5-trusted-ca-bundle\") pod \"console-579695d779-p47b4\" (UID: \"3e2a11eb-61da-4dd1-a03c-cb8810e610a5\") " pod="openshift-console/console-579695d779-p47b4"
Jul 29 12:26:54.452811 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.452667    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/d9e1e62f-87ee-4c09-b3f5-2393aede7247-secret-metrics-server-tls\") pod \"metrics-server-5479989fc7-qqwlm\" (UID: \"d9e1e62f-87ee-4c09-b3f5-2393aede7247\") " pod="openshift-monitoring/metrics-server-5479989fc7-qqwlm"
Jul 29 12:26:54.452811 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.452702    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3e2a11eb-61da-4dd1-a03c-cb8810e610a5-console-oauth-config\") pod \"console-579695d779-p47b4\" (UID: \"3e2a11eb-61da-4dd1-a03c-cb8810e610a5\") " pod="openshift-console/console-579695d779-p47b4"
Jul 29 12:26:54.475533 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.475491    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-rzxkb" podStartSLOduration=3.42716361 podStartE2EDuration="5.475477193s" totalImagesPullingTime="1.91713515s" totalInitContainerRuntime="131.178433ms" isStatefulPod=true podCreationTimestamp="2026-07-29 12:26:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:26:54.474175773 +0000 UTC m=+97.167864106" watchObservedRunningTime="2026-07-29 12:26:54.475477193 +0000 UTC m=+97.169165504"
Jul 29 12:26:54.553937 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.553911    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9lfm\" (UniqueName: \"kubernetes.io/projected/3e2a11eb-61da-4dd1-a03c-cb8810e610a5-kube-api-access-w9lfm\") pod \"console-579695d779-p47b4\" (UID: \"3e2a11eb-61da-4dd1-a03c-cb8810e610a5\") " pod="openshift-console/console-579695d779-p47b4"
Jul 29 12:26:54.554045 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.553950    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d9e1e62f-87ee-4c09-b3f5-2393aede7247-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5479989fc7-qqwlm\" (UID: \"d9e1e62f-87ee-4c09-b3f5-2393aede7247\") " pod="openshift-monitoring/metrics-server-5479989fc7-qqwlm"
Jul 29 12:26:54.554045 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.553974    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/d9e1e62f-87ee-4c09-b3f5-2393aede7247-metrics-server-audit-profiles\") pod \"metrics-server-5479989fc7-qqwlm\" (UID: \"d9e1e62f-87ee-4c09-b3f5-2393aede7247\") " pod="openshift-monitoring/metrics-server-5479989fc7-qqwlm"
Jul 29 12:26:54.554045 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.554030    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e2a11eb-61da-4dd1-a03c-cb8810e610a5-trusted-ca-bundle\") pod \"console-579695d779-p47b4\" (UID: \"3e2a11eb-61da-4dd1-a03c-cb8810e610a5\") " pod="openshift-console/console-579695d779-p47b4"
Jul 29 12:26:54.554246 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.554224    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/d9e1e62f-87ee-4c09-b3f5-2393aede7247-secret-metrics-server-tls\") pod \"metrics-server-5479989fc7-qqwlm\" (UID: \"d9e1e62f-87ee-4c09-b3f5-2393aede7247\") " pod="openshift-monitoring/metrics-server-5479989fc7-qqwlm"
Jul 29 12:26:54.554299 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.554273    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwdgp\" (UniqueName: \"kubernetes.io/projected/75171e3d-f745-40e6-807c-02518fbe54cc-kube-api-access-jwdgp\") pod \"monitoring-plugin-68b7fc9574-f79zm\" (UID: \"75171e3d-f745-40e6-807c-02518fbe54cc\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-f79zm"
Jul 29 12:26:54.554334 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.554305    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3e2a11eb-61da-4dd1-a03c-cb8810e610a5-console-oauth-config\") pod \"console-579695d779-p47b4\" (UID: \"3e2a11eb-61da-4dd1-a03c-cb8810e610a5\") " pod="openshift-console/console-579695d779-p47b4"
Jul 29 12:26:54.554455 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.554432    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d9e1e62f-87ee-4c09-b3f5-2393aede7247-client-ca-bundle\") pod \"metrics-server-5479989fc7-qqwlm\" (UID: \"d9e1e62f-87ee-4c09-b3f5-2393aede7247\") " pod="openshift-monitoring/metrics-server-5479989fc7-qqwlm"
Jul 29 12:26:54.554571 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.554479    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxvrj\" (UniqueName: \"kubernetes.io/projected/d9e1e62f-87ee-4c09-b3f5-2393aede7247-kube-api-access-nxvrj\") pod \"metrics-server-5479989fc7-qqwlm\" (UID: \"d9e1e62f-87ee-4c09-b3f5-2393aede7247\") " pod="openshift-monitoring/metrics-server-5479989fc7-qqwlm"
Jul 29 12:26:54.554571 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.554524    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/d9e1e62f-87ee-4c09-b3f5-2393aede7247-audit-log\") pod \"metrics-server-5479989fc7-qqwlm\" (UID: \"d9e1e62f-87ee-4c09-b3f5-2393aede7247\") " pod="openshift-monitoring/metrics-server-5479989fc7-qqwlm"
Jul 29 12:26:54.554571 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.554566    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3e2a11eb-61da-4dd1-a03c-cb8810e610a5-console-config\") pod \"console-579695d779-p47b4\" (UID: \"3e2a11eb-61da-4dd1-a03c-cb8810e610a5\") " pod="openshift-console/console-579695d779-p47b4"
Jul 29 12:26:54.554725 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.554635    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3e2a11eb-61da-4dd1-a03c-cb8810e610a5-service-ca\") pod \"console-579695d779-p47b4\" (UID: \"3e2a11eb-61da-4dd1-a03c-cb8810e610a5\") " pod="openshift-console/console-579695d779-p47b4"
Jul 29 12:26:54.554725 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.554663    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3e2a11eb-61da-4dd1-a03c-cb8810e610a5-console-serving-cert\") pod \"console-579695d779-p47b4\" (UID: \"3e2a11eb-61da-4dd1-a03c-cb8810e610a5\") " pod="openshift-console/console-579695d779-p47b4"
Jul 29 12:26:54.554828 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.554768    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3e2a11eb-61da-4dd1-a03c-cb8810e610a5-oauth-serving-cert\") pod \"console-579695d779-p47b4\" (UID: \"3e2a11eb-61da-4dd1-a03c-cb8810e610a5\") " pod="openshift-console/console-579695d779-p47b4"
Jul 29 12:26:54.554828 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.554778    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d9e1e62f-87ee-4c09-b3f5-2393aede7247-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5479989fc7-qqwlm\" (UID: \"d9e1e62f-87ee-4c09-b3f5-2393aede7247\") " pod="openshift-monitoring/metrics-server-5479989fc7-qqwlm"
Jul 29 12:26:54.554828 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.554807    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/d9e1e62f-87ee-4c09-b3f5-2393aede7247-secret-metrics-server-client-certs\") pod \"metrics-server-5479989fc7-qqwlm\" (UID: \"d9e1e62f-87ee-4c09-b3f5-2393aede7247\") " pod="openshift-monitoring/metrics-server-5479989fc7-qqwlm"
Jul 29 12:26:54.554964 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.554834    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/75171e3d-f745-40e6-807c-02518fbe54cc-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-f79zm\" (UID: \"75171e3d-f745-40e6-807c-02518fbe54cc\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-f79zm"
Jul 29 12:26:54.555020 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.554982    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/d9e1e62f-87ee-4c09-b3f5-2393aede7247-metrics-server-audit-profiles\") pod \"metrics-server-5479989fc7-qqwlm\" (UID: \"d9e1e62f-87ee-4c09-b3f5-2393aede7247\") " pod="openshift-monitoring/metrics-server-5479989fc7-qqwlm"
Jul 29 12:26:54.555167 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.555142    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e2a11eb-61da-4dd1-a03c-cb8810e610a5-trusted-ca-bundle\") pod \"console-579695d779-p47b4\" (UID: \"3e2a11eb-61da-4dd1-a03c-cb8810e610a5\") " pod="openshift-console/console-579695d779-p47b4"
Jul 29 12:26:54.555592 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.555572    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/d9e1e62f-87ee-4c09-b3f5-2393aede7247-audit-log\") pod \"metrics-server-5479989fc7-qqwlm\" (UID: \"d9e1e62f-87ee-4c09-b3f5-2393aede7247\") " pod="openshift-monitoring/metrics-server-5479989fc7-qqwlm"
Jul 29 12:26:54.556094 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.556072    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3e2a11eb-61da-4dd1-a03c-cb8810e610a5-oauth-serving-cert\") pod \"console-579695d779-p47b4\" (UID: \"3e2a11eb-61da-4dd1-a03c-cb8810e610a5\") " pod="openshift-console/console-579695d779-p47b4"
Jul 29 12:26:54.557002 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.556560    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3e2a11eb-61da-4dd1-a03c-cb8810e610a5-console-config\") pod \"console-579695d779-p47b4\" (UID: \"3e2a11eb-61da-4dd1-a03c-cb8810e610a5\") " pod="openshift-console/console-579695d779-p47b4"
Jul 29 12:26:54.557663 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.557624    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3e2a11eb-61da-4dd1-a03c-cb8810e610a5-service-ca\") pod \"console-579695d779-p47b4\" (UID: \"3e2a11eb-61da-4dd1-a03c-cb8810e610a5\") " pod="openshift-console/console-579695d779-p47b4"
Jul 29 12:26:54.566956 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.566090    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3e2a11eb-61da-4dd1-a03c-cb8810e610a5-console-serving-cert\") pod \"console-579695d779-p47b4\" (UID: \"3e2a11eb-61da-4dd1-a03c-cb8810e610a5\") " pod="openshift-console/console-579695d779-p47b4"
Jul 29 12:26:54.566956 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.566150    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/d9e1e62f-87ee-4c09-b3f5-2393aede7247-secret-metrics-server-client-certs\") pod \"metrics-server-5479989fc7-qqwlm\" (UID: \"d9e1e62f-87ee-4c09-b3f5-2393aede7247\") " pod="openshift-monitoring/metrics-server-5479989fc7-qqwlm"
Jul 29 12:26:54.566956 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.566205    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3e2a11eb-61da-4dd1-a03c-cb8810e610a5-console-oauth-config\") pod \"console-579695d779-p47b4\" (UID: \"3e2a11eb-61da-4dd1-a03c-cb8810e610a5\") " pod="openshift-console/console-579695d779-p47b4"
Jul 29 12:26:54.566956 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.566543    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d9e1e62f-87ee-4c09-b3f5-2393aede7247-client-ca-bundle\") pod \"metrics-server-5479989fc7-qqwlm\" (UID: \"d9e1e62f-87ee-4c09-b3f5-2393aede7247\") " pod="openshift-monitoring/metrics-server-5479989fc7-qqwlm"
Jul 29 12:26:54.566956 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.566655    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/d9e1e62f-87ee-4c09-b3f5-2393aede7247-secret-metrics-server-tls\") pod \"metrics-server-5479989fc7-qqwlm\" (UID: \"d9e1e62f-87ee-4c09-b3f5-2393aede7247\") " pod="openshift-monitoring/metrics-server-5479989fc7-qqwlm"
Jul 29 12:26:54.568969 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.568941    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9lfm\" (UniqueName: \"kubernetes.io/projected/3e2a11eb-61da-4dd1-a03c-cb8810e610a5-kube-api-access-w9lfm\") pod \"console-579695d779-p47b4\" (UID: \"3e2a11eb-61da-4dd1-a03c-cb8810e610a5\") " pod="openshift-console/console-579695d779-p47b4"
Jul 29 12:26:54.569751 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.569712    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxvrj\" (UniqueName: \"kubernetes.io/projected/d9e1e62f-87ee-4c09-b3f5-2393aede7247-kube-api-access-nxvrj\") pod \"metrics-server-5479989fc7-qqwlm\" (UID: \"d9e1e62f-87ee-4c09-b3f5-2393aede7247\") " pod="openshift-monitoring/metrics-server-5479989fc7-qqwlm"
Jul 29 12:26:54.655302 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.655240    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/75171e3d-f745-40e6-807c-02518fbe54cc-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-f79zm\" (UID: \"75171e3d-f745-40e6-807c-02518fbe54cc\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-f79zm"
Jul 29 12:26:54.655302 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.655285    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jwdgp\" (UniqueName: \"kubernetes.io/projected/75171e3d-f745-40e6-807c-02518fbe54cc-kube-api-access-jwdgp\") pod \"monitoring-plugin-68b7fc9574-f79zm\" (UID: \"75171e3d-f745-40e6-807c-02518fbe54cc\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-f79zm"
Jul 29 12:26:54.657620 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.657600    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/75171e3d-f745-40e6-807c-02518fbe54cc-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-f79zm\" (UID: \"75171e3d-f745-40e6-807c-02518fbe54cc\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-f79zm"
Jul 29 12:26:54.668190 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.668170    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwdgp\" (UniqueName: \"kubernetes.io/projected/75171e3d-f745-40e6-807c-02518fbe54cc-kube-api-access-jwdgp\") pod \"monitoring-plugin-68b7fc9574-f79zm\" (UID: \"75171e3d-f745-40e6-807c-02518fbe54cc\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-f79zm"
Jul 29 12:26:54.674994 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.674966    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-579695d779-p47b4"
Jul 29 12:26:54.695278 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.695254    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5479989fc7-qqwlm"
Jul 29 12:26:54.715674 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.715648    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-f79zm"
Jul 29 12:26:54.916652 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.913711    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-579695d779-p47b4"]
Jul 29 12:26:54.920773 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:26:54.920561    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e2a11eb_61da_4dd1_a03c_cb8810e610a5.slice/crio-36d3cb4b260b2a696d62448016811ff7915f9de86f576e45570172a4d24e9bf9 WatchSource:0}: Error finding container 36d3cb4b260b2a696d62448016811ff7915f9de86f576e45570172a4d24e9bf9: Status 404 returned error can't find the container with id 36d3cb4b260b2a696d62448016811ff7915f9de86f576e45570172a4d24e9bf9
Jul 29 12:26:54.955800 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.955734    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5479989fc7-qqwlm"]
Jul 29 12:26:54.958186 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:26:54.958138    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9e1e62f_87ee_4c09_b3f5_2393aede7247.slice/crio-79a26977d5b85579585e8833f265653b26bd36d429d7b387b1616725f994e5bc WatchSource:0}: Error finding container 79a26977d5b85579585e8833f265653b26bd36d429d7b387b1616725f994e5bc: Status 404 returned error can't find the container with id 79a26977d5b85579585e8833f265653b26bd36d429d7b387b1616725f994e5bc
Jul 29 12:26:54.976827 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:54.976768    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-f79zm"]
Jul 29 12:26:54.983732 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:26:54.983705    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75171e3d_f745_40e6_807c_02518fbe54cc.slice/crio-dabcd25d4efe7ddf3455878ecaf73a0fccf4091262d1b7e202afa368fce61d75 WatchSource:0}: Error finding container dabcd25d4efe7ddf3455878ecaf73a0fccf4091262d1b7e202afa368fce61d75: Status 404 returned error can't find the container with id dabcd25d4efe7ddf3455878ecaf73a0fccf4091262d1b7e202afa368fce61d75
Jul 29 12:26:55.405572 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:55.405495    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-f79zm" event={"ID":"75171e3d-f745-40e6-807c-02518fbe54cc","Type":"ContainerStarted","Data":"dabcd25d4efe7ddf3455878ecaf73a0fccf4091262d1b7e202afa368fce61d75"}
Jul 29 12:26:55.407647 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:55.407616    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5479989fc7-qqwlm" event={"ID":"d9e1e62f-87ee-4c09-b3f5-2393aede7247","Type":"ContainerStarted","Data":"79a26977d5b85579585e8833f265653b26bd36d429d7b387b1616725f994e5bc"}
Jul 29 12:26:55.409913 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:55.409817    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-579695d779-p47b4" event={"ID":"3e2a11eb-61da-4dd1-a03c-cb8810e610a5","Type":"ContainerStarted","Data":"78154e7c8b35c621186efc70e6a78db1c9f5384e0611cb0bbfe7ec69c66d089c"}
Jul 29 12:26:55.409913 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:55.409869    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-579695d779-p47b4" event={"ID":"3e2a11eb-61da-4dd1-a03c-cb8810e610a5","Type":"ContainerStarted","Data":"36d3cb4b260b2a696d62448016811ff7915f9de86f576e45570172a4d24e9bf9"}
Jul 29 12:26:55.432734 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:55.432672    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-579695d779-p47b4" podStartSLOduration=1.4326550980000001 podStartE2EDuration="1.432655098s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:26:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:26:55.431912945 +0000 UTC m=+98.125601302" watchObservedRunningTime="2026-07-29 12:26:55.432655098 +0000 UTC m=+98.126343410"
Jul 29 12:26:58.193805 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:58.193778    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-r4hjk"
Jul 29 12:26:58.427971 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:58.427942    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2118756c-36d8-4f5c-894a-d55d27deb59f","Type":"ContainerStarted","Data":"01c61cf2a01826475111354abc1a7f6bf1afbeb2362b0f6e4b50e08f64ba9133"}
Jul 29 12:26:58.428109 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:58.428020    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2118756c-36d8-4f5c-894a-d55d27deb59f","Type":"ContainerStarted","Data":"b20f0b1663747b0c27cee75734548c858177716de25276c1284523e73757dbed"}
Jul 29 12:26:58.429824 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:58.429797    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-f79zm" event={"ID":"75171e3d-f745-40e6-807c-02518fbe54cc","Type":"ContainerStarted","Data":"64fc774f076eb747ae2eed595c425c92fd364ec21b136273544dc8186426d4a4"}
Jul 29 12:26:58.430272 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:58.430222    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-f79zm"
Jul 29 12:26:58.431656 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:58.431629    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5479989fc7-qqwlm" event={"ID":"d9e1e62f-87ee-4c09-b3f5-2393aede7247","Type":"ContainerStarted","Data":"588d7698579cd942a99f5fac45b78e99f9d0f3c471178f1c6c2ae7c306996c4a"}
Jul 29 12:26:58.435763 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:58.435740    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-f79zm"
Jul 29 12:26:58.447881 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:58.447841    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-f79zm" podStartSLOduration=1.386087578 podStartE2EDuration="4.447825042s" totalImagesPullingTime="3.061737464s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:26:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:26:58.446272139 +0000 UTC m=+101.139960454" watchObservedRunningTime="2026-07-29 12:26:58.447825042 +0000 UTC m=+101.141513356"
Jul 29 12:26:58.483014 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:58.482659    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5479989fc7-qqwlm" podStartSLOduration=1.392718599 podStartE2EDuration="4.482638146s" totalImagesPullingTime="3.089919547s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:26:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:26:58.481369033 +0000 UTC m=+101.175057343" watchObservedRunningTime="2026-07-29 12:26:58.482638146 +0000 UTC m=+101.176326458"
Jul 29 12:26:59.438166 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:59.438131    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2118756c-36d8-4f5c-894a-d55d27deb59f","Type":"ContainerStarted","Data":"21c114537f45b82f9e078276992c694f812fc440a7e51c3dd08a995b9f1d17b7"}
Jul 29 12:26:59.438657 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:59.438210    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2118756c-36d8-4f5c-894a-d55d27deb59f","Type":"ContainerStarted","Data":"f79f6a4020e9d368184c942eb0f46b4aba964994afed265031b5d0f19c18e287"}
Jul 29 12:26:59.438657 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:26:59.438253    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2118756c-36d8-4f5c-894a-d55d27deb59f","Type":"ContainerStarted","Data":"5a2469d6121a750aaa88fbf7c540ad90f4ea168d7ab7cc4f45db551d625355b6"}
Jul 29 12:27:01.448682 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:01.448647    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2118756c-36d8-4f5c-894a-d55d27deb59f","Type":"ContainerStarted","Data":"c729b3ba3587bccee10113767e863f98bc11528a33b115c66d1a21231b80a8df"}
Jul 29 12:27:01.485371 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:01.485316    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.88409841 podStartE2EDuration="11.485293083s" totalImagesPullingTime="7.57498398s" totalInitContainerRuntime="26.210693ms" isStatefulPod=false podCreationTimestamp="2026-07-29 12:26:50 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:27:01.483564724 +0000 UTC m=+104.177253055" watchObservedRunningTime="2026-07-29 12:27:01.485293083 +0000 UTC m=+104.178981395"
Jul 29 12:27:04.675450 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:04.675388    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-579695d779-p47b4"
Jul 29 12:27:04.675909 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:04.675478    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-579695d779-p47b4"
Jul 29 12:27:04.680180 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:04.680152    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-579695d779-p47b4"
Jul 29 12:27:05.467954 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:05.467920    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-579695d779-p47b4"
Jul 29 12:27:05.518704 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:05.518669    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-66c6bd9f9c-fnvkh"]
Jul 29 12:27:14.695759 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:14.695724    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5479989fc7-qqwlm"
Jul 29 12:27:14.696232 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:14.695805    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5479989fc7-qqwlm"
Jul 29 12:27:15.162971 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:15.162933    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-w8d7p_5e7bd908-6123-4f1e-a5ef-109f1b3b0069/serve-healthcheck-canary/0.log"
Jul 29 12:27:16.118463 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:16.118425    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7f46c77f5c-tnln6" podUID="dff975a8-8807-4dbe-9124-7c96c00a1949" containerName="registry" containerID="cri-o://9467c35e85784519cf3bde26dda4711714434113d15345d5500dbe5d83644197" gracePeriod=30
Jul 29 12:27:16.384638 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:16.384586    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f46c77f5c-tnln6"
Jul 29 12:27:16.440043 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:16.440021    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/dff975a8-8807-4dbe-9124-7c96c00a1949-image-registry-private-configuration\") pod \"dff975a8-8807-4dbe-9124-7c96c00a1949\" (UID: \"dff975a8-8807-4dbe-9124-7c96c00a1949\") "
Jul 29 12:27:16.440160 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:16.440055    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/dff975a8-8807-4dbe-9124-7c96c00a1949-registry-certificates\") pod \"dff975a8-8807-4dbe-9124-7c96c00a1949\" (UID: \"dff975a8-8807-4dbe-9124-7c96c00a1949\") "
Jul 29 12:27:16.440160 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:16.440078    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/dff975a8-8807-4dbe-9124-7c96c00a1949-installation-pull-secrets\") pod \"dff975a8-8807-4dbe-9124-7c96c00a1949\" (UID: \"dff975a8-8807-4dbe-9124-7c96c00a1949\") "
Jul 29 12:27:16.440160 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:16.440100    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6bfs7\" (UniqueName: \"kubernetes.io/projected/dff975a8-8807-4dbe-9124-7c96c00a1949-kube-api-access-6bfs7\") pod \"dff975a8-8807-4dbe-9124-7c96c00a1949\" (UID: \"dff975a8-8807-4dbe-9124-7c96c00a1949\") "
Jul 29 12:27:16.440160 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:16.440124    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dff975a8-8807-4dbe-9124-7c96c00a1949-registry-tls\") pod \"dff975a8-8807-4dbe-9124-7c96c00a1949\" (UID: \"dff975a8-8807-4dbe-9124-7c96c00a1949\") "
Jul 29 12:27:16.440160 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:16.440160    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dff975a8-8807-4dbe-9124-7c96c00a1949-bound-sa-token\") pod \"dff975a8-8807-4dbe-9124-7c96c00a1949\" (UID: \"dff975a8-8807-4dbe-9124-7c96c00a1949\") "
Jul 29 12:27:16.440395 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:16.440201    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/dff975a8-8807-4dbe-9124-7c96c00a1949-ca-trust-extracted\") pod \"dff975a8-8807-4dbe-9124-7c96c00a1949\" (UID: \"dff975a8-8807-4dbe-9124-7c96c00a1949\") "
Jul 29 12:27:16.440395 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:16.440224    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dff975a8-8807-4dbe-9124-7c96c00a1949-trusted-ca\") pod \"dff975a8-8807-4dbe-9124-7c96c00a1949\" (UID: \"dff975a8-8807-4dbe-9124-7c96c00a1949\") "
Jul 29 12:27:16.440540 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:16.440514    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dff975a8-8807-4dbe-9124-7c96c00a1949-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "dff975a8-8807-4dbe-9124-7c96c00a1949" (UID: "dff975a8-8807-4dbe-9124-7c96c00a1949"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 12:27:16.440798 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:16.440738    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dff975a8-8807-4dbe-9124-7c96c00a1949-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "dff975a8-8807-4dbe-9124-7c96c00a1949" (UID: "dff975a8-8807-4dbe-9124-7c96c00a1949"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 12:27:16.442441 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:16.442394    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dff975a8-8807-4dbe-9124-7c96c00a1949-kube-api-access-6bfs7" (OuterVolumeSpecName: "kube-api-access-6bfs7") pod "dff975a8-8807-4dbe-9124-7c96c00a1949" (UID: "dff975a8-8807-4dbe-9124-7c96c00a1949"). InnerVolumeSpecName "kube-api-access-6bfs7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 12:27:16.442762 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:16.442738    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dff975a8-8807-4dbe-9124-7c96c00a1949-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "dff975a8-8807-4dbe-9124-7c96c00a1949" (UID: "dff975a8-8807-4dbe-9124-7c96c00a1949"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 12:27:16.442849 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:16.442758    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dff975a8-8807-4dbe-9124-7c96c00a1949-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "dff975a8-8807-4dbe-9124-7c96c00a1949" (UID: "dff975a8-8807-4dbe-9124-7c96c00a1949"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 12:27:16.442979 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:16.442960    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dff975a8-8807-4dbe-9124-7c96c00a1949-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "dff975a8-8807-4dbe-9124-7c96c00a1949" (UID: "dff975a8-8807-4dbe-9124-7c96c00a1949"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 12:27:16.443033 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:16.442999    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dff975a8-8807-4dbe-9124-7c96c00a1949-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "dff975a8-8807-4dbe-9124-7c96c00a1949" (UID: "dff975a8-8807-4dbe-9124-7c96c00a1949"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 12:27:16.447834 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:16.447812    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dff975a8-8807-4dbe-9124-7c96c00a1949-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "dff975a8-8807-4dbe-9124-7c96c00a1949" (UID: "dff975a8-8807-4dbe-9124-7c96c00a1949"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 12:27:16.509358 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:16.509333    2567 generic.go:356] "Generic (PLEG): container finished" podID="dff975a8-8807-4dbe-9124-7c96c00a1949" containerID="9467c35e85784519cf3bde26dda4711714434113d15345d5500dbe5d83644197" exitCode=0
Jul 29 12:27:16.509458 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:16.509371    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f46c77f5c-tnln6"
Jul 29 12:27:16.509458 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:16.509387    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7f46c77f5c-tnln6" event={"ID":"dff975a8-8807-4dbe-9124-7c96c00a1949","Type":"ContainerDied","Data":"9467c35e85784519cf3bde26dda4711714434113d15345d5500dbe5d83644197"}
Jul 29 12:27:16.509458 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:16.509443    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7f46c77f5c-tnln6" event={"ID":"dff975a8-8807-4dbe-9124-7c96c00a1949","Type":"ContainerDied","Data":"4e7f66dab42d4865a658cfc2fc77aa67486d5bb205266a92ffe72c7ae5ce537b"}
Jul 29 12:27:16.509562 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:16.509464    2567 scope.go:117] "RemoveContainer" containerID="9467c35e85784519cf3bde26dda4711714434113d15345d5500dbe5d83644197"
Jul 29 12:27:16.517391 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:16.517376    2567 scope.go:117] "RemoveContainer" containerID="9467c35e85784519cf3bde26dda4711714434113d15345d5500dbe5d83644197"
Jul 29 12:27:16.517662 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:27:16.517640    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9467c35e85784519cf3bde26dda4711714434113d15345d5500dbe5d83644197\": container with ID starting with 9467c35e85784519cf3bde26dda4711714434113d15345d5500dbe5d83644197 not found: ID does not exist" containerID="9467c35e85784519cf3bde26dda4711714434113d15345d5500dbe5d83644197"
Jul 29 12:27:16.517704 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:16.517670    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9467c35e85784519cf3bde26dda4711714434113d15345d5500dbe5d83644197"} err="failed to get container status \"9467c35e85784519cf3bde26dda4711714434113d15345d5500dbe5d83644197\": rpc error: code = NotFound desc = could not find container \"9467c35e85784519cf3bde26dda4711714434113d15345d5500dbe5d83644197\": container with ID starting with 9467c35e85784519cf3bde26dda4711714434113d15345d5500dbe5d83644197 not found: ID does not exist"
Jul 29 12:27:16.529846 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:16.529820    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7f46c77f5c-tnln6"]
Jul 29 12:27:16.531566 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:16.531547    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7f46c77f5c-tnln6"]
Jul 29 12:27:16.541647 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:16.541627    2567 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dff975a8-8807-4dbe-9124-7c96c00a1949-registry-tls\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:27:16.541717 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:16.541648    2567 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dff975a8-8807-4dbe-9124-7c96c00a1949-bound-sa-token\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:27:16.541717 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:16.541660    2567 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/dff975a8-8807-4dbe-9124-7c96c00a1949-ca-trust-extracted\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:27:16.541717 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:16.541668    2567 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dff975a8-8807-4dbe-9124-7c96c00a1949-trusted-ca\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:27:16.541717 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:16.541676    2567 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/dff975a8-8807-4dbe-9124-7c96c00a1949-image-registry-private-configuration\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:27:16.541717 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:16.541684    2567 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/dff975a8-8807-4dbe-9124-7c96c00a1949-registry-certificates\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:27:16.541717 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:16.541693    2567 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/dff975a8-8807-4dbe-9124-7c96c00a1949-installation-pull-secrets\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:27:16.541717 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:16.541701    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6bfs7\" (UniqueName: \"kubernetes.io/projected/dff975a8-8807-4dbe-9124-7c96c00a1949-kube-api-access-6bfs7\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:27:17.921078 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:17.921040    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dff975a8-8807-4dbe-9124-7c96c00a1949" path="/var/lib/kubelet/pods/dff975a8-8807-4dbe-9124-7c96c00a1949/volumes"
Jul 29 12:27:18.519789 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:18.519752    2567 generic.go:356] "Generic (PLEG): container finished" podID="74488661-38b3-4b24-b276-ae97f76f66fa" containerID="36f89f1f0fdf5afd8b4119a63c6eb915e66c18c28bcf22bacc999b4ed227228d" exitCode=0
Jul 29 12:27:18.519962 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:18.519832    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-q7w6t" event={"ID":"74488661-38b3-4b24-b276-ae97f76f66fa","Type":"ContainerDied","Data":"36f89f1f0fdf5afd8b4119a63c6eb915e66c18c28bcf22bacc999b4ed227228d"}
Jul 29 12:27:18.520250 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:18.520235    2567 scope.go:117] "RemoveContainer" containerID="36f89f1f0fdf5afd8b4119a63c6eb915e66c18c28bcf22bacc999b4ed227228d"
Jul 29 12:27:18.521454 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:18.521428    2567 generic.go:356] "Generic (PLEG): container finished" podID="c223418a-dbfe-4991-bc94-feebbfd40bfe" containerID="12c7ccd3a046f76ff06e8d212664b2d46a0900d9a6ae18df7f808af7a0b2d192" exitCode=0
Jul 29 12:27:18.521540 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:18.521470    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-g8xkd" event={"ID":"c223418a-dbfe-4991-bc94-feebbfd40bfe","Type":"ContainerDied","Data":"12c7ccd3a046f76ff06e8d212664b2d46a0900d9a6ae18df7f808af7a0b2d192"}
Jul 29 12:27:18.521706 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:18.521694    2567 scope.go:117] "RemoveContainer" containerID="12c7ccd3a046f76ff06e8d212664b2d46a0900d9a6ae18df7f808af7a0b2d192"
Jul 29 12:27:19.527796 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:19.527764    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-q7w6t" event={"ID":"74488661-38b3-4b24-b276-ae97f76f66fa","Type":"ContainerStarted","Data":"a8d5ea8a45df3950e1c028dc9a2ba6d6ba6679a9ef79a46b8e173b0758294ab4"}
Jul 29 12:27:19.529505 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:19.529483    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-g8xkd" event={"ID":"c223418a-dbfe-4991-bc94-feebbfd40bfe","Type":"ContainerStarted","Data":"18fd82e89a708b077b84d8bef41c51c7ef54dcbc7be97a037395d06b4458ecda"}
Jul 29 12:27:30.538189 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:30.538140    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-66c6bd9f9c-fnvkh" podUID="f88daa17-e943-49ad-b56a-f80ccabcebc0" containerName="console" containerID="cri-o://635324da685dcdba4e80404433f8e80f354073c60f367fb65c5a24b95743ac83" gracePeriod=15
Jul 29 12:27:30.811377 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:30.811326    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66c6bd9f9c-fnvkh"
Jul 29 12:27:30.852602 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:30.852577    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f88daa17-e943-49ad-b56a-f80ccabcebc0-console-serving-cert\") pod \"f88daa17-e943-49ad-b56a-f80ccabcebc0\" (UID: \"f88daa17-e943-49ad-b56a-f80ccabcebc0\") "
Jul 29 12:27:30.852707 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:30.852613    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f88daa17-e943-49ad-b56a-f80ccabcebc0-oauth-serving-cert\") pod \"f88daa17-e943-49ad-b56a-f80ccabcebc0\" (UID: \"f88daa17-e943-49ad-b56a-f80ccabcebc0\") "
Jul 29 12:27:30.852707 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:30.852634    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f88daa17-e943-49ad-b56a-f80ccabcebc0-console-oauth-config\") pod \"f88daa17-e943-49ad-b56a-f80ccabcebc0\" (UID: \"f88daa17-e943-49ad-b56a-f80ccabcebc0\") "
Jul 29 12:27:30.852707 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:30.852653    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-64sg8\" (UniqueName: \"kubernetes.io/projected/f88daa17-e943-49ad-b56a-f80ccabcebc0-kube-api-access-64sg8\") pod \"f88daa17-e943-49ad-b56a-f80ccabcebc0\" (UID: \"f88daa17-e943-49ad-b56a-f80ccabcebc0\") "
Jul 29 12:27:30.852880 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:30.852822    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f88daa17-e943-49ad-b56a-f80ccabcebc0-trusted-ca-bundle\") pod \"f88daa17-e943-49ad-b56a-f80ccabcebc0\" (UID: \"f88daa17-e943-49ad-b56a-f80ccabcebc0\") "
Jul 29 12:27:30.852933 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:30.852876    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f88daa17-e943-49ad-b56a-f80ccabcebc0-console-config\") pod \"f88daa17-e943-49ad-b56a-f80ccabcebc0\" (UID: \"f88daa17-e943-49ad-b56a-f80ccabcebc0\") "
Jul 29 12:27:30.852933 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:30.852926    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f88daa17-e943-49ad-b56a-f80ccabcebc0-service-ca\") pod \"f88daa17-e943-49ad-b56a-f80ccabcebc0\" (UID: \"f88daa17-e943-49ad-b56a-f80ccabcebc0\") "
Jul 29 12:27:30.853198 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:30.853166    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f88daa17-e943-49ad-b56a-f80ccabcebc0-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "f88daa17-e943-49ad-b56a-f80ccabcebc0" (UID: "f88daa17-e943-49ad-b56a-f80ccabcebc0"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 12:27:30.853354 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:30.853329    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f88daa17-e943-49ad-b56a-f80ccabcebc0-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "f88daa17-e943-49ad-b56a-f80ccabcebc0" (UID: "f88daa17-e943-49ad-b56a-f80ccabcebc0"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 12:27:30.853459 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:30.853336    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f88daa17-e943-49ad-b56a-f80ccabcebc0-console-config" (OuterVolumeSpecName: "console-config") pod "f88daa17-e943-49ad-b56a-f80ccabcebc0" (UID: "f88daa17-e943-49ad-b56a-f80ccabcebc0"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 12:27:30.853500 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:30.853463    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f88daa17-e943-49ad-b56a-f80ccabcebc0-service-ca" (OuterVolumeSpecName: "service-ca") pod "f88daa17-e943-49ad-b56a-f80ccabcebc0" (UID: "f88daa17-e943-49ad-b56a-f80ccabcebc0"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 12:27:30.854772 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:30.854751    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f88daa17-e943-49ad-b56a-f80ccabcebc0-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "f88daa17-e943-49ad-b56a-f80ccabcebc0" (UID: "f88daa17-e943-49ad-b56a-f80ccabcebc0"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 12:27:30.855037 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:30.855020    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f88daa17-e943-49ad-b56a-f80ccabcebc0-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "f88daa17-e943-49ad-b56a-f80ccabcebc0" (UID: "f88daa17-e943-49ad-b56a-f80ccabcebc0"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 12:27:30.855107 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:30.855043    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f88daa17-e943-49ad-b56a-f80ccabcebc0-kube-api-access-64sg8" (OuterVolumeSpecName: "kube-api-access-64sg8") pod "f88daa17-e943-49ad-b56a-f80ccabcebc0" (UID: "f88daa17-e943-49ad-b56a-f80ccabcebc0"). InnerVolumeSpecName "kube-api-access-64sg8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 12:27:30.953954 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:30.953930    2567 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f88daa17-e943-49ad-b56a-f80ccabcebc0-service-ca\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:27:30.953954 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:30.953950    2567 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f88daa17-e943-49ad-b56a-f80ccabcebc0-console-serving-cert\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:27:30.954073 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:30.953960    2567 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f88daa17-e943-49ad-b56a-f80ccabcebc0-oauth-serving-cert\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:27:30.954073 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:30.953969    2567 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f88daa17-e943-49ad-b56a-f80ccabcebc0-console-oauth-config\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:27:30.954073 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:30.953978    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-64sg8\" (UniqueName: \"kubernetes.io/projected/f88daa17-e943-49ad-b56a-f80ccabcebc0-kube-api-access-64sg8\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:27:30.954073 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:30.953986    2567 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f88daa17-e943-49ad-b56a-f80ccabcebc0-trusted-ca-bundle\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:27:30.954073 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:30.954002    2567 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f88daa17-e943-49ad-b56a-f80ccabcebc0-console-config\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:27:31.582165 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:31.582132    2567 generic.go:356] "Generic (PLEG): container finished" podID="f88daa17-e943-49ad-b56a-f80ccabcebc0" containerID="635324da685dcdba4e80404433f8e80f354073c60f367fb65c5a24b95743ac83" exitCode=0
Jul 29 12:27:31.582515 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:31.582175    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66c6bd9f9c-fnvkh"
Jul 29 12:27:31.582515 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:31.582223    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66c6bd9f9c-fnvkh" event={"ID":"f88daa17-e943-49ad-b56a-f80ccabcebc0","Type":"ContainerDied","Data":"635324da685dcdba4e80404433f8e80f354073c60f367fb65c5a24b95743ac83"}
Jul 29 12:27:31.582515 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:31.582278    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66c6bd9f9c-fnvkh" event={"ID":"f88daa17-e943-49ad-b56a-f80ccabcebc0","Type":"ContainerDied","Data":"c9ff5d27f2ce4022c6f1998ed0795f9beb24bc4e09a7dfb01fefbee848b22be0"}
Jul 29 12:27:31.582515 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:31.582305    2567 scope.go:117] "RemoveContainer" containerID="635324da685dcdba4e80404433f8e80f354073c60f367fb65c5a24b95743ac83"
Jul 29 12:27:31.599475 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:31.599380    2567 scope.go:117] "RemoveContainer" containerID="635324da685dcdba4e80404433f8e80f354073c60f367fb65c5a24b95743ac83"
Jul 29 12:27:31.599677 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:27:31.599659    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"635324da685dcdba4e80404433f8e80f354073c60f367fb65c5a24b95743ac83\": container with ID starting with 635324da685dcdba4e80404433f8e80f354073c60f367fb65c5a24b95743ac83 not found: ID does not exist" containerID="635324da685dcdba4e80404433f8e80f354073c60f367fb65c5a24b95743ac83"
Jul 29 12:27:31.599730 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:31.599685    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"635324da685dcdba4e80404433f8e80f354073c60f367fb65c5a24b95743ac83"} err="failed to get container status \"635324da685dcdba4e80404433f8e80f354073c60f367fb65c5a24b95743ac83\": rpc error: code = NotFound desc = could not find container \"635324da685dcdba4e80404433f8e80f354073c60f367fb65c5a24b95743ac83\": container with ID starting with 635324da685dcdba4e80404433f8e80f354073c60f367fb65c5a24b95743ac83 not found: ID does not exist"
Jul 29 12:27:31.608505 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:31.608483    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-66c6bd9f9c-fnvkh"]
Jul 29 12:27:31.625024 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:31.625004    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-66c6bd9f9c-fnvkh"]
Jul 29 12:27:31.920546 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:31.920467    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f88daa17-e943-49ad-b56a-f80ccabcebc0" path="/var/lib/kubelet/pods/f88daa17-e943-49ad-b56a-f80ccabcebc0/volumes"
Jul 29 12:27:34.700863 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:34.700831    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5479989fc7-qqwlm"
Jul 29 12:27:34.704837 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:27:34.704810    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5479989fc7-qqwlm"
Jul 29 12:28:14.018153 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:14.018076    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7c9474dfd4-fbwrc"]
Jul 29 12:28:14.018567 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:14.018332    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dff975a8-8807-4dbe-9124-7c96c00a1949" containerName="registry"
Jul 29 12:28:14.018567 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:14.018346    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="dff975a8-8807-4dbe-9124-7c96c00a1949" containerName="registry"
Jul 29 12:28:14.018567 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:14.018357    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f88daa17-e943-49ad-b56a-f80ccabcebc0" containerName="console"
Jul 29 12:28:14.018567 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:14.018362    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="f88daa17-e943-49ad-b56a-f80ccabcebc0" containerName="console"
Jul 29 12:28:14.018567 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:14.018439    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="dff975a8-8807-4dbe-9124-7c96c00a1949" containerName="registry"
Jul 29 12:28:14.018567 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:14.018450    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="f88daa17-e943-49ad-b56a-f80ccabcebc0" containerName="console"
Jul 29 12:28:14.020321 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:14.020305    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7c9474dfd4-fbwrc"
Jul 29 12:28:14.023469 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:14.023448    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 29 12:28:14.023611 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:14.023549    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 29 12:28:14.023825 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:14.023809    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 29 12:28:14.023908 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:14.023822    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 29 12:28:14.023966 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:14.023952    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-hd49m\""
Jul 29 12:28:14.024044 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:14.024028    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 29 12:28:14.030051 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:14.030033    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 29 12:28:14.037890 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:14.037871    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7c9474dfd4-fbwrc"]
Jul 29 12:28:14.176119 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:14.176098    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f02601fa-bbb1-4088-bc3f-c96dae91017f-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7c9474dfd4-fbwrc\" (UID: \"f02601fa-bbb1-4088-bc3f-c96dae91017f\") " pod="openshift-monitoring/telemeter-client-7c9474dfd4-fbwrc"
Jul 29 12:28:14.176226 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:14.176123    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/f02601fa-bbb1-4088-bc3f-c96dae91017f-telemeter-client-tls\") pod \"telemeter-client-7c9474dfd4-fbwrc\" (UID: \"f02601fa-bbb1-4088-bc3f-c96dae91017f\") " pod="openshift-monitoring/telemeter-client-7c9474dfd4-fbwrc"
Jul 29 12:28:14.176226 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:14.176163    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/f02601fa-bbb1-4088-bc3f-c96dae91017f-federate-client-tls\") pod \"telemeter-client-7c9474dfd4-fbwrc\" (UID: \"f02601fa-bbb1-4088-bc3f-c96dae91017f\") " pod="openshift-monitoring/telemeter-client-7c9474dfd4-fbwrc"
Jul 29 12:28:14.176226 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:14.176184    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f02601fa-bbb1-4088-bc3f-c96dae91017f-serving-certs-ca-bundle\") pod \"telemeter-client-7c9474dfd4-fbwrc\" (UID: \"f02601fa-bbb1-4088-bc3f-c96dae91017f\") " pod="openshift-monitoring/telemeter-client-7c9474dfd4-fbwrc"
Jul 29 12:28:14.176338 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:14.176246    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f02601fa-bbb1-4088-bc3f-c96dae91017f-metrics-client-ca\") pod \"telemeter-client-7c9474dfd4-fbwrc\" (UID: \"f02601fa-bbb1-4088-bc3f-c96dae91017f\") " pod="openshift-monitoring/telemeter-client-7c9474dfd4-fbwrc"
Jul 29 12:28:14.176338 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:14.176282    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f02601fa-bbb1-4088-bc3f-c96dae91017f-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7c9474dfd4-fbwrc\" (UID: \"f02601fa-bbb1-4088-bc3f-c96dae91017f\") " pod="openshift-monitoring/telemeter-client-7c9474dfd4-fbwrc"
Jul 29 12:28:14.176338 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:14.176303    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/f02601fa-bbb1-4088-bc3f-c96dae91017f-secret-telemeter-client\") pod \"telemeter-client-7c9474dfd4-fbwrc\" (UID: \"f02601fa-bbb1-4088-bc3f-c96dae91017f\") " pod="openshift-monitoring/telemeter-client-7c9474dfd4-fbwrc"
Jul 29 12:28:14.176338 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:14.176322    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmz8q\" (UniqueName: \"kubernetes.io/projected/f02601fa-bbb1-4088-bc3f-c96dae91017f-kube-api-access-wmz8q\") pod \"telemeter-client-7c9474dfd4-fbwrc\" (UID: \"f02601fa-bbb1-4088-bc3f-c96dae91017f\") " pod="openshift-monitoring/telemeter-client-7c9474dfd4-fbwrc"
Jul 29 12:28:14.277185 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:14.277159    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f02601fa-bbb1-4088-bc3f-c96dae91017f-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7c9474dfd4-fbwrc\" (UID: \"f02601fa-bbb1-4088-bc3f-c96dae91017f\") " pod="openshift-monitoring/telemeter-client-7c9474dfd4-fbwrc"
Jul 29 12:28:14.277280 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:14.277208    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/f02601fa-bbb1-4088-bc3f-c96dae91017f-telemeter-client-tls\") pod \"telemeter-client-7c9474dfd4-fbwrc\" (UID: \"f02601fa-bbb1-4088-bc3f-c96dae91017f\") " pod="openshift-monitoring/telemeter-client-7c9474dfd4-fbwrc"
Jul 29 12:28:14.277280 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:14.277247    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/f02601fa-bbb1-4088-bc3f-c96dae91017f-federate-client-tls\") pod \"telemeter-client-7c9474dfd4-fbwrc\" (UID: \"f02601fa-bbb1-4088-bc3f-c96dae91017f\") " pod="openshift-monitoring/telemeter-client-7c9474dfd4-fbwrc"
Jul 29 12:28:14.277361 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:14.277281    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f02601fa-bbb1-4088-bc3f-c96dae91017f-serving-certs-ca-bundle\") pod \"telemeter-client-7c9474dfd4-fbwrc\" (UID: \"f02601fa-bbb1-4088-bc3f-c96dae91017f\") " pod="openshift-monitoring/telemeter-client-7c9474dfd4-fbwrc"
Jul 29 12:28:14.277361 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:14.277313    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f02601fa-bbb1-4088-bc3f-c96dae91017f-metrics-client-ca\") pod \"telemeter-client-7c9474dfd4-fbwrc\" (UID: \"f02601fa-bbb1-4088-bc3f-c96dae91017f\") " pod="openshift-monitoring/telemeter-client-7c9474dfd4-fbwrc"
Jul 29 12:28:14.277510 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:14.277469    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f02601fa-bbb1-4088-bc3f-c96dae91017f-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7c9474dfd4-fbwrc\" (UID: \"f02601fa-bbb1-4088-bc3f-c96dae91017f\") " pod="openshift-monitoring/telemeter-client-7c9474dfd4-fbwrc"
Jul 29 12:28:14.277568 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:14.277516    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/f02601fa-bbb1-4088-bc3f-c96dae91017f-secret-telemeter-client\") pod \"telemeter-client-7c9474dfd4-fbwrc\" (UID: \"f02601fa-bbb1-4088-bc3f-c96dae91017f\") " pod="openshift-monitoring/telemeter-client-7c9474dfd4-fbwrc"
Jul 29 12:28:14.277568 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:14.277549    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmz8q\" (UniqueName: \"kubernetes.io/projected/f02601fa-bbb1-4088-bc3f-c96dae91017f-kube-api-access-wmz8q\") pod \"telemeter-client-7c9474dfd4-fbwrc\" (UID: \"f02601fa-bbb1-4088-bc3f-c96dae91017f\") " pod="openshift-monitoring/telemeter-client-7c9474dfd4-fbwrc"
Jul 29 12:28:14.278128 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:14.278096    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f02601fa-bbb1-4088-bc3f-c96dae91017f-serving-certs-ca-bundle\") pod \"telemeter-client-7c9474dfd4-fbwrc\" (UID: \"f02601fa-bbb1-4088-bc3f-c96dae91017f\") " pod="openshift-monitoring/telemeter-client-7c9474dfd4-fbwrc"
Jul 29 12:28:14.278259 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:14.278172    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f02601fa-bbb1-4088-bc3f-c96dae91017f-metrics-client-ca\") pod \"telemeter-client-7c9474dfd4-fbwrc\" (UID: \"f02601fa-bbb1-4088-bc3f-c96dae91017f\") " pod="openshift-monitoring/telemeter-client-7c9474dfd4-fbwrc"
Jul 29 12:28:14.278494 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:14.278462    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f02601fa-bbb1-4088-bc3f-c96dae91017f-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7c9474dfd4-fbwrc\" (UID: \"f02601fa-bbb1-4088-bc3f-c96dae91017f\") " pod="openshift-monitoring/telemeter-client-7c9474dfd4-fbwrc"
Jul 29 12:28:14.280014 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:14.279980    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f02601fa-bbb1-4088-bc3f-c96dae91017f-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7c9474dfd4-fbwrc\" (UID: \"f02601fa-bbb1-4088-bc3f-c96dae91017f\") " pod="openshift-monitoring/telemeter-client-7c9474dfd4-fbwrc"
Jul 29 12:28:14.280127 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:14.280095    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/f02601fa-bbb1-4088-bc3f-c96dae91017f-telemeter-client-tls\") pod \"telemeter-client-7c9474dfd4-fbwrc\" (UID: \"f02601fa-bbb1-4088-bc3f-c96dae91017f\") " pod="openshift-monitoring/telemeter-client-7c9474dfd4-fbwrc"
Jul 29 12:28:14.280364 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:14.280342    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/f02601fa-bbb1-4088-bc3f-c96dae91017f-federate-client-tls\") pod \"telemeter-client-7c9474dfd4-fbwrc\" (UID: \"f02601fa-bbb1-4088-bc3f-c96dae91017f\") " pod="openshift-monitoring/telemeter-client-7c9474dfd4-fbwrc"
Jul 29 12:28:14.280456 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:14.280436    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/f02601fa-bbb1-4088-bc3f-c96dae91017f-secret-telemeter-client\") pod \"telemeter-client-7c9474dfd4-fbwrc\" (UID: \"f02601fa-bbb1-4088-bc3f-c96dae91017f\") " pod="openshift-monitoring/telemeter-client-7c9474dfd4-fbwrc"
Jul 29 12:28:14.285982 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:14.285957    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmz8q\" (UniqueName: \"kubernetes.io/projected/f02601fa-bbb1-4088-bc3f-c96dae91017f-kube-api-access-wmz8q\") pod \"telemeter-client-7c9474dfd4-fbwrc\" (UID: \"f02601fa-bbb1-4088-bc3f-c96dae91017f\") " pod="openshift-monitoring/telemeter-client-7c9474dfd4-fbwrc"
Jul 29 12:28:14.330352 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:14.330319    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7c9474dfd4-fbwrc"
Jul 29 12:28:14.468488 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:14.468453    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7c9474dfd4-fbwrc"]
Jul 29 12:28:14.471460 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:28:14.471437    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf02601fa_bbb1_4088_bc3f_c96dae91017f.slice/crio-e796d9ead8426e40219dd6eee3f9f2d7a9d74157e2de68be6daf085d3b441823 WatchSource:0}: Error finding container e796d9ead8426e40219dd6eee3f9f2d7a9d74157e2de68be6daf085d3b441823: Status 404 returned error can't find the container with id e796d9ead8426e40219dd6eee3f9f2d7a9d74157e2de68be6daf085d3b441823
Jul 29 12:28:14.749485 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:14.749420    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7c9474dfd4-fbwrc" event={"ID":"f02601fa-bbb1-4088-bc3f-c96dae91017f","Type":"ContainerStarted","Data":"e796d9ead8426e40219dd6eee3f9f2d7a9d74157e2de68be6daf085d3b441823"}
Jul 29 12:28:17.765444 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:17.765388    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7c9474dfd4-fbwrc" event={"ID":"f02601fa-bbb1-4088-bc3f-c96dae91017f","Type":"ContainerStarted","Data":"131edc7e27b5cfbe18270094bfff5ab53886dc799696b956eb38d47d92f4f307"}
Jul 29 12:28:17.765839 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:17.765479    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7c9474dfd4-fbwrc" event={"ID":"f02601fa-bbb1-4088-bc3f-c96dae91017f","Type":"ContainerStarted","Data":"053874344bc41b1a293af543c2d9fc90db5a0ac120beccb2df09315b0a38e3ce"}
Jul 29 12:28:17.765839 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:17.765511    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7c9474dfd4-fbwrc" event={"ID":"f02601fa-bbb1-4088-bc3f-c96dae91017f","Type":"ContainerStarted","Data":"3de661aee1995f35543fdc8efeaa447b2e115e726ef1df3ab2dc9900a43f1257"}
Jul 29 12:28:17.787589 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:17.787546    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7c9474dfd4-fbwrc" podStartSLOduration=2.465010677 podStartE2EDuration="4.787529554s" totalImagesPullingTime="2.322518877s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:28:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:28:17.786232044 +0000 UTC m=+180.479920375" watchObservedRunningTime="2026-07-29 12:28:17.787529554 +0000 UTC m=+180.481217864"
Jul 29 12:28:18.610476 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:18.610437    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-96c76976-9z4gt"]
Jul 29 12:28:18.613275 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:18.613254    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-96c76976-9z4gt"
Jul 29 12:28:18.627190 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:18.627156    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-96c76976-9z4gt"]
Jul 29 12:28:18.711310 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:18.711279    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b1749b1a-92e5-4543-b92f-62120445b013-service-ca\") pod \"console-96c76976-9z4gt\" (UID: \"b1749b1a-92e5-4543-b92f-62120445b013\") " pod="openshift-console/console-96c76976-9z4gt"
Jul 29 12:28:18.711476 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:18.711313    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b1749b1a-92e5-4543-b92f-62120445b013-console-serving-cert\") pod \"console-96c76976-9z4gt\" (UID: \"b1749b1a-92e5-4543-b92f-62120445b013\") " pod="openshift-console/console-96c76976-9z4gt"
Jul 29 12:28:18.711476 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:18.711337    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kffjt\" (UniqueName: \"kubernetes.io/projected/b1749b1a-92e5-4543-b92f-62120445b013-kube-api-access-kffjt\") pod \"console-96c76976-9z4gt\" (UID: \"b1749b1a-92e5-4543-b92f-62120445b013\") " pod="openshift-console/console-96c76976-9z4gt"
Jul 29 12:28:18.711612 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:18.711482    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b1749b1a-92e5-4543-b92f-62120445b013-console-config\") pod \"console-96c76976-9z4gt\" (UID: \"b1749b1a-92e5-4543-b92f-62120445b013\") " pod="openshift-console/console-96c76976-9z4gt"
Jul 29 12:28:18.711612 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:18.711538    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b1749b1a-92e5-4543-b92f-62120445b013-trusted-ca-bundle\") pod \"console-96c76976-9z4gt\" (UID: \"b1749b1a-92e5-4543-b92f-62120445b013\") " pod="openshift-console/console-96c76976-9z4gt"
Jul 29 12:28:18.711612 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:18.711570    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b1749b1a-92e5-4543-b92f-62120445b013-console-oauth-config\") pod \"console-96c76976-9z4gt\" (UID: \"b1749b1a-92e5-4543-b92f-62120445b013\") " pod="openshift-console/console-96c76976-9z4gt"
Jul 29 12:28:18.711612 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:18.711600    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b1749b1a-92e5-4543-b92f-62120445b013-oauth-serving-cert\") pod \"console-96c76976-9z4gt\" (UID: \"b1749b1a-92e5-4543-b92f-62120445b013\") " pod="openshift-console/console-96c76976-9z4gt"
Jul 29 12:28:18.812601 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:18.812572    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b1749b1a-92e5-4543-b92f-62120445b013-console-oauth-config\") pod \"console-96c76976-9z4gt\" (UID: \"b1749b1a-92e5-4543-b92f-62120445b013\") " pod="openshift-console/console-96c76976-9z4gt"
Jul 29 12:28:18.812947 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:18.812606    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b1749b1a-92e5-4543-b92f-62120445b013-oauth-serving-cert\") pod \"console-96c76976-9z4gt\" (UID: \"b1749b1a-92e5-4543-b92f-62120445b013\") " pod="openshift-console/console-96c76976-9z4gt"
Jul 29 12:28:18.812947 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:18.812669    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b1749b1a-92e5-4543-b92f-62120445b013-service-ca\") pod \"console-96c76976-9z4gt\" (UID: \"b1749b1a-92e5-4543-b92f-62120445b013\") " pod="openshift-console/console-96c76976-9z4gt"
Jul 29 12:28:18.812947 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:18.812696    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b1749b1a-92e5-4543-b92f-62120445b013-console-serving-cert\") pod \"console-96c76976-9z4gt\" (UID: \"b1749b1a-92e5-4543-b92f-62120445b013\") " pod="openshift-console/console-96c76976-9z4gt"
Jul 29 12:28:18.812947 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:18.812725    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kffjt\" (UniqueName: \"kubernetes.io/projected/b1749b1a-92e5-4543-b92f-62120445b013-kube-api-access-kffjt\") pod \"console-96c76976-9z4gt\" (UID: \"b1749b1a-92e5-4543-b92f-62120445b013\") " pod="openshift-console/console-96c76976-9z4gt"
Jul 29 12:28:18.812947 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:18.812844    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b1749b1a-92e5-4543-b92f-62120445b013-console-config\") pod \"console-96c76976-9z4gt\" (UID: \"b1749b1a-92e5-4543-b92f-62120445b013\") " pod="openshift-console/console-96c76976-9z4gt"
Jul 29 12:28:18.812947 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:18.812894    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b1749b1a-92e5-4543-b92f-62120445b013-trusted-ca-bundle\") pod \"console-96c76976-9z4gt\" (UID: \"b1749b1a-92e5-4543-b92f-62120445b013\") " pod="openshift-console/console-96c76976-9z4gt"
Jul 29 12:28:18.813602 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:18.813570    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b1749b1a-92e5-4543-b92f-62120445b013-service-ca\") pod \"console-96c76976-9z4gt\" (UID: \"b1749b1a-92e5-4543-b92f-62120445b013\") " pod="openshift-console/console-96c76976-9z4gt"
Jul 29 12:28:18.813707 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:18.813622    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b1749b1a-92e5-4543-b92f-62120445b013-console-config\") pod \"console-96c76976-9z4gt\" (UID: \"b1749b1a-92e5-4543-b92f-62120445b013\") " pod="openshift-console/console-96c76976-9z4gt"
Jul 29 12:28:18.813788 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:18.813763    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b1749b1a-92e5-4543-b92f-62120445b013-oauth-serving-cert\") pod \"console-96c76976-9z4gt\" (UID: \"b1749b1a-92e5-4543-b92f-62120445b013\") " pod="openshift-console/console-96c76976-9z4gt"
Jul 29 12:28:18.813941 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:18.813917    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b1749b1a-92e5-4543-b92f-62120445b013-trusted-ca-bundle\") pod \"console-96c76976-9z4gt\" (UID: \"b1749b1a-92e5-4543-b92f-62120445b013\") " pod="openshift-console/console-96c76976-9z4gt"
Jul 29 12:28:18.815315 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:18.815285    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b1749b1a-92e5-4543-b92f-62120445b013-console-serving-cert\") pod \"console-96c76976-9z4gt\" (UID: \"b1749b1a-92e5-4543-b92f-62120445b013\") " pod="openshift-console/console-96c76976-9z4gt"
Jul 29 12:28:18.815496 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:18.815473    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b1749b1a-92e5-4543-b92f-62120445b013-console-oauth-config\") pod \"console-96c76976-9z4gt\" (UID: \"b1749b1a-92e5-4543-b92f-62120445b013\") " pod="openshift-console/console-96c76976-9z4gt"
Jul 29 12:28:18.820211 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:18.820185    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kffjt\" (UniqueName: \"kubernetes.io/projected/b1749b1a-92e5-4543-b92f-62120445b013-kube-api-access-kffjt\") pod \"console-96c76976-9z4gt\" (UID: \"b1749b1a-92e5-4543-b92f-62120445b013\") " pod="openshift-console/console-96c76976-9z4gt"
Jul 29 12:28:18.923271 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:18.923206    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-96c76976-9z4gt"
Jul 29 12:28:19.049699 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:19.049568    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-96c76976-9z4gt"]
Jul 29 12:28:19.052101 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:28:19.052070    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1749b1a_92e5_4543_b92f_62120445b013.slice/crio-50b461bb19f38b939c43e54bf5f63ee314b9d7bd9d99063e4a35d18e814017ff WatchSource:0}: Error finding container 50b461bb19f38b939c43e54bf5f63ee314b9d7bd9d99063e4a35d18e814017ff: Status 404 returned error can't find the container with id 50b461bb19f38b939c43e54bf5f63ee314b9d7bd9d99063e4a35d18e814017ff
Jul 29 12:28:19.776467 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:19.776431    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-96c76976-9z4gt" event={"ID":"b1749b1a-92e5-4543-b92f-62120445b013","Type":"ContainerStarted","Data":"46aa841c75a8edc87994bbc324a855d7648b61d408c207b77f5bbd397d0d7891"}
Jul 29 12:28:19.776734 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:19.776525    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-96c76976-9z4gt" event={"ID":"b1749b1a-92e5-4543-b92f-62120445b013","Type":"ContainerStarted","Data":"50b461bb19f38b939c43e54bf5f63ee314b9d7bd9d99063e4a35d18e814017ff"}
Jul 29 12:28:19.799793 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:19.799737    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-96c76976-9z4gt" podStartSLOduration=1.79971909 podStartE2EDuration="1.79971909s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:28:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:28:19.798918558 +0000 UTC m=+182.492606889" watchObservedRunningTime="2026-07-29 12:28:19.79971909 +0000 UTC m=+182.493407400"
Jul 29 12:28:25.711777 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:25.711738    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-96c76976-9z4gt"]
Jul 29 12:28:28.923329 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:28.923295    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-96c76976-9z4gt"
Jul 29 12:28:36.627842 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:36.627802    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-579695d779-p47b4"]
Jul 29 12:28:50.731053 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:50.731003    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-96c76976-9z4gt" podUID="b1749b1a-92e5-4543-b92f-62120445b013" containerName="console" containerID="cri-o://46aa841c75a8edc87994bbc324a855d7648b61d408c207b77f5bbd397d0d7891" gracePeriod=15
Jul 29 12:28:50.907769 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:50.907733    2567 generic.go:356] "Generic (PLEG): container finished" podID="b1749b1a-92e5-4543-b92f-62120445b013" containerID="46aa841c75a8edc87994bbc324a855d7648b61d408c207b77f5bbd397d0d7891" exitCode=0
Jul 29 12:28:50.907930 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:50.907836    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-96c76976-9z4gt" event={"ID":"b1749b1a-92e5-4543-b92f-62120445b013","Type":"ContainerDied","Data":"46aa841c75a8edc87994bbc324a855d7648b61d408c207b77f5bbd397d0d7891"}
Jul 29 12:28:50.969129 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:50.969109    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-96c76976-9z4gt"
Jul 29 12:28:51.037796 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:51.037773    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b1749b1a-92e5-4543-b92f-62120445b013-oauth-serving-cert\") pod \"b1749b1a-92e5-4543-b92f-62120445b013\" (UID: \"b1749b1a-92e5-4543-b92f-62120445b013\") "
Jul 29 12:28:51.037912 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:51.037803    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kffjt\" (UniqueName: \"kubernetes.io/projected/b1749b1a-92e5-4543-b92f-62120445b013-kube-api-access-kffjt\") pod \"b1749b1a-92e5-4543-b92f-62120445b013\" (UID: \"b1749b1a-92e5-4543-b92f-62120445b013\") "
Jul 29 12:28:51.037912 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:51.037834    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b1749b1a-92e5-4543-b92f-62120445b013-console-config\") pod \"b1749b1a-92e5-4543-b92f-62120445b013\" (UID: \"b1749b1a-92e5-4543-b92f-62120445b013\") "
Jul 29 12:28:51.037912 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:51.037879    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b1749b1a-92e5-4543-b92f-62120445b013-console-oauth-config\") pod \"b1749b1a-92e5-4543-b92f-62120445b013\" (UID: \"b1749b1a-92e5-4543-b92f-62120445b013\") "
Jul 29 12:28:51.038074 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:51.037926    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b1749b1a-92e5-4543-b92f-62120445b013-service-ca\") pod \"b1749b1a-92e5-4543-b92f-62120445b013\" (UID: \"b1749b1a-92e5-4543-b92f-62120445b013\") "
Jul 29 12:28:51.038074 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:51.037952    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b1749b1a-92e5-4543-b92f-62120445b013-console-serving-cert\") pod \"b1749b1a-92e5-4543-b92f-62120445b013\" (UID: \"b1749b1a-92e5-4543-b92f-62120445b013\") "
Jul 29 12:28:51.038074 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:51.037971    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b1749b1a-92e5-4543-b92f-62120445b013-trusted-ca-bundle\") pod \"b1749b1a-92e5-4543-b92f-62120445b013\" (UID: \"b1749b1a-92e5-4543-b92f-62120445b013\") "
Jul 29 12:28:51.038252 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:51.038230    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1749b1a-92e5-4543-b92f-62120445b013-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b1749b1a-92e5-4543-b92f-62120445b013" (UID: "b1749b1a-92e5-4543-b92f-62120445b013"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 12:28:51.038309 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:51.038267    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1749b1a-92e5-4543-b92f-62120445b013-console-config" (OuterVolumeSpecName: "console-config") pod "b1749b1a-92e5-4543-b92f-62120445b013" (UID: "b1749b1a-92e5-4543-b92f-62120445b013"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 12:28:51.038358 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:51.038332    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1749b1a-92e5-4543-b92f-62120445b013-service-ca" (OuterVolumeSpecName: "service-ca") pod "b1749b1a-92e5-4543-b92f-62120445b013" (UID: "b1749b1a-92e5-4543-b92f-62120445b013"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 12:28:51.038635 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:51.038611    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1749b1a-92e5-4543-b92f-62120445b013-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "b1749b1a-92e5-4543-b92f-62120445b013" (UID: "b1749b1a-92e5-4543-b92f-62120445b013"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 12:28:51.039925 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:51.039901    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1749b1a-92e5-4543-b92f-62120445b013-kube-api-access-kffjt" (OuterVolumeSpecName: "kube-api-access-kffjt") pod "b1749b1a-92e5-4543-b92f-62120445b013" (UID: "b1749b1a-92e5-4543-b92f-62120445b013"). InnerVolumeSpecName "kube-api-access-kffjt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 12:28:51.040008 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:51.039982    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1749b1a-92e5-4543-b92f-62120445b013-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b1749b1a-92e5-4543-b92f-62120445b013" (UID: "b1749b1a-92e5-4543-b92f-62120445b013"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 12:28:51.040046 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:51.040027    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1749b1a-92e5-4543-b92f-62120445b013-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b1749b1a-92e5-4543-b92f-62120445b013" (UID: "b1749b1a-92e5-4543-b92f-62120445b013"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 12:28:51.139052 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:51.139031    2567 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b1749b1a-92e5-4543-b92f-62120445b013-trusted-ca-bundle\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:28:51.139052 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:51.139049    2567 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b1749b1a-92e5-4543-b92f-62120445b013-oauth-serving-cert\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:28:51.139159 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:51.139058    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kffjt\" (UniqueName: \"kubernetes.io/projected/b1749b1a-92e5-4543-b92f-62120445b013-kube-api-access-kffjt\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:28:51.139159 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:51.139067    2567 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b1749b1a-92e5-4543-b92f-62120445b013-console-config\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:28:51.139159 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:51.139076    2567 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b1749b1a-92e5-4543-b92f-62120445b013-console-oauth-config\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:28:51.139159 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:51.139085    2567 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b1749b1a-92e5-4543-b92f-62120445b013-service-ca\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:28:51.139159 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:51.139093    2567 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b1749b1a-92e5-4543-b92f-62120445b013-console-serving-cert\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:28:51.913407 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:51.913380    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-96c76976-9z4gt"
Jul 29 12:28:51.913773 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:51.913433    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-96c76976-9z4gt" event={"ID":"b1749b1a-92e5-4543-b92f-62120445b013","Type":"ContainerDied","Data":"50b461bb19f38b939c43e54bf5f63ee314b9d7bd9d99063e4a35d18e814017ff"}
Jul 29 12:28:51.913773 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:51.913498    2567 scope.go:117] "RemoveContainer" containerID="46aa841c75a8edc87994bbc324a855d7648b61d408c207b77f5bbd397d0d7891"
Jul 29 12:28:51.939059 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:51.939035    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-96c76976-9z4gt"]
Jul 29 12:28:51.943577 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:51.943553    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-96c76976-9z4gt"]
Jul 29 12:28:53.920971 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:28:53.920931    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1749b1a-92e5-4543-b92f-62120445b013" path="/var/lib/kubelet/pods/b1749b1a-92e5-4543-b92f-62120445b013/volumes"
Jul 29 12:29:01.646665 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:29:01.646615    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-579695d779-p47b4" podUID="3e2a11eb-61da-4dd1-a03c-cb8810e610a5" containerName="console" containerID="cri-o://78154e7c8b35c621186efc70e6a78db1c9f5384e0611cb0bbfe7ec69c66d089c" gracePeriod=15
Jul 29 12:29:01.886737 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:29:01.886713    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-579695d779-p47b4"
Jul 29 12:29:01.953165 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:29:01.953103    2567 generic.go:356] "Generic (PLEG): container finished" podID="3e2a11eb-61da-4dd1-a03c-cb8810e610a5" containerID="78154e7c8b35c621186efc70e6a78db1c9f5384e0611cb0bbfe7ec69c66d089c" exitCode=0
Jul 29 12:29:01.953165 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:29:01.953144    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-579695d779-p47b4"
Jul 29 12:29:01.953333 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:29:01.953186    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-579695d779-p47b4" event={"ID":"3e2a11eb-61da-4dd1-a03c-cb8810e610a5","Type":"ContainerDied","Data":"78154e7c8b35c621186efc70e6a78db1c9f5384e0611cb0bbfe7ec69c66d089c"}
Jul 29 12:29:01.953333 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:29:01.953231    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-579695d779-p47b4" event={"ID":"3e2a11eb-61da-4dd1-a03c-cb8810e610a5","Type":"ContainerDied","Data":"36d3cb4b260b2a696d62448016811ff7915f9de86f576e45570172a4d24e9bf9"}
Jul 29 12:29:01.953333 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:29:01.953252    2567 scope.go:117] "RemoveContainer" containerID="78154e7c8b35c621186efc70e6a78db1c9f5384e0611cb0bbfe7ec69c66d089c"
Jul 29 12:29:01.962388 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:29:01.962361    2567 scope.go:117] "RemoveContainer" containerID="78154e7c8b35c621186efc70e6a78db1c9f5384e0611cb0bbfe7ec69c66d089c"
Jul 29 12:29:01.962702 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:29:01.962682    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78154e7c8b35c621186efc70e6a78db1c9f5384e0611cb0bbfe7ec69c66d089c\": container with ID starting with 78154e7c8b35c621186efc70e6a78db1c9f5384e0611cb0bbfe7ec69c66d089c not found: ID does not exist" containerID="78154e7c8b35c621186efc70e6a78db1c9f5384e0611cb0bbfe7ec69c66d089c"
Jul 29 12:29:01.962760 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:29:01.962714    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78154e7c8b35c621186efc70e6a78db1c9f5384e0611cb0bbfe7ec69c66d089c"} err="failed to get container status \"78154e7c8b35c621186efc70e6a78db1c9f5384e0611cb0bbfe7ec69c66d089c\": rpc error: code = NotFound desc = could not find container \"78154e7c8b35c621186efc70e6a78db1c9f5384e0611cb0bbfe7ec69c66d089c\": container with ID starting with 78154e7c8b35c621186efc70e6a78db1c9f5384e0611cb0bbfe7ec69c66d089c not found: ID does not exist"
Jul 29 12:29:02.012958 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:29:02.012934    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3e2a11eb-61da-4dd1-a03c-cb8810e610a5-console-oauth-config\") pod \"3e2a11eb-61da-4dd1-a03c-cb8810e610a5\" (UID: \"3e2a11eb-61da-4dd1-a03c-cb8810e610a5\") "
Jul 29 12:29:02.013040 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:29:02.012966    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3e2a11eb-61da-4dd1-a03c-cb8810e610a5-service-ca\") pod \"3e2a11eb-61da-4dd1-a03c-cb8810e610a5\" (UID: \"3e2a11eb-61da-4dd1-a03c-cb8810e610a5\") "
Jul 29 12:29:02.013040 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:29:02.012988    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w9lfm\" (UniqueName: \"kubernetes.io/projected/3e2a11eb-61da-4dd1-a03c-cb8810e610a5-kube-api-access-w9lfm\") pod \"3e2a11eb-61da-4dd1-a03c-cb8810e610a5\" (UID: \"3e2a11eb-61da-4dd1-a03c-cb8810e610a5\") "
Jul 29 12:29:02.013040 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:29:02.013029    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e2a11eb-61da-4dd1-a03c-cb8810e610a5-trusted-ca-bundle\") pod \"3e2a11eb-61da-4dd1-a03c-cb8810e610a5\" (UID: \"3e2a11eb-61da-4dd1-a03c-cb8810e610a5\") "
Jul 29 12:29:02.013181 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:29:02.013051    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3e2a11eb-61da-4dd1-a03c-cb8810e610a5-oauth-serving-cert\") pod \"3e2a11eb-61da-4dd1-a03c-cb8810e610a5\" (UID: \"3e2a11eb-61da-4dd1-a03c-cb8810e610a5\") "
Jul 29 12:29:02.013181 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:29:02.013072    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3e2a11eb-61da-4dd1-a03c-cb8810e610a5-console-serving-cert\") pod \"3e2a11eb-61da-4dd1-a03c-cb8810e610a5\" (UID: \"3e2a11eb-61da-4dd1-a03c-cb8810e610a5\") "
Jul 29 12:29:02.013181 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:29:02.013101    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3e2a11eb-61da-4dd1-a03c-cb8810e610a5-console-config\") pod \"3e2a11eb-61da-4dd1-a03c-cb8810e610a5\" (UID: \"3e2a11eb-61da-4dd1-a03c-cb8810e610a5\") "
Jul 29 12:29:02.013385 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:29:02.013359    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e2a11eb-61da-4dd1-a03c-cb8810e610a5-service-ca" (OuterVolumeSpecName: "service-ca") pod "3e2a11eb-61da-4dd1-a03c-cb8810e610a5" (UID: "3e2a11eb-61da-4dd1-a03c-cb8810e610a5"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 12:29:02.013523 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:29:02.013494    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e2a11eb-61da-4dd1-a03c-cb8810e610a5-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "3e2a11eb-61da-4dd1-a03c-cb8810e610a5" (UID: "3e2a11eb-61da-4dd1-a03c-cb8810e610a5"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 12:29:02.013523 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:29:02.013505    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e2a11eb-61da-4dd1-a03c-cb8810e610a5-console-config" (OuterVolumeSpecName: "console-config") pod "3e2a11eb-61da-4dd1-a03c-cb8810e610a5" (UID: "3e2a11eb-61da-4dd1-a03c-cb8810e610a5"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 12:29:02.013964 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:29:02.013942    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e2a11eb-61da-4dd1-a03c-cb8810e610a5-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "3e2a11eb-61da-4dd1-a03c-cb8810e610a5" (UID: "3e2a11eb-61da-4dd1-a03c-cb8810e610a5"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 12:29:02.015111 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:29:02.015081    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e2a11eb-61da-4dd1-a03c-cb8810e610a5-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "3e2a11eb-61da-4dd1-a03c-cb8810e610a5" (UID: "3e2a11eb-61da-4dd1-a03c-cb8810e610a5"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 12:29:02.015542 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:29:02.015520    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e2a11eb-61da-4dd1-a03c-cb8810e610a5-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "3e2a11eb-61da-4dd1-a03c-cb8810e610a5" (UID: "3e2a11eb-61da-4dd1-a03c-cb8810e610a5"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 12:29:02.015542 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:29:02.015528    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e2a11eb-61da-4dd1-a03c-cb8810e610a5-kube-api-access-w9lfm" (OuterVolumeSpecName: "kube-api-access-w9lfm") pod "3e2a11eb-61da-4dd1-a03c-cb8810e610a5" (UID: "3e2a11eb-61da-4dd1-a03c-cb8810e610a5"). InnerVolumeSpecName "kube-api-access-w9lfm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 12:29:02.114620 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:29:02.114595    2567 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3e2a11eb-61da-4dd1-a03c-cb8810e610a5-console-config\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:29:02.114620 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:29:02.114615    2567 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3e2a11eb-61da-4dd1-a03c-cb8810e610a5-console-oauth-config\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:29:02.114736 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:29:02.114625    2567 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3e2a11eb-61da-4dd1-a03c-cb8810e610a5-service-ca\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:29:02.114736 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:29:02.114634    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w9lfm\" (UniqueName: \"kubernetes.io/projected/3e2a11eb-61da-4dd1-a03c-cb8810e610a5-kube-api-access-w9lfm\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:29:02.114736 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:29:02.114642    2567 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e2a11eb-61da-4dd1-a03c-cb8810e610a5-trusted-ca-bundle\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:29:02.114736 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:29:02.114650    2567 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3e2a11eb-61da-4dd1-a03c-cb8810e610a5-oauth-serving-cert\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:29:02.114736 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:29:02.114658    2567 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3e2a11eb-61da-4dd1-a03c-cb8810e610a5-console-serving-cert\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:29:02.275025 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:29:02.274997    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-579695d779-p47b4"]
Jul 29 12:29:02.279617 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:29:02.279586    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-579695d779-p47b4"]
Jul 29 12:29:03.921444 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:29:03.921375    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e2a11eb-61da-4dd1-a03c-cb8810e610a5" path="/var/lib/kubelet/pods/3e2a11eb-61da-4dd1-a03c-cb8810e610a5/volumes"
Jul 29 12:29:45.727691 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:29:45.727601    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gj8zz"]
Jul 29 12:29:45.728161 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:29:45.727916    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e2a11eb-61da-4dd1-a03c-cb8810e610a5" containerName="console"
Jul 29 12:29:45.728161 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:29:45.727928    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e2a11eb-61da-4dd1-a03c-cb8810e610a5" containerName="console"
Jul 29 12:29:45.728161 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:29:45.727945    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1749b1a-92e5-4543-b92f-62120445b013" containerName="console"
Jul 29 12:29:45.728161 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:29:45.727951    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1749b1a-92e5-4543-b92f-62120445b013" containerName="console"
Jul 29 12:29:45.728161 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:29:45.728009    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e2a11eb-61da-4dd1-a03c-cb8810e610a5" containerName="console"
Jul 29 12:29:45.728161 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:29:45.728017    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1749b1a-92e5-4543-b92f-62120445b013" containerName="console"
Jul 29 12:29:45.731138 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:29:45.731118    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gj8zz"
Jul 29 12:29:45.734081 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:29:45.734056    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-ktz4v\""
Jul 29 12:29:45.734247 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:29:45.734226    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 29 12:29:45.735032 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:29:45.735002    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 29 12:29:45.744439 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:29:45.744394    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gj8zz"]
Jul 29 12:29:45.830832 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:29:45.830786    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fstbv\" (UniqueName: \"kubernetes.io/projected/af6645cd-231f-481c-8554-f94a8873adff-kube-api-access-fstbv\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gj8zz\" (UID: \"af6645cd-231f-481c-8554-f94a8873adff\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gj8zz"
Jul 29 12:29:45.830933 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:29:45.830872    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/af6645cd-231f-481c-8554-f94a8873adff-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gj8zz\" (UID: \"af6645cd-231f-481c-8554-f94a8873adff\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gj8zz"
Jul 29 12:29:45.830933 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:29:45.830904    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/af6645cd-231f-481c-8554-f94a8873adff-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gj8zz\" (UID: \"af6645cd-231f-481c-8554-f94a8873adff\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gj8zz"
Jul 29 12:29:45.931866 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:29:45.931837    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/af6645cd-231f-481c-8554-f94a8873adff-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gj8zz\" (UID: \"af6645cd-231f-481c-8554-f94a8873adff\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gj8zz"
Jul 29 12:29:45.931956 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:29:45.931882    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/af6645cd-231f-481c-8554-f94a8873adff-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gj8zz\" (UID: \"af6645cd-231f-481c-8554-f94a8873adff\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gj8zz"
Jul 29 12:29:45.931956 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:29:45.931922    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fstbv\" (UniqueName: \"kubernetes.io/projected/af6645cd-231f-481c-8554-f94a8873adff-kube-api-access-fstbv\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gj8zz\" (UID: \"af6645cd-231f-481c-8554-f94a8873adff\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gj8zz"
Jul 29 12:29:45.932316 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:29:45.932294    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/af6645cd-231f-481c-8554-f94a8873adff-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gj8zz\" (UID: \"af6645cd-231f-481c-8554-f94a8873adff\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gj8zz"
Jul 29 12:29:45.932475 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:29:45.932342    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/af6645cd-231f-481c-8554-f94a8873adff-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gj8zz\" (UID: \"af6645cd-231f-481c-8554-f94a8873adff\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gj8zz"
Jul 29 12:29:45.940236 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:29:45.940210    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fstbv\" (UniqueName: \"kubernetes.io/projected/af6645cd-231f-481c-8554-f94a8873adff-kube-api-access-fstbv\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gj8zz\" (UID: \"af6645cd-231f-481c-8554-f94a8873adff\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gj8zz"
Jul 29 12:29:46.040468 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:29:46.040439    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gj8zz"
Jul 29 12:29:46.169337 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:29:46.169280    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gj8zz"]
Jul 29 12:29:46.171593 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:29:46.171567    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf6645cd_231f_481c_8554_f94a8873adff.slice/crio-cc748500143bb9b8f83769ef4d4475ac9f83852956b896129eeb1d536e4c253e WatchSource:0}: Error finding container cc748500143bb9b8f83769ef4d4475ac9f83852956b896129eeb1d536e4c253e: Status 404 returned error can't find the container with id cc748500143bb9b8f83769ef4d4475ac9f83852956b896129eeb1d536e4c253e
Jul 29 12:29:47.139536 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:29:47.139504    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gj8zz" event={"ID":"af6645cd-231f-481c-8554-f94a8873adff","Type":"ContainerStarted","Data":"cc748500143bb9b8f83769ef4d4475ac9f83852956b896129eeb1d536e4c253e"}
Jul 29 12:29:53.164778 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:29:53.164743    2567 generic.go:356] "Generic (PLEG): container finished" podID="af6645cd-231f-481c-8554-f94a8873adff" containerID="af310dbd4c8f7e8175d2d8c82d19777ce3f5ea61ec3f72ce58ea8dc2b90d7c06" exitCode=0
Jul 29 12:29:53.165140 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:29:53.164837    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gj8zz" event={"ID":"af6645cd-231f-481c-8554-f94a8873adff","Type":"ContainerDied","Data":"af310dbd4c8f7e8175d2d8c82d19777ce3f5ea61ec3f72ce58ea8dc2b90d7c06"}
Jul 29 12:29:56.180475 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:29:56.180447    2567 generic.go:356] "Generic (PLEG): container finished" podID="af6645cd-231f-481c-8554-f94a8873adff" containerID="7429d4e1728b0576547d365ec2dd99811ac71e2febcedf506d1c9ad44a155911" exitCode=0
Jul 29 12:29:56.180853 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:29:56.180543    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gj8zz" event={"ID":"af6645cd-231f-481c-8554-f94a8873adff","Type":"ContainerDied","Data":"7429d4e1728b0576547d365ec2dd99811ac71e2febcedf506d1c9ad44a155911"}
Jul 29 12:30:04.215997 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:04.215953    2567 generic.go:356] "Generic (PLEG): container finished" podID="af6645cd-231f-481c-8554-f94a8873adff" containerID="8015232c170166cbd35b004b8c0b9c6fe2b385e113ad3e4138b884cf2fc7f09d" exitCode=0
Jul 29 12:30:04.216335 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:04.216032    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gj8zz" event={"ID":"af6645cd-231f-481c-8554-f94a8873adff","Type":"ContainerDied","Data":"8015232c170166cbd35b004b8c0b9c6fe2b385e113ad3e4138b884cf2fc7f09d"}
Jul 29 12:30:05.341380 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:05.341357    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gj8zz"
Jul 29 12:30:05.375858 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:05.375839    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fstbv\" (UniqueName: \"kubernetes.io/projected/af6645cd-231f-481c-8554-f94a8873adff-kube-api-access-fstbv\") pod \"af6645cd-231f-481c-8554-f94a8873adff\" (UID: \"af6645cd-231f-481c-8554-f94a8873adff\") "
Jul 29 12:30:05.375954 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:05.375866    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/af6645cd-231f-481c-8554-f94a8873adff-util\") pod \"af6645cd-231f-481c-8554-f94a8873adff\" (UID: \"af6645cd-231f-481c-8554-f94a8873adff\") "
Jul 29 12:30:05.375954 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:05.375883    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/af6645cd-231f-481c-8554-f94a8873adff-bundle\") pod \"af6645cd-231f-481c-8554-f94a8873adff\" (UID: \"af6645cd-231f-481c-8554-f94a8873adff\") "
Jul 29 12:30:05.376734 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:05.376703    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af6645cd-231f-481c-8554-f94a8873adff-bundle" (OuterVolumeSpecName: "bundle") pod "af6645cd-231f-481c-8554-f94a8873adff" (UID: "af6645cd-231f-481c-8554-f94a8873adff"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 12:30:05.377884 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:05.377862    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af6645cd-231f-481c-8554-f94a8873adff-kube-api-access-fstbv" (OuterVolumeSpecName: "kube-api-access-fstbv") pod "af6645cd-231f-481c-8554-f94a8873adff" (UID: "af6645cd-231f-481c-8554-f94a8873adff"). InnerVolumeSpecName "kube-api-access-fstbv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 12:30:05.381917 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:05.381890    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af6645cd-231f-481c-8554-f94a8873adff-util" (OuterVolumeSpecName: "util") pod "af6645cd-231f-481c-8554-f94a8873adff" (UID: "af6645cd-231f-481c-8554-f94a8873adff"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 12:30:05.477417 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:05.477346    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fstbv\" (UniqueName: \"kubernetes.io/projected/af6645cd-231f-481c-8554-f94a8873adff-kube-api-access-fstbv\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:30:05.477417 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:05.477367    2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/af6645cd-231f-481c-8554-f94a8873adff-util\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:30:05.477417 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:05.477377    2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/af6645cd-231f-481c-8554-f94a8873adff-bundle\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:30:06.226019 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:06.225960    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gj8zz"
Jul 29 12:30:06.226019 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:06.225988    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gj8zz" event={"ID":"af6645cd-231f-481c-8554-f94a8873adff","Type":"ContainerDied","Data":"cc748500143bb9b8f83769ef4d4475ac9f83852956b896129eeb1d536e4c253e"}
Jul 29 12:30:06.226161 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:06.226037    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cc748500143bb9b8f83769ef4d4475ac9f83852956b896129eeb1d536e4c253e"
Jul 29 12:30:14.144598 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:14.144564    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6hwdt"]
Jul 29 12:30:14.145029 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:14.144859    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af6645cd-231f-481c-8554-f94a8873adff" containerName="util"
Jul 29 12:30:14.145029 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:14.144871    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="af6645cd-231f-481c-8554-f94a8873adff" containerName="util"
Jul 29 12:30:14.145029 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:14.144892    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af6645cd-231f-481c-8554-f94a8873adff" containerName="pull"
Jul 29 12:30:14.145029 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:14.144897    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="af6645cd-231f-481c-8554-f94a8873adff" containerName="pull"
Jul 29 12:30:14.145029 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:14.144906    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af6645cd-231f-481c-8554-f94a8873adff" containerName="extract"
Jul 29 12:30:14.145029 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:14.144911    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="af6645cd-231f-481c-8554-f94a8873adff" containerName="extract"
Jul 29 12:30:14.145029 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:14.144961    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="af6645cd-231f-481c-8554-f94a8873adff" containerName="extract"
Jul 29 12:30:14.165741 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:14.165720    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6hwdt"
Jul 29 12:30:14.168658 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:14.168632    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 29 12:30:14.168999 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:14.168977    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-zkdhg\""
Jul 29 12:30:14.169298 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:14.169068    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 29 12:30:14.171144 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:14.171121    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6hwdt"]
Jul 29 12:30:14.237235 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:14.237212    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5201bb08-705e-48f4-8dc0-adaa1800b1a7-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-6hwdt\" (UID: \"5201bb08-705e-48f4-8dc0-adaa1800b1a7\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6hwdt"
Jul 29 12:30:14.237340 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:14.237279    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rvr8\" (UniqueName: \"kubernetes.io/projected/5201bb08-705e-48f4-8dc0-adaa1800b1a7-kube-api-access-5rvr8\") pod \"cert-manager-operator-controller-manager-55c444bf86-6hwdt\" (UID: \"5201bb08-705e-48f4-8dc0-adaa1800b1a7\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6hwdt"
Jul 29 12:30:14.338280 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:14.338256    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5201bb08-705e-48f4-8dc0-adaa1800b1a7-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-6hwdt\" (UID: \"5201bb08-705e-48f4-8dc0-adaa1800b1a7\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6hwdt"
Jul 29 12:30:14.338359 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:14.338305    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rvr8\" (UniqueName: \"kubernetes.io/projected/5201bb08-705e-48f4-8dc0-adaa1800b1a7-kube-api-access-5rvr8\") pod \"cert-manager-operator-controller-manager-55c444bf86-6hwdt\" (UID: \"5201bb08-705e-48f4-8dc0-adaa1800b1a7\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6hwdt"
Jul 29 12:30:14.338685 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:14.338666    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5201bb08-705e-48f4-8dc0-adaa1800b1a7-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-6hwdt\" (UID: \"5201bb08-705e-48f4-8dc0-adaa1800b1a7\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6hwdt"
Jul 29 12:30:14.347198 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:14.347180    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rvr8\" (UniqueName: \"kubernetes.io/projected/5201bb08-705e-48f4-8dc0-adaa1800b1a7-kube-api-access-5rvr8\") pod \"cert-manager-operator-controller-manager-55c444bf86-6hwdt\" (UID: \"5201bb08-705e-48f4-8dc0-adaa1800b1a7\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6hwdt"
Jul 29 12:30:14.476371 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:14.476326    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6hwdt"
Jul 29 12:30:14.609181 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:14.609155    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6hwdt"]
Jul 29 12:30:14.610707 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:30:14.610682    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5201bb08_705e_48f4_8dc0_adaa1800b1a7.slice/crio-aac619f5b9900d78e1e30d5189af01ede1125e9e742e34817b8df1126ffe85e3 WatchSource:0}: Error finding container aac619f5b9900d78e1e30d5189af01ede1125e9e742e34817b8df1126ffe85e3: Status 404 returned error can't find the container with id aac619f5b9900d78e1e30d5189af01ede1125e9e742e34817b8df1126ffe85e3
Jul 29 12:30:15.263880 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:15.263837    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6hwdt" event={"ID":"5201bb08-705e-48f4-8dc0-adaa1800b1a7","Type":"ContainerStarted","Data":"aac619f5b9900d78e1e30d5189af01ede1125e9e742e34817b8df1126ffe85e3"}
Jul 29 12:30:16.269492 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:16.269421    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6hwdt" event={"ID":"5201bb08-705e-48f4-8dc0-adaa1800b1a7","Type":"ContainerStarted","Data":"367adae3319f78bee51decd6dddfddae55d5af0866535ecd16e37a417f7660e9"}
Jul 29 12:30:16.294657 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:16.294616    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6hwdt" podStartSLOduration=0.931868529 podStartE2EDuration="2.294600181s" totalImagesPullingTime="1.362731652s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:30:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:30:16.293428555 +0000 UTC m=+298.987116863" watchObservedRunningTime="2026-07-29 12:30:16.294600181 +0000 UTC m=+298.988288494"
Jul 29 12:30:17.735222 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:17.735184    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpkl4t"]
Jul 29 12:30:17.737593 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:17.737573    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpkl4t"
Jul 29 12:30:17.740373 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:17.740341    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-ktz4v\""
Jul 29 12:30:17.740547 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:17.740396    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 29 12:30:17.741302 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:17.741281    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 29 12:30:17.752809 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:17.752783    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpkl4t"]
Jul 29 12:30:17.788446 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:17.788418    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kj82f_a6d5f3a2-26cc-40a7-9841-2fbfc247661a/ovn-acl-logging/0.log"
Jul 29 12:30:17.788554 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:17.788522    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kj82f_a6d5f3a2-26cc-40a7-9841-2fbfc247661a/ovn-acl-logging/0.log"
Jul 29 12:30:17.792523 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:17.792502    2567 kubelet.go:1648] "Image garbage collection succeeded"
Jul 29 12:30:17.863356 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:17.863329    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/98774f30-7323-47ca-b4e6-e402869f29e6-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpkl4t\" (UID: \"98774f30-7323-47ca-b4e6-e402869f29e6\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpkl4t"
Jul 29 12:30:17.863478 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:17.863367    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdwn9\" (UniqueName: \"kubernetes.io/projected/98774f30-7323-47ca-b4e6-e402869f29e6-kube-api-access-kdwn9\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpkl4t\" (UID: \"98774f30-7323-47ca-b4e6-e402869f29e6\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpkl4t"
Jul 29 12:30:17.863478 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:17.863412    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/98774f30-7323-47ca-b4e6-e402869f29e6-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpkl4t\" (UID: \"98774f30-7323-47ca-b4e6-e402869f29e6\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpkl4t"
Jul 29 12:30:17.964532 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:17.964413    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/98774f30-7323-47ca-b4e6-e402869f29e6-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpkl4t\" (UID: \"98774f30-7323-47ca-b4e6-e402869f29e6\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpkl4t"
Jul 29 12:30:17.964532 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:17.964454    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdwn9\" (UniqueName: \"kubernetes.io/projected/98774f30-7323-47ca-b4e6-e402869f29e6-kube-api-access-kdwn9\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpkl4t\" (UID: \"98774f30-7323-47ca-b4e6-e402869f29e6\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpkl4t"
Jul 29 12:30:17.964532 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:17.964483    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/98774f30-7323-47ca-b4e6-e402869f29e6-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpkl4t\" (UID: \"98774f30-7323-47ca-b4e6-e402869f29e6\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpkl4t"
Jul 29 12:30:17.973243 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:17.964753    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/98774f30-7323-47ca-b4e6-e402869f29e6-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpkl4t\" (UID: \"98774f30-7323-47ca-b4e6-e402869f29e6\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpkl4t"
Jul 29 12:30:17.973243 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:17.964789    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/98774f30-7323-47ca-b4e6-e402869f29e6-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpkl4t\" (UID: \"98774f30-7323-47ca-b4e6-e402869f29e6\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpkl4t"
Jul 29 12:30:17.975802 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:17.975784    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 29 12:30:17.986047 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:17.985996    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 29 12:30:17.996009 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:17.995989    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdwn9\" (UniqueName: \"kubernetes.io/projected/98774f30-7323-47ca-b4e6-e402869f29e6-kube-api-access-kdwn9\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpkl4t\" (UID: \"98774f30-7323-47ca-b4e6-e402869f29e6\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpkl4t"
Jul 29 12:30:18.050661 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:18.050641    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-ktz4v\""
Jul 29 12:30:18.058458 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:18.058438    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpkl4t"
Jul 29 12:30:18.183427 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:18.183386    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpkl4t"]
Jul 29 12:30:18.185103 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:30:18.185075    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98774f30_7323_47ca_b4e6_e402869f29e6.slice/crio-0c16eecca6b0dcec05c870bf21353e3d21dc8168c7c86b7d950fca1687324499 WatchSource:0}: Error finding container 0c16eecca6b0dcec05c870bf21353e3d21dc8168c7c86b7d950fca1687324499: Status 404 returned error can't find the container with id 0c16eecca6b0dcec05c870bf21353e3d21dc8168c7c86b7d950fca1687324499
Jul 29 12:30:18.283336 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:18.283309    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpkl4t" event={"ID":"98774f30-7323-47ca-b4e6-e402869f29e6","Type":"ContainerStarted","Data":"e0db964f99a39175dd20dcc65c7fc70bc33047e5bc22659a76f72bfe4c7928f6"}
Jul 29 12:30:18.283459 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:18.283363    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpkl4t" event={"ID":"98774f30-7323-47ca-b4e6-e402869f29e6","Type":"ContainerStarted","Data":"0c16eecca6b0dcec05c870bf21353e3d21dc8168c7c86b7d950fca1687324499"}
Jul 29 12:30:18.782473 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:18.782442    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-6t2x2"]
Jul 29 12:30:18.784367 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:18.784352    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-6t2x2"
Jul 29 12:30:18.788295 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:18.788278    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 29 12:30:18.789190 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:18.789174    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 29 12:30:18.789190 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:18.789181    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-pk7f4\""
Jul 29 12:30:18.794808 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:18.794787    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-6t2x2"]
Jul 29 12:30:18.871877 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:18.871853    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/169e9854-bb86-49d1-b9b6-d25d11f87634-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-6t2x2\" (UID: \"169e9854-bb86-49d1-b9b6-d25d11f87634\") " pod="cert-manager/cert-manager-webhook-6486ffd997-6t2x2"
Jul 29 12:30:18.871966 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:18.871901    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cq94v\" (UniqueName: \"kubernetes.io/projected/169e9854-bb86-49d1-b9b6-d25d11f87634-kube-api-access-cq94v\") pod \"cert-manager-webhook-6486ffd997-6t2x2\" (UID: \"169e9854-bb86-49d1-b9b6-d25d11f87634\") " pod="cert-manager/cert-manager-webhook-6486ffd997-6t2x2"
Jul 29 12:30:18.972767 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:18.972743    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/169e9854-bb86-49d1-b9b6-d25d11f87634-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-6t2x2\" (UID: \"169e9854-bb86-49d1-b9b6-d25d11f87634\") " pod="cert-manager/cert-manager-webhook-6486ffd997-6t2x2"
Jul 29 12:30:18.972875 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:18.972804    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cq94v\" (UniqueName: \"kubernetes.io/projected/169e9854-bb86-49d1-b9b6-d25d11f87634-kube-api-access-cq94v\") pod \"cert-manager-webhook-6486ffd997-6t2x2\" (UID: \"169e9854-bb86-49d1-b9b6-d25d11f87634\") " pod="cert-manager/cert-manager-webhook-6486ffd997-6t2x2"
Jul 29 12:30:18.984838 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:18.984790    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/169e9854-bb86-49d1-b9b6-d25d11f87634-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-6t2x2\" (UID: \"169e9854-bb86-49d1-b9b6-d25d11f87634\") " pod="cert-manager/cert-manager-webhook-6486ffd997-6t2x2"
Jul 29 12:30:18.984975 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:18.984957    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cq94v\" (UniqueName: \"kubernetes.io/projected/169e9854-bb86-49d1-b9b6-d25d11f87634-kube-api-access-cq94v\") pod \"cert-manager-webhook-6486ffd997-6t2x2\" (UID: \"169e9854-bb86-49d1-b9b6-d25d11f87634\") " pod="cert-manager/cert-manager-webhook-6486ffd997-6t2x2"
Jul 29 12:30:19.104875 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:19.104808    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-6t2x2"
Jul 29 12:30:19.253670 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:19.253644    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-6t2x2"]
Jul 29 12:30:19.255128 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:30:19.255095    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod169e9854_bb86_49d1_b9b6_d25d11f87634.slice/crio-f96b07cb3435b42463dc1af2463f699decb7370b23dc8b61f9bee065d7a34a14 WatchSource:0}: Error finding container f96b07cb3435b42463dc1af2463f699decb7370b23dc8b61f9bee065d7a34a14: Status 404 returned error can't find the container with id f96b07cb3435b42463dc1af2463f699decb7370b23dc8b61f9bee065d7a34a14
Jul 29 12:30:19.257485 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:19.257463    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 12:30:19.290550 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:19.290528    2567 generic.go:356] "Generic (PLEG): container finished" podID="98774f30-7323-47ca-b4e6-e402869f29e6" containerID="e0db964f99a39175dd20dcc65c7fc70bc33047e5bc22659a76f72bfe4c7928f6" exitCode=0
Jul 29 12:30:19.290659 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:19.290608    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpkl4t" event={"ID":"98774f30-7323-47ca-b4e6-e402869f29e6","Type":"ContainerDied","Data":"e0db964f99a39175dd20dcc65c7fc70bc33047e5bc22659a76f72bfe4c7928f6"}
Jul 29 12:30:19.292008 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:19.291978    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-6t2x2" event={"ID":"169e9854-bb86-49d1-b9b6-d25d11f87634","Type":"ContainerStarted","Data":"f96b07cb3435b42463dc1af2463f699decb7370b23dc8b61f9bee065d7a34a14"}
Jul 29 12:30:20.298284 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:20.298250    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpkl4t" event={"ID":"98774f30-7323-47ca-b4e6-e402869f29e6","Type":"ContainerStarted","Data":"88caeda363e343d4986ffd6496432f3e9072aeb30aecfa6c76678ffc3dada5e3"}
Jul 29 12:30:21.304027 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:21.303995    2567 generic.go:356] "Generic (PLEG): container finished" podID="98774f30-7323-47ca-b4e6-e402869f29e6" containerID="88caeda363e343d4986ffd6496432f3e9072aeb30aecfa6c76678ffc3dada5e3" exitCode=0
Jul 29 12:30:21.304471 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:21.304066    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpkl4t" event={"ID":"98774f30-7323-47ca-b4e6-e402869f29e6","Type":"ContainerDied","Data":"88caeda363e343d4986ffd6496432f3e9072aeb30aecfa6c76678ffc3dada5e3"}
Jul 29 12:30:22.310968 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:22.310931    2567 generic.go:356] "Generic (PLEG): container finished" podID="98774f30-7323-47ca-b4e6-e402869f29e6" containerID="dba0d16acf038a537ae5207787cfaccc81eb4024f3f63f3c544b937b69e100c6" exitCode=0
Jul 29 12:30:22.311422 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:22.311023    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpkl4t" event={"ID":"98774f30-7323-47ca-b4e6-e402869f29e6","Type":"ContainerDied","Data":"dba0d16acf038a537ae5207787cfaccc81eb4024f3f63f3c544b937b69e100c6"}
Jul 29 12:30:23.443558 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:23.443536    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpkl4t"
Jul 29 12:30:23.510501 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:23.510475    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/98774f30-7323-47ca-b4e6-e402869f29e6-bundle\") pod \"98774f30-7323-47ca-b4e6-e402869f29e6\" (UID: \"98774f30-7323-47ca-b4e6-e402869f29e6\") "
Jul 29 12:30:23.510595 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:23.510535    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kdwn9\" (UniqueName: \"kubernetes.io/projected/98774f30-7323-47ca-b4e6-e402869f29e6-kube-api-access-kdwn9\") pod \"98774f30-7323-47ca-b4e6-e402869f29e6\" (UID: \"98774f30-7323-47ca-b4e6-e402869f29e6\") "
Jul 29 12:30:23.510595 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:23.510567    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/98774f30-7323-47ca-b4e6-e402869f29e6-util\") pod \"98774f30-7323-47ca-b4e6-e402869f29e6\" (UID: \"98774f30-7323-47ca-b4e6-e402869f29e6\") "
Jul 29 12:30:23.510912 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:23.510881    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98774f30-7323-47ca-b4e6-e402869f29e6-bundle" (OuterVolumeSpecName: "bundle") pod "98774f30-7323-47ca-b4e6-e402869f29e6" (UID: "98774f30-7323-47ca-b4e6-e402869f29e6"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 12:30:23.512531 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:23.512509    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98774f30-7323-47ca-b4e6-e402869f29e6-kube-api-access-kdwn9" (OuterVolumeSpecName: "kube-api-access-kdwn9") pod "98774f30-7323-47ca-b4e6-e402869f29e6" (UID: "98774f30-7323-47ca-b4e6-e402869f29e6"). InnerVolumeSpecName "kube-api-access-kdwn9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 12:30:23.516121 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:23.516103    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98774f30-7323-47ca-b4e6-e402869f29e6-util" (OuterVolumeSpecName: "util") pod "98774f30-7323-47ca-b4e6-e402869f29e6" (UID: "98774f30-7323-47ca-b4e6-e402869f29e6"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 12:30:23.612014 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:23.611965    2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/98774f30-7323-47ca-b4e6-e402869f29e6-bundle\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:30:23.612014 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:23.611986    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kdwn9\" (UniqueName: \"kubernetes.io/projected/98774f30-7323-47ca-b4e6-e402869f29e6-kube-api-access-kdwn9\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:30:23.612014 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:23.611997    2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/98774f30-7323-47ca-b4e6-e402869f29e6-util\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:30:24.196645 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:24.196585    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-5kbz2"]
Jul 29 12:30:24.196866 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:24.196854    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98774f30-7323-47ca-b4e6-e402869f29e6" containerName="extract"
Jul 29 12:30:24.196950 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:24.196867    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="98774f30-7323-47ca-b4e6-e402869f29e6" containerName="extract"
Jul 29 12:30:24.196950 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:24.196882    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98774f30-7323-47ca-b4e6-e402869f29e6" containerName="pull"
Jul 29 12:30:24.196950 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:24.196888    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="98774f30-7323-47ca-b4e6-e402869f29e6" containerName="pull"
Jul 29 12:30:24.196950 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:24.196906    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98774f30-7323-47ca-b4e6-e402869f29e6" containerName="util"
Jul 29 12:30:24.196950 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:24.196912    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="98774f30-7323-47ca-b4e6-e402869f29e6" containerName="util"
Jul 29 12:30:24.197117 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:24.196958    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="98774f30-7323-47ca-b4e6-e402869f29e6" containerName="extract"
Jul 29 12:30:24.199009 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:24.198988    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-5kbz2"
Jul 29 12:30:24.204493 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:24.204470    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-s5bvp\""
Jul 29 12:30:24.211323 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:24.211302    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-5kbz2"]
Jul 29 12:30:24.316185 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:24.316161    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bc87f59e-8b01-4442-a459-2e3a89e32830-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-5kbz2\" (UID: \"bc87f59e-8b01-4442-a459-2e3a89e32830\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-5kbz2"
Jul 29 12:30:24.316281 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:24.316241    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqc5r\" (UniqueName: \"kubernetes.io/projected/bc87f59e-8b01-4442-a459-2e3a89e32830-kube-api-access-dqc5r\") pod \"cert-manager-cainjector-559cdd8c87-5kbz2\" (UID: \"bc87f59e-8b01-4442-a459-2e3a89e32830\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-5kbz2"
Jul 29 12:30:24.321645 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:24.321628    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpkl4t"
Jul 29 12:30:24.321735 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:24.321627    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpkl4t" event={"ID":"98774f30-7323-47ca-b4e6-e402869f29e6","Type":"ContainerDied","Data":"0c16eecca6b0dcec05c870bf21353e3d21dc8168c7c86b7d950fca1687324499"}
Jul 29 12:30:24.321775 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:24.321741    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0c16eecca6b0dcec05c870bf21353e3d21dc8168c7c86b7d950fca1687324499"
Jul 29 12:30:24.416718 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:24.416697    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqc5r\" (UniqueName: \"kubernetes.io/projected/bc87f59e-8b01-4442-a459-2e3a89e32830-kube-api-access-dqc5r\") pod \"cert-manager-cainjector-559cdd8c87-5kbz2\" (UID: \"bc87f59e-8b01-4442-a459-2e3a89e32830\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-5kbz2"
Jul 29 12:30:24.416813 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:24.416749    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bc87f59e-8b01-4442-a459-2e3a89e32830-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-5kbz2\" (UID: \"bc87f59e-8b01-4442-a459-2e3a89e32830\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-5kbz2"
Jul 29 12:30:24.425452 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:24.425428    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqc5r\" (UniqueName: \"kubernetes.io/projected/bc87f59e-8b01-4442-a459-2e3a89e32830-kube-api-access-dqc5r\") pod \"cert-manager-cainjector-559cdd8c87-5kbz2\" (UID: \"bc87f59e-8b01-4442-a459-2e3a89e32830\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-5kbz2"
Jul 29 12:30:24.425670 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:24.425650    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bc87f59e-8b01-4442-a459-2e3a89e32830-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-5kbz2\" (UID: \"bc87f59e-8b01-4442-a459-2e3a89e32830\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-5kbz2"
Jul 29 12:30:24.507706 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:24.507656    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-5kbz2"
Jul 29 12:30:24.644391 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:24.644369    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-5kbz2"]
Jul 29 12:30:24.646740 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:30:24.646718    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbc87f59e_8b01_4442_a459_2e3a89e32830.slice/crio-c2eea1ab7b7c5f802101a24d0bc9e633e386dc0ce9d40b68d2c28f67d3f1905c WatchSource:0}: Error finding container c2eea1ab7b7c5f802101a24d0bc9e633e386dc0ce9d40b68d2c28f67d3f1905c: Status 404 returned error can't find the container with id c2eea1ab7b7c5f802101a24d0bc9e633e386dc0ce9d40b68d2c28f67d3f1905c
Jul 29 12:30:25.327348 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:25.327317    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-5kbz2" event={"ID":"bc87f59e-8b01-4442-a459-2e3a89e32830","Type":"ContainerStarted","Data":"c2eea1ab7b7c5f802101a24d0bc9e633e386dc0ce9d40b68d2c28f67d3f1905c"}
Jul 29 12:30:27.338271 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:27.338228    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-5kbz2" event={"ID":"bc87f59e-8b01-4442-a459-2e3a89e32830","Type":"ContainerStarted","Data":"1db814ad96ba0573a8d831c7fa6d59e1e48f472ae78569a6bd2e8a49cf65ba3c"}
Jul 29 12:30:27.357297 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:27.357252    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-5kbz2" podStartSLOduration=0.896083166 podStartE2EDuration="3.357236112s" totalImagesPullingTime="2.461152946s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:30:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:30:27.355544333 +0000 UTC m=+310.049232647" watchObservedRunningTime="2026-07-29 12:30:27.357236112 +0000 UTC m=+310.050924423"
Jul 29 12:30:30.848324 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:30.848286    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-b7fxs"]
Jul 29 12:30:30.850624 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:30.850604    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-b7fxs"
Jul 29 12:30:30.853372 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:30.853339    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 29 12:30:30.853372 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:30.853355    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-82q4p\""
Jul 29 12:30:30.853372 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:30.853339    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 29 12:30:30.861080 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:30.861057    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-b7fxs"]
Jul 29 12:30:30.965902 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:30.965841    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldp2p\" (UniqueName: \"kubernetes.io/projected/763a3c4a-bd0e-4f64-bbc5-c98f492f36d4-kube-api-access-ldp2p\") pod \"openshift-lws-operator-fd8ccff4c-b7fxs\" (UID: \"763a3c4a-bd0e-4f64-bbc5-c98f492f36d4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-b7fxs"
Jul 29 12:30:30.965902 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:30.965887    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/763a3c4a-bd0e-4f64-bbc5-c98f492f36d4-tmp\") pod \"openshift-lws-operator-fd8ccff4c-b7fxs\" (UID: \"763a3c4a-bd0e-4f64-bbc5-c98f492f36d4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-b7fxs"
Jul 29 12:30:31.066683 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:31.066658    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldp2p\" (UniqueName: \"kubernetes.io/projected/763a3c4a-bd0e-4f64-bbc5-c98f492f36d4-kube-api-access-ldp2p\") pod \"openshift-lws-operator-fd8ccff4c-b7fxs\" (UID: \"763a3c4a-bd0e-4f64-bbc5-c98f492f36d4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-b7fxs"
Jul 29 12:30:31.066777 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:31.066691    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/763a3c4a-bd0e-4f64-bbc5-c98f492f36d4-tmp\") pod \"openshift-lws-operator-fd8ccff4c-b7fxs\" (UID: \"763a3c4a-bd0e-4f64-bbc5-c98f492f36d4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-b7fxs"
Jul 29 12:30:31.067029 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:31.067012    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/763a3c4a-bd0e-4f64-bbc5-c98f492f36d4-tmp\") pod \"openshift-lws-operator-fd8ccff4c-b7fxs\" (UID: \"763a3c4a-bd0e-4f64-bbc5-c98f492f36d4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-b7fxs"
Jul 29 12:30:31.085261 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:31.085235    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldp2p\" (UniqueName: \"kubernetes.io/projected/763a3c4a-bd0e-4f64-bbc5-c98f492f36d4-kube-api-access-ldp2p\") pod \"openshift-lws-operator-fd8ccff4c-b7fxs\" (UID: \"763a3c4a-bd0e-4f64-bbc5-c98f492f36d4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-b7fxs"
Jul 29 12:30:31.159104 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:31.159081    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-b7fxs"
Jul 29 12:30:31.282569 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:31.282540    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-b7fxs"]
Jul 29 12:30:31.285724 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:30:31.285697    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod763a3c4a_bd0e_4f64_bbc5_c98f492f36d4.slice/crio-6c7dfb5c61b23efaf2a22cb5c629c09680a212ee0e2937795483c7c538615a03 WatchSource:0}: Error finding container 6c7dfb5c61b23efaf2a22cb5c629c09680a212ee0e2937795483c7c538615a03: Status 404 returned error can't find the container with id 6c7dfb5c61b23efaf2a22cb5c629c09680a212ee0e2937795483c7c538615a03
Jul 29 12:30:31.356709 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:31.356684    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-b7fxs" event={"ID":"763a3c4a-bd0e-4f64-bbc5-c98f492f36d4","Type":"ContainerStarted","Data":"6c7dfb5c61b23efaf2a22cb5c629c09680a212ee0e2937795483c7c538615a03"}
Jul 29 12:30:31.358111 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:31.358091    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-6t2x2" event={"ID":"169e9854-bb86-49d1-b9b6-d25d11f87634","Type":"ContainerStarted","Data":"3681f84484215576cc91342b7bd2357bc1a51158a206ce7ca1a092687483577b"}
Jul 29 12:30:31.358227 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:31.358209    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-6t2x2"
Jul 29 12:30:31.377088 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:31.377047    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-6t2x2" podStartSLOduration=1.944514575 podStartE2EDuration="13.377035104s" totalImagesPullingTime="11.432520529s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:30:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:30:31.375475615 +0000 UTC m=+314.069163927" watchObservedRunningTime="2026-07-29 12:30:31.377035104 +0000 UTC m=+314.070723415"
Jul 29 12:30:33.368565 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:33.368526    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-b7fxs" event={"ID":"763a3c4a-bd0e-4f64-bbc5-c98f492f36d4","Type":"ContainerStarted","Data":"4094d9805538a3966fe3cf43f55efd09335d16ee4896ffff485b3295695d0136"}
Jul 29 12:30:33.385769 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:33.385705    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-b7fxs" podStartSLOduration=1.8556317089999999 podStartE2EDuration="3.38569248s" totalImagesPullingTime="1.530060771s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:30:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:30:33.384680556 +0000 UTC m=+316.078368868" watchObservedRunningTime="2026-07-29 12:30:33.38569248 +0000 UTC m=+316.079380839"
Jul 29 12:30:37.364425 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:37.364379    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-6t2x2"
Jul 29 12:30:38.726385 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:38.726348    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4d9j5"]
Jul 29 12:30:38.729015 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:38.728990    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4d9j5"
Jul 29 12:30:38.731856 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:38.731831    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 29 12:30:38.731856 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:38.731849    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 29 12:30:38.732813 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:38.732794    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-ktz4v\""
Jul 29 12:30:38.740345 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:38.740325    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4d9j5"]
Jul 29 12:30:38.819723 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:38.819694    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b5a9e42c-4ce7-4623-9677-eee0f77123bb-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4d9j5\" (UID: \"b5a9e42c-4ce7-4623-9677-eee0f77123bb\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4d9j5"
Jul 29 12:30:38.819817 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:38.819746    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b5a9e42c-4ce7-4623-9677-eee0f77123bb-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4d9j5\" (UID: \"b5a9e42c-4ce7-4623-9677-eee0f77123bb\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4d9j5"
Jul 29 12:30:38.819866 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:38.819825    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnpn9\" (UniqueName: \"kubernetes.io/projected/b5a9e42c-4ce7-4623-9677-eee0f77123bb-kube-api-access-xnpn9\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4d9j5\" (UID: \"b5a9e42c-4ce7-4623-9677-eee0f77123bb\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4d9j5"
Jul 29 12:30:38.921133 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:38.921106    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b5a9e42c-4ce7-4623-9677-eee0f77123bb-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4d9j5\" (UID: \"b5a9e42c-4ce7-4623-9677-eee0f77123bb\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4d9j5"
Jul 29 12:30:38.921257 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:38.921141    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnpn9\" (UniqueName: \"kubernetes.io/projected/b5a9e42c-4ce7-4623-9677-eee0f77123bb-kube-api-access-xnpn9\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4d9j5\" (UID: \"b5a9e42c-4ce7-4623-9677-eee0f77123bb\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4d9j5"
Jul 29 12:30:38.921257 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:38.921186    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b5a9e42c-4ce7-4623-9677-eee0f77123bb-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4d9j5\" (UID: \"b5a9e42c-4ce7-4623-9677-eee0f77123bb\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4d9j5"
Jul 29 12:30:38.921597 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:38.921569    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b5a9e42c-4ce7-4623-9677-eee0f77123bb-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4d9j5\" (UID: \"b5a9e42c-4ce7-4623-9677-eee0f77123bb\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4d9j5"
Jul 29 12:30:38.921683 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:38.921602    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b5a9e42c-4ce7-4623-9677-eee0f77123bb-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4d9j5\" (UID: \"b5a9e42c-4ce7-4623-9677-eee0f77123bb\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4d9j5"
Jul 29 12:30:38.933091 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:38.933058    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnpn9\" (UniqueName: \"kubernetes.io/projected/b5a9e42c-4ce7-4623-9677-eee0f77123bb-kube-api-access-xnpn9\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4d9j5\" (UID: \"b5a9e42c-4ce7-4623-9677-eee0f77123bb\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4d9j5"
Jul 29 12:30:39.039220 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:39.039198    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4d9j5"
Jul 29 12:30:39.172869 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:39.172844    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4d9j5"]
Jul 29 12:30:39.175215 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:30:39.175182    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5a9e42c_4ce7_4623_9677_eee0f77123bb.slice/crio-de6bf9e7ed08ecd4e7bb636d0718de5195ac808be6f544a3704425136c262f64 WatchSource:0}: Error finding container de6bf9e7ed08ecd4e7bb636d0718de5195ac808be6f544a3704425136c262f64: Status 404 returned error can't find the container with id de6bf9e7ed08ecd4e7bb636d0718de5195ac808be6f544a3704425136c262f64
Jul 29 12:30:39.396851 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:39.396795    2567 generic.go:356] "Generic (PLEG): container finished" podID="b5a9e42c-4ce7-4623-9677-eee0f77123bb" containerID="699dbaf76105abc96bb9b81c6172e6dcab7c042c7406054a408578ac01920a29" exitCode=0
Jul 29 12:30:39.396944 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:39.396850    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4d9j5" event={"ID":"b5a9e42c-4ce7-4623-9677-eee0f77123bb","Type":"ContainerDied","Data":"699dbaf76105abc96bb9b81c6172e6dcab7c042c7406054a408578ac01920a29"}
Jul 29 12:30:39.396944 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:39.396900    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4d9j5" event={"ID":"b5a9e42c-4ce7-4623-9677-eee0f77123bb","Type":"ContainerStarted","Data":"de6bf9e7ed08ecd4e7bb636d0718de5195ac808be6f544a3704425136c262f64"}
Jul 29 12:30:40.403811 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:40.403777    2567 generic.go:356] "Generic (PLEG): container finished" podID="b5a9e42c-4ce7-4623-9677-eee0f77123bb" containerID="cb8db6a6b9fbbb2b17cefe1de237442348f67aa8209ac4dbb5399b0c52b0547a" exitCode=0
Jul 29 12:30:40.404113 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:40.403866    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4d9j5" event={"ID":"b5a9e42c-4ce7-4623-9677-eee0f77123bb","Type":"ContainerDied","Data":"cb8db6a6b9fbbb2b17cefe1de237442348f67aa8209ac4dbb5399b0c52b0547a"}
Jul 29 12:30:41.414315 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:41.414278    2567 generic.go:356] "Generic (PLEG): container finished" podID="b5a9e42c-4ce7-4623-9677-eee0f77123bb" containerID="220b65b1d67bf9afcae3cf013ba637434a988f80ba5ff3f62bb69afe137b5451" exitCode=0
Jul 29 12:30:41.414707 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:41.414372    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4d9j5" event={"ID":"b5a9e42c-4ce7-4623-9677-eee0f77123bb","Type":"ContainerDied","Data":"220b65b1d67bf9afcae3cf013ba637434a988f80ba5ff3f62bb69afe137b5451"}
Jul 29 12:30:42.542381 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:42.542361    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4d9j5"
Jul 29 12:30:42.651226 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:42.651206    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b5a9e42c-4ce7-4623-9677-eee0f77123bb-bundle\") pod \"b5a9e42c-4ce7-4623-9677-eee0f77123bb\" (UID: \"b5a9e42c-4ce7-4623-9677-eee0f77123bb\") "
Jul 29 12:30:42.651348 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:42.651246    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b5a9e42c-4ce7-4623-9677-eee0f77123bb-util\") pod \"b5a9e42c-4ce7-4623-9677-eee0f77123bb\" (UID: \"b5a9e42c-4ce7-4623-9677-eee0f77123bb\") "
Jul 29 12:30:42.651348 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:42.651289    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xnpn9\" (UniqueName: \"kubernetes.io/projected/b5a9e42c-4ce7-4623-9677-eee0f77123bb-kube-api-access-xnpn9\") pod \"b5a9e42c-4ce7-4623-9677-eee0f77123bb\" (UID: \"b5a9e42c-4ce7-4623-9677-eee0f77123bb\") "
Jul 29 12:30:42.651991 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:42.651962    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5a9e42c-4ce7-4623-9677-eee0f77123bb-bundle" (OuterVolumeSpecName: "bundle") pod "b5a9e42c-4ce7-4623-9677-eee0f77123bb" (UID: "b5a9e42c-4ce7-4623-9677-eee0f77123bb"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 12:30:42.653367 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:42.653337    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5a9e42c-4ce7-4623-9677-eee0f77123bb-kube-api-access-xnpn9" (OuterVolumeSpecName: "kube-api-access-xnpn9") pod "b5a9e42c-4ce7-4623-9677-eee0f77123bb" (UID: "b5a9e42c-4ce7-4623-9677-eee0f77123bb"). InnerVolumeSpecName "kube-api-access-xnpn9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 12:30:42.656551 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:42.656513    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5a9e42c-4ce7-4623-9677-eee0f77123bb-util" (OuterVolumeSpecName: "util") pod "b5a9e42c-4ce7-4623-9677-eee0f77123bb" (UID: "b5a9e42c-4ce7-4623-9677-eee0f77123bb"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 12:30:42.752593 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:42.752545    2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b5a9e42c-4ce7-4623-9677-eee0f77123bb-util\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:30:42.752593 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:42.752565    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xnpn9\" (UniqueName: \"kubernetes.io/projected/b5a9e42c-4ce7-4623-9677-eee0f77123bb-kube-api-access-xnpn9\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:30:42.752593 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:42.752575    2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b5a9e42c-4ce7-4623-9677-eee0f77123bb-bundle\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:30:43.425646 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:43.425620    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4d9j5"
Jul 29 12:30:43.425815 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:43.425657    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4d9j5" event={"ID":"b5a9e42c-4ce7-4623-9677-eee0f77123bb","Type":"ContainerDied","Data":"de6bf9e7ed08ecd4e7bb636d0718de5195ac808be6f544a3704425136c262f64"}
Jul 29 12:30:43.425815 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:43.425716    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="de6bf9e7ed08ecd4e7bb636d0718de5195ac808be6f544a3704425136c262f64"
Jul 29 12:30:52.167594 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:52.167558    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-556f8bf875-2lg9f"]
Jul 29 12:30:52.167980 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:52.167899    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5a9e42c-4ce7-4623-9677-eee0f77123bb" containerName="util"
Jul 29 12:30:52.167980 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:52.167911    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5a9e42c-4ce7-4623-9677-eee0f77123bb" containerName="util"
Jul 29 12:30:52.167980 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:52.167920    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5a9e42c-4ce7-4623-9677-eee0f77123bb" containerName="pull"
Jul 29 12:30:52.167980 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:52.167925    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5a9e42c-4ce7-4623-9677-eee0f77123bb" containerName="pull"
Jul 29 12:30:52.167980 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:52.167939    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5a9e42c-4ce7-4623-9677-eee0f77123bb" containerName="extract"
Jul 29 12:30:52.167980 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:52.167945    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5a9e42c-4ce7-4623-9677-eee0f77123bb" containerName="extract"
Jul 29 12:30:52.168146 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:52.167992    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5a9e42c-4ce7-4623-9677-eee0f77123bb" containerName="extract"
Jul 29 12:30:52.172779 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:52.172759    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-556f8bf875-2lg9f"
Jul 29 12:30:52.179190 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:52.178997    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 29 12:30:52.179190 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:52.179094    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-kbp6j\""
Jul 29 12:30:52.179489 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:52.179465    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 29 12:30:52.180068 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:52.180050    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 29 12:30:52.182322 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:52.182303    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 29 12:30:52.233426 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:52.233391    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-556f8bf875-2lg9f"]
Jul 29 12:30:52.325511 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:52.325484    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/078c30c5-e329-4de6-85e6-97e5fcef0a7f-webhook-cert\") pod \"opendatahub-operator-controller-manager-556f8bf875-2lg9f\" (UID: \"078c30c5-e329-4de6-85e6-97e5fcef0a7f\") " pod="opendatahub/opendatahub-operator-controller-manager-556f8bf875-2lg9f"
Jul 29 12:30:52.325612 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:52.325526    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/078c30c5-e329-4de6-85e6-97e5fcef0a7f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-556f8bf875-2lg9f\" (UID: \"078c30c5-e329-4de6-85e6-97e5fcef0a7f\") " pod="opendatahub/opendatahub-operator-controller-manager-556f8bf875-2lg9f"
Jul 29 12:30:52.325612 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:52.325556    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/078c30c5-e329-4de6-85e6-97e5fcef0a7f-manifests\") pod \"opendatahub-operator-controller-manager-556f8bf875-2lg9f\" (UID: \"078c30c5-e329-4de6-85e6-97e5fcef0a7f\") " pod="opendatahub/opendatahub-operator-controller-manager-556f8bf875-2lg9f"
Jul 29 12:30:52.325684 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:52.325630    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvdw4\" (UniqueName: \"kubernetes.io/projected/078c30c5-e329-4de6-85e6-97e5fcef0a7f-kube-api-access-wvdw4\") pod \"opendatahub-operator-controller-manager-556f8bf875-2lg9f\" (UID: \"078c30c5-e329-4de6-85e6-97e5fcef0a7f\") " pod="opendatahub/opendatahub-operator-controller-manager-556f8bf875-2lg9f"
Jul 29 12:30:52.325684 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:52.325668    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/078c30c5-e329-4de6-85e6-97e5fcef0a7f-tmp\") pod \"opendatahub-operator-controller-manager-556f8bf875-2lg9f\" (UID: \"078c30c5-e329-4de6-85e6-97e5fcef0a7f\") " pod="opendatahub/opendatahub-operator-controller-manager-556f8bf875-2lg9f"
Jul 29 12:30:52.426250 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:52.426194    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvdw4\" (UniqueName: \"kubernetes.io/projected/078c30c5-e329-4de6-85e6-97e5fcef0a7f-kube-api-access-wvdw4\") pod \"opendatahub-operator-controller-manager-556f8bf875-2lg9f\" (UID: \"078c30c5-e329-4de6-85e6-97e5fcef0a7f\") " pod="opendatahub/opendatahub-operator-controller-manager-556f8bf875-2lg9f"
Jul 29 12:30:52.426250 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:52.426234    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/078c30c5-e329-4de6-85e6-97e5fcef0a7f-tmp\") pod \"opendatahub-operator-controller-manager-556f8bf875-2lg9f\" (UID: \"078c30c5-e329-4de6-85e6-97e5fcef0a7f\") " pod="opendatahub/opendatahub-operator-controller-manager-556f8bf875-2lg9f"
Jul 29 12:30:52.426371 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:52.426265    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/078c30c5-e329-4de6-85e6-97e5fcef0a7f-webhook-cert\") pod \"opendatahub-operator-controller-manager-556f8bf875-2lg9f\" (UID: \"078c30c5-e329-4de6-85e6-97e5fcef0a7f\") " pod="opendatahub/opendatahub-operator-controller-manager-556f8bf875-2lg9f"
Jul 29 12:30:52.426371 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:52.426296    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/078c30c5-e329-4de6-85e6-97e5fcef0a7f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-556f8bf875-2lg9f\" (UID: \"078c30c5-e329-4de6-85e6-97e5fcef0a7f\") " pod="opendatahub/opendatahub-operator-controller-manager-556f8bf875-2lg9f"
Jul 29 12:30:52.426371 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:52.426331    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/078c30c5-e329-4de6-85e6-97e5fcef0a7f-manifests\") pod \"opendatahub-operator-controller-manager-556f8bf875-2lg9f\" (UID: \"078c30c5-e329-4de6-85e6-97e5fcef0a7f\") " pod="opendatahub/opendatahub-operator-controller-manager-556f8bf875-2lg9f"
Jul 29 12:30:52.426711 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:52.426693    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/078c30c5-e329-4de6-85e6-97e5fcef0a7f-manifests\") pod \"opendatahub-operator-controller-manager-556f8bf875-2lg9f\" (UID: \"078c30c5-e329-4de6-85e6-97e5fcef0a7f\") " pod="opendatahub/opendatahub-operator-controller-manager-556f8bf875-2lg9f"
Jul 29 12:30:52.428375 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:52.428356    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/078c30c5-e329-4de6-85e6-97e5fcef0a7f-tmp\") pod \"opendatahub-operator-controller-manager-556f8bf875-2lg9f\" (UID: \"078c30c5-e329-4de6-85e6-97e5fcef0a7f\") " pod="opendatahub/opendatahub-operator-controller-manager-556f8bf875-2lg9f"
Jul 29 12:30:52.428767 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:52.428748    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/078c30c5-e329-4de6-85e6-97e5fcef0a7f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-556f8bf875-2lg9f\" (UID: \"078c30c5-e329-4de6-85e6-97e5fcef0a7f\") " pod="opendatahub/opendatahub-operator-controller-manager-556f8bf875-2lg9f"
Jul 29 12:30:52.428803 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:52.428764    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/078c30c5-e329-4de6-85e6-97e5fcef0a7f-webhook-cert\") pod \"opendatahub-operator-controller-manager-556f8bf875-2lg9f\" (UID: \"078c30c5-e329-4de6-85e6-97e5fcef0a7f\") " pod="opendatahub/opendatahub-operator-controller-manager-556f8bf875-2lg9f"
Jul 29 12:30:52.436977 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:52.436956    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvdw4\" (UniqueName: \"kubernetes.io/projected/078c30c5-e329-4de6-85e6-97e5fcef0a7f-kube-api-access-wvdw4\") pod \"opendatahub-operator-controller-manager-556f8bf875-2lg9f\" (UID: \"078c30c5-e329-4de6-85e6-97e5fcef0a7f\") " pod="opendatahub/opendatahub-operator-controller-manager-556f8bf875-2lg9f"
Jul 29 12:30:52.482845 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:52.482824    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-556f8bf875-2lg9f"
Jul 29 12:30:52.623707 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:52.623682    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-556f8bf875-2lg9f"]
Jul 29 12:30:52.625616 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:30:52.625583    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod078c30c5_e329_4de6_85e6_97e5fcef0a7f.slice/crio-2d247bd9f3e1c8c115737e31e4ec3dd84539ce61f6bc9959731a0a18be9d441d WatchSource:0}: Error finding container 2d247bd9f3e1c8c115737e31e4ec3dd84539ce61f6bc9959731a0a18be9d441d: Status 404 returned error can't find the container with id 2d247bd9f3e1c8c115737e31e4ec3dd84539ce61f6bc9959731a0a18be9d441d
Jul 29 12:30:53.476002 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:53.475962    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-556f8bf875-2lg9f" event={"ID":"078c30c5-e329-4de6-85e6-97e5fcef0a7f","Type":"ContainerStarted","Data":"2d247bd9f3e1c8c115737e31e4ec3dd84539ce61f6bc9959731a0a18be9d441d"}
Jul 29 12:30:56.492198 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:56.492165    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-556f8bf875-2lg9f" event={"ID":"078c30c5-e329-4de6-85e6-97e5fcef0a7f","Type":"ContainerStarted","Data":"cf9a73d6e2f3dd5a04bff04c0ba90a96e6dfc317dcad6c23189b182cb64e179c"}
Jul 29 12:30:57.499272 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:57.499212    2567 generic.go:356] "Generic (PLEG): container finished" podID="078c30c5-e329-4de6-85e6-97e5fcef0a7f" containerID="cf9a73d6e2f3dd5a04bff04c0ba90a96e6dfc317dcad6c23189b182cb64e179c" exitCode=0
Jul 29 12:30:57.499609 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:57.499280    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-556f8bf875-2lg9f" event={"ID":"078c30c5-e329-4de6-85e6-97e5fcef0a7f","Type":"ContainerDied","Data":"cf9a73d6e2f3dd5a04bff04c0ba90a96e6dfc317dcad6c23189b182cb64e179c"}
Jul 29 12:30:58.507315 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:58.507272    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-556f8bf875-2lg9f" event={"ID":"078c30c5-e329-4de6-85e6-97e5fcef0a7f","Type":"ContainerStarted","Data":"54249f284d5caa4cf4a356dd76fa183e4d06a42fd28a488241b50bb5ff9df957"}
Jul 29 12:30:58.507695 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:58.507430    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-556f8bf875-2lg9f"
Jul 29 12:30:58.549316 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:30:58.549265    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-556f8bf875-2lg9f" podStartSLOduration=2.077806837 podStartE2EDuration="6.549252106s" totalImagesPullingTime="3.442435785s" totalInitContainerRuntime="1.029009484s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:30:52 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:30:58.548217193 +0000 UTC m=+341.241905505" watchObservedRunningTime="2026-07-29 12:30:58.549252106 +0000 UTC m=+341.242940417"
Jul 29 12:31:09.515250 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:09.515218    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-556f8bf875-2lg9f"
Jul 29 12:31:12.234657 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:12.234573    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-hnbhw"]
Jul 29 12:31:12.237110 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:12.237091    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-hnbhw"
Jul 29 12:31:12.240419 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:12.240381    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 29 12:31:12.241767 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:12.241748    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-cw4wg\""
Jul 29 12:31:12.253427 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:12.250844    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-hnbhw"]
Jul 29 12:31:12.265578 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:12.265550    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/02f86dac-d455-4b0e-af6a-0cdfc0a8b9f2-cert\") pod \"odh-model-controller-7998c67dc4-hnbhw\" (UID: \"02f86dac-d455-4b0e-af6a-0cdfc0a8b9f2\") " pod="opendatahub/odh-model-controller-7998c67dc4-hnbhw"
Jul 29 12:31:12.265676 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:12.265615    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2clx\" (UniqueName: \"kubernetes.io/projected/02f86dac-d455-4b0e-af6a-0cdfc0a8b9f2-kube-api-access-f2clx\") pod \"odh-model-controller-7998c67dc4-hnbhw\" (UID: \"02f86dac-d455-4b0e-af6a-0cdfc0a8b9f2\") " pod="opendatahub/odh-model-controller-7998c67dc4-hnbhw"
Jul 29 12:31:12.365917 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:12.365898    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2clx\" (UniqueName: \"kubernetes.io/projected/02f86dac-d455-4b0e-af6a-0cdfc0a8b9f2-kube-api-access-f2clx\") pod \"odh-model-controller-7998c67dc4-hnbhw\" (UID: \"02f86dac-d455-4b0e-af6a-0cdfc0a8b9f2\") " pod="opendatahub/odh-model-controller-7998c67dc4-hnbhw"
Jul 29 12:31:12.366016 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:12.365944    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/02f86dac-d455-4b0e-af6a-0cdfc0a8b9f2-cert\") pod \"odh-model-controller-7998c67dc4-hnbhw\" (UID: \"02f86dac-d455-4b0e-af6a-0cdfc0a8b9f2\") " pod="opendatahub/odh-model-controller-7998c67dc4-hnbhw"
Jul 29 12:31:12.366057 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:31:12.366032    2567 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 29 12:31:12.366089 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:31:12.366082    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02f86dac-d455-4b0e-af6a-0cdfc0a8b9f2-cert podName:02f86dac-d455-4b0e-af6a-0cdfc0a8b9f2 nodeName:}" failed. No retries permitted until 2026-07-29 12:31:12.866066203 +0000 UTC m=+355.559754492 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/02f86dac-d455-4b0e-af6a-0cdfc0a8b9f2-cert") pod "odh-model-controller-7998c67dc4-hnbhw" (UID: "02f86dac-d455-4b0e-af6a-0cdfc0a8b9f2") : secret "odh-model-controller-webhook-cert" not found
Jul 29 12:31:12.404133 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:12.404103    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2clx\" (UniqueName: \"kubernetes.io/projected/02f86dac-d455-4b0e-af6a-0cdfc0a8b9f2-kube-api-access-f2clx\") pod \"odh-model-controller-7998c67dc4-hnbhw\" (UID: \"02f86dac-d455-4b0e-af6a-0cdfc0a8b9f2\") " pod="opendatahub/odh-model-controller-7998c67dc4-hnbhw"
Jul 29 12:31:12.870001 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:12.869968    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/02f86dac-d455-4b0e-af6a-0cdfc0a8b9f2-cert\") pod \"odh-model-controller-7998c67dc4-hnbhw\" (UID: \"02f86dac-d455-4b0e-af6a-0cdfc0a8b9f2\") " pod="opendatahub/odh-model-controller-7998c67dc4-hnbhw"
Jul 29 12:31:12.872288 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:12.872267    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/02f86dac-d455-4b0e-af6a-0cdfc0a8b9f2-cert\") pod \"odh-model-controller-7998c67dc4-hnbhw\" (UID: \"02f86dac-d455-4b0e-af6a-0cdfc0a8b9f2\") " pod="opendatahub/odh-model-controller-7998c67dc4-hnbhw"
Jul 29 12:31:13.148352 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:13.148281    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-hnbhw"
Jul 29 12:31:13.308471 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:13.308450    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-hnbhw"]
Jul 29 12:31:13.310034 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:31:13.310009    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02f86dac_d455_4b0e_af6a_0cdfc0a8b9f2.slice/crio-30059acdbfe8d3b7cca21c0b1cbc8847d3cea7eecbd654a63049f4bc76a33f04 WatchSource:0}: Error finding container 30059acdbfe8d3b7cca21c0b1cbc8847d3cea7eecbd654a63049f4bc76a33f04: Status 404 returned error can't find the container with id 30059acdbfe8d3b7cca21c0b1cbc8847d3cea7eecbd654a63049f4bc76a33f04
Jul 29 12:31:13.397430 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:13.397391    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-cbbgq"]
Jul 29 12:31:13.400230 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:13.400187    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-cbbgq"
Jul 29 12:31:13.413583 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:13.413560    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 29 12:31:13.426550 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:13.426532    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-cbbgq"]
Jul 29 12:31:13.475342 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:13.475317    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bcp74\" (UniqueName: \"kubernetes.io/projected/8c660dce-7015-4cee-bb2d-e3a9ec0c2228-kube-api-access-bcp74\") pod \"model-serving-api-54d89bd5fc-cbbgq\" (UID: \"8c660dce-7015-4cee-bb2d-e3a9ec0c2228\") " pod="opendatahub/model-serving-api-54d89bd5fc-cbbgq"
Jul 29 12:31:13.475453 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:13.475346    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c660dce-7015-4cee-bb2d-e3a9ec0c2228-tls-certs\") pod \"model-serving-api-54d89bd5fc-cbbgq\" (UID: \"8c660dce-7015-4cee-bb2d-e3a9ec0c2228\") " pod="opendatahub/model-serving-api-54d89bd5fc-cbbgq"
Jul 29 12:31:13.575814 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:13.575790    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c660dce-7015-4cee-bb2d-e3a9ec0c2228-tls-certs\") pod \"model-serving-api-54d89bd5fc-cbbgq\" (UID: \"8c660dce-7015-4cee-bb2d-e3a9ec0c2228\") " pod="opendatahub/model-serving-api-54d89bd5fc-cbbgq"
Jul 29 12:31:13.575916 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:13.575853    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcp74\" (UniqueName: \"kubernetes.io/projected/8c660dce-7015-4cee-bb2d-e3a9ec0c2228-kube-api-access-bcp74\") pod \"model-serving-api-54d89bd5fc-cbbgq\" (UID: \"8c660dce-7015-4cee-bb2d-e3a9ec0c2228\") " pod="opendatahub/model-serving-api-54d89bd5fc-cbbgq"
Jul 29 12:31:13.578119 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:13.578096    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-hnbhw" event={"ID":"02f86dac-d455-4b0e-af6a-0cdfc0a8b9f2","Type":"ContainerStarted","Data":"30059acdbfe8d3b7cca21c0b1cbc8847d3cea7eecbd654a63049f4bc76a33f04"}
Jul 29 12:31:13.578326 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:13.578307    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c660dce-7015-4cee-bb2d-e3a9ec0c2228-tls-certs\") pod \"model-serving-api-54d89bd5fc-cbbgq\" (UID: \"8c660dce-7015-4cee-bb2d-e3a9ec0c2228\") " pod="opendatahub/model-serving-api-54d89bd5fc-cbbgq"
Jul 29 12:31:13.586322 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:13.586296    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bcp74\" (UniqueName: \"kubernetes.io/projected/8c660dce-7015-4cee-bb2d-e3a9ec0c2228-kube-api-access-bcp74\") pod \"model-serving-api-54d89bd5fc-cbbgq\" (UID: \"8c660dce-7015-4cee-bb2d-e3a9ec0c2228\") " pod="opendatahub/model-serving-api-54d89bd5fc-cbbgq"
Jul 29 12:31:13.710237 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:13.710187    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-cbbgq"
Jul 29 12:31:13.834612 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:13.834588    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-cbbgq"]
Jul 29 12:31:13.835968 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:31:13.835945    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c660dce_7015_4cee_bb2d_e3a9ec0c2228.slice/crio-03fec43a0743892292e0a2706cdb2e28efaa699804386c2c4ff0ce2f64e02841 WatchSource:0}: Error finding container 03fec43a0743892292e0a2706cdb2e28efaa699804386c2c4ff0ce2f64e02841: Status 404 returned error can't find the container with id 03fec43a0743892292e0a2706cdb2e28efaa699804386c2c4ff0ce2f64e02841
Jul 29 12:31:14.586233 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:14.586193    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-cbbgq" event={"ID":"8c660dce-7015-4cee-bb2d-e3a9ec0c2228","Type":"ContainerStarted","Data":"03fec43a0743892292e0a2706cdb2e28efaa699804386c2c4ff0ce2f64e02841"}
Jul 29 12:31:16.597529 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:16.597490    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-cbbgq" event={"ID":"8c660dce-7015-4cee-bb2d-e3a9ec0c2228","Type":"ContainerStarted","Data":"6223c1c7eefc4e2a9c2aae54ce3fb7f7397629ac7bee8526900b3960b7b884b8"}
Jul 29 12:31:16.597903 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:16.597619    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-cbbgq"
Jul 29 12:31:16.599088 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:16.599065    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-hnbhw" event={"ID":"02f86dac-d455-4b0e-af6a-0cdfc0a8b9f2","Type":"ContainerStarted","Data":"5085f18db734f1caad8f97f5ddbfbaf53904a25912b963bf709ee75bc7943ab5"}
Jul 29 12:31:16.599211 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:16.599185    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-hnbhw"
Jul 29 12:31:16.616766 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:16.616699    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-cbbgq" podStartSLOduration=1.091262863 podStartE2EDuration="3.616686311s" totalImagesPullingTime="2.525423448s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:31:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:31:16.61487951 +0000 UTC m=+359.308567838" watchObservedRunningTime="2026-07-29 12:31:16.616686311 +0000 UTC m=+359.310374600"
Jul 29 12:31:16.632586 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:16.632546    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-hnbhw" podStartSLOduration=1.571572759 podStartE2EDuration="4.632533735s" totalImagesPullingTime="3.060960976s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:31:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:31:16.63147274 +0000 UTC m=+359.325161053" watchObservedRunningTime="2026-07-29 12:31:16.632533735 +0000 UTC m=+359.326222045"
Jul 29 12:31:22.482966 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:22.482933    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6bb9c5fc49-n286g"]
Jul 29 12:31:22.486228 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:22.486208    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6bb9c5fc49-n286g"
Jul 29 12:31:22.489476 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:22.489457    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 29 12:31:22.490378 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:22.490360    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 29 12:31:22.493091 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:22.493073    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 29 12:31:22.507514 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:22.507495    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6bb9c5fc49-n286g"]
Jul 29 12:31:22.507666 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:22.507650    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 29 12:31:22.550356 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:22.550331    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7w52z\" (UniqueName: \"kubernetes.io/projected/52a11e4e-9288-433e-993f-987b09206bf2-kube-api-access-7w52z\") pod \"kube-auth-proxy-6bb9c5fc49-n286g\" (UID: \"52a11e4e-9288-433e-993f-987b09206bf2\") " pod="openshift-ingress/kube-auth-proxy-6bb9c5fc49-n286g"
Jul 29 12:31:22.550470 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:22.550361    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52a11e4e-9288-433e-993f-987b09206bf2-tls-certs\") pod \"kube-auth-proxy-6bb9c5fc49-n286g\" (UID: \"52a11e4e-9288-433e-993f-987b09206bf2\") " pod="openshift-ingress/kube-auth-proxy-6bb9c5fc49-n286g"
Jul 29 12:31:22.550470 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:22.550377    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/52a11e4e-9288-433e-993f-987b09206bf2-tmp\") pod \"kube-auth-proxy-6bb9c5fc49-n286g\" (UID: \"52a11e4e-9288-433e-993f-987b09206bf2\") " pod="openshift-ingress/kube-auth-proxy-6bb9c5fc49-n286g"
Jul 29 12:31:22.651252 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:22.651228    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52a11e4e-9288-433e-993f-987b09206bf2-tls-certs\") pod \"kube-auth-proxy-6bb9c5fc49-n286g\" (UID: \"52a11e4e-9288-433e-993f-987b09206bf2\") " pod="openshift-ingress/kube-auth-proxy-6bb9c5fc49-n286g"
Jul 29 12:31:22.651335 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:22.651255    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/52a11e4e-9288-433e-993f-987b09206bf2-tmp\") pod \"kube-auth-proxy-6bb9c5fc49-n286g\" (UID: \"52a11e4e-9288-433e-993f-987b09206bf2\") " pod="openshift-ingress/kube-auth-proxy-6bb9c5fc49-n286g"
Jul 29 12:31:22.651378 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:22.651339    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7w52z\" (UniqueName: \"kubernetes.io/projected/52a11e4e-9288-433e-993f-987b09206bf2-kube-api-access-7w52z\") pod \"kube-auth-proxy-6bb9c5fc49-n286g\" (UID: \"52a11e4e-9288-433e-993f-987b09206bf2\") " pod="openshift-ingress/kube-auth-proxy-6bb9c5fc49-n286g"
Jul 29 12:31:22.653638 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:22.653608    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/52a11e4e-9288-433e-993f-987b09206bf2-tmp\") pod \"kube-auth-proxy-6bb9c5fc49-n286g\" (UID: \"52a11e4e-9288-433e-993f-987b09206bf2\") " pod="openshift-ingress/kube-auth-proxy-6bb9c5fc49-n286g"
Jul 29 12:31:22.653787 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:22.653773    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52a11e4e-9288-433e-993f-987b09206bf2-tls-certs\") pod \"kube-auth-proxy-6bb9c5fc49-n286g\" (UID: \"52a11e4e-9288-433e-993f-987b09206bf2\") " pod="openshift-ingress/kube-auth-proxy-6bb9c5fc49-n286g"
Jul 29 12:31:22.671113 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:22.671087    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7w52z\" (UniqueName: \"kubernetes.io/projected/52a11e4e-9288-433e-993f-987b09206bf2-kube-api-access-7w52z\") pod \"kube-auth-proxy-6bb9c5fc49-n286g\" (UID: \"52a11e4e-9288-433e-993f-987b09206bf2\") " pod="openshift-ingress/kube-auth-proxy-6bb9c5fc49-n286g"
Jul 29 12:31:22.795250 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:22.795227    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6bb9c5fc49-n286g"
Jul 29 12:31:22.948981 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:22.948958    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6bb9c5fc49-n286g"]
Jul 29 12:31:22.950231 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:31:22.950199    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52a11e4e_9288_433e_993f_987b09206bf2.slice/crio-4fb95e4a99bc5438f2a57f71b98f0939bd18ddf56d6c847de2284839d985bf76 WatchSource:0}: Error finding container 4fb95e4a99bc5438f2a57f71b98f0939bd18ddf56d6c847de2284839d985bf76: Status 404 returned error can't find the container with id 4fb95e4a99bc5438f2a57f71b98f0939bd18ddf56d6c847de2284839d985bf76
Jul 29 12:31:23.183492 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:23.183416    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-zn4kr"]
Jul 29 12:31:23.186195 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:23.186176    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-zn4kr"
Jul 29 12:31:23.189154 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:23.189131    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 29 12:31:23.189435 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:23.189417    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-62dzk\""
Jul 29 12:31:23.201065 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:23.201043    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-zn4kr"]
Jul 29 12:31:23.204985 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:23.204964    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-6cn2w"]
Jul 29 12:31:23.207146 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:23.207131    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6cn2w"
Jul 29 12:31:23.209792 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:23.209771    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 29 12:31:23.209859 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:23.209828    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-sd8dq\""
Jul 29 12:31:23.220239 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:23.220221    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-6cn2w"]
Jul 29 12:31:23.256944 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:23.256924    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3a610ba1-bc8d-4484-890b-89646572c27b-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-6cn2w\" (UID: \"3a610ba1-bc8d-4484-890b-89646572c27b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6cn2w"
Jul 29 12:31:23.257020 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:23.256956    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0890babf-8771-4220-ad44-be722eb595e3-cert\") pod \"kserve-controller-manager-d6f4b65b8-zn4kr\" (UID: \"0890babf-8771-4220-ad44-be722eb595e3\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-zn4kr"
Jul 29 12:31:23.257020 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:23.256973    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2w8c8\" (UniqueName: \"kubernetes.io/projected/0890babf-8771-4220-ad44-be722eb595e3-kube-api-access-2w8c8\") pod \"kserve-controller-manager-d6f4b65b8-zn4kr\" (UID: \"0890babf-8771-4220-ad44-be722eb595e3\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-zn4kr"
Jul 29 12:31:23.257109 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:23.257056    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4f9l\" (UniqueName: \"kubernetes.io/projected/3a610ba1-bc8d-4484-890b-89646572c27b-kube-api-access-v4f9l\") pod \"llmisvc-controller-manager-777c6bd7d5-6cn2w\" (UID: \"3a610ba1-bc8d-4484-890b-89646572c27b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6cn2w"
Jul 29 12:31:23.358025 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:23.358000    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4f9l\" (UniqueName: \"kubernetes.io/projected/3a610ba1-bc8d-4484-890b-89646572c27b-kube-api-access-v4f9l\") pod \"llmisvc-controller-manager-777c6bd7d5-6cn2w\" (UID: \"3a610ba1-bc8d-4484-890b-89646572c27b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6cn2w"
Jul 29 12:31:23.358110 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:23.358045    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3a610ba1-bc8d-4484-890b-89646572c27b-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-6cn2w\" (UID: \"3a610ba1-bc8d-4484-890b-89646572c27b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6cn2w"
Jul 29 12:31:23.358110 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:23.358077    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0890babf-8771-4220-ad44-be722eb595e3-cert\") pod \"kserve-controller-manager-d6f4b65b8-zn4kr\" (UID: \"0890babf-8771-4220-ad44-be722eb595e3\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-zn4kr"
Jul 29 12:31:23.358110 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:23.358101    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2w8c8\" (UniqueName: \"kubernetes.io/projected/0890babf-8771-4220-ad44-be722eb595e3-kube-api-access-2w8c8\") pod \"kserve-controller-manager-d6f4b65b8-zn4kr\" (UID: \"0890babf-8771-4220-ad44-be722eb595e3\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-zn4kr"
Jul 29 12:31:23.360321 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:23.360303    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3a610ba1-bc8d-4484-890b-89646572c27b-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-6cn2w\" (UID: \"3a610ba1-bc8d-4484-890b-89646572c27b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6cn2w"
Jul 29 12:31:23.360380 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:23.360367    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0890babf-8771-4220-ad44-be722eb595e3-cert\") pod \"kserve-controller-manager-d6f4b65b8-zn4kr\" (UID: \"0890babf-8771-4220-ad44-be722eb595e3\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-zn4kr"
Jul 29 12:31:23.381814 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:23.381793    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4f9l\" (UniqueName: \"kubernetes.io/projected/3a610ba1-bc8d-4484-890b-89646572c27b-kube-api-access-v4f9l\") pod \"llmisvc-controller-manager-777c6bd7d5-6cn2w\" (UID: \"3a610ba1-bc8d-4484-890b-89646572c27b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6cn2w"
Jul 29 12:31:23.392542 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:23.392521    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2w8c8\" (UniqueName: \"kubernetes.io/projected/0890babf-8771-4220-ad44-be722eb595e3-kube-api-access-2w8c8\") pod \"kserve-controller-manager-d6f4b65b8-zn4kr\" (UID: \"0890babf-8771-4220-ad44-be722eb595e3\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-zn4kr"
Jul 29 12:31:23.498196 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:23.498142    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-zn4kr"
Jul 29 12:31:23.516740 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:23.516721    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6cn2w"
Jul 29 12:31:23.636295 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:23.636261    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6bb9c5fc49-n286g" event={"ID":"52a11e4e-9288-433e-993f-987b09206bf2","Type":"ContainerStarted","Data":"4fb95e4a99bc5438f2a57f71b98f0939bd18ddf56d6c847de2284839d985bf76"}
Jul 29 12:31:23.652128 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:23.652106    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-zn4kr"]
Jul 29 12:31:23.653233 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:31:23.653207    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0890babf_8771_4220_ad44_be722eb595e3.slice/crio-ac6ff0d61e85df75edaf41448327ff0363a9d347bf357b16e1b357a8eaa8d61c WatchSource:0}: Error finding container ac6ff0d61e85df75edaf41448327ff0363a9d347bf357b16e1b357a8eaa8d61c: Status 404 returned error can't find the container with id ac6ff0d61e85df75edaf41448327ff0363a9d347bf357b16e1b357a8eaa8d61c
Jul 29 12:31:23.676211 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:23.676192    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-6cn2w"]
Jul 29 12:31:23.677670 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:31:23.677651    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod3a610ba1_bc8d_4484_890b_89646572c27b.slice/crio-6b45c11b16216cf70b6e3b2bac09222951323e9fc050ad9d72133857580434b8 WatchSource:0}: Error finding container 6b45c11b16216cf70b6e3b2bac09222951323e9fc050ad9d72133857580434b8: Status 404 returned error can't find the container with id 6b45c11b16216cf70b6e3b2bac09222951323e9fc050ad9d72133857580434b8
Jul 29 12:31:24.649567 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:24.649430    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6cn2w" event={"ID":"3a610ba1-bc8d-4484-890b-89646572c27b","Type":"ContainerStarted","Data":"6b45c11b16216cf70b6e3b2bac09222951323e9fc050ad9d72133857580434b8"}
Jul 29 12:31:24.652468 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:24.652425    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-zn4kr" event={"ID":"0890babf-8771-4220-ad44-be722eb595e3","Type":"ContainerStarted","Data":"ac6ff0d61e85df75edaf41448327ff0363a9d347bf357b16e1b357a8eaa8d61c"}
Jul 29 12:31:26.139711 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:26.139678    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-hwvfh"]
Jul 29 12:31:26.154141 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:26.154116    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hwvfh"
Jul 29 12:31:26.157108 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:26.157031    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-22xmb\""
Jul 29 12:31:26.157381 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:26.157333    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 29 12:31:26.157650 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:26.157628    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 29 12:31:26.157854 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:26.157829    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 29 12:31:26.158028 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:26.157835    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 29 12:31:26.165257 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:26.163677    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-hwvfh"]
Jul 29 12:31:26.184960 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:26.184936    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7377791-4ee0-4583-a6a9-4339db5c7c40-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-hwvfh\" (UID: \"b7377791-4ee0-4583-a6a9-4339db5c7c40\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hwvfh"
Jul 29 12:31:26.185061 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:26.184979    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jdqb\" (UniqueName: \"kubernetes.io/projected/b7377791-4ee0-4583-a6a9-4339db5c7c40-kube-api-access-4jdqb\") pod \"istiod-openshift-gateway-5677ccb49b-hwvfh\" (UID: \"b7377791-4ee0-4583-a6a9-4339db5c7c40\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hwvfh"
Jul 29 12:31:26.185061 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:26.185002    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b7377791-4ee0-4583-a6a9-4339db5c7c40-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-hwvfh\" (UID: \"b7377791-4ee0-4583-a6a9-4339db5c7c40\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hwvfh"
Jul 29 12:31:26.185147 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:26.185078    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7377791-4ee0-4583-a6a9-4339db5c7c40-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-hwvfh\" (UID: \"b7377791-4ee0-4583-a6a9-4339db5c7c40\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hwvfh"
Jul 29 12:31:26.185147 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:26.185117    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b7377791-4ee0-4583-a6a9-4339db5c7c40-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-hwvfh\" (UID: \"b7377791-4ee0-4583-a6a9-4339db5c7c40\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hwvfh"
Jul 29 12:31:26.185210 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:26.185187    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b7377791-4ee0-4583-a6a9-4339db5c7c40-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-hwvfh\" (UID: \"b7377791-4ee0-4583-a6a9-4339db5c7c40\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hwvfh"
Jul 29 12:31:26.185261 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:26.185243    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b7377791-4ee0-4583-a6a9-4339db5c7c40-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-hwvfh\" (UID: \"b7377791-4ee0-4583-a6a9-4339db5c7c40\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hwvfh"
Jul 29 12:31:26.286559 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:26.286166    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7377791-4ee0-4583-a6a9-4339db5c7c40-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-hwvfh\" (UID: \"b7377791-4ee0-4583-a6a9-4339db5c7c40\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hwvfh"
Jul 29 12:31:26.286559 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:26.286209    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b7377791-4ee0-4583-a6a9-4339db5c7c40-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-hwvfh\" (UID: \"b7377791-4ee0-4583-a6a9-4339db5c7c40\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hwvfh"
Jul 29 12:31:26.286559 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:26.286271    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b7377791-4ee0-4583-a6a9-4339db5c7c40-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-hwvfh\" (UID: \"b7377791-4ee0-4583-a6a9-4339db5c7c40\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hwvfh"
Jul 29 12:31:26.286559 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:26.286324    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b7377791-4ee0-4583-a6a9-4339db5c7c40-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-hwvfh\" (UID: \"b7377791-4ee0-4583-a6a9-4339db5c7c40\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hwvfh"
Jul 29 12:31:26.286559 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:26.286365    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7377791-4ee0-4583-a6a9-4339db5c7c40-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-hwvfh\" (UID: \"b7377791-4ee0-4583-a6a9-4339db5c7c40\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hwvfh"
Jul 29 12:31:26.286559 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:26.286427    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jdqb\" (UniqueName: \"kubernetes.io/projected/b7377791-4ee0-4583-a6a9-4339db5c7c40-kube-api-access-4jdqb\") pod \"istiod-openshift-gateway-5677ccb49b-hwvfh\" (UID: \"b7377791-4ee0-4583-a6a9-4339db5c7c40\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hwvfh"
Jul 29 12:31:26.286559 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:26.286457    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b7377791-4ee0-4583-a6a9-4339db5c7c40-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-hwvfh\" (UID: \"b7377791-4ee0-4583-a6a9-4339db5c7c40\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hwvfh"
Jul 29 12:31:26.288006 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:26.287968    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b7377791-4ee0-4583-a6a9-4339db5c7c40-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-hwvfh\" (UID: \"b7377791-4ee0-4583-a6a9-4339db5c7c40\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hwvfh"
Jul 29 12:31:26.289975 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:26.289942    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b7377791-4ee0-4583-a6a9-4339db5c7c40-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-hwvfh\" (UID: \"b7377791-4ee0-4583-a6a9-4339db5c7c40\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hwvfh"
Jul 29 12:31:26.290210 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:26.290167    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b7377791-4ee0-4583-a6a9-4339db5c7c40-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-hwvfh\" (UID: \"b7377791-4ee0-4583-a6a9-4339db5c7c40\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hwvfh"
Jul 29 12:31:26.291601 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:26.291560    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7377791-4ee0-4583-a6a9-4339db5c7c40-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-hwvfh\" (UID: \"b7377791-4ee0-4583-a6a9-4339db5c7c40\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hwvfh"
Jul 29 12:31:26.291697 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:26.291622    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b7377791-4ee0-4583-a6a9-4339db5c7c40-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-hwvfh\" (UID: \"b7377791-4ee0-4583-a6a9-4339db5c7c40\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hwvfh"
Jul 29 12:31:26.301916 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:26.301893    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jdqb\" (UniqueName: \"kubernetes.io/projected/b7377791-4ee0-4583-a6a9-4339db5c7c40-kube-api-access-4jdqb\") pod \"istiod-openshift-gateway-5677ccb49b-hwvfh\" (UID: \"b7377791-4ee0-4583-a6a9-4339db5c7c40\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hwvfh"
Jul 29 12:31:26.328145 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:26.328126    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7377791-4ee0-4583-a6a9-4339db5c7c40-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-hwvfh\" (UID: \"b7377791-4ee0-4583-a6a9-4339db5c7c40\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hwvfh"
Jul 29 12:31:26.469513 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:26.469443    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hwvfh"
Jul 29 12:31:27.605686 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:27.605661    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-hnbhw"
Jul 29 12:31:27.607798 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:27.607780    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-cbbgq"
Jul 29 12:31:27.846634 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:27.846608    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-hwvfh"]
Jul 29 12:31:28.315535 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:31:28.315500    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7377791_4ee0_4583_a6a9_4339db5c7c40.slice/crio-ecbda06342e971980f8385d9304821e9d158693d602486384f7d1097280114e9 WatchSource:0}: Error finding container ecbda06342e971980f8385d9304821e9d158693d602486384f7d1097280114e9: Status 404 returned error can't find the container with id ecbda06342e971980f8385d9304821e9d158693d602486384f7d1097280114e9
Jul 29 12:31:28.678179 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:28.678100    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6cn2w" event={"ID":"3a610ba1-bc8d-4484-890b-89646572c27b","Type":"ContainerStarted","Data":"91598b95fccbd4ed9b09a4c7e73100113f00dd685cdb3dc3b06b59ef4a7c3c8f"}
Jul 29 12:31:28.678179 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:28.678162    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6cn2w"
Jul 29 12:31:28.679534 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:28.679507    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6bb9c5fc49-n286g" event={"ID":"52a11e4e-9288-433e-993f-987b09206bf2","Type":"ContainerStarted","Data":"eb6d74201758a4dab9294807d65032efc086709b68ae4d4028239b61a5f163e0"}
Jul 29 12:31:28.680913 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:28.680890    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-zn4kr" event={"ID":"0890babf-8771-4220-ad44-be722eb595e3","Type":"ContainerStarted","Data":"fb7df90c3ad6fde888d637aeae86459bdff0f4563ecb3a730f85976a8ca76d89"}
Jul 29 12:31:28.680987 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:28.680933    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-zn4kr"
Jul 29 12:31:28.682061 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:28.682043    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hwvfh" event={"ID":"b7377791-4ee0-4583-a6a9-4339db5c7c40","Type":"ContainerStarted","Data":"ecbda06342e971980f8385d9304821e9d158693d602486384f7d1097280114e9"}
Jul 29 12:31:28.709102 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:28.709058    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6cn2w" podStartSLOduration=1.031440107 podStartE2EDuration="5.709045945s" totalImagesPullingTime="4.677605838s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:31:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:31:28.708738904 +0000 UTC m=+371.402427239" watchObservedRunningTime="2026-07-29 12:31:28.709045945 +0000 UTC m=+371.402734255"
Jul 29 12:31:28.754631 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:28.754583    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6bb9c5fc49-n286g" podStartSLOduration=2.102837604 podStartE2EDuration="6.75457064s" totalImagesPullingTime="4.651733036s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:31:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:31:28.753704141 +0000 UTC m=+371.447392451" watchObservedRunningTime="2026-07-29 12:31:28.75457064 +0000 UTC m=+371.448258989"
Jul 29 12:31:28.780331 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:28.780289    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-zn4kr" podStartSLOduration=1.789289724 podStartE2EDuration="5.780276423s" totalImagesPullingTime="3.990986699s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:31:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:31:28.77836182 +0000 UTC m=+371.472050134" watchObservedRunningTime="2026-07-29 12:31:28.780276423 +0000 UTC m=+371.473964734"
Jul 29 12:31:30.756853 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:30.756816    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 12:31:30.757075 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:30.756884    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 12:31:31.701641 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:31.701597    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hwvfh" event={"ID":"b7377791-4ee0-4583-a6a9-4339db5c7c40","Type":"ContainerStarted","Data":"c1f6928daf734557008435532c5b4220081f06f8082792a5c9e5df809a9915d5"}
Jul 29 12:31:31.701802 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:31.701727    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hwvfh"
Jul 29 12:31:31.703244 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:31.703214    2567 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-hwvfh container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 29 12:31:31.703346 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:31.703268    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hwvfh" podUID="b7377791-4ee0-4583-a6a9-4339db5c7c40" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 29 12:31:31.730531 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:31.730483    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hwvfh" podStartSLOduration=3.292691593 podStartE2EDuration="5.730466888s" totalImagesPullingTime="2.437775295s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:31:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:31:31.727783267 +0000 UTC m=+374.421471577" watchObservedRunningTime="2026-07-29 12:31:31.730466888 +0000 UTC m=+374.424155200"
Jul 29 12:31:32.707781 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:32.707756    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hwvfh"
Jul 29 12:31:59.690608 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:59.690575    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6cn2w"
Jul 29 12:31:59.693920 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:31:59.693887    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-zn4kr"
Jul 29 12:32:28.731508 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:28.731462    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zd7m6"]
Jul 29 12:32:28.734274 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:28.734254    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zd7m6"
Jul 29 12:32:28.736859 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:28.736830    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 29 12:32:28.737010 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:28.736915    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 29 12:32:28.737837 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:28.737814    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-jw82m\""
Jul 29 12:32:28.744509 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:28.744483    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zd7m6"]
Jul 29 12:32:28.844518 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:28.844492    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-759g2\" (UniqueName: \"kubernetes.io/projected/e9c400e5-5a3f-4c97-a133-af20186cbb59-kube-api-access-759g2\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zd7m6\" (UID: \"e9c400e5-5a3f-4c97-a133-af20186cbb59\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zd7m6"
Jul 29 12:32:28.844641 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:28.844544    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e9c400e5-5a3f-4c97-a133-af20186cbb59-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zd7m6\" (UID: \"e9c400e5-5a3f-4c97-a133-af20186cbb59\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zd7m6"
Jul 29 12:32:28.844641 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:28.844570    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e9c400e5-5a3f-4c97-a133-af20186cbb59-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zd7m6\" (UID: \"e9c400e5-5a3f-4c97-a133-af20186cbb59\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zd7m6"
Jul 29 12:32:28.945318 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:28.945288    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e9c400e5-5a3f-4c97-a133-af20186cbb59-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zd7m6\" (UID: \"e9c400e5-5a3f-4c97-a133-af20186cbb59\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zd7m6"
Jul 29 12:32:28.945464 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:28.945349    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-759g2\" (UniqueName: \"kubernetes.io/projected/e9c400e5-5a3f-4c97-a133-af20186cbb59-kube-api-access-759g2\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zd7m6\" (UID: \"e9c400e5-5a3f-4c97-a133-af20186cbb59\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zd7m6"
Jul 29 12:32:28.945464 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:28.945437    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e9c400e5-5a3f-4c97-a133-af20186cbb59-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zd7m6\" (UID: \"e9c400e5-5a3f-4c97-a133-af20186cbb59\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zd7m6"
Jul 29 12:32:28.945743 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:28.945725    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e9c400e5-5a3f-4c97-a133-af20186cbb59-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zd7m6\" (UID: \"e9c400e5-5a3f-4c97-a133-af20186cbb59\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zd7m6"
Jul 29 12:32:28.945811 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:28.945740    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e9c400e5-5a3f-4c97-a133-af20186cbb59-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zd7m6\" (UID: \"e9c400e5-5a3f-4c97-a133-af20186cbb59\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zd7m6"
Jul 29 12:32:28.955049 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:28.955025    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-759g2\" (UniqueName: \"kubernetes.io/projected/e9c400e5-5a3f-4c97-a133-af20186cbb59-kube-api-access-759g2\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zd7m6\" (UID: \"e9c400e5-5a3f-4c97-a133-af20186cbb59\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zd7m6"
Jul 29 12:32:29.045557 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:29.045535    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zd7m6"
Jul 29 12:32:29.171974 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:29.171948    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zd7m6"]
Jul 29 12:32:29.173074 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:32:29.173051    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9c400e5_5a3f_4c97_a133_af20186cbb59.slice/crio-e689ff2e4ed4451b0a7c4b9577305bf37debe270f85fa020be94d78e6b297a7e WatchSource:0}: Error finding container e689ff2e4ed4451b0a7c4b9577305bf37debe270f85fa020be94d78e6b297a7e: Status 404 returned error can't find the container with id e689ff2e4ed4451b0a7c4b9577305bf37debe270f85fa020be94d78e6b297a7e
Jul 29 12:32:29.335911 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:29.335858    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595psw8"]
Jul 29 12:32:29.338545 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:29.338529    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595psw8"
Jul 29 12:32:29.350955 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:29.350933    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595psw8"]
Jul 29 12:32:29.449799 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:29.449777    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/41b3c368-9b73-4152-9cba-3b722aedf467-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595psw8\" (UID: \"41b3c368-9b73-4152-9cba-3b722aedf467\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595psw8"
Jul 29 12:32:29.449913 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:29.449809    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzs7r\" (UniqueName: \"kubernetes.io/projected/41b3c368-9b73-4152-9cba-3b722aedf467-kube-api-access-zzs7r\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595psw8\" (UID: \"41b3c368-9b73-4152-9cba-3b722aedf467\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595psw8"
Jul 29 12:32:29.449913 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:29.449892    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/41b3c368-9b73-4152-9cba-3b722aedf467-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595psw8\" (UID: \"41b3c368-9b73-4152-9cba-3b722aedf467\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595psw8"
Jul 29 12:32:29.551035 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:29.551005    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/41b3c368-9b73-4152-9cba-3b722aedf467-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595psw8\" (UID: \"41b3c368-9b73-4152-9cba-3b722aedf467\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595psw8"
Jul 29 12:32:29.551137 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:29.551042    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzs7r\" (UniqueName: \"kubernetes.io/projected/41b3c368-9b73-4152-9cba-3b722aedf467-kube-api-access-zzs7r\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595psw8\" (UID: \"41b3c368-9b73-4152-9cba-3b722aedf467\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595psw8"
Jul 29 12:32:29.551137 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:29.551109    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/41b3c368-9b73-4152-9cba-3b722aedf467-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595psw8\" (UID: \"41b3c368-9b73-4152-9cba-3b722aedf467\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595psw8"
Jul 29 12:32:29.551477 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:29.551456    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/41b3c368-9b73-4152-9cba-3b722aedf467-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595psw8\" (UID: \"41b3c368-9b73-4152-9cba-3b722aedf467\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595psw8"
Jul 29 12:32:29.551557 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:29.551465    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/41b3c368-9b73-4152-9cba-3b722aedf467-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595psw8\" (UID: \"41b3c368-9b73-4152-9cba-3b722aedf467\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595psw8"
Jul 29 12:32:29.560089 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:29.560068    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzs7r\" (UniqueName: \"kubernetes.io/projected/41b3c368-9b73-4152-9cba-3b722aedf467-kube-api-access-zzs7r\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595psw8\" (UID: \"41b3c368-9b73-4152-9cba-3b722aedf467\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595psw8"
Jul 29 12:32:29.647579 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:29.647532    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595psw8"
Jul 29 12:32:29.772473 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:29.772444    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595psw8"]
Jul 29 12:32:29.773996 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:32:29.773975    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41b3c368_9b73_4152_9cba_3b722aedf467.slice/crio-ceb18c7b0b457b4ee5d0530f4eac76cddb5243695190fa55f63dbbb1041aaf9a WatchSource:0}: Error finding container ceb18c7b0b457b4ee5d0530f4eac76cddb5243695190fa55f63dbbb1041aaf9a: Status 404 returned error can't find the container with id ceb18c7b0b457b4ee5d0530f4eac76cddb5243695190fa55f63dbbb1041aaf9a
Jul 29 12:32:29.932107 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:29.932057    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73czhfd"]
Jul 29 12:32:29.934335 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:29.934320    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73czhfd"
Jul 29 12:32:29.945940 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:29.945919    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73czhfd"]
Jul 29 12:32:30.011301 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:30.011274    2567 generic.go:356] "Generic (PLEG): container finished" podID="41b3c368-9b73-4152-9cba-3b722aedf467" containerID="019178a36bc4499f95b4bb3e3142bd53af8c81facd70c6e0736fb9a4bfa5a75c" exitCode=0
Jul 29 12:32:30.011449 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:30.011353    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595psw8" event={"ID":"41b3c368-9b73-4152-9cba-3b722aedf467","Type":"ContainerDied","Data":"019178a36bc4499f95b4bb3e3142bd53af8c81facd70c6e0736fb9a4bfa5a75c"}
Jul 29 12:32:30.011449 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:30.011421    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595psw8" event={"ID":"41b3c368-9b73-4152-9cba-3b722aedf467","Type":"ContainerStarted","Data":"ceb18c7b0b457b4ee5d0530f4eac76cddb5243695190fa55f63dbbb1041aaf9a"}
Jul 29 12:32:30.013157 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:30.013135    2567 generic.go:356] "Generic (PLEG): container finished" podID="e9c400e5-5a3f-4c97-a133-af20186cbb59" containerID="40c80e648bb6a067804a08066da73d97304085f8b109c91cad78925777beaef0" exitCode=0
Jul 29 12:32:30.013251 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:30.013217    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zd7m6" event={"ID":"e9c400e5-5a3f-4c97-a133-af20186cbb59","Type":"ContainerDied","Data":"40c80e648bb6a067804a08066da73d97304085f8b109c91cad78925777beaef0"}
Jul 29 12:32:30.013300 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:30.013257    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zd7m6" event={"ID":"e9c400e5-5a3f-4c97-a133-af20186cbb59","Type":"ContainerStarted","Data":"e689ff2e4ed4451b0a7c4b9577305bf37debe270f85fa020be94d78e6b297a7e"}
Jul 29 12:32:30.054986 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:30.054955    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0d4a9b8d-ec51-4957-a386-67046db0072f-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73czhfd\" (UID: \"0d4a9b8d-ec51-4957-a386-67046db0072f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73czhfd"
Jul 29 12:32:30.055087 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:30.055027    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkt2k\" (UniqueName: \"kubernetes.io/projected/0d4a9b8d-ec51-4957-a386-67046db0072f-kube-api-access-pkt2k\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73czhfd\" (UID: \"0d4a9b8d-ec51-4957-a386-67046db0072f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73czhfd"
Jul 29 12:32:30.055087 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:30.055068    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0d4a9b8d-ec51-4957-a386-67046db0072f-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73czhfd\" (UID: \"0d4a9b8d-ec51-4957-a386-67046db0072f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73czhfd"
Jul 29 12:32:30.156180 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:30.156155    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0d4a9b8d-ec51-4957-a386-67046db0072f-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73czhfd\" (UID: \"0d4a9b8d-ec51-4957-a386-67046db0072f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73czhfd"
Jul 29 12:32:30.156294 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:30.156194    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pkt2k\" (UniqueName: \"kubernetes.io/projected/0d4a9b8d-ec51-4957-a386-67046db0072f-kube-api-access-pkt2k\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73czhfd\" (UID: \"0d4a9b8d-ec51-4957-a386-67046db0072f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73czhfd"
Jul 29 12:32:30.156294 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:30.156226    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0d4a9b8d-ec51-4957-a386-67046db0072f-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73czhfd\" (UID: \"0d4a9b8d-ec51-4957-a386-67046db0072f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73czhfd"
Jul 29 12:32:30.156579 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:30.156561    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0d4a9b8d-ec51-4957-a386-67046db0072f-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73czhfd\" (UID: \"0d4a9b8d-ec51-4957-a386-67046db0072f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73czhfd"
Jul 29 12:32:30.156646 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:30.156599    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0d4a9b8d-ec51-4957-a386-67046db0072f-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73czhfd\" (UID: \"0d4a9b8d-ec51-4957-a386-67046db0072f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73czhfd"
Jul 29 12:32:30.165233 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:30.165207    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkt2k\" (UniqueName: \"kubernetes.io/projected/0d4a9b8d-ec51-4957-a386-67046db0072f-kube-api-access-pkt2k\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73czhfd\" (UID: \"0d4a9b8d-ec51-4957-a386-67046db0072f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73czhfd"
Jul 29 12:32:30.242904 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:30.242860    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73czhfd"
Jul 29 12:32:30.344680 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:30.344654    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zknzl"]
Jul 29 12:32:30.347918 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:30.347897    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zknzl"
Jul 29 12:32:30.362274 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:30.362252    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zknzl"]
Jul 29 12:32:30.378164 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:30.378146    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73czhfd"]
Jul 29 12:32:30.379563 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:32:30.379538    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d4a9b8d_ec51_4957_a386_67046db0072f.slice/crio-17cc9c50371fff14d1b4e611102f1f123c9284b2210e829d9c751f5d76c10a83 WatchSource:0}: Error finding container 17cc9c50371fff14d1b4e611102f1f123c9284b2210e829d9c751f5d76c10a83: Status 404 returned error can't find the container with id 17cc9c50371fff14d1b4e611102f1f123c9284b2210e829d9c751f5d76c10a83
Jul 29 12:32:30.458605 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:30.458576    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e0a1ea4f-b4d2-4e30-8de8-521a4b0b8616-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zknzl\" (UID: \"e0a1ea4f-b4d2-4e30-8de8-521a4b0b8616\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zknzl"
Jul 29 12:32:30.458724 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:30.458631    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgqm2\" (UniqueName: \"kubernetes.io/projected/e0a1ea4f-b4d2-4e30-8de8-521a4b0b8616-kube-api-access-jgqm2\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zknzl\" (UID: \"e0a1ea4f-b4d2-4e30-8de8-521a4b0b8616\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zknzl"
Jul 29 12:32:30.458784 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:30.458759    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e0a1ea4f-b4d2-4e30-8de8-521a4b0b8616-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zknzl\" (UID: \"e0a1ea4f-b4d2-4e30-8de8-521a4b0b8616\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zknzl"
Jul 29 12:32:30.559709 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:30.559682    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e0a1ea4f-b4d2-4e30-8de8-521a4b0b8616-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zknzl\" (UID: \"e0a1ea4f-b4d2-4e30-8de8-521a4b0b8616\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zknzl"
Jul 29 12:32:30.559824 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:30.559722    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgqm2\" (UniqueName: \"kubernetes.io/projected/e0a1ea4f-b4d2-4e30-8de8-521a4b0b8616-kube-api-access-jgqm2\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zknzl\" (UID: \"e0a1ea4f-b4d2-4e30-8de8-521a4b0b8616\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zknzl"
Jul 29 12:32:30.559824 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:30.559767    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e0a1ea4f-b4d2-4e30-8de8-521a4b0b8616-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zknzl\" (UID: \"e0a1ea4f-b4d2-4e30-8de8-521a4b0b8616\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zknzl"
Jul 29 12:32:30.560094 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:30.560073    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e0a1ea4f-b4d2-4e30-8de8-521a4b0b8616-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zknzl\" (UID: \"e0a1ea4f-b4d2-4e30-8de8-521a4b0b8616\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zknzl"
Jul 29 12:32:30.560159 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:30.560110    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e0a1ea4f-b4d2-4e30-8de8-521a4b0b8616-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zknzl\" (UID: \"e0a1ea4f-b4d2-4e30-8de8-521a4b0b8616\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zknzl"
Jul 29 12:32:30.569141 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:30.569113    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgqm2\" (UniqueName: \"kubernetes.io/projected/e0a1ea4f-b4d2-4e30-8de8-521a4b0b8616-kube-api-access-jgqm2\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zknzl\" (UID: \"e0a1ea4f-b4d2-4e30-8de8-521a4b0b8616\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zknzl"
Jul 29 12:32:30.657665 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:30.657642    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zknzl"
Jul 29 12:32:30.782523 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:30.782497    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zknzl"]
Jul 29 12:32:30.783591 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:32:30.783568    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0a1ea4f_b4d2_4e30_8de8_521a4b0b8616.slice/crio-49174aecc21867839889d7347afffdea9bd17f270a4457506a97e3417c8e0f71 WatchSource:0}: Error finding container 49174aecc21867839889d7347afffdea9bd17f270a4457506a97e3417c8e0f71: Status 404 returned error can't find the container with id 49174aecc21867839889d7347afffdea9bd17f270a4457506a97e3417c8e0f71
Jul 29 12:32:31.020533 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:31.020501    2567 generic.go:356] "Generic (PLEG): container finished" podID="e0a1ea4f-b4d2-4e30-8de8-521a4b0b8616" containerID="0024a0ff69a296c75a50894790a5e41bea05e220fbf02351d0743bbe93a81e2c" exitCode=0
Jul 29 12:32:31.020710 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:31.020580    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zknzl" event={"ID":"e0a1ea4f-b4d2-4e30-8de8-521a4b0b8616","Type":"ContainerDied","Data":"0024a0ff69a296c75a50894790a5e41bea05e220fbf02351d0743bbe93a81e2c"}
Jul 29 12:32:31.020710 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:31.020632    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zknzl" event={"ID":"e0a1ea4f-b4d2-4e30-8de8-521a4b0b8616","Type":"ContainerStarted","Data":"49174aecc21867839889d7347afffdea9bd17f270a4457506a97e3417c8e0f71"}
Jul 29 12:32:31.022307 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:31.022261    2567 generic.go:356] "Generic (PLEG): container finished" podID="0d4a9b8d-ec51-4957-a386-67046db0072f" containerID="40718c0e9d7b51cecebd2ab65b920d6def9d281d8fe926aabeb8013c81659fa8" exitCode=0
Jul 29 12:32:31.022450 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:31.022428    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73czhfd" event={"ID":"0d4a9b8d-ec51-4957-a386-67046db0072f","Type":"ContainerDied","Data":"40718c0e9d7b51cecebd2ab65b920d6def9d281d8fe926aabeb8013c81659fa8"}
Jul 29 12:32:31.022532 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:31.022472    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73czhfd" event={"ID":"0d4a9b8d-ec51-4957-a386-67046db0072f","Type":"ContainerStarted","Data":"17cc9c50371fff14d1b4e611102f1f123c9284b2210e829d9c751f5d76c10a83"}
Jul 29 12:32:32.030766 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:32.030728    2567 generic.go:356] "Generic (PLEG): container finished" podID="41b3c368-9b73-4152-9cba-3b722aedf467" containerID="377f21fcdb73f48d5d33ce47f277f4c27c8caae68761d5c7a4a3dfcae2920ade" exitCode=0
Jul 29 12:32:32.031163 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:32.030815    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595psw8" event={"ID":"41b3c368-9b73-4152-9cba-3b722aedf467","Type":"ContainerDied","Data":"377f21fcdb73f48d5d33ce47f277f4c27c8caae68761d5c7a4a3dfcae2920ade"}
Jul 29 12:32:32.032750 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:32.032727    2567 generic.go:356] "Generic (PLEG): container finished" podID="e9c400e5-5a3f-4c97-a133-af20186cbb59" containerID="f27d83d6b56e34c87b43b36a4c14ddcb06a057465f141337055c3cb36933afbd" exitCode=0
Jul 29 12:32:32.032842 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:32.032763    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zd7m6" event={"ID":"e9c400e5-5a3f-4c97-a133-af20186cbb59","Type":"ContainerDied","Data":"f27d83d6b56e34c87b43b36a4c14ddcb06a057465f141337055c3cb36933afbd"}
Jul 29 12:32:33.040326 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:33.040286    2567 generic.go:356] "Generic (PLEG): container finished" podID="e0a1ea4f-b4d2-4e30-8de8-521a4b0b8616" containerID="38bd005552644fb52156d61cee0bea14f26f2ff6f533001efea99af060bb2527" exitCode=0
Jul 29 12:32:33.040734 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:33.040373    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zknzl" event={"ID":"e0a1ea4f-b4d2-4e30-8de8-521a4b0b8616","Type":"ContainerDied","Data":"38bd005552644fb52156d61cee0bea14f26f2ff6f533001efea99af060bb2527"}
Jul 29 12:32:33.042966 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:33.042940    2567 generic.go:356] "Generic (PLEG): container finished" podID="0d4a9b8d-ec51-4957-a386-67046db0072f" containerID="08845852e90ce6efb1595032bf452f8f515af86e9f835f688a709687db1a2d46" exitCode=0
Jul 29 12:32:33.043090 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:33.043031    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73czhfd" event={"ID":"0d4a9b8d-ec51-4957-a386-67046db0072f","Type":"ContainerDied","Data":"08845852e90ce6efb1595032bf452f8f515af86e9f835f688a709687db1a2d46"}
Jul 29 12:32:33.045928 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:33.045905    2567 generic.go:356] "Generic (PLEG): container finished" podID="41b3c368-9b73-4152-9cba-3b722aedf467" containerID="20be7b99bfeb0f110551b13c3e1a32fb7ef1107ea1b4a97d208c16ec4e69ea84" exitCode=0
Jul 29 12:32:33.046057 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:33.045977    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595psw8" event={"ID":"41b3c368-9b73-4152-9cba-3b722aedf467","Type":"ContainerDied","Data":"20be7b99bfeb0f110551b13c3e1a32fb7ef1107ea1b4a97d208c16ec4e69ea84"}
Jul 29 12:32:33.048527 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:33.048490    2567 generic.go:356] "Generic (PLEG): container finished" podID="e9c400e5-5a3f-4c97-a133-af20186cbb59" containerID="bff5c139282de593e6be679cdb4251979730ec669565768c596d1461f430e0d4" exitCode=0
Jul 29 12:32:33.048636 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:33.048553    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zd7m6" event={"ID":"e9c400e5-5a3f-4c97-a133-af20186cbb59","Type":"ContainerDied","Data":"bff5c139282de593e6be679cdb4251979730ec669565768c596d1461f430e0d4"}
Jul 29 12:32:34.056004 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:34.055965    2567 generic.go:356] "Generic (PLEG): container finished" podID="e0a1ea4f-b4d2-4e30-8de8-521a4b0b8616" containerID="994d0460b4ada2eb04f8e0dd2d9d6bc2767b226a5be94a2bdd448225544615d0" exitCode=0
Jul 29 12:32:34.056553 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:34.056010    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zknzl" event={"ID":"e0a1ea4f-b4d2-4e30-8de8-521a4b0b8616","Type":"ContainerDied","Data":"994d0460b4ada2eb04f8e0dd2d9d6bc2767b226a5be94a2bdd448225544615d0"}
Jul 29 12:32:34.058580 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:34.058556    2567 generic.go:356] "Generic (PLEG): container finished" podID="0d4a9b8d-ec51-4957-a386-67046db0072f" containerID="d857fc5bd59d714d9ce1f152bddd2afa3fdad6d84e6d67a7ec666a9641a8257a" exitCode=0
Jul 29 12:32:34.058700 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:34.058633    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73czhfd" event={"ID":"0d4a9b8d-ec51-4957-a386-67046db0072f","Type":"ContainerDied","Data":"d857fc5bd59d714d9ce1f152bddd2afa3fdad6d84e6d67a7ec666a9641a8257a"}
Jul 29 12:32:34.190562 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:34.190540    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zd7m6"
Jul 29 12:32:34.223112 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:34.223085    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595psw8"
Jul 29 12:32:34.294260 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:34.294230    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e9c400e5-5a3f-4c97-a133-af20186cbb59-util\") pod \"e9c400e5-5a3f-4c97-a133-af20186cbb59\" (UID: \"e9c400e5-5a3f-4c97-a133-af20186cbb59\") "
Jul 29 12:32:34.294375 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:34.294282    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-759g2\" (UniqueName: \"kubernetes.io/projected/e9c400e5-5a3f-4c97-a133-af20186cbb59-kube-api-access-759g2\") pod \"e9c400e5-5a3f-4c97-a133-af20186cbb59\" (UID: \"e9c400e5-5a3f-4c97-a133-af20186cbb59\") "
Jul 29 12:32:34.294375 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:34.294325    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e9c400e5-5a3f-4c97-a133-af20186cbb59-bundle\") pod \"e9c400e5-5a3f-4c97-a133-af20186cbb59\" (UID: \"e9c400e5-5a3f-4c97-a133-af20186cbb59\") "
Jul 29 12:32:34.294848 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:34.294827    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9c400e5-5a3f-4c97-a133-af20186cbb59-bundle" (OuterVolumeSpecName: "bundle") pod "e9c400e5-5a3f-4c97-a133-af20186cbb59" (UID: "e9c400e5-5a3f-4c97-a133-af20186cbb59"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 12:32:34.296474 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:34.296449    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9c400e5-5a3f-4c97-a133-af20186cbb59-kube-api-access-759g2" (OuterVolumeSpecName: "kube-api-access-759g2") pod "e9c400e5-5a3f-4c97-a133-af20186cbb59" (UID: "e9c400e5-5a3f-4c97-a133-af20186cbb59"). InnerVolumeSpecName "kube-api-access-759g2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 12:32:34.395812 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:34.395758    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zzs7r\" (UniqueName: \"kubernetes.io/projected/41b3c368-9b73-4152-9cba-3b722aedf467-kube-api-access-zzs7r\") pod \"41b3c368-9b73-4152-9cba-3b722aedf467\" (UID: \"41b3c368-9b73-4152-9cba-3b722aedf467\") "
Jul 29 12:32:34.395909 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:34.395879    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/41b3c368-9b73-4152-9cba-3b722aedf467-bundle\") pod \"41b3c368-9b73-4152-9cba-3b722aedf467\" (UID: \"41b3c368-9b73-4152-9cba-3b722aedf467\") "
Jul 29 12:32:34.395974 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:34.395910    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/41b3c368-9b73-4152-9cba-3b722aedf467-util\") pod \"41b3c368-9b73-4152-9cba-3b722aedf467\" (UID: \"41b3c368-9b73-4152-9cba-3b722aedf467\") "
Jul 29 12:32:34.396116 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:34.396102    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-759g2\" (UniqueName: \"kubernetes.io/projected/e9c400e5-5a3f-4c97-a133-af20186cbb59-kube-api-access-759g2\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:32:34.396174 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:34.396123    2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e9c400e5-5a3f-4c97-a133-af20186cbb59-bundle\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:32:34.396300 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:34.396273    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41b3c368-9b73-4152-9cba-3b722aedf467-bundle" (OuterVolumeSpecName: "bundle") pod "41b3c368-9b73-4152-9cba-3b722aedf467" (UID: "41b3c368-9b73-4152-9cba-3b722aedf467"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 12:32:34.397568 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:34.397549    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41b3c368-9b73-4152-9cba-3b722aedf467-kube-api-access-zzs7r" (OuterVolumeSpecName: "kube-api-access-zzs7r") pod "41b3c368-9b73-4152-9cba-3b722aedf467" (UID: "41b3c368-9b73-4152-9cba-3b722aedf467"). InnerVolumeSpecName "kube-api-access-zzs7r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 12:32:34.400570 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:34.400547    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41b3c368-9b73-4152-9cba-3b722aedf467-util" (OuterVolumeSpecName: "util") pod "41b3c368-9b73-4152-9cba-3b722aedf467" (UID: "41b3c368-9b73-4152-9cba-3b722aedf467"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 12:32:34.497341 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:34.497321    2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/41b3c368-9b73-4152-9cba-3b722aedf467-bundle\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:32:34.497341 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:34.497339    2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/41b3c368-9b73-4152-9cba-3b722aedf467-util\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:32:34.497472 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:34.497348    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zzs7r\" (UniqueName: \"kubernetes.io/projected/41b3c368-9b73-4152-9cba-3b722aedf467-kube-api-access-zzs7r\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:32:34.608380 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:34.608354    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9c400e5-5a3f-4c97-a133-af20186cbb59-util" (OuterVolumeSpecName: "util") pod "e9c400e5-5a3f-4c97-a133-af20186cbb59" (UID: "e9c400e5-5a3f-4c97-a133-af20186cbb59"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 12:32:34.698987 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:34.698940    2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e9c400e5-5a3f-4c97-a133-af20186cbb59-util\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:32:35.066876 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:35.066845    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595psw8" event={"ID":"41b3c368-9b73-4152-9cba-3b722aedf467","Type":"ContainerDied","Data":"ceb18c7b0b457b4ee5d0530f4eac76cddb5243695190fa55f63dbbb1041aaf9a"}
Jul 29 12:32:35.067285 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:35.066904    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ceb18c7b0b457b4ee5d0530f4eac76cddb5243695190fa55f63dbbb1041aaf9a"
Jul 29 12:32:35.067285 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:35.066851    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595psw8"
Jul 29 12:32:35.069189 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:35.069169    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zd7m6"
Jul 29 12:32:35.069308 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:35.069244    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zd7m6" event={"ID":"e9c400e5-5a3f-4c97-a133-af20186cbb59","Type":"ContainerDied","Data":"e689ff2e4ed4451b0a7c4b9577305bf37debe270f85fa020be94d78e6b297a7e"}
Jul 29 12:32:35.069308 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:35.069290    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e689ff2e4ed4451b0a7c4b9577305bf37debe270f85fa020be94d78e6b297a7e"
Jul 29 12:32:35.207322 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:35.207298    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73czhfd"
Jul 29 12:32:35.231174 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:35.231153    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zknzl"
Jul 29 12:32:35.303522 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:35.303498    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0d4a9b8d-ec51-4957-a386-67046db0072f-bundle\") pod \"0d4a9b8d-ec51-4957-a386-67046db0072f\" (UID: \"0d4a9b8d-ec51-4957-a386-67046db0072f\") "
Jul 29 12:32:35.303632 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:35.303563    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pkt2k\" (UniqueName: \"kubernetes.io/projected/0d4a9b8d-ec51-4957-a386-67046db0072f-kube-api-access-pkt2k\") pod \"0d4a9b8d-ec51-4957-a386-67046db0072f\" (UID: \"0d4a9b8d-ec51-4957-a386-67046db0072f\") "
Jul 29 12:32:35.303632 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:35.303617    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0d4a9b8d-ec51-4957-a386-67046db0072f-util\") pod \"0d4a9b8d-ec51-4957-a386-67046db0072f\" (UID: \"0d4a9b8d-ec51-4957-a386-67046db0072f\") "
Jul 29 12:32:35.304083 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:35.304054    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d4a9b8d-ec51-4957-a386-67046db0072f-bundle" (OuterVolumeSpecName: "bundle") pod "0d4a9b8d-ec51-4957-a386-67046db0072f" (UID: "0d4a9b8d-ec51-4957-a386-67046db0072f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 12:32:35.305480 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:35.305455    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d4a9b8d-ec51-4957-a386-67046db0072f-kube-api-access-pkt2k" (OuterVolumeSpecName: "kube-api-access-pkt2k") pod "0d4a9b8d-ec51-4957-a386-67046db0072f" (UID: "0d4a9b8d-ec51-4957-a386-67046db0072f"). InnerVolumeSpecName "kube-api-access-pkt2k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 12:32:35.309005 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:35.308974    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d4a9b8d-ec51-4957-a386-67046db0072f-util" (OuterVolumeSpecName: "util") pod "0d4a9b8d-ec51-4957-a386-67046db0072f" (UID: "0d4a9b8d-ec51-4957-a386-67046db0072f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 12:32:35.404374 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:35.404325    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e0a1ea4f-b4d2-4e30-8de8-521a4b0b8616-util\") pod \"e0a1ea4f-b4d2-4e30-8de8-521a4b0b8616\" (UID: \"e0a1ea4f-b4d2-4e30-8de8-521a4b0b8616\") "
Jul 29 12:32:35.404374 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:35.404373    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e0a1ea4f-b4d2-4e30-8de8-521a4b0b8616-bundle\") pod \"e0a1ea4f-b4d2-4e30-8de8-521a4b0b8616\" (UID: \"e0a1ea4f-b4d2-4e30-8de8-521a4b0b8616\") "
Jul 29 12:32:35.404549 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:35.404423    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jgqm2\" (UniqueName: \"kubernetes.io/projected/e0a1ea4f-b4d2-4e30-8de8-521a4b0b8616-kube-api-access-jgqm2\") pod \"e0a1ea4f-b4d2-4e30-8de8-521a4b0b8616\" (UID: \"e0a1ea4f-b4d2-4e30-8de8-521a4b0b8616\") "
Jul 29 12:32:35.404606 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:35.404578    2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0d4a9b8d-ec51-4957-a386-67046db0072f-util\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:32:35.404606 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:35.404587    2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0d4a9b8d-ec51-4957-a386-67046db0072f-bundle\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:32:35.404606 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:35.404597    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pkt2k\" (UniqueName: \"kubernetes.io/projected/0d4a9b8d-ec51-4957-a386-67046db0072f-kube-api-access-pkt2k\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:32:35.404783 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:35.404765    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0a1ea4f-b4d2-4e30-8de8-521a4b0b8616-bundle" (OuterVolumeSpecName: "bundle") pod "e0a1ea4f-b4d2-4e30-8de8-521a4b0b8616" (UID: "e0a1ea4f-b4d2-4e30-8de8-521a4b0b8616"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 12:32:35.406319 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:35.406297    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0a1ea4f-b4d2-4e30-8de8-521a4b0b8616-kube-api-access-jgqm2" (OuterVolumeSpecName: "kube-api-access-jgqm2") pod "e0a1ea4f-b4d2-4e30-8de8-521a4b0b8616" (UID: "e0a1ea4f-b4d2-4e30-8de8-521a4b0b8616"). InnerVolumeSpecName "kube-api-access-jgqm2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 12:32:35.409679 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:35.409658    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0a1ea4f-b4d2-4e30-8de8-521a4b0b8616-util" (OuterVolumeSpecName: "util") pod "e0a1ea4f-b4d2-4e30-8de8-521a4b0b8616" (UID: "e0a1ea4f-b4d2-4e30-8de8-521a4b0b8616"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 12:32:35.505223 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:35.505200    2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e0a1ea4f-b4d2-4e30-8de8-521a4b0b8616-util\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:32:35.505223 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:35.505221    2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e0a1ea4f-b4d2-4e30-8de8-521a4b0b8616-bundle\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:32:35.505346 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:35.505231    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jgqm2\" (UniqueName: \"kubernetes.io/projected/e0a1ea4f-b4d2-4e30-8de8-521a4b0b8616-kube-api-access-jgqm2\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:32:36.076571 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:36.076545    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zknzl"
Jul 29 12:32:36.076911 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:36.076540    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zknzl" event={"ID":"e0a1ea4f-b4d2-4e30-8de8-521a4b0b8616","Type":"ContainerDied","Data":"49174aecc21867839889d7347afffdea9bd17f270a4457506a97e3417c8e0f71"}
Jul 29 12:32:36.076911 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:36.076676    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="49174aecc21867839889d7347afffdea9bd17f270a4457506a97e3417c8e0f71"
Jul 29 12:32:36.078996 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:36.078972    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73czhfd"
Jul 29 12:32:36.078996 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:36.078976    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73czhfd" event={"ID":"0d4a9b8d-ec51-4957-a386-67046db0072f","Type":"ContainerDied","Data":"17cc9c50371fff14d1b4e611102f1f123c9284b2210e829d9c751f5d76c10a83"}
Jul 29 12:32:36.079122 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:36.079019    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="17cc9c50371fff14d1b4e611102f1f123c9284b2210e829d9c751f5d76c10a83"
Jul 29 12:32:52.893303 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:52.893223    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6trq6"]
Jul 29 12:32:52.893675 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:52.893574    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d4a9b8d-ec51-4957-a386-67046db0072f" containerName="util"
Jul 29 12:32:52.893675 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:52.893586    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d4a9b8d-ec51-4957-a386-67046db0072f" containerName="util"
Jul 29 12:32:52.893675 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:52.893594    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d4a9b8d-ec51-4957-a386-67046db0072f" containerName="pull"
Jul 29 12:32:52.893675 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:52.893599    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d4a9b8d-ec51-4957-a386-67046db0072f" containerName="pull"
Jul 29 12:32:52.893675 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:52.893605    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0a1ea4f-b4d2-4e30-8de8-521a4b0b8616" containerName="extract"
Jul 29 12:32:52.893675 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:52.893610    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0a1ea4f-b4d2-4e30-8de8-521a4b0b8616" containerName="extract"
Jul 29 12:32:52.893675 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:52.893617    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9c400e5-5a3f-4c97-a133-af20186cbb59" containerName="util"
Jul 29 12:32:52.893675 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:52.893622    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9c400e5-5a3f-4c97-a133-af20186cbb59" containerName="util"
Jul 29 12:32:52.893675 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:52.893634    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41b3c368-9b73-4152-9cba-3b722aedf467" containerName="pull"
Jul 29 12:32:52.893675 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:52.893639    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="41b3c368-9b73-4152-9cba-3b722aedf467" containerName="pull"
Jul 29 12:32:52.893675 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:52.893645    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0a1ea4f-b4d2-4e30-8de8-521a4b0b8616" containerName="util"
Jul 29 12:32:52.893675 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:52.893650    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0a1ea4f-b4d2-4e30-8de8-521a4b0b8616" containerName="util"
Jul 29 12:32:52.893675 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:52.893656    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9c400e5-5a3f-4c97-a133-af20186cbb59" containerName="pull"
Jul 29 12:32:52.893675 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:52.893660    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9c400e5-5a3f-4c97-a133-af20186cbb59" containerName="pull"
Jul 29 12:32:52.893675 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:52.893666    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9c400e5-5a3f-4c97-a133-af20186cbb59" containerName="extract"
Jul 29 12:32:52.893675 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:52.893670    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9c400e5-5a3f-4c97-a133-af20186cbb59" containerName="extract"
Jul 29 12:32:52.893675 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:52.893675    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d4a9b8d-ec51-4957-a386-67046db0072f" containerName="extract"
Jul 29 12:32:52.893675 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:52.893680    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d4a9b8d-ec51-4957-a386-67046db0072f" containerName="extract"
Jul 29 12:32:52.894258 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:52.893687    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41b3c368-9b73-4152-9cba-3b722aedf467" containerName="extract"
Jul 29 12:32:52.894258 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:52.893692    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="41b3c368-9b73-4152-9cba-3b722aedf467" containerName="extract"
Jul 29 12:32:52.894258 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:52.893699    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41b3c368-9b73-4152-9cba-3b722aedf467" containerName="util"
Jul 29 12:32:52.894258 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:52.893704    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="41b3c368-9b73-4152-9cba-3b722aedf467" containerName="util"
Jul 29 12:32:52.894258 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:52.893709    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0a1ea4f-b4d2-4e30-8de8-521a4b0b8616" containerName="pull"
Jul 29 12:32:52.894258 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:52.893714    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0a1ea4f-b4d2-4e30-8de8-521a4b0b8616" containerName="pull"
Jul 29 12:32:52.894258 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:52.893770    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d4a9b8d-ec51-4957-a386-67046db0072f" containerName="extract"
Jul 29 12:32:52.894258 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:52.893779    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="41b3c368-9b73-4152-9cba-3b722aedf467" containerName="extract"
Jul 29 12:32:52.894258 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:52.893784    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0a1ea4f-b4d2-4e30-8de8-521a4b0b8616" containerName="extract"
Jul 29 12:32:52.894258 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:52.893789    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9c400e5-5a3f-4c97-a133-af20186cbb59" containerName="extract"
Jul 29 12:32:52.895597 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:52.895581    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6trq6"
Jul 29 12:32:52.902122 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:52.902095    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 29 12:32:52.902874 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:52.902852    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-vfd8q\""
Jul 29 12:32:52.902874 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:52.902862    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 29 12:32:52.911261 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:52.911239    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6trq6"]
Jul 29 12:32:52.931363 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:52.931335    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c22b7\" (UniqueName: \"kubernetes.io/projected/9ba608f9-7409-4305-9089-2c4edb2d2c30-kube-api-access-c22b7\") pod \"limitador-operator-controller-manager-859d7b78bd-6trq6\" (UID: \"9ba608f9-7409-4305-9089-2c4edb2d2c30\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6trq6"
Jul 29 12:32:53.032167 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:53.032145    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c22b7\" (UniqueName: \"kubernetes.io/projected/9ba608f9-7409-4305-9089-2c4edb2d2c30-kube-api-access-c22b7\") pod \"limitador-operator-controller-manager-859d7b78bd-6trq6\" (UID: \"9ba608f9-7409-4305-9089-2c4edb2d2c30\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6trq6"
Jul 29 12:32:53.044852 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:53.044834    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c22b7\" (UniqueName: \"kubernetes.io/projected/9ba608f9-7409-4305-9089-2c4edb2d2c30-kube-api-access-c22b7\") pod \"limitador-operator-controller-manager-859d7b78bd-6trq6\" (UID: \"9ba608f9-7409-4305-9089-2c4edb2d2c30\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6trq6"
Jul 29 12:32:53.211348 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:53.211287    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6trq6"
Jul 29 12:32:53.360842 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:32:53.360813    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ba608f9_7409_4305_9089_2c4edb2d2c30.slice/crio-1e28689e2295c9a8be335c1320cd7a5b399b7d9e96ad850e2349b29e81797462 WatchSource:0}: Error finding container 1e28689e2295c9a8be335c1320cd7a5b399b7d9e96ad850e2349b29e81797462: Status 404 returned error can't find the container with id 1e28689e2295c9a8be335c1320cd7a5b399b7d9e96ad850e2349b29e81797462
Jul 29 12:32:53.364577 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:53.363146    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6trq6"]
Jul 29 12:32:54.177177 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:54.177140    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6trq6" event={"ID":"9ba608f9-7409-4305-9089-2c4edb2d2c30","Type":"ContainerStarted","Data":"1e28689e2295c9a8be335c1320cd7a5b399b7d9e96ad850e2349b29e81797462"}
Jul 29 12:32:55.185790 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:55.185764    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6trq6" event={"ID":"9ba608f9-7409-4305-9089-2c4edb2d2c30","Type":"ContainerStarted","Data":"d8926741d8341e96b8d62ad442223925d775cab42cd4374c3154ec2c4a8e3c13"}
Jul 29 12:32:55.186052 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:55.185822    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6trq6"
Jul 29 12:32:55.223891 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:32:55.223846    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6trq6" podStartSLOduration=1.518185118 podStartE2EDuration="3.223831231s" totalImagesPullingTime="1.705646113s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:32:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:32:55.221626714 +0000 UTC m=+457.915315045" watchObservedRunningTime="2026-07-29 12:32:55.223831231 +0000 UTC m=+457.917519541"
Jul 29 12:33:06.194628 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:06.194600    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6trq6"
Jul 29 12:33:15.683622 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:15.683569    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-m4jp5"]
Jul 29 12:33:15.687912 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:15.687881    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-m4jp5"
Jul 29 12:33:15.690962 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:15.690931    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-9h9ms\""
Jul 29 12:33:15.708217 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:15.708188    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-m4jp5"]
Jul 29 12:33:15.808774 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:15.808739    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/658b1eda-5f0e-4028-88b1-015a2f059b02-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-m4jp5\" (UID: \"658b1eda-5f0e-4028-88b1-015a2f059b02\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-m4jp5"
Jul 29 12:33:15.808919 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:15.808792    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvtxq\" (UniqueName: \"kubernetes.io/projected/658b1eda-5f0e-4028-88b1-015a2f059b02-kube-api-access-tvtxq\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-m4jp5\" (UID: \"658b1eda-5f0e-4028-88b1-015a2f059b02\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-m4jp5"
Jul 29 12:33:15.810940 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:15.810919    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-m4jp5"]
Jul 29 12:33:15.811109 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:33:15.811091    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-tvtxq], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-m4jp5" podUID="658b1eda-5f0e-4028-88b1-015a2f059b02"
Jul 29 12:33:15.909555 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:15.909529    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/658b1eda-5f0e-4028-88b1-015a2f059b02-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-m4jp5\" (UID: \"658b1eda-5f0e-4028-88b1-015a2f059b02\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-m4jp5"
Jul 29 12:33:15.909645 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:15.909562    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvtxq\" (UniqueName: \"kubernetes.io/projected/658b1eda-5f0e-4028-88b1-015a2f059b02-kube-api-access-tvtxq\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-m4jp5\" (UID: \"658b1eda-5f0e-4028-88b1-015a2f059b02\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-m4jp5"
Jul 29 12:33:15.909855 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:15.909838    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/658b1eda-5f0e-4028-88b1-015a2f059b02-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-m4jp5\" (UID: \"658b1eda-5f0e-4028-88b1-015a2f059b02\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-m4jp5"
Jul 29 12:33:15.931754 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:15.931734    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvtxq\" (UniqueName: \"kubernetes.io/projected/658b1eda-5f0e-4028-88b1-015a2f059b02-kube-api-access-tvtxq\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-m4jp5\" (UID: \"658b1eda-5f0e-4028-88b1-015a2f059b02\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-m4jp5"
Jul 29 12:33:16.307002 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:16.306973    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-m4jp5"
Jul 29 12:33:16.313744 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:16.313720    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-m4jp5"
Jul 29 12:33:16.414023 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:16.414001    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tvtxq\" (UniqueName: \"kubernetes.io/projected/658b1eda-5f0e-4028-88b1-015a2f059b02-kube-api-access-tvtxq\") pod \"658b1eda-5f0e-4028-88b1-015a2f059b02\" (UID: \"658b1eda-5f0e-4028-88b1-015a2f059b02\") "
Jul 29 12:33:16.414128 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:16.414049    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/658b1eda-5f0e-4028-88b1-015a2f059b02-extensions-socket-volume\") pod \"658b1eda-5f0e-4028-88b1-015a2f059b02\" (UID: \"658b1eda-5f0e-4028-88b1-015a2f059b02\") "
Jul 29 12:33:16.414345 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:16.414321    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/658b1eda-5f0e-4028-88b1-015a2f059b02-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "658b1eda-5f0e-4028-88b1-015a2f059b02" (UID: "658b1eda-5f0e-4028-88b1-015a2f059b02"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 12:33:16.416027 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:16.416001    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/658b1eda-5f0e-4028-88b1-015a2f059b02-kube-api-access-tvtxq" (OuterVolumeSpecName: "kube-api-access-tvtxq") pod "658b1eda-5f0e-4028-88b1-015a2f059b02" (UID: "658b1eda-5f0e-4028-88b1-015a2f059b02"). InnerVolumeSpecName "kube-api-access-tvtxq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 12:33:16.483329 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:16.483302    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-m4jp5"]
Jul 29 12:33:16.501396 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:16.501372    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-m4jp5"]
Jul 29 12:33:16.515435 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:16.515413    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tvtxq\" (UniqueName: \"kubernetes.io/projected/658b1eda-5f0e-4028-88b1-015a2f059b02-kube-api-access-tvtxq\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:33:16.515435 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:16.515435    2567 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/658b1eda-5f0e-4028-88b1-015a2f059b02-extensions-socket-volume\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:33:16.556065 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:16.556042    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6trq6"]
Jul 29 12:33:16.556258 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:16.556234    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6trq6" podUID="9ba608f9-7409-4305-9089-2c4edb2d2c30" containerName="manager" containerID="cri-o://d8926741d8341e96b8d62ad442223925d775cab42cd4374c3154ec2c4a8e3c13" gracePeriod=2
Jul 29 12:33:16.568064 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:16.568044    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6trq6"]
Jul 29 12:33:16.619796 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:16.619770    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l8jlz"]
Jul 29 12:33:16.620178 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:16.620161    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ba608f9-7409-4305-9089-2c4edb2d2c30" containerName="manager"
Jul 29 12:33:16.620223 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:16.620184    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ba608f9-7409-4305-9089-2c4edb2d2c30" containerName="manager"
Jul 29 12:33:16.620305 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:16.620294    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ba608f9-7409-4305-9089-2c4edb2d2c30" containerName="manager"
Jul 29 12:33:16.622484 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:16.622460    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l8jlz"
Jul 29 12:33:16.637989 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:16.637967    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l8jlz"]
Jul 29 12:33:16.658604 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:33:16.658575    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-6trq6\" is forbidden: User \"system:node:ip-10-0-128-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-25.ec2.internal' and this object" podUID="9ba608f9-7409-4305-9089-2c4edb2d2c30" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6trq6"
Jul 29 12:33:16.717313 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:16.717282    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k898q\" (UniqueName: \"kubernetes.io/projected/a39801db-46e2-4be7-b99a-083d68ff26fc-kube-api-access-k898q\") pod \"limitador-operator-controller-manager-859d7b78bd-l8jlz\" (UID: \"a39801db-46e2-4be7-b99a-083d68ff26fc\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l8jlz"
Jul 29 12:33:16.818118 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:16.818095    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k898q\" (UniqueName: \"kubernetes.io/projected/a39801db-46e2-4be7-b99a-083d68ff26fc-kube-api-access-k898q\") pod \"limitador-operator-controller-manager-859d7b78bd-l8jlz\" (UID: \"a39801db-46e2-4be7-b99a-083d68ff26fc\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l8jlz"
Jul 29 12:33:16.828883 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:16.828864    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6trq6"
Jul 29 12:33:16.831252 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:16.831233    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k898q\" (UniqueName: \"kubernetes.io/projected/a39801db-46e2-4be7-b99a-083d68ff26fc-kube-api-access-k898q\") pod \"limitador-operator-controller-manager-859d7b78bd-l8jlz\" (UID: \"a39801db-46e2-4be7-b99a-083d68ff26fc\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l8jlz"
Jul 29 12:33:16.831452 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:33:16.831435    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-6trq6\" is forbidden: User \"system:node:ip-10-0-128-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-25.ec2.internal' and this object" podUID="9ba608f9-7409-4305-9089-2c4edb2d2c30" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6trq6"
Jul 29 12:33:16.919306 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:16.919288    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c22b7\" (UniqueName: \"kubernetes.io/projected/9ba608f9-7409-4305-9089-2c4edb2d2c30-kube-api-access-c22b7\") pod \"9ba608f9-7409-4305-9089-2c4edb2d2c30\" (UID: \"9ba608f9-7409-4305-9089-2c4edb2d2c30\") "
Jul 29 12:33:16.921047 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:16.921027    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ba608f9-7409-4305-9089-2c4edb2d2c30-kube-api-access-c22b7" (OuterVolumeSpecName: "kube-api-access-c22b7") pod "9ba608f9-7409-4305-9089-2c4edb2d2c30" (UID: "9ba608f9-7409-4305-9089-2c4edb2d2c30"). InnerVolumeSpecName "kube-api-access-c22b7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 12:33:16.932175 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:16.932152    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l8jlz"
Jul 29 12:33:17.020149 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:17.020117    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c22b7\" (UniqueName: \"kubernetes.io/projected/9ba608f9-7409-4305-9089-2c4edb2d2c30-kube-api-access-c22b7\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:33:17.087155 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:17.087062    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l8jlz"]
Jul 29 12:33:17.089289 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:33:17.089262    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda39801db_46e2_4be7_b99a_083d68ff26fc.slice/crio-5ac529807889a687d40a37888629b7007e688afd8ca5ae806f2af424f5fbf10c WatchSource:0}: Error finding container 5ac529807889a687d40a37888629b7007e688afd8ca5ae806f2af424f5fbf10c: Status 404 returned error can't find the container with id 5ac529807889a687d40a37888629b7007e688afd8ca5ae806f2af424f5fbf10c
Jul 29 12:33:17.315088 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:17.315058    2567 generic.go:356] "Generic (PLEG): container finished" podID="9ba608f9-7409-4305-9089-2c4edb2d2c30" containerID="d8926741d8341e96b8d62ad442223925d775cab42cd4374c3154ec2c4a8e3c13" exitCode=0
Jul 29 12:33:17.315193 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:17.315097    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6trq6"
Jul 29 12:33:17.315247 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:17.315197    2567 scope.go:117] "RemoveContainer" containerID="d8926741d8341e96b8d62ad442223925d775cab42cd4374c3154ec2c4a8e3c13"
Jul 29 12:33:17.316982 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:17.316960    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-m4jp5"
Jul 29 12:33:17.317109 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:17.316983    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l8jlz" event={"ID":"a39801db-46e2-4be7-b99a-083d68ff26fc","Type":"ContainerStarted","Data":"feefb265f58971f3025567340a11909920e488273f842a0486979234f0004475"}
Jul 29 12:33:17.317109 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:17.317037    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l8jlz" event={"ID":"a39801db-46e2-4be7-b99a-083d68ff26fc","Type":"ContainerStarted","Data":"5ac529807889a687d40a37888629b7007e688afd8ca5ae806f2af424f5fbf10c"}
Jul 29 12:33:17.317203 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:17.317139    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l8jlz"
Jul 29 12:33:17.317731 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:33:17.317705    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-6trq6\" is forbidden: User \"system:node:ip-10-0-128-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-25.ec2.internal' and this object" podUID="9ba608f9-7409-4305-9089-2c4edb2d2c30" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6trq6"
Jul 29 12:33:17.325287 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:17.325203    2567 scope.go:117] "RemoveContainer" containerID="d8926741d8341e96b8d62ad442223925d775cab42cd4374c3154ec2c4a8e3c13"
Jul 29 12:33:17.325503 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:33:17.325479    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8926741d8341e96b8d62ad442223925d775cab42cd4374c3154ec2c4a8e3c13\": container with ID starting with d8926741d8341e96b8d62ad442223925d775cab42cd4374c3154ec2c4a8e3c13 not found: ID does not exist" containerID="d8926741d8341e96b8d62ad442223925d775cab42cd4374c3154ec2c4a8e3c13"
Jul 29 12:33:17.325587 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:17.325513    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8926741d8341e96b8d62ad442223925d775cab42cd4374c3154ec2c4a8e3c13"} err="failed to get container status \"d8926741d8341e96b8d62ad442223925d775cab42cd4374c3154ec2c4a8e3c13\": rpc error: code = NotFound desc = could not find container \"d8926741d8341e96b8d62ad442223925d775cab42cd4374c3154ec2c4a8e3c13\": container with ID starting with d8926741d8341e96b8d62ad442223925d775cab42cd4374c3154ec2c4a8e3c13 not found: ID does not exist"
Jul 29 12:33:17.343578 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:17.343509    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l8jlz" podStartSLOduration=1.343497043 podStartE2EDuration="1.343497043s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:33:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:33:17.342836259 +0000 UTC m=+480.036524570" watchObservedRunningTime="2026-07-29 12:33:17.343497043 +0000 UTC m=+480.037185354"
Jul 29 12:33:17.345626 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:33:17.345604    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-6trq6\" is forbidden: User \"system:node:ip-10-0-128-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-25.ec2.internal' and this object" podUID="9ba608f9-7409-4305-9089-2c4edb2d2c30" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6trq6"
Jul 29 12:33:17.348305 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:33:17.348278    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-m4jp5\" is forbidden: User \"system:node:ip-10-0-128-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-25.ec2.internal' and this object" podUID="658b1eda-5f0e-4028-88b1-015a2f059b02" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-m4jp5"
Jul 29 12:33:17.351065 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:33:17.351043    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-6trq6\" is forbidden: User \"system:node:ip-10-0-128-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-25.ec2.internal' and this object" podUID="9ba608f9-7409-4305-9089-2c4edb2d2c30" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6trq6"
Jul 29 12:33:17.356686 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:33:17.356659    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-m4jp5\" is forbidden: User \"system:node:ip-10-0-128-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-25.ec2.internal' and this object" podUID="658b1eda-5f0e-4028-88b1-015a2f059b02" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-m4jp5"
Jul 29 12:33:17.920314 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:33:17.920287    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-6trq6\" is forbidden: User \"system:node:ip-10-0-128-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-25.ec2.internal' and this object" podUID="9ba608f9-7409-4305-9089-2c4edb2d2c30" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6trq6"
Jul 29 12:33:17.922493 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:33:17.922455    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-m4jp5\" is forbidden: User \"system:node:ip-10-0-128-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-25.ec2.internal' and this object" podUID="658b1eda-5f0e-4028-88b1-015a2f059b02" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-m4jp5"
Jul 29 12:33:17.924436 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:17.923323    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="658b1eda-5f0e-4028-88b1-015a2f059b02" path="/var/lib/kubelet/pods/658b1eda-5f0e-4028-88b1-015a2f059b02/volumes"
Jul 29 12:33:17.924436 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:17.923697    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ba608f9-7409-4305-9089-2c4edb2d2c30" path="/var/lib/kubelet/pods/9ba608f9-7409-4305-9089-2c4edb2d2c30/volumes"
Jul 29 12:33:26.516301 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:26.516258    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-76bb87fbc7-6khpt"]
Jul 29 12:33:26.523792 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:26.523763    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76bb87fbc7-6khpt"
Jul 29 12:33:26.526711 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:26.526687    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 29 12:33:26.527764 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:26.527728    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 29 12:33:26.527916 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:26.527803    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-gwdkx\""
Jul 29 12:33:26.527916 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:26.527731    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 29 12:33:26.527916 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:26.527871    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 29 12:33:26.527916 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:26.527728    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 29 12:33:26.537189 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:26.537165    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 29 12:33:26.550629 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:26.550605    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76bb87fbc7-6khpt"]
Jul 29 12:33:26.596681 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:26.596653    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/32b1ff08-d437-4015-85d5-dff245655e8b-console-oauth-config\") pod \"console-76bb87fbc7-6khpt\" (UID: \"32b1ff08-d437-4015-85d5-dff245655e8b\") " pod="openshift-console/console-76bb87fbc7-6khpt"
Jul 29 12:33:26.596765 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:26.596695    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/32b1ff08-d437-4015-85d5-dff245655e8b-trusted-ca-bundle\") pod \"console-76bb87fbc7-6khpt\" (UID: \"32b1ff08-d437-4015-85d5-dff245655e8b\") " pod="openshift-console/console-76bb87fbc7-6khpt"
Jul 29 12:33:26.596765 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:26.596724    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/32b1ff08-d437-4015-85d5-dff245655e8b-console-config\") pod \"console-76bb87fbc7-6khpt\" (UID: \"32b1ff08-d437-4015-85d5-dff245655e8b\") " pod="openshift-console/console-76bb87fbc7-6khpt"
Jul 29 12:33:26.596834 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:26.596800    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/32b1ff08-d437-4015-85d5-dff245655e8b-console-serving-cert\") pod \"console-76bb87fbc7-6khpt\" (UID: \"32b1ff08-d437-4015-85d5-dff245655e8b\") " pod="openshift-console/console-76bb87fbc7-6khpt"
Jul 29 12:33:26.596867 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:26.596851    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpvtr\" (UniqueName: \"kubernetes.io/projected/32b1ff08-d437-4015-85d5-dff245655e8b-kube-api-access-zpvtr\") pod \"console-76bb87fbc7-6khpt\" (UID: \"32b1ff08-d437-4015-85d5-dff245655e8b\") " pod="openshift-console/console-76bb87fbc7-6khpt"
Jul 29 12:33:26.596897 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:26.596881    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/32b1ff08-d437-4015-85d5-dff245655e8b-service-ca\") pod \"console-76bb87fbc7-6khpt\" (UID: \"32b1ff08-d437-4015-85d5-dff245655e8b\") " pod="openshift-console/console-76bb87fbc7-6khpt"
Jul 29 12:33:26.596929 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:26.596921    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/32b1ff08-d437-4015-85d5-dff245655e8b-oauth-serving-cert\") pod \"console-76bb87fbc7-6khpt\" (UID: \"32b1ff08-d437-4015-85d5-dff245655e8b\") " pod="openshift-console/console-76bb87fbc7-6khpt"
Jul 29 12:33:26.697212 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:26.697187    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/32b1ff08-d437-4015-85d5-dff245655e8b-oauth-serving-cert\") pod \"console-76bb87fbc7-6khpt\" (UID: \"32b1ff08-d437-4015-85d5-dff245655e8b\") " pod="openshift-console/console-76bb87fbc7-6khpt"
Jul 29 12:33:26.697299 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:26.697221    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/32b1ff08-d437-4015-85d5-dff245655e8b-console-oauth-config\") pod \"console-76bb87fbc7-6khpt\" (UID: \"32b1ff08-d437-4015-85d5-dff245655e8b\") " pod="openshift-console/console-76bb87fbc7-6khpt"
Jul 29 12:33:26.697299 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:26.697247    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/32b1ff08-d437-4015-85d5-dff245655e8b-trusted-ca-bundle\") pod \"console-76bb87fbc7-6khpt\" (UID: \"32b1ff08-d437-4015-85d5-dff245655e8b\") " pod="openshift-console/console-76bb87fbc7-6khpt"
Jul 29 12:33:26.697299 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:26.697278    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/32b1ff08-d437-4015-85d5-dff245655e8b-console-config\") pod \"console-76bb87fbc7-6khpt\" (UID: \"32b1ff08-d437-4015-85d5-dff245655e8b\") " pod="openshift-console/console-76bb87fbc7-6khpt"
Jul 29 12:33:26.697454 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:26.697307    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/32b1ff08-d437-4015-85d5-dff245655e8b-console-serving-cert\") pod \"console-76bb87fbc7-6khpt\" (UID: \"32b1ff08-d437-4015-85d5-dff245655e8b\") " pod="openshift-console/console-76bb87fbc7-6khpt"
Jul 29 12:33:26.697454 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:26.697330    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpvtr\" (UniqueName: \"kubernetes.io/projected/32b1ff08-d437-4015-85d5-dff245655e8b-kube-api-access-zpvtr\") pod \"console-76bb87fbc7-6khpt\" (UID: \"32b1ff08-d437-4015-85d5-dff245655e8b\") " pod="openshift-console/console-76bb87fbc7-6khpt"
Jul 29 12:33:26.697602 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:26.697568    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/32b1ff08-d437-4015-85d5-dff245655e8b-service-ca\") pod \"console-76bb87fbc7-6khpt\" (UID: \"32b1ff08-d437-4015-85d5-dff245655e8b\") " pod="openshift-console/console-76bb87fbc7-6khpt"
Jul 29 12:33:26.698029 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:26.698002    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/32b1ff08-d437-4015-85d5-dff245655e8b-console-config\") pod \"console-76bb87fbc7-6khpt\" (UID: \"32b1ff08-d437-4015-85d5-dff245655e8b\") " pod="openshift-console/console-76bb87fbc7-6khpt"
Jul 29 12:33:26.698138 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:26.698002    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/32b1ff08-d437-4015-85d5-dff245655e8b-oauth-serving-cert\") pod \"console-76bb87fbc7-6khpt\" (UID: \"32b1ff08-d437-4015-85d5-dff245655e8b\") " pod="openshift-console/console-76bb87fbc7-6khpt"
Jul 29 12:33:26.698200 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:26.698183    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/32b1ff08-d437-4015-85d5-dff245655e8b-service-ca\") pod \"console-76bb87fbc7-6khpt\" (UID: \"32b1ff08-d437-4015-85d5-dff245655e8b\") " pod="openshift-console/console-76bb87fbc7-6khpt"
Jul 29 12:33:26.698279 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:26.698254    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/32b1ff08-d437-4015-85d5-dff245655e8b-trusted-ca-bundle\") pod \"console-76bb87fbc7-6khpt\" (UID: \"32b1ff08-d437-4015-85d5-dff245655e8b\") " pod="openshift-console/console-76bb87fbc7-6khpt"
Jul 29 12:33:26.699775 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:26.699757    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/32b1ff08-d437-4015-85d5-dff245655e8b-console-oauth-config\") pod \"console-76bb87fbc7-6khpt\" (UID: \"32b1ff08-d437-4015-85d5-dff245655e8b\") " pod="openshift-console/console-76bb87fbc7-6khpt"
Jul 29 12:33:26.700023 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:26.699999    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/32b1ff08-d437-4015-85d5-dff245655e8b-console-serving-cert\") pod \"console-76bb87fbc7-6khpt\" (UID: \"32b1ff08-d437-4015-85d5-dff245655e8b\") " pod="openshift-console/console-76bb87fbc7-6khpt"
Jul 29 12:33:26.709465 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:26.709445    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpvtr\" (UniqueName: \"kubernetes.io/projected/32b1ff08-d437-4015-85d5-dff245655e8b-kube-api-access-zpvtr\") pod \"console-76bb87fbc7-6khpt\" (UID: \"32b1ff08-d437-4015-85d5-dff245655e8b\") " pod="openshift-console/console-76bb87fbc7-6khpt"
Jul 29 12:33:26.833784 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:26.833753    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76bb87fbc7-6khpt"
Jul 29 12:33:26.975274 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:26.975244    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76bb87fbc7-6khpt"]
Jul 29 12:33:26.978933 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:33:26.978905    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32b1ff08_d437_4015_85d5_dff245655e8b.slice/crio-7429fc396eb87eccd890e4d128ed64f3c070406dda88b3e81353cbaa520f9954 WatchSource:0}: Error finding container 7429fc396eb87eccd890e4d128ed64f3c070406dda88b3e81353cbaa520f9954: Status 404 returned error can't find the container with id 7429fc396eb87eccd890e4d128ed64f3c070406dda88b3e81353cbaa520f9954
Jul 29 12:33:27.372632 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:27.372596    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76bb87fbc7-6khpt" event={"ID":"32b1ff08-d437-4015-85d5-dff245655e8b","Type":"ContainerStarted","Data":"a7ad91e0c5e386558cae2fe618d22f06ef3954c467d79f99522fb52397d660ee"}
Jul 29 12:33:27.372806 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:27.372652    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76bb87fbc7-6khpt" event={"ID":"32b1ff08-d437-4015-85d5-dff245655e8b","Type":"ContainerStarted","Data":"7429fc396eb87eccd890e4d128ed64f3c070406dda88b3e81353cbaa520f9954"}
Jul 29 12:33:27.400171 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:27.400124    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-76bb87fbc7-6khpt" podStartSLOduration=1.400106951 podStartE2EDuration="1.400106951s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:33:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:33:27.398558207 +0000 UTC m=+490.092246539" watchObservedRunningTime="2026-07-29 12:33:27.400106951 +0000 UTC m=+490.093795261"
Jul 29 12:33:28.324488 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:28.324454    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l8jlz"
Jul 29 12:33:36.834340 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:36.834302    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-76bb87fbc7-6khpt"
Jul 29 12:33:36.834781 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:36.834363    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-76bb87fbc7-6khpt"
Jul 29 12:33:36.838953 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:36.838926    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-76bb87fbc7-6khpt"
Jul 29 12:33:37.432389 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:33:37.432362    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-76bb87fbc7-6khpt"
Jul 29 12:34:04.273131 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:04.273095    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-pgl9v"]
Jul 29 12:34:04.277098 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:04.277076    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-pgl9v"
Jul 29 12:34:04.282049 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:04.282029    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 29 12:34:04.282278 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:04.282257    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-jw82m\""
Jul 29 12:34:04.296931 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:04.296905    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-pgl9v"]
Jul 29 12:34:04.376114 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:04.376087    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52gf5\" (UniqueName: \"kubernetes.io/projected/8bc5f1ad-fc6b-4b48-8435-8b337957d152-kube-api-access-52gf5\") pod \"limitador-limitador-64c94fd64f-pgl9v\" (UID: \"8bc5f1ad-fc6b-4b48-8435-8b337957d152\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-pgl9v"
Jul 29 12:34:04.376267 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:04.376227    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8bc5f1ad-fc6b-4b48-8435-8b337957d152-config-file\") pod \"limitador-limitador-64c94fd64f-pgl9v\" (UID: \"8bc5f1ad-fc6b-4b48-8435-8b337957d152\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-pgl9v"
Jul 29 12:34:04.378327 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:04.378311    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-pgl9v"]
Jul 29 12:34:04.476901 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:04.476882    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52gf5\" (UniqueName: \"kubernetes.io/projected/8bc5f1ad-fc6b-4b48-8435-8b337957d152-kube-api-access-52gf5\") pod \"limitador-limitador-64c94fd64f-pgl9v\" (UID: \"8bc5f1ad-fc6b-4b48-8435-8b337957d152\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-pgl9v"
Jul 29 12:34:04.477004 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:04.476943    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8bc5f1ad-fc6b-4b48-8435-8b337957d152-config-file\") pod \"limitador-limitador-64c94fd64f-pgl9v\" (UID: \"8bc5f1ad-fc6b-4b48-8435-8b337957d152\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-pgl9v"
Jul 29 12:34:04.477515 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:04.477498    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8bc5f1ad-fc6b-4b48-8435-8b337957d152-config-file\") pod \"limitador-limitador-64c94fd64f-pgl9v\" (UID: \"8bc5f1ad-fc6b-4b48-8435-8b337957d152\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-pgl9v"
Jul 29 12:34:04.493425 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:04.493381    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-52gf5\" (UniqueName: \"kubernetes.io/projected/8bc5f1ad-fc6b-4b48-8435-8b337957d152-kube-api-access-52gf5\") pod \"limitador-limitador-64c94fd64f-pgl9v\" (UID: \"8bc5f1ad-fc6b-4b48-8435-8b337957d152\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-pgl9v"
Jul 29 12:34:04.588590 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:04.588571    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-pgl9v"
Jul 29 12:34:04.725363 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:04.725338    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-pgl9v"]
Jul 29 12:34:04.726055 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:34:04.726031    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bc5f1ad_fc6b_4b48_8435_8b337957d152.slice/crio-28f0a13cd192a09e3377db588d4350fa575fb4bda2672fd71810e66439ad3cc6 WatchSource:0}: Error finding container 28f0a13cd192a09e3377db588d4350fa575fb4bda2672fd71810e66439ad3cc6: Status 404 returned error can't find the container with id 28f0a13cd192a09e3377db588d4350fa575fb4bda2672fd71810e66439ad3cc6
Jul 29 12:34:05.589212 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:05.589132    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-pgl9v" event={"ID":"8bc5f1ad-fc6b-4b48-8435-8b337957d152","Type":"ContainerStarted","Data":"28f0a13cd192a09e3377db588d4350fa575fb4bda2672fd71810e66439ad3cc6"}
Jul 29 12:34:07.601297 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:07.601265    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-pgl9v" event={"ID":"8bc5f1ad-fc6b-4b48-8435-8b337957d152","Type":"ContainerStarted","Data":"a6a25e5bd7d8dfcfb626de5789ff6b7799a7fb1de9c72a8e854f414164ae4883"}
Jul 29 12:34:07.601623 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:07.601411    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-pgl9v"
Jul 29 12:34:07.637929 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:07.637857    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-pgl9v" podStartSLOduration=0.990775204 podStartE2EDuration="3.637841813s" totalImagesPullingTime="2.647066609s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:34:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:34:07.637507259 +0000 UTC m=+530.331195590" watchObservedRunningTime="2026-07-29 12:34:07.637841813 +0000 UTC m=+530.331530184"
Jul 29 12:34:18.607803 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:18.607721    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-pgl9v"
Jul 29 12:34:21.072474 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:21.072428    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-pgl9v"]
Jul 29 12:34:21.072946 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:21.072643    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-pgl9v" podUID="8bc5f1ad-fc6b-4b48-8435-8b337957d152" containerName="limitador" containerID="cri-o://a6a25e5bd7d8dfcfb626de5789ff6b7799a7fb1de9c72a8e854f414164ae4883" gracePeriod=30
Jul 29 12:34:21.619813 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:21.619787    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-pgl9v"
Jul 29 12:34:21.688430 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:21.688367    2567 generic.go:356] "Generic (PLEG): container finished" podID="8bc5f1ad-fc6b-4b48-8435-8b337957d152" containerID="a6a25e5bd7d8dfcfb626de5789ff6b7799a7fb1de9c72a8e854f414164ae4883" exitCode=0
Jul 29 12:34:21.688430 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:21.688416    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-pgl9v"
Jul 29 12:34:21.688548 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:21.688436    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-pgl9v" event={"ID":"8bc5f1ad-fc6b-4b48-8435-8b337957d152","Type":"ContainerDied","Data":"a6a25e5bd7d8dfcfb626de5789ff6b7799a7fb1de9c72a8e854f414164ae4883"}
Jul 29 12:34:21.688548 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:21.688498    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-pgl9v" event={"ID":"8bc5f1ad-fc6b-4b48-8435-8b337957d152","Type":"ContainerDied","Data":"28f0a13cd192a09e3377db588d4350fa575fb4bda2672fd71810e66439ad3cc6"}
Jul 29 12:34:21.688548 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:21.688521    2567 scope.go:117] "RemoveContainer" containerID="a6a25e5bd7d8dfcfb626de5789ff6b7799a7fb1de9c72a8e854f414164ae4883"
Jul 29 12:34:21.697240 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:21.697224    2567 scope.go:117] "RemoveContainer" containerID="a6a25e5bd7d8dfcfb626de5789ff6b7799a7fb1de9c72a8e854f414164ae4883"
Jul 29 12:34:21.697505 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:34:21.697488    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6a25e5bd7d8dfcfb626de5789ff6b7799a7fb1de9c72a8e854f414164ae4883\": container with ID starting with a6a25e5bd7d8dfcfb626de5789ff6b7799a7fb1de9c72a8e854f414164ae4883 not found: ID does not exist" containerID="a6a25e5bd7d8dfcfb626de5789ff6b7799a7fb1de9c72a8e854f414164ae4883"
Jul 29 12:34:21.697558 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:21.697514    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6a25e5bd7d8dfcfb626de5789ff6b7799a7fb1de9c72a8e854f414164ae4883"} err="failed to get container status \"a6a25e5bd7d8dfcfb626de5789ff6b7799a7fb1de9c72a8e854f414164ae4883\": rpc error: code = NotFound desc = could not find container \"a6a25e5bd7d8dfcfb626de5789ff6b7799a7fb1de9c72a8e854f414164ae4883\": container with ID starting with a6a25e5bd7d8dfcfb626de5789ff6b7799a7fb1de9c72a8e854f414164ae4883 not found: ID does not exist"
Jul 29 12:34:21.704809 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:21.704789    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-52gf5\" (UniqueName: \"kubernetes.io/projected/8bc5f1ad-fc6b-4b48-8435-8b337957d152-kube-api-access-52gf5\") pod \"8bc5f1ad-fc6b-4b48-8435-8b337957d152\" (UID: \"8bc5f1ad-fc6b-4b48-8435-8b337957d152\") "
Jul 29 12:34:21.704899 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:21.704817    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8bc5f1ad-fc6b-4b48-8435-8b337957d152-config-file\") pod \"8bc5f1ad-fc6b-4b48-8435-8b337957d152\" (UID: \"8bc5f1ad-fc6b-4b48-8435-8b337957d152\") "
Jul 29 12:34:21.705200 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:21.705181    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8bc5f1ad-fc6b-4b48-8435-8b337957d152-config-file" (OuterVolumeSpecName: "config-file") pod "8bc5f1ad-fc6b-4b48-8435-8b337957d152" (UID: "8bc5f1ad-fc6b-4b48-8435-8b337957d152"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 12:34:21.706606 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:21.706586    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8bc5f1ad-fc6b-4b48-8435-8b337957d152-kube-api-access-52gf5" (OuterVolumeSpecName: "kube-api-access-52gf5") pod "8bc5f1ad-fc6b-4b48-8435-8b337957d152" (UID: "8bc5f1ad-fc6b-4b48-8435-8b337957d152"). InnerVolumeSpecName "kube-api-access-52gf5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 12:34:21.806356 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:21.806332    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-52gf5\" (UniqueName: \"kubernetes.io/projected/8bc5f1ad-fc6b-4b48-8435-8b337957d152-kube-api-access-52gf5\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:34:21.806356 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:21.806353    2567 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8bc5f1ad-fc6b-4b48-8435-8b337957d152-config-file\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:34:22.005190 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:22.005130    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-pgl9v"]
Jul 29 12:34:22.009756 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:22.009731    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-pgl9v"]
Jul 29 12:34:23.921742 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:23.921703    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8bc5f1ad-fc6b-4b48-8435-8b337957d152" path="/var/lib/kubelet/pods/8bc5f1ad-fc6b-4b48-8435-8b337957d152/volumes"
Jul 29 12:34:24.629563 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:24.629532    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-ng554"]
Jul 29 12:34:24.629929 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:24.629915    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bc5f1ad-fc6b-4b48-8435-8b337957d152" containerName="limitador"
Jul 29 12:34:24.629977 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:24.629931    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bc5f1ad-fc6b-4b48-8435-8b337957d152" containerName="limitador"
Jul 29 12:34:24.630011 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:24.630003    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bc5f1ad-fc6b-4b48-8435-8b337957d152" containerName="limitador"
Jul 29 12:34:24.634444 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:24.634427    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ng554"
Jul 29 12:34:24.640037 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:24.639997    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 29 12:34:24.641056 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:24.641020    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 29 12:34:24.641178 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:24.641024    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 29 12:34:24.641178 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:24.641023    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-478wv\""
Jul 29 12:34:24.643469 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:24.643449    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-ng554"]
Jul 29 12:34:24.729263 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:24.729241    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/4e94fe7d-6d68-436f-addf-5cb8c2f893c7-data\") pod \"postgres-69fb4594cf-ng554\" (UID: \"4e94fe7d-6d68-436f-addf-5cb8c2f893c7\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-ng554"
Jul 29 12:34:24.729353 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:24.729269    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5z4w\" (UniqueName: \"kubernetes.io/projected/4e94fe7d-6d68-436f-addf-5cb8c2f893c7-kube-api-access-m5z4w\") pod \"postgres-69fb4594cf-ng554\" (UID: \"4e94fe7d-6d68-436f-addf-5cb8c2f893c7\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-ng554"
Jul 29 12:34:24.829770 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:24.829744    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/4e94fe7d-6d68-436f-addf-5cb8c2f893c7-data\") pod \"postgres-69fb4594cf-ng554\" (UID: \"4e94fe7d-6d68-436f-addf-5cb8c2f893c7\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-ng554"
Jul 29 12:34:24.829870 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:24.829774    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5z4w\" (UniqueName: \"kubernetes.io/projected/4e94fe7d-6d68-436f-addf-5cb8c2f893c7-kube-api-access-m5z4w\") pod \"postgres-69fb4594cf-ng554\" (UID: \"4e94fe7d-6d68-436f-addf-5cb8c2f893c7\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-ng554"
Jul 29 12:34:24.830154 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:24.830136    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/4e94fe7d-6d68-436f-addf-5cb8c2f893c7-data\") pod \"postgres-69fb4594cf-ng554\" (UID: \"4e94fe7d-6d68-436f-addf-5cb8c2f893c7\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-ng554"
Jul 29 12:34:24.841055 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:24.841029    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5z4w\" (UniqueName: \"kubernetes.io/projected/4e94fe7d-6d68-436f-addf-5cb8c2f893c7-kube-api-access-m5z4w\") pod \"postgres-69fb4594cf-ng554\" (UID: \"4e94fe7d-6d68-436f-addf-5cb8c2f893c7\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-ng554"
Jul 29 12:34:24.945221 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:24.945160    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ng554"
Jul 29 12:34:25.077071 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:25.077051    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-ng554"]
Jul 29 12:34:25.078737 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:34:25.078712    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e94fe7d_6d68_436f_addf_5cb8c2f893c7.slice/crio-0a2c8ba275403a3a733582569173da07e1b679b1c01ed9766e087d0699f6947d WatchSource:0}: Error finding container 0a2c8ba275403a3a733582569173da07e1b679b1c01ed9766e087d0699f6947d: Status 404 returned error can't find the container with id 0a2c8ba275403a3a733582569173da07e1b679b1c01ed9766e087d0699f6947d
Jul 29 12:34:25.715352 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:25.715322    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ng554" event={"ID":"4e94fe7d-6d68-436f-addf-5cb8c2f893c7","Type":"ContainerStarted","Data":"0a2c8ba275403a3a733582569173da07e1b679b1c01ed9766e087d0699f6947d"}
Jul 29 12:34:30.752472 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:30.752436    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ng554" event={"ID":"4e94fe7d-6d68-436f-addf-5cb8c2f893c7","Type":"ContainerStarted","Data":"fa1b18bbaeca7b890629faf38d77375ff0887eac0be92445db2ec6d145db36c0"}
Jul 29 12:34:30.752831 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:30.752536    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ng554"
Jul 29 12:34:30.772804 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:30.772763    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ng554" podStartSLOduration=1.7736777 podStartE2EDuration="6.77275048s" totalImagesPullingTime="4.99907278s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:34:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:34:30.772502048 +0000 UTC m=+553.466190393" watchObservedRunningTime="2026-07-29 12:34:30.77275048 +0000 UTC m=+553.466438790"
Jul 29 12:34:36.787447 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:36.787395    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ng554"
Jul 29 12:34:43.824948 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:43.824914    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5f686dc7cf-8gdrd"]
Jul 29 12:34:43.964648 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:43.964618    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f686dc7cf-8gdrd"
Jul 29 12:34:43.968755 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:43.968727    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 29 12:34:43.968755 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:43.968750    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 29 12:34:43.968989 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:43.968798    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-kjvkj\""
Jul 29 12:34:43.971712 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:43.971691    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5f686dc7cf-8gdrd"]
Jul 29 12:34:44.016394 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:44.016369    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-65d577d9d5-w4w8z"]
Jul 29 12:34:44.035600 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:44.035579    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65d577d9d5-w4w8z"
Jul 29 12:34:44.035781 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:44.035763    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-65d577d9d5-w4w8z"]
Jul 29 12:34:44.086484 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:44.086436    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9e4c74f9-5499-4d95-8eca-0fa2296d23ce-webhook-cert\") pod \"maas-controller-5f686dc7cf-8gdrd\" (UID: \"9e4c74f9-5499-4d95-8eca-0fa2296d23ce\") " pod="opendatahub/maas-controller-5f686dc7cf-8gdrd"
Jul 29 12:34:44.086484 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:44.086482    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjlpq\" (UniqueName: \"kubernetes.io/projected/9e4c74f9-5499-4d95-8eca-0fa2296d23ce-kube-api-access-rjlpq\") pod \"maas-controller-5f686dc7cf-8gdrd\" (UID: \"9e4c74f9-5499-4d95-8eca-0fa2296d23ce\") " pod="opendatahub/maas-controller-5f686dc7cf-8gdrd"
Jul 29 12:34:44.186820 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:44.186801    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9e4c74f9-5499-4d95-8eca-0fa2296d23ce-webhook-cert\") pod \"maas-controller-5f686dc7cf-8gdrd\" (UID: \"9e4c74f9-5499-4d95-8eca-0fa2296d23ce\") " pod="opendatahub/maas-controller-5f686dc7cf-8gdrd"
Jul 29 12:34:44.186936 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:44.186832    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e5179166-12eb-429c-9fda-e8ab32f31f52-webhook-cert\") pod \"maas-controller-65d577d9d5-w4w8z\" (UID: \"e5179166-12eb-429c-9fda-e8ab32f31f52\") " pod="opendatahub/maas-controller-65d577d9d5-w4w8z"
Jul 29 12:34:44.186936 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:44.186858    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wgzm7\" (UniqueName: \"kubernetes.io/projected/e5179166-12eb-429c-9fda-e8ab32f31f52-kube-api-access-wgzm7\") pod \"maas-controller-65d577d9d5-w4w8z\" (UID: \"e5179166-12eb-429c-9fda-e8ab32f31f52\") " pod="opendatahub/maas-controller-65d577d9d5-w4w8z"
Jul 29 12:34:44.186936 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:44.186883    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjlpq\" (UniqueName: \"kubernetes.io/projected/9e4c74f9-5499-4d95-8eca-0fa2296d23ce-kube-api-access-rjlpq\") pod \"maas-controller-5f686dc7cf-8gdrd\" (UID: \"9e4c74f9-5499-4d95-8eca-0fa2296d23ce\") " pod="opendatahub/maas-controller-5f686dc7cf-8gdrd"
Jul 29 12:34:44.187092 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:34:44.186957    2567 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Jul 29 12:34:44.187092 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:34:44.187020    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e4c74f9-5499-4d95-8eca-0fa2296d23ce-webhook-cert podName:9e4c74f9-5499-4d95-8eca-0fa2296d23ce nodeName:}" failed. No retries permitted until 2026-07-29 12:34:44.687000876 +0000 UTC m=+567.380689167 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/9e4c74f9-5499-4d95-8eca-0fa2296d23ce-webhook-cert") pod "maas-controller-5f686dc7cf-8gdrd" (UID: "9e4c74f9-5499-4d95-8eca-0fa2296d23ce") : secret "maas-controller-webhook-cert" not found
Jul 29 12:34:44.198120 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:44.198093    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjlpq\" (UniqueName: \"kubernetes.io/projected/9e4c74f9-5499-4d95-8eca-0fa2296d23ce-kube-api-access-rjlpq\") pod \"maas-controller-5f686dc7cf-8gdrd\" (UID: \"9e4c74f9-5499-4d95-8eca-0fa2296d23ce\") " pod="opendatahub/maas-controller-5f686dc7cf-8gdrd"
Jul 29 12:34:44.287919 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:44.287897    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e5179166-12eb-429c-9fda-e8ab32f31f52-webhook-cert\") pod \"maas-controller-65d577d9d5-w4w8z\" (UID: \"e5179166-12eb-429c-9fda-e8ab32f31f52\") " pod="opendatahub/maas-controller-65d577d9d5-w4w8z"
Jul 29 12:34:44.288011 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:44.287929    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wgzm7\" (UniqueName: \"kubernetes.io/projected/e5179166-12eb-429c-9fda-e8ab32f31f52-kube-api-access-wgzm7\") pod \"maas-controller-65d577d9d5-w4w8z\" (UID: \"e5179166-12eb-429c-9fda-e8ab32f31f52\") " pod="opendatahub/maas-controller-65d577d9d5-w4w8z"
Jul 29 12:34:44.290101 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:44.290078    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e5179166-12eb-429c-9fda-e8ab32f31f52-webhook-cert\") pod \"maas-controller-65d577d9d5-w4w8z\" (UID: \"e5179166-12eb-429c-9fda-e8ab32f31f52\") " pod="opendatahub/maas-controller-65d577d9d5-w4w8z"
Jul 29 12:34:44.296877 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:44.296857    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wgzm7\" (UniqueName: \"kubernetes.io/projected/e5179166-12eb-429c-9fda-e8ab32f31f52-kube-api-access-wgzm7\") pod \"maas-controller-65d577d9d5-w4w8z\" (UID: \"e5179166-12eb-429c-9fda-e8ab32f31f52\") " pod="opendatahub/maas-controller-65d577d9d5-w4w8z"
Jul 29 12:34:44.345767 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:44.345713    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65d577d9d5-w4w8z"
Jul 29 12:34:44.482670 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:44.482565    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-65d577d9d5-w4w8z"]
Jul 29 12:34:44.485115 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:34:44.485088    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5179166_12eb_429c_9fda_e8ab32f31f52.slice/crio-137f2cf1017082f99f12e078b287e6f0d7b42d063982f7a519f577eee04e487c WatchSource:0}: Error finding container 137f2cf1017082f99f12e078b287e6f0d7b42d063982f7a519f577eee04e487c: Status 404 returned error can't find the container with id 137f2cf1017082f99f12e078b287e6f0d7b42d063982f7a519f577eee04e487c
Jul 29 12:34:44.691431 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:44.691357    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9e4c74f9-5499-4d95-8eca-0fa2296d23ce-webhook-cert\") pod \"maas-controller-5f686dc7cf-8gdrd\" (UID: \"9e4c74f9-5499-4d95-8eca-0fa2296d23ce\") " pod="opendatahub/maas-controller-5f686dc7cf-8gdrd"
Jul 29 12:34:44.693610 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:44.693589    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9e4c74f9-5499-4d95-8eca-0fa2296d23ce-webhook-cert\") pod \"maas-controller-5f686dc7cf-8gdrd\" (UID: \"9e4c74f9-5499-4d95-8eca-0fa2296d23ce\") " pod="opendatahub/maas-controller-5f686dc7cf-8gdrd"
Jul 29 12:34:44.842356 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:44.842327    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65d577d9d5-w4w8z" event={"ID":"e5179166-12eb-429c-9fda-e8ab32f31f52","Type":"ContainerStarted","Data":"137f2cf1017082f99f12e078b287e6f0d7b42d063982f7a519f577eee04e487c"}
Jul 29 12:34:44.875474 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:44.875448    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f686dc7cf-8gdrd"
Jul 29 12:34:45.003775 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:45.003754    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5f686dc7cf-8gdrd"]
Jul 29 12:34:45.004913 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:34:45.004888    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e4c74f9_5499_4d95_8eca_0fa2296d23ce.slice/crio-0c455b51ed1302fa333b7647c32023c9e5a7e11562e212d7db0b3494b26f98b8 WatchSource:0}: Error finding container 0c455b51ed1302fa333b7647c32023c9e5a7e11562e212d7db0b3494b26f98b8: Status 404 returned error can't find the container with id 0c455b51ed1302fa333b7647c32023c9e5a7e11562e212d7db0b3494b26f98b8
Jul 29 12:34:45.854609 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:45.854570    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f686dc7cf-8gdrd" event={"ID":"9e4c74f9-5499-4d95-8eca-0fa2296d23ce","Type":"ContainerStarted","Data":"0c455b51ed1302fa333b7647c32023c9e5a7e11562e212d7db0b3494b26f98b8"}
Jul 29 12:34:47.867970 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:47.867933    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f686dc7cf-8gdrd" event={"ID":"9e4c74f9-5499-4d95-8eca-0fa2296d23ce","Type":"ContainerStarted","Data":"e1b63ea537db0f457baf3c98fc75caf74b381b75375f4e80046e1693e01d0973"}
Jul 29 12:34:47.868421 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:47.867991    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5f686dc7cf-8gdrd"
Jul 29 12:34:47.869441 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:47.869381    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65d577d9d5-w4w8z" event={"ID":"e5179166-12eb-429c-9fda-e8ab32f31f52","Type":"ContainerStarted","Data":"1a2862659dd92db89e13702cdb2a3813190df194de6f682aa43cd6401bd02945"}
Jul 29 12:34:47.869563 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:47.869454    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-65d577d9d5-w4w8z"
Jul 29 12:34:47.888025 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:47.887983    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5f686dc7cf-8gdrd" podStartSLOduration=2.750390939 podStartE2EDuration="4.887971121s" totalImagesPullingTime="2.137580182s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:34:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:34:47.886752625 +0000 UTC m=+570.580440949" watchObservedRunningTime="2026-07-29 12:34:47.887971121 +0000 UTC m=+570.581659432"
Jul 29 12:34:47.911566 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:47.911525    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-65d577d9d5-w4w8z" podStartSLOduration=2.2607613779999998 podStartE2EDuration="4.911512009s" totalImagesPullingTime="2.650750631s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:34:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:34:47.909371545 +0000 UTC m=+570.603059858" watchObservedRunningTime="2026-07-29 12:34:47.911512009 +0000 UTC m=+570.605200320"
Jul 29 12:34:58.876424 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:58.876379    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-65d577d9d5-w4w8z"
Jul 29 12:34:58.876866 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:58.876565    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5f686dc7cf-8gdrd"
Jul 29 12:34:58.949569 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:58.949532    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5f686dc7cf-8gdrd"]
Jul 29 12:34:58.949747 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:58.949713    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5f686dc7cf-8gdrd" podUID="9e4c74f9-5499-4d95-8eca-0fa2296d23ce" containerName="manager" containerID="cri-o://e1b63ea537db0f457baf3c98fc75caf74b381b75375f4e80046e1693e01d0973" gracePeriod=30
Jul 29 12:34:59.187588 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:59.187564    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f686dc7cf-8gdrd"
Jul 29 12:34:59.317537 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:59.317511    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9e4c74f9-5499-4d95-8eca-0fa2296d23ce-webhook-cert\") pod \"9e4c74f9-5499-4d95-8eca-0fa2296d23ce\" (UID: \"9e4c74f9-5499-4d95-8eca-0fa2296d23ce\") "
Jul 29 12:34:59.317675 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:59.317566    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rjlpq\" (UniqueName: \"kubernetes.io/projected/9e4c74f9-5499-4d95-8eca-0fa2296d23ce-kube-api-access-rjlpq\") pod \"9e4c74f9-5499-4d95-8eca-0fa2296d23ce\" (UID: \"9e4c74f9-5499-4d95-8eca-0fa2296d23ce\") "
Jul 29 12:34:59.319673 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:59.319646    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e4c74f9-5499-4d95-8eca-0fa2296d23ce-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "9e4c74f9-5499-4d95-8eca-0fa2296d23ce" (UID: "9e4c74f9-5499-4d95-8eca-0fa2296d23ce"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 12:34:59.319843 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:59.319816    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e4c74f9-5499-4d95-8eca-0fa2296d23ce-kube-api-access-rjlpq" (OuterVolumeSpecName: "kube-api-access-rjlpq") pod "9e4c74f9-5499-4d95-8eca-0fa2296d23ce" (UID: "9e4c74f9-5499-4d95-8eca-0fa2296d23ce"). InnerVolumeSpecName "kube-api-access-rjlpq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 12:34:59.418692 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:59.418635    2567 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9e4c74f9-5499-4d95-8eca-0fa2296d23ce-webhook-cert\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:34:59.418692 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:59.418660    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rjlpq\" (UniqueName: \"kubernetes.io/projected/9e4c74f9-5499-4d95-8eca-0fa2296d23ce-kube-api-access-rjlpq\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:34:59.944319 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:59.944288    2567 generic.go:356] "Generic (PLEG): container finished" podID="9e4c74f9-5499-4d95-8eca-0fa2296d23ce" containerID="e1b63ea537db0f457baf3c98fc75caf74b381b75375f4e80046e1693e01d0973" exitCode=0
Jul 29 12:34:59.944698 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:59.944363    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f686dc7cf-8gdrd"
Jul 29 12:34:59.944698 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:59.944376    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f686dc7cf-8gdrd" event={"ID":"9e4c74f9-5499-4d95-8eca-0fa2296d23ce","Type":"ContainerDied","Data":"e1b63ea537db0f457baf3c98fc75caf74b381b75375f4e80046e1693e01d0973"}
Jul 29 12:34:59.944698 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:59.944445    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f686dc7cf-8gdrd" event={"ID":"9e4c74f9-5499-4d95-8eca-0fa2296d23ce","Type":"ContainerDied","Data":"0c455b51ed1302fa333b7647c32023c9e5a7e11562e212d7db0b3494b26f98b8"}
Jul 29 12:34:59.944698 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:59.944469    2567 scope.go:117] "RemoveContainer" containerID="e1b63ea537db0f457baf3c98fc75caf74b381b75375f4e80046e1693e01d0973"
Jul 29 12:34:59.954095 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:59.953932    2567 scope.go:117] "RemoveContainer" containerID="e1b63ea537db0f457baf3c98fc75caf74b381b75375f4e80046e1693e01d0973"
Jul 29 12:34:59.954246 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:34:59.954226    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1b63ea537db0f457baf3c98fc75caf74b381b75375f4e80046e1693e01d0973\": container with ID starting with e1b63ea537db0f457baf3c98fc75caf74b381b75375f4e80046e1693e01d0973 not found: ID does not exist" containerID="e1b63ea537db0f457baf3c98fc75caf74b381b75375f4e80046e1693e01d0973"
Jul 29 12:34:59.954310 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:59.954254    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1b63ea537db0f457baf3c98fc75caf74b381b75375f4e80046e1693e01d0973"} err="failed to get container status \"e1b63ea537db0f457baf3c98fc75caf74b381b75375f4e80046e1693e01d0973\": rpc error: code = NotFound desc = could not find container \"e1b63ea537db0f457baf3c98fc75caf74b381b75375f4e80046e1693e01d0973\": container with ID starting with e1b63ea537db0f457baf3c98fc75caf74b381b75375f4e80046e1693e01d0973 not found: ID does not exist"
Jul 29 12:34:59.963885 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:59.963861    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5f686dc7cf-8gdrd"]
Jul 29 12:34:59.967801 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:34:59.967778    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5f686dc7cf-8gdrd"]
Jul 29 12:35:01.921300 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:01.921257    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e4c74f9-5499-4d95-8eca-0fa2296d23ce" path="/var/lib/kubelet/pods/9e4c74f9-5499-4d95-8eca-0fa2296d23ce/volumes"
Jul 29 12:35:17.837150 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:17.837115    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kj82f_a6d5f3a2-26cc-40a7-9841-2fbfc247661a/ovn-acl-logging/0.log"
Jul 29 12:35:17.838884 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:17.838852    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kj82f_a6d5f3a2-26cc-40a7-9841-2fbfc247661a/ovn-acl-logging/0.log"
Jul 29 12:35:20.014032 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:20.013989    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-847fd45f6b-njnmj"]
Jul 29 12:35:20.016392 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:20.014437    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e4c74f9-5499-4d95-8eca-0fa2296d23ce" containerName="manager"
Jul 29 12:35:20.016392 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:20.014451    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e4c74f9-5499-4d95-8eca-0fa2296d23ce" containerName="manager"
Jul 29 12:35:20.016392 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:20.014516    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e4c74f9-5499-4d95-8eca-0fa2296d23ce" containerName="manager"
Jul 29 12:35:20.017648 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:20.017629    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-847fd45f6b-njnmj"
Jul 29 12:35:20.025918 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:20.025892    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 29 12:35:20.026024 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:20.025971    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-6znmc\""
Jul 29 12:35:20.036125 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:20.036103    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-847fd45f6b-njnmj"]
Jul 29 12:35:20.066144 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:20.066122    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e789990-05eb-467f-8a5d-f543fd824811-maas-api-tls\") pod \"maas-api-847fd45f6b-njnmj\" (UID: \"4e789990-05eb-467f-8a5d-f543fd824811\") " pod="odh-ai-gateway-infra/maas-api-847fd45f6b-njnmj"
Jul 29 12:35:20.066233 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:20.066168    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqxmh\" (UniqueName: \"kubernetes.io/projected/4e789990-05eb-467f-8a5d-f543fd824811-kube-api-access-lqxmh\") pod \"maas-api-847fd45f6b-njnmj\" (UID: \"4e789990-05eb-467f-8a5d-f543fd824811\") " pod="odh-ai-gateway-infra/maas-api-847fd45f6b-njnmj"
Jul 29 12:35:20.066274 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:20.066251    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e789990-05eb-467f-8a5d-f543fd824811-openshift-service-ca\") pod \"maas-api-847fd45f6b-njnmj\" (UID: \"4e789990-05eb-467f-8a5d-f543fd824811\") " pod="odh-ai-gateway-infra/maas-api-847fd45f6b-njnmj"
Jul 29 12:35:20.166791 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:20.166769    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e789990-05eb-467f-8a5d-f543fd824811-openshift-service-ca\") pod \"maas-api-847fd45f6b-njnmj\" (UID: \"4e789990-05eb-467f-8a5d-f543fd824811\") " pod="odh-ai-gateway-infra/maas-api-847fd45f6b-njnmj"
Jul 29 12:35:20.166889 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:20.166805    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e789990-05eb-467f-8a5d-f543fd824811-maas-api-tls\") pod \"maas-api-847fd45f6b-njnmj\" (UID: \"4e789990-05eb-467f-8a5d-f543fd824811\") " pod="odh-ai-gateway-infra/maas-api-847fd45f6b-njnmj"
Jul 29 12:35:20.166889 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:20.166846    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqxmh\" (UniqueName: \"kubernetes.io/projected/4e789990-05eb-467f-8a5d-f543fd824811-kube-api-access-lqxmh\") pod \"maas-api-847fd45f6b-njnmj\" (UID: \"4e789990-05eb-467f-8a5d-f543fd824811\") " pod="odh-ai-gateway-infra/maas-api-847fd45f6b-njnmj"
Jul 29 12:35:20.167423 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:20.167380    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e789990-05eb-467f-8a5d-f543fd824811-openshift-service-ca\") pod \"maas-api-847fd45f6b-njnmj\" (UID: \"4e789990-05eb-467f-8a5d-f543fd824811\") " pod="odh-ai-gateway-infra/maas-api-847fd45f6b-njnmj"
Jul 29 12:35:20.169106 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:20.169086    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e789990-05eb-467f-8a5d-f543fd824811-maas-api-tls\") pod \"maas-api-847fd45f6b-njnmj\" (UID: \"4e789990-05eb-467f-8a5d-f543fd824811\") " pod="odh-ai-gateway-infra/maas-api-847fd45f6b-njnmj"
Jul 29 12:35:20.179316 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:20.179294    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqxmh\" (UniqueName: \"kubernetes.io/projected/4e789990-05eb-467f-8a5d-f543fd824811-kube-api-access-lqxmh\") pod \"maas-api-847fd45f6b-njnmj\" (UID: \"4e789990-05eb-467f-8a5d-f543fd824811\") " pod="odh-ai-gateway-infra/maas-api-847fd45f6b-njnmj"
Jul 29 12:35:20.327653 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:20.327628    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-847fd45f6b-njnmj"
Jul 29 12:35:20.472972 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:35:20.472946    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e789990_05eb_467f_8a5d_f543fd824811.slice/crio-0affd1cec4bdfc3e75486a0c07a73a2232ec5512b318292e9364d7f4fb2ef75d WatchSource:0}: Error finding container 0affd1cec4bdfc3e75486a0c07a73a2232ec5512b318292e9364d7f4fb2ef75d: Status 404 returned error can't find the container with id 0affd1cec4bdfc3e75486a0c07a73a2232ec5512b318292e9364d7f4fb2ef75d
Jul 29 12:35:20.473164 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:20.473142    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-847fd45f6b-njnmj"]
Jul 29 12:35:20.474430 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:20.474382    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 12:35:20.939342 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:20.939313    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-k7kgk"]
Jul 29 12:35:20.947862 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:20.947837    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-k7kgk"
Jul 29 12:35:20.950467 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:20.950445    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 29 12:35:20.950608 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:20.950513    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-kwmrt\""
Jul 29 12:35:20.956021 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:20.955980    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-k7kgk"]
Jul 29 12:35:20.973714 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:20.973687    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-456mt\" (UniqueName: \"kubernetes.io/projected/c261451c-033e-4484-ae79-c1a2ebb7d81b-kube-api-access-456mt\") pod \"payload-processing-5cc6cd444b-k7kgk\" (UID: \"c261451c-033e-4484-ae79-c1a2ebb7d81b\") " pod="openshift-ingress/payload-processing-5cc6cd444b-k7kgk"
Jul 29 12:35:20.973827 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:20.973758    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c261451c-033e-4484-ae79-c1a2ebb7d81b-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-k7kgk\" (UID: \"c261451c-033e-4484-ae79-c1a2ebb7d81b\") " pod="openshift-ingress/payload-processing-5cc6cd444b-k7kgk"
Jul 29 12:35:21.060710 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:21.060671    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-847fd45f6b-njnmj" event={"ID":"4e789990-05eb-467f-8a5d-f543fd824811","Type":"ContainerStarted","Data":"0affd1cec4bdfc3e75486a0c07a73a2232ec5512b318292e9364d7f4fb2ef75d"}
Jul 29 12:35:21.075183 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:21.075155    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c261451c-033e-4484-ae79-c1a2ebb7d81b-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-k7kgk\" (UID: \"c261451c-033e-4484-ae79-c1a2ebb7d81b\") " pod="openshift-ingress/payload-processing-5cc6cd444b-k7kgk"
Jul 29 12:35:21.075308 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:21.075225    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-456mt\" (UniqueName: \"kubernetes.io/projected/c261451c-033e-4484-ae79-c1a2ebb7d81b-kube-api-access-456mt\") pod \"payload-processing-5cc6cd444b-k7kgk\" (UID: \"c261451c-033e-4484-ae79-c1a2ebb7d81b\") " pod="openshift-ingress/payload-processing-5cc6cd444b-k7kgk"
Jul 29 12:35:21.076043 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:21.076022    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c261451c-033e-4484-ae79-c1a2ebb7d81b-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-k7kgk\" (UID: \"c261451c-033e-4484-ae79-c1a2ebb7d81b\") " pod="openshift-ingress/payload-processing-5cc6cd444b-k7kgk"
Jul 29 12:35:21.093150 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:21.093127    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-456mt\" (UniqueName: \"kubernetes.io/projected/c261451c-033e-4484-ae79-c1a2ebb7d81b-kube-api-access-456mt\") pod \"payload-processing-5cc6cd444b-k7kgk\" (UID: \"c261451c-033e-4484-ae79-c1a2ebb7d81b\") " pod="openshift-ingress/payload-processing-5cc6cd444b-k7kgk"
Jul 29 12:35:21.260162 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:21.260106    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-k7kgk"
Jul 29 12:35:21.411384 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:21.411355    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-k7kgk"]
Jul 29 12:35:21.413434 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:35:21.413382    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc261451c_033e_4484_ae79_c1a2ebb7d81b.slice/crio-0c3424da8669402c97615deb4e5deee56178e785ff76210065eddb818e79ea6d WatchSource:0}: Error finding container 0c3424da8669402c97615deb4e5deee56178e785ff76210065eddb818e79ea6d: Status 404 returned error can't find the container with id 0c3424da8669402c97615deb4e5deee56178e785ff76210065eddb818e79ea6d
Jul 29 12:35:22.066862 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:22.066829    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-k7kgk" event={"ID":"c261451c-033e-4484-ae79-c1a2ebb7d81b","Type":"ContainerStarted","Data":"0c3424da8669402c97615deb4e5deee56178e785ff76210065eddb818e79ea6d"}
Jul 29 12:35:23.074961 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:23.074927    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-847fd45f6b-njnmj" event={"ID":"4e789990-05eb-467f-8a5d-f543fd824811","Type":"ContainerStarted","Data":"ad247745454d2de9f095b28fd2c184ad63f4548532603dbfdd86fc95c59dd7eb"}
Jul 29 12:35:23.075352 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:23.075010    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-847fd45f6b-njnmj"
Jul 29 12:35:23.095254 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:23.095207    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-847fd45f6b-njnmj" podStartSLOduration=2.4805225650000002 podStartE2EDuration="4.095194063s" totalImagesPullingTime="1.614671498s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:35:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:35:23.093497405 +0000 UTC m=+605.787185740" watchObservedRunningTime="2026-07-29 12:35:23.095194063 +0000 UTC m=+605.788882373"
Jul 29 12:35:24.081931 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:24.081893    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-k7kgk" event={"ID":"c261451c-033e-4484-ae79-c1a2ebb7d81b","Type":"ContainerStarted","Data":"b55333f68d643affebe66b84d2a19c910ea134a493de4a064a18833d5c5ccfe2"}
Jul 29 12:35:24.082368 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:24.082038    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5cc6cd444b-k7kgk"
Jul 29 12:35:24.106000 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:24.105953    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5cc6cd444b-k7kgk" podStartSLOduration=2.421303439 podStartE2EDuration="4.105936624s" totalImagesPullingTime="1.684633185s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:35:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:35:24.10431478 +0000 UTC m=+606.798003090" watchObservedRunningTime="2026-07-29 12:35:24.105936624 +0000 UTC m=+606.799624935"
Jul 29 12:35:29.088224 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:29.088187    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-847fd45f6b-njnmj"
Jul 29 12:35:35.089568 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:35.089533    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5cc6cd444b-k7kgk"
Jul 29 12:35:38.055389 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:38.055358    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-c2qbk"]
Jul 29 12:35:38.060234 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:38.060210    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-c2qbk"
Jul 29 12:35:38.065268 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:38.065249    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-mh475\""
Jul 29 12:35:38.065826 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:38.065808    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 29 12:35:38.066440 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:38.066396    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 29 12:35:38.066543 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:38.066528    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 29 12:35:38.077262 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:38.077243    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-c2qbk"]
Jul 29 12:35:38.115296 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:38.115271    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6a26f56b-1c7c-44fb-acc5-9a5a00a93a3f-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-c2qbk\" (UID: \"6a26f56b-1c7c-44fb-acc5-9a5a00a93a3f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-c2qbk"
Jul 29 12:35:38.115385 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:38.115319    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6a26f56b-1c7c-44fb-acc5-9a5a00a93a3f-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-c2qbk\" (UID: \"6a26f56b-1c7c-44fb-acc5-9a5a00a93a3f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-c2qbk"
Jul 29 12:35:38.115385 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:38.115356    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xp54\" (UniqueName: \"kubernetes.io/projected/6a26f56b-1c7c-44fb-acc5-9a5a00a93a3f-kube-api-access-6xp54\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-c2qbk\" (UID: \"6a26f56b-1c7c-44fb-acc5-9a5a00a93a3f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-c2qbk"
Jul 29 12:35:38.115503 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:38.115427    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a26f56b-1c7c-44fb-acc5-9a5a00a93a3f-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-c2qbk\" (UID: \"6a26f56b-1c7c-44fb-acc5-9a5a00a93a3f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-c2qbk"
Jul 29 12:35:38.115503 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:38.115478    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a26f56b-1c7c-44fb-acc5-9a5a00a93a3f-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-c2qbk\" (UID: \"6a26f56b-1c7c-44fb-acc5-9a5a00a93a3f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-c2qbk"
Jul 29 12:35:38.115503 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:38.115495    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6a26f56b-1c7c-44fb-acc5-9a5a00a93a3f-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-c2qbk\" (UID: \"6a26f56b-1c7c-44fb-acc5-9a5a00a93a3f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-c2qbk"
Jul 29 12:35:38.216721 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:38.216690    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a26f56b-1c7c-44fb-acc5-9a5a00a93a3f-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-c2qbk\" (UID: \"6a26f56b-1c7c-44fb-acc5-9a5a00a93a3f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-c2qbk"
Jul 29 12:35:38.216862 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:38.216749    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a26f56b-1c7c-44fb-acc5-9a5a00a93a3f-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-c2qbk\" (UID: \"6a26f56b-1c7c-44fb-acc5-9a5a00a93a3f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-c2qbk"
Jul 29 12:35:38.216929 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:38.216862    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6a26f56b-1c7c-44fb-acc5-9a5a00a93a3f-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-c2qbk\" (UID: \"6a26f56b-1c7c-44fb-acc5-9a5a00a93a3f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-c2qbk"
Jul 29 12:35:38.216993 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:38.216924    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6a26f56b-1c7c-44fb-acc5-9a5a00a93a3f-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-c2qbk\" (UID: \"6a26f56b-1c7c-44fb-acc5-9a5a00a93a3f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-c2qbk"
Jul 29 12:35:38.216993 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:38.216979    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6a26f56b-1c7c-44fb-acc5-9a5a00a93a3f-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-c2qbk\" (UID: \"6a26f56b-1c7c-44fb-acc5-9a5a00a93a3f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-c2qbk"
Jul 29 12:35:38.217103 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:38.217013    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xp54\" (UniqueName: \"kubernetes.io/projected/6a26f56b-1c7c-44fb-acc5-9a5a00a93a3f-kube-api-access-6xp54\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-c2qbk\" (UID: \"6a26f56b-1c7c-44fb-acc5-9a5a00a93a3f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-c2qbk"
Jul 29 12:35:38.217162 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:38.217137    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a26f56b-1c7c-44fb-acc5-9a5a00a93a3f-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-c2qbk\" (UID: \"6a26f56b-1c7c-44fb-acc5-9a5a00a93a3f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-c2qbk"
Jul 29 12:35:38.217208 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:38.217174    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6a26f56b-1c7c-44fb-acc5-9a5a00a93a3f-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-c2qbk\" (UID: \"6a26f56b-1c7c-44fb-acc5-9a5a00a93a3f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-c2qbk"
Jul 29 12:35:38.217365 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:38.217344    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6a26f56b-1c7c-44fb-acc5-9a5a00a93a3f-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-c2qbk\" (UID: \"6a26f56b-1c7c-44fb-acc5-9a5a00a93a3f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-c2qbk"
Jul 29 12:35:38.219029 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:38.219006    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6a26f56b-1c7c-44fb-acc5-9a5a00a93a3f-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-c2qbk\" (UID: \"6a26f56b-1c7c-44fb-acc5-9a5a00a93a3f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-c2qbk"
Jul 29 12:35:38.219309 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:38.219287    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a26f56b-1c7c-44fb-acc5-9a5a00a93a3f-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-c2qbk\" (UID: \"6a26f56b-1c7c-44fb-acc5-9a5a00a93a3f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-c2qbk"
Jul 29 12:35:38.231347 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:38.231324    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xp54\" (UniqueName: \"kubernetes.io/projected/6a26f56b-1c7c-44fb-acc5-9a5a00a93a3f-kube-api-access-6xp54\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-c2qbk\" (UID: \"6a26f56b-1c7c-44fb-acc5-9a5a00a93a3f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-c2qbk"
Jul 29 12:35:38.370970 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:38.370921    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-c2qbk"
Jul 29 12:35:38.504859 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:38.504836    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-c2qbk"]
Jul 29 12:35:38.505499 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:35:38.505473    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a26f56b_1c7c_44fb_acc5_9a5a00a93a3f.slice/crio-31cdd796fd7680cc46c6d1c86a481480d432f8bae7559aa627c29dbd96bd95bf WatchSource:0}: Error finding container 31cdd796fd7680cc46c6d1c86a481480d432f8bae7559aa627c29dbd96bd95bf: Status 404 returned error can't find the container with id 31cdd796fd7680cc46c6d1c86a481480d432f8bae7559aa627c29dbd96bd95bf
Jul 29 12:35:39.164143 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:39.164112    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-c2qbk" event={"ID":"6a26f56b-1c7c-44fb-acc5-9a5a00a93a3f","Type":"ContainerStarted","Data":"31cdd796fd7680cc46c6d1c86a481480d432f8bae7559aa627c29dbd96bd95bf"}
Jul 29 12:35:41.175958 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:41.175922    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-c2qbk" event={"ID":"6a26f56b-1c7c-44fb-acc5-9a5a00a93a3f","Type":"ContainerStarted","Data":"8540ecd712a3444c0214e0fe43ba763d1a1febe6055d3eb367cafc1c3efa7878"}
Jul 29 12:35:41.198335 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:41.198288    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-c2qbk" podStartSLOduration=1.030918166 podStartE2EDuration="3.19827485s" totalImagesPullingTime="2.167356684s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:35:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:35:41.195812055 +0000 UTC m=+623.889500365" watchObservedRunningTime="2026-07-29 12:35:41.19827485 +0000 UTC m=+623.891963161"
Jul 29 12:35:48.371805 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:48.371773    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-c2qbk"
Jul 29 12:35:48.372287 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:48.371864    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-c2qbk"
Jul 29 12:35:48.384093 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:48.384072    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-c2qbk"
Jul 29 12:35:49.233667 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:35:49.233638    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-c2qbk"
Jul 29 12:36:02.967217 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:36:02.967182    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-vbkqz"]
Jul 29 12:36:02.973280 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:36:02.973258    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vbkqz"
Jul 29 12:36:02.977416 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:36:02.977387    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 29 12:36:02.988130 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:36:02.988110    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-vbkqz"]
Jul 29 12:36:03.115705 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:36:03.115678    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a273005c-fe77-4c36-b343-6aa5726ea755-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vbkqz\" (UID: \"a273005c-fe77-4c36-b343-6aa5726ea755\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vbkqz"
Jul 29 12:36:03.115815 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:36:03.115717    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wz9xg\" (UniqueName: \"kubernetes.io/projected/a273005c-fe77-4c36-b343-6aa5726ea755-kube-api-access-wz9xg\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vbkqz\" (UID: \"a273005c-fe77-4c36-b343-6aa5726ea755\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vbkqz"
Jul 29 12:36:03.115815 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:36:03.115762    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a273005c-fe77-4c36-b343-6aa5726ea755-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vbkqz\" (UID: \"a273005c-fe77-4c36-b343-6aa5726ea755\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vbkqz"
Jul 29 12:36:03.115815 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:36:03.115793    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a273005c-fe77-4c36-b343-6aa5726ea755-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vbkqz\" (UID: \"a273005c-fe77-4c36-b343-6aa5726ea755\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vbkqz"
Jul 29 12:36:03.115815 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:36:03.115811    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a273005c-fe77-4c36-b343-6aa5726ea755-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vbkqz\" (UID: \"a273005c-fe77-4c36-b343-6aa5726ea755\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vbkqz"
Jul 29 12:36:03.115947 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:36:03.115895    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a273005c-fe77-4c36-b343-6aa5726ea755-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vbkqz\" (UID: \"a273005c-fe77-4c36-b343-6aa5726ea755\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vbkqz"
Jul 29 12:36:03.216581 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:36:03.216551    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wz9xg\" (UniqueName: \"kubernetes.io/projected/a273005c-fe77-4c36-b343-6aa5726ea755-kube-api-access-wz9xg\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vbkqz\" (UID: \"a273005c-fe77-4c36-b343-6aa5726ea755\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vbkqz"
Jul 29 12:36:03.216696 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:36:03.216596    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a273005c-fe77-4c36-b343-6aa5726ea755-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vbkqz\" (UID: \"a273005c-fe77-4c36-b343-6aa5726ea755\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vbkqz"
Jul 29 12:36:03.216696 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:36:03.216624    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a273005c-fe77-4c36-b343-6aa5726ea755-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vbkqz\" (UID: \"a273005c-fe77-4c36-b343-6aa5726ea755\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vbkqz"
Jul 29 12:36:03.216696 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:36:03.216642    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a273005c-fe77-4c36-b343-6aa5726ea755-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vbkqz\" (UID: \"a273005c-fe77-4c36-b343-6aa5726ea755\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vbkqz"
Jul 29 12:36:03.216696 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:36:03.216674    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a273005c-fe77-4c36-b343-6aa5726ea755-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vbkqz\" (UID: \"a273005c-fe77-4c36-b343-6aa5726ea755\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vbkqz"
Jul 29 12:36:03.216696 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:36:03.216696    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a273005c-fe77-4c36-b343-6aa5726ea755-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vbkqz\" (UID: \"a273005c-fe77-4c36-b343-6aa5726ea755\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vbkqz"
Jul 29 12:36:03.217058 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:36:03.217040    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a273005c-fe77-4c36-b343-6aa5726ea755-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vbkqz\" (UID: \"a273005c-fe77-4c36-b343-6aa5726ea755\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vbkqz"
Jul 29 12:36:03.217135 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:36:03.217079    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a273005c-fe77-4c36-b343-6aa5726ea755-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vbkqz\" (UID: \"a273005c-fe77-4c36-b343-6aa5726ea755\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vbkqz"
Jul 29 12:36:03.217174 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:36:03.217151    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a273005c-fe77-4c36-b343-6aa5726ea755-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vbkqz\" (UID: \"a273005c-fe77-4c36-b343-6aa5726ea755\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vbkqz"
Jul 29 12:36:03.218840 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:36:03.218792    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a273005c-fe77-4c36-b343-6aa5726ea755-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vbkqz\" (UID: \"a273005c-fe77-4c36-b343-6aa5726ea755\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vbkqz"
Jul 29 12:36:03.219168 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:36:03.219146    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a273005c-fe77-4c36-b343-6aa5726ea755-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vbkqz\" (UID: \"a273005c-fe77-4c36-b343-6aa5726ea755\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vbkqz"
Jul 29 12:36:03.227353 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:36:03.227330    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wz9xg\" (UniqueName: \"kubernetes.io/projected/a273005c-fe77-4c36-b343-6aa5726ea755-kube-api-access-wz9xg\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vbkqz\" (UID: \"a273005c-fe77-4c36-b343-6aa5726ea755\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vbkqz"
Jul 29 12:36:03.283313 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:36:03.283288    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vbkqz"
Jul 29 12:36:03.421318 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:36:03.421295    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-vbkqz"]
Jul 29 12:36:03.422357 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:36:03.422331    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda273005c_fe77_4c36_b343_6aa5726ea755.slice/crio-e872377d1060527bc129a931862f592f955bda5393584f5a0526c6bfce0f3dca WatchSource:0}: Error finding container e872377d1060527bc129a931862f592f955bda5393584f5a0526c6bfce0f3dca: Status 404 returned error can't find the container with id e872377d1060527bc129a931862f592f955bda5393584f5a0526c6bfce0f3dca
Jul 29 12:36:04.310779 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:36:04.310744    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vbkqz" event={"ID":"a273005c-fe77-4c36-b343-6aa5726ea755","Type":"ContainerStarted","Data":"3bc437d28ff57c6f573de20e2c1cddc1f8646171399ebeb7974802c871c28c4a"}
Jul 29 12:36:04.311130 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:36:04.310791    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vbkqz" event={"ID":"a273005c-fe77-4c36-b343-6aa5726ea755","Type":"ContainerStarted","Data":"e872377d1060527bc129a931862f592f955bda5393584f5a0526c6bfce0f3dca"}
Jul 29 12:36:04.334713 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:36:04.334664    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vbkqz" podStartSLOduration=2.119739897 podStartE2EDuration="2.334650212s" totalImagesPullingTime="214.910315ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:36:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:36:04.331249369 +0000 UTC m=+647.024937679" watchObservedRunningTime="2026-07-29 12:36:04.334650212 +0000 UTC m=+647.028338523"
Jul 29 12:36:13.284168 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:36:13.284130    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vbkqz"
Jul 29 12:36:13.284666 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:36:13.284191    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vbkqz"
Jul 29 12:36:13.296232 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:36:13.296204    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vbkqz"
Jul 29 12:36:13.369503 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:36:13.369479    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vbkqz"
Jul 29 12:37:59.827583 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:37:59.827550    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-65d577d9d5-w4w8z"]
Jul 29 12:37:59.828001 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:37:59.827750    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-65d577d9d5-w4w8z" podUID="e5179166-12eb-429c-9fda-e8ab32f31f52" containerName="manager" containerID="cri-o://1a2862659dd92db89e13702cdb2a3813190df194de6f682aa43cd6401bd02945" gracePeriod=30
Jul 29 12:37:59.981101 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:37:59.981074    2567 generic.go:356] "Generic (PLEG): container finished" podID="e5179166-12eb-429c-9fda-e8ab32f31f52" containerID="1a2862659dd92db89e13702cdb2a3813190df194de6f682aa43cd6401bd02945" exitCode=0
Jul 29 12:37:59.981230 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:37:59.981145    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65d577d9d5-w4w8z" event={"ID":"e5179166-12eb-429c-9fda-e8ab32f31f52","Type":"ContainerDied","Data":"1a2862659dd92db89e13702cdb2a3813190df194de6f682aa43cd6401bd02945"}
Jul 29 12:38:00.082894 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:38:00.082827    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65d577d9d5-w4w8z"
Jul 29 12:38:00.228486 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:38:00.228461    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e5179166-12eb-429c-9fda-e8ab32f31f52-webhook-cert\") pod \"e5179166-12eb-429c-9fda-e8ab32f31f52\" (UID: \"e5179166-12eb-429c-9fda-e8ab32f31f52\") "
Jul 29 12:38:00.228656 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:38:00.228497    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wgzm7\" (UniqueName: \"kubernetes.io/projected/e5179166-12eb-429c-9fda-e8ab32f31f52-kube-api-access-wgzm7\") pod \"e5179166-12eb-429c-9fda-e8ab32f31f52\" (UID: \"e5179166-12eb-429c-9fda-e8ab32f31f52\") "
Jul 29 12:38:00.230397 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:38:00.230367    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5179166-12eb-429c-9fda-e8ab32f31f52-kube-api-access-wgzm7" (OuterVolumeSpecName: "kube-api-access-wgzm7") pod "e5179166-12eb-429c-9fda-e8ab32f31f52" (UID: "e5179166-12eb-429c-9fda-e8ab32f31f52"). InnerVolumeSpecName "kube-api-access-wgzm7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 12:38:00.230397 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:38:00.230389    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5179166-12eb-429c-9fda-e8ab32f31f52-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "e5179166-12eb-429c-9fda-e8ab32f31f52" (UID: "e5179166-12eb-429c-9fda-e8ab32f31f52"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 12:38:00.329441 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:38:00.329395    2567 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e5179166-12eb-429c-9fda-e8ab32f31f52-webhook-cert\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:38:00.329441 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:38:00.329442    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wgzm7\" (UniqueName: \"kubernetes.io/projected/e5179166-12eb-429c-9fda-e8ab32f31f52-kube-api-access-wgzm7\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:38:00.989232 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:38:00.989201    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65d577d9d5-w4w8z"
Jul 29 12:38:00.989758 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:38:00.989231    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65d577d9d5-w4w8z" event={"ID":"e5179166-12eb-429c-9fda-e8ab32f31f52","Type":"ContainerDied","Data":"137f2cf1017082f99f12e078b287e6f0d7b42d063982f7a519f577eee04e487c"}
Jul 29 12:38:00.989758 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:38:00.989292    2567 scope.go:117] "RemoveContainer" containerID="1a2862659dd92db89e13702cdb2a3813190df194de6f682aa43cd6401bd02945"
Jul 29 12:38:01.015455 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:38:01.015430    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-65d577d9d5-w4w8z"]
Jul 29 12:38:01.019170 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:38:01.019144    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-65d577d9d5-w4w8z"]
Jul 29 12:38:01.922834 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:38:01.922803    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5179166-12eb-429c-9fda-e8ab32f31f52" path="/var/lib/kubelet/pods/e5179166-12eb-429c-9fda-e8ab32f31f52/volumes"
Jul 29 12:39:01.870038 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:39:01.869999    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-65d577d9d5-mffbn"]
Jul 29 12:39:01.870649 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:39:01.870629    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5179166-12eb-429c-9fda-e8ab32f31f52" containerName="manager"
Jul 29 12:39:01.870735 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:39:01.870652    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5179166-12eb-429c-9fda-e8ab32f31f52" containerName="manager"
Jul 29 12:39:01.870787 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:39:01.870743    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5179166-12eb-429c-9fda-e8ab32f31f52" containerName="manager"
Jul 29 12:39:01.873822 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:39:01.873803    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65d577d9d5-mffbn"
Jul 29 12:39:01.880223 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:39:01.879906    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 29 12:39:01.880223 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:39:01.879915    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-kjvkj\""
Jul 29 12:39:01.880223 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:39:01.879996    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 29 12:39:01.887921 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:39:01.887896    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-65d577d9d5-mffbn"]
Jul 29 12:39:01.924999 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:39:01.924969    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5k4g\" (UniqueName: \"kubernetes.io/projected/d670fcee-dba4-48e2-9482-ba0f05823678-kube-api-access-w5k4g\") pod \"maas-controller-65d577d9d5-mffbn\" (UID: \"d670fcee-dba4-48e2-9482-ba0f05823678\") " pod="opendatahub/maas-controller-65d577d9d5-mffbn"
Jul 29 12:39:01.925153 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:39:01.925003    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d670fcee-dba4-48e2-9482-ba0f05823678-webhook-cert\") pod \"maas-controller-65d577d9d5-mffbn\" (UID: \"d670fcee-dba4-48e2-9482-ba0f05823678\") " pod="opendatahub/maas-controller-65d577d9d5-mffbn"
Jul 29 12:39:02.026201 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:39:02.026161    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5k4g\" (UniqueName: \"kubernetes.io/projected/d670fcee-dba4-48e2-9482-ba0f05823678-kube-api-access-w5k4g\") pod \"maas-controller-65d577d9d5-mffbn\" (UID: \"d670fcee-dba4-48e2-9482-ba0f05823678\") " pod="opendatahub/maas-controller-65d577d9d5-mffbn"
Jul 29 12:39:02.026201 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:39:02.026207    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d670fcee-dba4-48e2-9482-ba0f05823678-webhook-cert\") pod \"maas-controller-65d577d9d5-mffbn\" (UID: \"d670fcee-dba4-48e2-9482-ba0f05823678\") " pod="opendatahub/maas-controller-65d577d9d5-mffbn"
Jul 29 12:39:02.028760 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:39:02.028736    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d670fcee-dba4-48e2-9482-ba0f05823678-webhook-cert\") pod \"maas-controller-65d577d9d5-mffbn\" (UID: \"d670fcee-dba4-48e2-9482-ba0f05823678\") " pod="opendatahub/maas-controller-65d577d9d5-mffbn"
Jul 29 12:39:02.035035 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:39:02.035009    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5k4g\" (UniqueName: \"kubernetes.io/projected/d670fcee-dba4-48e2-9482-ba0f05823678-kube-api-access-w5k4g\") pod \"maas-controller-65d577d9d5-mffbn\" (UID: \"d670fcee-dba4-48e2-9482-ba0f05823678\") " pod="opendatahub/maas-controller-65d577d9d5-mffbn"
Jul 29 12:39:02.183499 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:39:02.183392    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65d577d9d5-mffbn"
Jul 29 12:39:02.327011 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:39:02.326979    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-65d577d9d5-mffbn"]
Jul 29 12:39:02.327521 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:39:02.327494    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd670fcee_dba4_48e2_9482_ba0f05823678.slice/crio-6dc9f14363f948e95f6dff97ba0b5e6c071910eef2c10539c113bb01ab261b06 WatchSource:0}: Error finding container 6dc9f14363f948e95f6dff97ba0b5e6c071910eef2c10539c113bb01ab261b06: Status 404 returned error can't find the container with id 6dc9f14363f948e95f6dff97ba0b5e6c071910eef2c10539c113bb01ab261b06
Jul 29 12:39:02.337523 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:39:02.337495    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65d577d9d5-mffbn" event={"ID":"d670fcee-dba4-48e2-9482-ba0f05823678","Type":"ContainerStarted","Data":"6dc9f14363f948e95f6dff97ba0b5e6c071910eef2c10539c113bb01ab261b06"}
Jul 29 12:39:03.344614 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:39:03.344577    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65d577d9d5-mffbn" event={"ID":"d670fcee-dba4-48e2-9482-ba0f05823678","Type":"ContainerStarted","Data":"57b39aab1073a4715d90b8bfef2a4193f2ea097846546c7671b1bd2d127c5958"}
Jul 29 12:39:03.345035 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:39:03.344696    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-65d577d9d5-mffbn"
Jul 29 12:39:03.367741 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:39:03.367682    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-65d577d9d5-mffbn" podStartSLOduration=1.8567817930000001 podStartE2EDuration="2.367661237s" totalImagesPullingTime="510.879444ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:39:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:39:03.363673351 +0000 UTC m=+826.057361665" watchObservedRunningTime="2026-07-29 12:39:03.367661237 +0000 UTC m=+826.061349549"
Jul 29 12:39:14.351912 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:39:14.351872    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-65d577d9d5-mffbn"
Jul 29 12:39:19.078319 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:39:19.078274    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-65d577d9d5-mffbn"]
Jul 29 12:39:19.078795 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:39:19.078519    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-65d577d9d5-mffbn" podUID="d670fcee-dba4-48e2-9482-ba0f05823678" containerName="manager" containerID="cri-o://57b39aab1073a4715d90b8bfef2a4193f2ea097846546c7671b1bd2d127c5958" gracePeriod=30
Jul 29 12:39:19.322838 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:39:19.322818    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65d577d9d5-mffbn"
Jul 29 12:39:19.441748 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:39:19.441678    2567 generic.go:356] "Generic (PLEG): container finished" podID="d670fcee-dba4-48e2-9482-ba0f05823678" containerID="57b39aab1073a4715d90b8bfef2a4193f2ea097846546c7671b1bd2d127c5958" exitCode=0
Jul 29 12:39:19.441748 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:39:19.441733    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65d577d9d5-mffbn"
Jul 29 12:39:19.441748 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:39:19.441744    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65d577d9d5-mffbn" event={"ID":"d670fcee-dba4-48e2-9482-ba0f05823678","Type":"ContainerDied","Data":"57b39aab1073a4715d90b8bfef2a4193f2ea097846546c7671b1bd2d127c5958"}
Jul 29 12:39:19.441945 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:39:19.441803    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65d577d9d5-mffbn" event={"ID":"d670fcee-dba4-48e2-9482-ba0f05823678","Type":"ContainerDied","Data":"6dc9f14363f948e95f6dff97ba0b5e6c071910eef2c10539c113bb01ab261b06"}
Jul 29 12:39:19.441945 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:39:19.441824    2567 scope.go:117] "RemoveContainer" containerID="57b39aab1073a4715d90b8bfef2a4193f2ea097846546c7671b1bd2d127c5958"
Jul 29 12:39:19.450438 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:39:19.450387    2567 scope.go:117] "RemoveContainer" containerID="57b39aab1073a4715d90b8bfef2a4193f2ea097846546c7671b1bd2d127c5958"
Jul 29 12:39:19.450665 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:39:19.450649    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57b39aab1073a4715d90b8bfef2a4193f2ea097846546c7671b1bd2d127c5958\": container with ID starting with 57b39aab1073a4715d90b8bfef2a4193f2ea097846546c7671b1bd2d127c5958 not found: ID does not exist" containerID="57b39aab1073a4715d90b8bfef2a4193f2ea097846546c7671b1bd2d127c5958"
Jul 29 12:39:19.450721 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:39:19.450676    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57b39aab1073a4715d90b8bfef2a4193f2ea097846546c7671b1bd2d127c5958"} err="failed to get container status \"57b39aab1073a4715d90b8bfef2a4193f2ea097846546c7671b1bd2d127c5958\": rpc error: code = NotFound desc = could not find container \"57b39aab1073a4715d90b8bfef2a4193f2ea097846546c7671b1bd2d127c5958\": container with ID starting with 57b39aab1073a4715d90b8bfef2a4193f2ea097846546c7671b1bd2d127c5958 not found: ID does not exist"
Jul 29 12:39:19.475506 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:39:19.475487    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w5k4g\" (UniqueName: \"kubernetes.io/projected/d670fcee-dba4-48e2-9482-ba0f05823678-kube-api-access-w5k4g\") pod \"d670fcee-dba4-48e2-9482-ba0f05823678\" (UID: \"d670fcee-dba4-48e2-9482-ba0f05823678\") "
Jul 29 12:39:19.475627 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:39:19.475614    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d670fcee-dba4-48e2-9482-ba0f05823678-webhook-cert\") pod \"d670fcee-dba4-48e2-9482-ba0f05823678\" (UID: \"d670fcee-dba4-48e2-9482-ba0f05823678\") "
Jul 29 12:39:19.477347 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:39:19.477323    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d670fcee-dba4-48e2-9482-ba0f05823678-kube-api-access-w5k4g" (OuterVolumeSpecName: "kube-api-access-w5k4g") pod "d670fcee-dba4-48e2-9482-ba0f05823678" (UID: "d670fcee-dba4-48e2-9482-ba0f05823678"). InnerVolumeSpecName "kube-api-access-w5k4g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 12:39:19.477439 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:39:19.477355    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d670fcee-dba4-48e2-9482-ba0f05823678-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "d670fcee-dba4-48e2-9482-ba0f05823678" (UID: "d670fcee-dba4-48e2-9482-ba0f05823678"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 12:39:19.577019 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:39:19.576997    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w5k4g\" (UniqueName: \"kubernetes.io/projected/d670fcee-dba4-48e2-9482-ba0f05823678-kube-api-access-w5k4g\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:39:19.577097 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:39:19.577023    2567 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d670fcee-dba4-48e2-9482-ba0f05823678-webhook-cert\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:39:19.769135 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:39:19.769111    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-65d577d9d5-mffbn"]
Jul 29 12:39:19.774116 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:39:19.774097    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-65d577d9d5-mffbn"]
Jul 29 12:39:19.921902 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:39:19.921875    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d670fcee-dba4-48e2-9482-ba0f05823678" path="/var/lib/kubelet/pods/d670fcee-dba4-48e2-9482-ba0f05823678/volumes"
Jul 29 12:40:17.872832 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:40:17.872800    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kj82f_a6d5f3a2-26cc-40a7-9841-2fbfc247661a/ovn-acl-logging/0.log"
Jul 29 12:40:17.875805 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:40:17.875786    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kj82f_a6d5f3a2-26cc-40a7-9841-2fbfc247661a/ovn-acl-logging/0.log"
Jul 29 12:45:00.170418 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:45:00.170327    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29755485-gzrqx"]
Jul 29 12:45:00.170837 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:45:00.170718    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d670fcee-dba4-48e2-9482-ba0f05823678" containerName="manager"
Jul 29 12:45:00.170837 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:45:00.170732    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d670fcee-dba4-48e2-9482-ba0f05823678" containerName="manager"
Jul 29 12:45:00.170837 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:45:00.170796    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="d670fcee-dba4-48e2-9482-ba0f05823678" containerName="manager"
Jul 29 12:45:00.173824 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:45:00.173807    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755485-gzrqx"
Jul 29 12:45:00.176419 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:45:00.176388    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7gpd2\""
Jul 29 12:45:00.183332 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:45:00.183315    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755485-gzrqx"
Jul 29 12:45:00.244282 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:45:00.244253    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29755485-gzrqx"]
Jul 29 12:45:00.515326 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:45:00.515299    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29755485-gzrqx"]
Jul 29 12:45:00.516866 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:45:00.516839    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc836e992_7c0a_4870_a012_68d4a43a4697.slice/crio-144334357fb01c8f51b4bb8c2742e72d08a97556d03265691f85a77c52528320 WatchSource:0}: Error finding container 144334357fb01c8f51b4bb8c2742e72d08a97556d03265691f85a77c52528320: Status 404 returned error can't find the container with id 144334357fb01c8f51b4bb8c2742e72d08a97556d03265691f85a77c52528320
Jul 29 12:45:00.518760 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:45:00.518745    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 12:45:01.430451 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:45:01.430414    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755485-gzrqx" event={"ID":"c836e992-7c0a-4870-a012-68d4a43a4697","Type":"ContainerStarted","Data":"144334357fb01c8f51b4bb8c2742e72d08a97556d03265691f85a77c52528320"}
Jul 29 12:45:03.442971 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:45:03.442947    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755485-gzrqx" event={"ID":"c836e992-7c0a-4870-a012-68d4a43a4697","Type":"ContainerStarted","Data":"f00d0147c17a4473f0a352d70b4993d0c8001cb7800f705986c9607bbf6b622a"}
Jul 29 12:45:04.450489 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:45:04.450423    2567 generic.go:356] "Generic (PLEG): container finished" podID="c836e992-7c0a-4870-a012-68d4a43a4697" containerID="f00d0147c17a4473f0a352d70b4993d0c8001cb7800f705986c9607bbf6b622a" exitCode=0
Jul 29 12:45:04.450781 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:45:04.450502    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755485-gzrqx" event={"ID":"c836e992-7c0a-4870-a012-68d4a43a4697","Type":"ContainerDied","Data":"f00d0147c17a4473f0a352d70b4993d0c8001cb7800f705986c9607bbf6b622a"}
Jul 29 12:45:05.583913 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:45:05.583892    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755485-gzrqx"
Jul 29 12:45:06.463182 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:45:06.463150    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755485-gzrqx"
Jul 29 12:45:06.463343 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:45:06.463182    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755485-gzrqx" event={"ID":"c836e992-7c0a-4870-a012-68d4a43a4697","Type":"ContainerDied","Data":"144334357fb01c8f51b4bb8c2742e72d08a97556d03265691f85a77c52528320"}
Jul 29 12:45:06.463343 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:45:06.463234    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="144334357fb01c8f51b4bb8c2742e72d08a97556d03265691f85a77c52528320"
Jul 29 12:45:17.914689 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:45:17.914656    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kj82f_a6d5f3a2-26cc-40a7-9841-2fbfc247661a/ovn-acl-logging/0.log"
Jul 29 12:45:17.926885 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:45:17.926854    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kj82f_a6d5f3a2-26cc-40a7-9841-2fbfc247661a/ovn-acl-logging/0.log"
Jul 29 12:50:17.961451 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:50:17.961424    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kj82f_a6d5f3a2-26cc-40a7-9841-2fbfc247661a/ovn-acl-logging/0.log"
Jul 29 12:50:17.964979 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:50:17.964958    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kj82f_a6d5f3a2-26cc-40a7-9841-2fbfc247661a/ovn-acl-logging/0.log"
Jul 29 12:55:18.000797 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:55:18.000772    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kj82f_a6d5f3a2-26cc-40a7-9841-2fbfc247661a/ovn-acl-logging/0.log"
Jul 29 12:55:18.005589 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:55:18.005570    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kj82f_a6d5f3a2-26cc-40a7-9841-2fbfc247661a/ovn-acl-logging/0.log"
Jul 29 12:56:38.940415 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:56:38.940374    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-3cf3b144-c7cfddd9-ntszv"]
Jul 29 12:56:38.940873 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:56:38.940814    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c836e992-7c0a-4870-a012-68d4a43a4697" containerName="cleanup"
Jul 29 12:56:38.940873 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:56:38.940827    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c836e992-7c0a-4870-a012-68d4a43a4697" containerName="cleanup"
Jul 29 12:56:38.940950 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:56:38.940890    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="c836e992-7c0a-4870-a012-68d4a43a4697" containerName="cleanup"
Jul 29 12:56:38.943855 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:56:38.943840    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3cf3b144-c7cfddd9-ntszv"
Jul 29 12:56:38.946352 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:56:38.946333    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-3cf3b144-serving-cert\""
Jul 29 12:56:38.955127 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:56:38.955106    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-3cf3b144-c7cfddd9-ntszv"]
Jul 29 12:56:39.091463 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:56:39.091438    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1cf1fac-a527-43a7-bca1-ad3dc211a9fb-openshift-service-ca\") pod \"maas-api-e2e-ait-3cf3b144-c7cfddd9-ntszv\" (UID: \"a1cf1fac-a527-43a7-bca1-ad3dc211a9fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3cf3b144-c7cfddd9-ntszv"
Jul 29 12:56:39.091589 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:56:39.091491    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a1cf1fac-a527-43a7-bca1-ad3dc211a9fb-maas-api-tls\") pod \"maas-api-e2e-ait-3cf3b144-c7cfddd9-ntszv\" (UID: \"a1cf1fac-a527-43a7-bca1-ad3dc211a9fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3cf3b144-c7cfddd9-ntszv"
Jul 29 12:56:39.091589 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:56:39.091567    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8bnj\" (UniqueName: \"kubernetes.io/projected/a1cf1fac-a527-43a7-bca1-ad3dc211a9fb-kube-api-access-q8bnj\") pod \"maas-api-e2e-ait-3cf3b144-c7cfddd9-ntszv\" (UID: \"a1cf1fac-a527-43a7-bca1-ad3dc211a9fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3cf3b144-c7cfddd9-ntszv"
Jul 29 12:56:39.192734 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:56:39.192656    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8bnj\" (UniqueName: \"kubernetes.io/projected/a1cf1fac-a527-43a7-bca1-ad3dc211a9fb-kube-api-access-q8bnj\") pod \"maas-api-e2e-ait-3cf3b144-c7cfddd9-ntszv\" (UID: \"a1cf1fac-a527-43a7-bca1-ad3dc211a9fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3cf3b144-c7cfddd9-ntszv"
Jul 29 12:56:39.192734 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:56:39.192712    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1cf1fac-a527-43a7-bca1-ad3dc211a9fb-openshift-service-ca\") pod \"maas-api-e2e-ait-3cf3b144-c7cfddd9-ntszv\" (UID: \"a1cf1fac-a527-43a7-bca1-ad3dc211a9fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3cf3b144-c7cfddd9-ntszv"
Jul 29 12:56:39.192867 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:56:39.192755    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a1cf1fac-a527-43a7-bca1-ad3dc211a9fb-maas-api-tls\") pod \"maas-api-e2e-ait-3cf3b144-c7cfddd9-ntszv\" (UID: \"a1cf1fac-a527-43a7-bca1-ad3dc211a9fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3cf3b144-c7cfddd9-ntszv"
Jul 29 12:56:39.192939 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:56:39.192866    2567 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-3cf3b144-serving-cert: secret "maas-api-e2e-ait-3cf3b144-serving-cert" not found
Jul 29 12:56:39.192939 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:56:39.192931    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a1cf1fac-a527-43a7-bca1-ad3dc211a9fb-maas-api-tls podName:a1cf1fac-a527-43a7-bca1-ad3dc211a9fb nodeName:}" failed. No retries permitted until 2026-07-29 12:56:39.692915702 +0000 UTC m=+1882.386603994 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a1cf1fac-a527-43a7-bca1-ad3dc211a9fb-maas-api-tls") pod "maas-api-e2e-ait-3cf3b144-c7cfddd9-ntszv" (UID: "a1cf1fac-a527-43a7-bca1-ad3dc211a9fb") : secret "maas-api-e2e-ait-3cf3b144-serving-cert" not found
Jul 29 12:56:39.193368 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:56:39.193349    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1cf1fac-a527-43a7-bca1-ad3dc211a9fb-openshift-service-ca\") pod \"maas-api-e2e-ait-3cf3b144-c7cfddd9-ntszv\" (UID: \"a1cf1fac-a527-43a7-bca1-ad3dc211a9fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3cf3b144-c7cfddd9-ntszv"
Jul 29 12:56:39.203355 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:56:39.203332    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8bnj\" (UniqueName: \"kubernetes.io/projected/a1cf1fac-a527-43a7-bca1-ad3dc211a9fb-kube-api-access-q8bnj\") pod \"maas-api-e2e-ait-3cf3b144-c7cfddd9-ntszv\" (UID: \"a1cf1fac-a527-43a7-bca1-ad3dc211a9fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3cf3b144-c7cfddd9-ntszv"
Jul 29 12:56:39.696542 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:56:39.696513    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a1cf1fac-a527-43a7-bca1-ad3dc211a9fb-maas-api-tls\") pod \"maas-api-e2e-ait-3cf3b144-c7cfddd9-ntszv\" (UID: \"a1cf1fac-a527-43a7-bca1-ad3dc211a9fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3cf3b144-c7cfddd9-ntszv"
Jul 29 12:56:39.698910 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:56:39.698886    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a1cf1fac-a527-43a7-bca1-ad3dc211a9fb-maas-api-tls\") pod \"maas-api-e2e-ait-3cf3b144-c7cfddd9-ntszv\" (UID: \"a1cf1fac-a527-43a7-bca1-ad3dc211a9fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3cf3b144-c7cfddd9-ntszv"
Jul 29 12:56:39.854952 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:56:39.854919    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3cf3b144-c7cfddd9-ntszv"
Jul 29 12:56:40.204791 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:56:40.204757    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-3cf3b144-c7cfddd9-ntszv"]
Jul 29 12:56:40.207906 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:56:40.207878    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1cf1fac_a527_43a7_bca1_ad3dc211a9fb.slice/crio-eda2b570db58a127169857a60f02fd372b37820c3ee823828343143a2046e5a6 WatchSource:0}: Error finding container eda2b570db58a127169857a60f02fd372b37820c3ee823828343143a2046e5a6: Status 404 returned error can't find the container with id eda2b570db58a127169857a60f02fd372b37820c3ee823828343143a2046e5a6
Jul 29 12:56:40.209511 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:56:40.209494    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 12:56:40.474181 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:56:40.474120    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3cf3b144-c7cfddd9-ntszv" event={"ID":"a1cf1fac-a527-43a7-bca1-ad3dc211a9fb","Type":"ContainerStarted","Data":"eda2b570db58a127169857a60f02fd372b37820c3ee823828343143a2046e5a6"}
Jul 29 12:56:41.482326 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:56:41.482293    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3cf3b144-c7cfddd9-ntszv" event={"ID":"a1cf1fac-a527-43a7-bca1-ad3dc211a9fb","Type":"ContainerStarted","Data":"b8945b6f9051bd3c691bfe960390c66110ac38835a7d122685e1d7b5e134756c"}
Jul 29 12:56:41.482713 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:56:41.482433    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3cf3b144-c7cfddd9-ntszv"
Jul 29 12:56:41.499601 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:56:41.499557    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3cf3b144-c7cfddd9-ntszv" podStartSLOduration=2.781652782 podStartE2EDuration="3.499544226s" totalImagesPullingTime="717.891444ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:56:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:56:41.498692606 +0000 UTC m=+1884.192380926" watchObservedRunningTime="2026-07-29 12:56:41.499544226 +0000 UTC m=+1884.193232535"
Jul 29 12:56:47.494306 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:56:47.494273    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3cf3b144-c7cfddd9-ntszv"
Jul 29 12:57:04.456614 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:04.456575    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3cf3b144-pmkf5"]
Jul 29 12:57:04.460250 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:04.460230    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3cf3b144-pmkf5"
Jul 29 12:57:04.462975 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:04.462951    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7gpd2\""
Jul 29 12:57:04.468870 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:04.468839    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3cf3b144-pmkf5"]
Jul 29 12:57:04.491253 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:04.491225    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85c958c3-e33d-46c4-be5f-06536bc8cb6e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3cf3b144-pmkf5\" (UID: \"85c958c3-e33d-46c4-be5f-06536bc8cb6e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3cf3b144-pmkf5"
Jul 29 12:57:04.592574 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:04.592548    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85c958c3-e33d-46c4-be5f-06536bc8cb6e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3cf3b144-pmkf5\" (UID: \"85c958c3-e33d-46c4-be5f-06536bc8cb6e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3cf3b144-pmkf5"
Jul 29 12:57:04.593219 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:04.593198    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85c958c3-e33d-46c4-be5f-06536bc8cb6e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3cf3b144-pmkf5\" (UID: \"85c958c3-e33d-46c4-be5f-06536bc8cb6e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3cf3b144-pmkf5"
Jul 29 12:57:04.773844 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:04.773767    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3cf3b144-pmkf5"
Jul 29 12:57:04.907289 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:04.907262    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3cf3b144-pmkf5"]
Jul 29 12:57:04.908444 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:57:04.908382    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85c958c3_e33d_46c4_be5f_06536bc8cb6e.slice/crio-9ac904e2f6e3860b11bb229529bbeb45af62c95346dde7e4356738dea2c94c62 WatchSource:0}: Error finding container 9ac904e2f6e3860b11bb229529bbeb45af62c95346dde7e4356738dea2c94c62: Status 404 returned error can't find the container with id 9ac904e2f6e3860b11bb229529bbeb45af62c95346dde7e4356738dea2c94c62
Jul 29 12:57:05.624966 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:05.624935    2567 generic.go:356] "Generic (PLEG): container finished" podID="85c958c3-e33d-46c4-be5f-06536bc8cb6e" containerID="453fdf3e95a9f883da6602871f28883a01d3069fa4ae67961bf1b8b940d5d45b" exitCode=0
Jul 29 12:57:05.625457 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:05.624975    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3cf3b144-pmkf5" event={"ID":"85c958c3-e33d-46c4-be5f-06536bc8cb6e","Type":"ContainerDied","Data":"453fdf3e95a9f883da6602871f28883a01d3069fa4ae67961bf1b8b940d5d45b"}
Jul 29 12:57:05.625457 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:05.625021    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3cf3b144-pmkf5" event={"ID":"85c958c3-e33d-46c4-be5f-06536bc8cb6e","Type":"ContainerStarted","Data":"9ac904e2f6e3860b11bb229529bbeb45af62c95346dde7e4356738dea2c94c62"}
Jul 29 12:57:06.763938 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:06.763918    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3cf3b144-pmkf5"
Jul 29 12:57:06.814005 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:06.813981    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85c958c3-e33d-46c4-be5f-06536bc8cb6e-maas-api-service-ca\") pod \"85c958c3-e33d-46c4-be5f-06536bc8cb6e\" (UID: \"85c958c3-e33d-46c4-be5f-06536bc8cb6e\") "
Jul 29 12:57:06.814319 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:06.814300    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85c958c3-e33d-46c4-be5f-06536bc8cb6e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "85c958c3-e33d-46c4-be5f-06536bc8cb6e" (UID: "85c958c3-e33d-46c4-be5f-06536bc8cb6e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 12:57:06.915141 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:06.915076    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85c958c3-e33d-46c4-be5f-06536bc8cb6e-maas-api-service-ca\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:57:07.637600 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:07.637574    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3cf3b144-pmkf5"
Jul 29 12:57:07.637749 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:07.637612    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3cf3b144-pmkf5" event={"ID":"85c958c3-e33d-46c4-be5f-06536bc8cb6e","Type":"ContainerDied","Data":"9ac904e2f6e3860b11bb229529bbeb45af62c95346dde7e4356738dea2c94c62"}
Jul 29 12:57:07.637749 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:07.637662    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9ac904e2f6e3860b11bb229529bbeb45af62c95346dde7e4356738dea2c94c62"
Jul 29 12:57:09.617979 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:09.617942    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-3cf3b144-c7cfddd9-ntszv"]
Jul 29 12:57:09.618521 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:09.618169    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3cf3b144-c7cfddd9-ntszv" podUID="a1cf1fac-a527-43a7-bca1-ad3dc211a9fb" containerName="maas-api" containerID="cri-o://b8945b6f9051bd3c691bfe960390c66110ac38835a7d122685e1d7b5e134756c" gracePeriod=30
Jul 29 12:57:09.808242 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:09.808214    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3cf3b144-pmkf5"]
Jul 29 12:57:09.810615 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:09.810596    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3cf3b144-pmkf5"]
Jul 29 12:57:09.871361 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:09.871308    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3cf3b144-c7cfddd9-ntszv"
Jul 29 12:57:09.924462 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:09.924439    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="85c958c3-e33d-46c4-be5f-06536bc8cb6e" path="/var/lib/kubelet/pods/85c958c3-e33d-46c4-be5f-06536bc8cb6e/volumes"
Jul 29 12:57:09.941531 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:09.941504    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1cf1fac-a527-43a7-bca1-ad3dc211a9fb-openshift-service-ca\") pod \"a1cf1fac-a527-43a7-bca1-ad3dc211a9fb\" (UID: \"a1cf1fac-a527-43a7-bca1-ad3dc211a9fb\") "
Jul 29 12:57:09.941665 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:09.941573    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a1cf1fac-a527-43a7-bca1-ad3dc211a9fb-maas-api-tls\") pod \"a1cf1fac-a527-43a7-bca1-ad3dc211a9fb\" (UID: \"a1cf1fac-a527-43a7-bca1-ad3dc211a9fb\") "
Jul 29 12:57:09.941728 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:09.941684    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q8bnj\" (UniqueName: \"kubernetes.io/projected/a1cf1fac-a527-43a7-bca1-ad3dc211a9fb-kube-api-access-q8bnj\") pod \"a1cf1fac-a527-43a7-bca1-ad3dc211a9fb\" (UID: \"a1cf1fac-a527-43a7-bca1-ad3dc211a9fb\") "
Jul 29 12:57:09.941888 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:09.941862    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1cf1fac-a527-43a7-bca1-ad3dc211a9fb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a1cf1fac-a527-43a7-bca1-ad3dc211a9fb" (UID: "a1cf1fac-a527-43a7-bca1-ad3dc211a9fb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 12:57:09.942052 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:09.942032    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1cf1fac-a527-43a7-bca1-ad3dc211a9fb-openshift-service-ca\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:57:09.944165 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:09.944142    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a1cf1fac-a527-43a7-bca1-ad3dc211a9fb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a1cf1fac-a527-43a7-bca1-ad3dc211a9fb" (UID: "a1cf1fac-a527-43a7-bca1-ad3dc211a9fb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 12:57:09.944302 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:09.944283    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1cf1fac-a527-43a7-bca1-ad3dc211a9fb-kube-api-access-q8bnj" (OuterVolumeSpecName: "kube-api-access-q8bnj") pod "a1cf1fac-a527-43a7-bca1-ad3dc211a9fb" (UID: "a1cf1fac-a527-43a7-bca1-ad3dc211a9fb"). InnerVolumeSpecName "kube-api-access-q8bnj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 12:57:10.043111 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:10.043072    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a1cf1fac-a527-43a7-bca1-ad3dc211a9fb-maas-api-tls\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:57:10.043111 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:10.043103    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q8bnj\" (UniqueName: \"kubernetes.io/projected/a1cf1fac-a527-43a7-bca1-ad3dc211a9fb-kube-api-access-q8bnj\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:57:10.656163 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:10.656132    2567 generic.go:356] "Generic (PLEG): container finished" podID="a1cf1fac-a527-43a7-bca1-ad3dc211a9fb" containerID="b8945b6f9051bd3c691bfe960390c66110ac38835a7d122685e1d7b5e134756c" exitCode=0
Jul 29 12:57:10.656569 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:10.656177    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3cf3b144-c7cfddd9-ntszv"
Jul 29 12:57:10.656569 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:10.656214    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3cf3b144-c7cfddd9-ntszv" event={"ID":"a1cf1fac-a527-43a7-bca1-ad3dc211a9fb","Type":"ContainerDied","Data":"b8945b6f9051bd3c691bfe960390c66110ac38835a7d122685e1d7b5e134756c"}
Jul 29 12:57:10.656569 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:10.656268    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3cf3b144-c7cfddd9-ntszv" event={"ID":"a1cf1fac-a527-43a7-bca1-ad3dc211a9fb","Type":"ContainerDied","Data":"eda2b570db58a127169857a60f02fd372b37820c3ee823828343143a2046e5a6"}
Jul 29 12:57:10.656569 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:10.656297    2567 scope.go:117] "RemoveContainer" containerID="b8945b6f9051bd3c691bfe960390c66110ac38835a7d122685e1d7b5e134756c"
Jul 29 12:57:10.666629 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:10.666614    2567 scope.go:117] "RemoveContainer" containerID="b8945b6f9051bd3c691bfe960390c66110ac38835a7d122685e1d7b5e134756c"
Jul 29 12:57:10.666862 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:57:10.666843    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b8945b6f9051bd3c691bfe960390c66110ac38835a7d122685e1d7b5e134756c\": container with ID starting with b8945b6f9051bd3c691bfe960390c66110ac38835a7d122685e1d7b5e134756c not found: ID does not exist" containerID="b8945b6f9051bd3c691bfe960390c66110ac38835a7d122685e1d7b5e134756c"
Jul 29 12:57:10.666915 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:10.666872    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8945b6f9051bd3c691bfe960390c66110ac38835a7d122685e1d7b5e134756c"} err="failed to get container status \"b8945b6f9051bd3c691bfe960390c66110ac38835a7d122685e1d7b5e134756c\": rpc error: code = NotFound desc = could not find container \"b8945b6f9051bd3c691bfe960390c66110ac38835a7d122685e1d7b5e134756c\": container with ID starting with b8945b6f9051bd3c691bfe960390c66110ac38835a7d122685e1d7b5e134756c not found: ID does not exist"
Jul 29 12:57:10.679887 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:10.679862    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-3cf3b144-c7cfddd9-ntszv"]
Jul 29 12:57:10.685505 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:10.685483    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-3cf3b144-c7cfddd9-ntszv"]
Jul 29 12:57:11.921937 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:11.921903    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1cf1fac-a527-43a7-bca1-ad3dc211a9fb" path="/var/lib/kubelet/pods/a1cf1fac-a527-43a7-bca1-ad3dc211a9fb/volumes"
Jul 29 12:57:13.210188 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:13.210144    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-95d13da9-85c7c6d684-7k4tz"]
Jul 29 12:57:13.210840 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:13.210814    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1cf1fac-a527-43a7-bca1-ad3dc211a9fb" containerName="maas-api"
Jul 29 12:57:13.210886 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:13.210846    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1cf1fac-a527-43a7-bca1-ad3dc211a9fb" containerName="maas-api"
Jul 29 12:57:13.210886 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:13.210882    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85c958c3-e33d-46c4-be5f-06536bc8cb6e" containerName="revoke-keys"
Jul 29 12:57:13.210952 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:13.210891    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="85c958c3-e33d-46c4-be5f-06536bc8cb6e" containerName="revoke-keys"
Jul 29 12:57:13.211026 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:13.211012    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="85c958c3-e33d-46c4-be5f-06536bc8cb6e" containerName="revoke-keys"
Jul 29 12:57:13.211064 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:13.211031    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1cf1fac-a527-43a7-bca1-ad3dc211a9fb" containerName="maas-api"
Jul 29 12:57:13.216386 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:13.216361    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-95d13da9-85c7c6d684-7k4tz"
Jul 29 12:57:13.219135 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:13.219105    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-95d13da9-serving-cert\""
Jul 29 12:57:13.222702 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:13.222669    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-95d13da9-85c7c6d684-7k4tz"]
Jul 29 12:57:13.271997 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:13.271973    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xffw\" (UniqueName: \"kubernetes.io/projected/91298c38-7845-4af9-bff5-80bebb8865f4-kube-api-access-5xffw\") pod \"maas-api-e2e-migrate-95d13da9-85c7c6d684-7k4tz\" (UID: \"91298c38-7845-4af9-bff5-80bebb8865f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-95d13da9-85c7c6d684-7k4tz"
Jul 29 12:57:13.272103 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:13.272020    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91298c38-7845-4af9-bff5-80bebb8865f4-openshift-service-ca\") pod \"maas-api-e2e-migrate-95d13da9-85c7c6d684-7k4tz\" (UID: \"91298c38-7845-4af9-bff5-80bebb8865f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-95d13da9-85c7c6d684-7k4tz"
Jul 29 12:57:13.272103 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:13.272064    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91298c38-7845-4af9-bff5-80bebb8865f4-maas-api-tls\") pod \"maas-api-e2e-migrate-95d13da9-85c7c6d684-7k4tz\" (UID: \"91298c38-7845-4af9-bff5-80bebb8865f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-95d13da9-85c7c6d684-7k4tz"
Jul 29 12:57:13.373187 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:13.373158    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xffw\" (UniqueName: \"kubernetes.io/projected/91298c38-7845-4af9-bff5-80bebb8865f4-kube-api-access-5xffw\") pod \"maas-api-e2e-migrate-95d13da9-85c7c6d684-7k4tz\" (UID: \"91298c38-7845-4af9-bff5-80bebb8865f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-95d13da9-85c7c6d684-7k4tz"
Jul 29 12:57:13.373289 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:13.373210    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91298c38-7845-4af9-bff5-80bebb8865f4-openshift-service-ca\") pod \"maas-api-e2e-migrate-95d13da9-85c7c6d684-7k4tz\" (UID: \"91298c38-7845-4af9-bff5-80bebb8865f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-95d13da9-85c7c6d684-7k4tz"
Jul 29 12:57:13.373289 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:13.373239    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91298c38-7845-4af9-bff5-80bebb8865f4-maas-api-tls\") pod \"maas-api-e2e-migrate-95d13da9-85c7c6d684-7k4tz\" (UID: \"91298c38-7845-4af9-bff5-80bebb8865f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-95d13da9-85c7c6d684-7k4tz"
Jul 29 12:57:13.373939 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:13.373908    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91298c38-7845-4af9-bff5-80bebb8865f4-openshift-service-ca\") pod \"maas-api-e2e-migrate-95d13da9-85c7c6d684-7k4tz\" (UID: \"91298c38-7845-4af9-bff5-80bebb8865f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-95d13da9-85c7c6d684-7k4tz"
Jul 29 12:57:13.375541 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:13.375517    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91298c38-7845-4af9-bff5-80bebb8865f4-maas-api-tls\") pod \"maas-api-e2e-migrate-95d13da9-85c7c6d684-7k4tz\" (UID: \"91298c38-7845-4af9-bff5-80bebb8865f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-95d13da9-85c7c6d684-7k4tz"
Jul 29 12:57:13.381829 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:13.381808    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xffw\" (UniqueName: \"kubernetes.io/projected/91298c38-7845-4af9-bff5-80bebb8865f4-kube-api-access-5xffw\") pod \"maas-api-e2e-migrate-95d13da9-85c7c6d684-7k4tz\" (UID: \"91298c38-7845-4af9-bff5-80bebb8865f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-95d13da9-85c7c6d684-7k4tz"
Jul 29 12:57:13.529950 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:13.529928    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-95d13da9-85c7c6d684-7k4tz"
Jul 29 12:57:13.674700 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:13.674676    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-95d13da9-85c7c6d684-7k4tz"]
Jul 29 12:57:13.676554 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:57:13.676529    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91298c38_7845_4af9_bff5_80bebb8865f4.slice/crio-d6d3ac2db3a3fb3c3e4bf006beccf13e7b788cab0f6eca1f68014caf6e068388 WatchSource:0}: Error finding container d6d3ac2db3a3fb3c3e4bf006beccf13e7b788cab0f6eca1f68014caf6e068388: Status 404 returned error can't find the container with id d6d3ac2db3a3fb3c3e4bf006beccf13e7b788cab0f6eca1f68014caf6e068388
Jul 29 12:57:14.684814 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:14.684782    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-95d13da9-85c7c6d684-7k4tz" event={"ID":"91298c38-7845-4af9-bff5-80bebb8865f4","Type":"ContainerStarted","Data":"298c121b21b78f883b6f4bf67c32c074b41fe2f95e7990adbb3e1ea891b4424a"}
Jul 29 12:57:14.685388 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:14.684835    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-95d13da9-85c7c6d684-7k4tz" event={"ID":"91298c38-7845-4af9-bff5-80bebb8865f4","Type":"ContainerStarted","Data":"d6d3ac2db3a3fb3c3e4bf006beccf13e7b788cab0f6eca1f68014caf6e068388"}
Jul 29 12:57:14.685388 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:14.684858    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-95d13da9-85c7c6d684-7k4tz"
Jul 29 12:57:14.707645 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:14.707598    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-95d13da9-85c7c6d684-7k4tz" podStartSLOduration=1.258116842 podStartE2EDuration="1.707585168s" totalImagesPullingTime="449.468326ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:57:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:57:14.704454106 +0000 UTC m=+1917.398142428" watchObservedRunningTime="2026-07-29 12:57:14.707585168 +0000 UTC m=+1917.401273477"
Jul 29 12:57:20.697327 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:20.697300    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-95d13da9-85c7c6d684-7k4tz"
Jul 29 12:57:38.147734 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:38.147681    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-95d13da9-nl8g5"]
Jul 29 12:57:38.151382 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:38.151361    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-95d13da9-nl8g5"
Jul 29 12:57:38.153903 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:38.153876    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7gpd2\""
Jul 29 12:57:38.159034 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:38.159007    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-95d13da9-nl8g5"]
Jul 29 12:57:38.273325 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:38.273298    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6deb1ad-78da-417f-aff6-fb98787660c3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-95d13da9-nl8g5\" (UID: \"a6deb1ad-78da-417f-aff6-fb98787660c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-95d13da9-nl8g5"
Jul 29 12:57:38.373742 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:38.373703    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6deb1ad-78da-417f-aff6-fb98787660c3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-95d13da9-nl8g5\" (UID: \"a6deb1ad-78da-417f-aff6-fb98787660c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-95d13da9-nl8g5"
Jul 29 12:57:38.374339 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:38.374315    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6deb1ad-78da-417f-aff6-fb98787660c3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-95d13da9-nl8g5\" (UID: \"a6deb1ad-78da-417f-aff6-fb98787660c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-95d13da9-nl8g5"
Jul 29 12:57:38.464096 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:38.464045    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-95d13da9-nl8g5"
Jul 29 12:57:38.596085 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:38.596058    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-95d13da9-nl8g5"]
Jul 29 12:57:38.597311 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:57:38.597282    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6deb1ad_78da_417f_aff6_fb98787660c3.slice/crio-c9518443ea41c17b23aa94005709fc347a2a81a44cd6ff24054cb2df0a089cbe WatchSource:0}: Error finding container c9518443ea41c17b23aa94005709fc347a2a81a44cd6ff24054cb2df0a089cbe: Status 404 returned error can't find the container with id c9518443ea41c17b23aa94005709fc347a2a81a44cd6ff24054cb2df0a089cbe
Jul 29 12:57:38.828032 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:38.828004    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-95d13da9-nl8g5" event={"ID":"a6deb1ad-78da-417f-aff6-fb98787660c3","Type":"ContainerStarted","Data":"c9518443ea41c17b23aa94005709fc347a2a81a44cd6ff24054cb2df0a089cbe"}
Jul 29 12:57:39.836291 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:39.836249    2567 generic.go:356] "Generic (PLEG): container finished" podID="a6deb1ad-78da-417f-aff6-fb98787660c3" containerID="d6879196a62bd7e7cdaef21ba6bd55d49fb075c1fd6e5755da979fd3e2b6b02e" exitCode=0
Jul 29 12:57:39.836707 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:39.836333    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-95d13da9-nl8g5" event={"ID":"a6deb1ad-78da-417f-aff6-fb98787660c3","Type":"ContainerDied","Data":"d6879196a62bd7e7cdaef21ba6bd55d49fb075c1fd6e5755da979fd3e2b6b02e"}
Jul 29 12:57:40.981515 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:40.981489    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-95d13da9-nl8g5"
Jul 29 12:57:41.096028 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:41.095970    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6deb1ad-78da-417f-aff6-fb98787660c3-maas-api-service-ca\") pod \"a6deb1ad-78da-417f-aff6-fb98787660c3\" (UID: \"a6deb1ad-78da-417f-aff6-fb98787660c3\") "
Jul 29 12:57:41.096278 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:41.096254    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a6deb1ad-78da-417f-aff6-fb98787660c3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a6deb1ad-78da-417f-aff6-fb98787660c3" (UID: "a6deb1ad-78da-417f-aff6-fb98787660c3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 12:57:41.197454 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:41.197430    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6deb1ad-78da-417f-aff6-fb98787660c3-maas-api-service-ca\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:57:41.851079 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:41.851052    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-95d13da9-nl8g5"
Jul 29 12:57:41.851219 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:41.851050    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-95d13da9-nl8g5" event={"ID":"a6deb1ad-78da-417f-aff6-fb98787660c3","Type":"ContainerDied","Data":"c9518443ea41c17b23aa94005709fc347a2a81a44cd6ff24054cb2df0a089cbe"}
Jul 29 12:57:41.851219 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:41.851188    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c9518443ea41c17b23aa94005709fc347a2a81a44cd6ff24054cb2df0a089cbe"
Jul 29 12:57:44.537794 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:44.537763    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-95d13da9-85c7c6d684-7k4tz"]
Jul 29 12:57:44.538262 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:44.538003    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-95d13da9-85c7c6d684-7k4tz" podUID="91298c38-7845-4af9-bff5-80bebb8865f4" containerName="maas-api" containerID="cri-o://298c121b21b78f883b6f4bf67c32c074b41fe2f95e7990adbb3e1ea891b4424a" gracePeriod=30
Jul 29 12:57:44.749094 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:44.749061    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-95d13da9-nl8g5"]
Jul 29 12:57:44.751435 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:44.751395    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-95d13da9-nl8g5"]
Jul 29 12:57:44.791138 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:44.791091    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-95d13da9-85c7c6d684-7k4tz"
Jul 29 12:57:44.873120 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:44.873089    2567 generic.go:356] "Generic (PLEG): container finished" podID="91298c38-7845-4af9-bff5-80bebb8865f4" containerID="298c121b21b78f883b6f4bf67c32c074b41fe2f95e7990adbb3e1ea891b4424a" exitCode=0
Jul 29 12:57:44.873265 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:44.873142    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-95d13da9-85c7c6d684-7k4tz"
Jul 29 12:57:44.873265 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:44.873171    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-95d13da9-85c7c6d684-7k4tz" event={"ID":"91298c38-7845-4af9-bff5-80bebb8865f4","Type":"ContainerDied","Data":"298c121b21b78f883b6f4bf67c32c074b41fe2f95e7990adbb3e1ea891b4424a"}
Jul 29 12:57:44.873384 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:44.873346    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-95d13da9-85c7c6d684-7k4tz" event={"ID":"91298c38-7845-4af9-bff5-80bebb8865f4","Type":"ContainerDied","Data":"d6d3ac2db3a3fb3c3e4bf006beccf13e7b788cab0f6eca1f68014caf6e068388"}
Jul 29 12:57:44.873384 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:44.873382    2567 scope.go:117] "RemoveContainer" containerID="298c121b21b78f883b6f4bf67c32c074b41fe2f95e7990adbb3e1ea891b4424a"
Jul 29 12:57:44.883002 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:44.882976    2567 scope.go:117] "RemoveContainer" containerID="298c121b21b78f883b6f4bf67c32c074b41fe2f95e7990adbb3e1ea891b4424a"
Jul 29 12:57:44.883263 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:57:44.883244    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"298c121b21b78f883b6f4bf67c32c074b41fe2f95e7990adbb3e1ea891b4424a\": container with ID starting with 298c121b21b78f883b6f4bf67c32c074b41fe2f95e7990adbb3e1ea891b4424a not found: ID does not exist" containerID="298c121b21b78f883b6f4bf67c32c074b41fe2f95e7990adbb3e1ea891b4424a"
Jul 29 12:57:44.883333 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:44.883272    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"298c121b21b78f883b6f4bf67c32c074b41fe2f95e7990adbb3e1ea891b4424a"} err="failed to get container status \"298c121b21b78f883b6f4bf67c32c074b41fe2f95e7990adbb3e1ea891b4424a\": rpc error: code = NotFound desc = could not find container \"298c121b21b78f883b6f4bf67c32c074b41fe2f95e7990adbb3e1ea891b4424a\": container with ID starting with 298c121b21b78f883b6f4bf67c32c074b41fe2f95e7990adbb3e1ea891b4424a not found: ID does not exist"
Jul 29 12:57:44.929590 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:44.929564    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91298c38-7845-4af9-bff5-80bebb8865f4-maas-api-tls\") pod \"91298c38-7845-4af9-bff5-80bebb8865f4\" (UID: \"91298c38-7845-4af9-bff5-80bebb8865f4\") "
Jul 29 12:57:44.929695 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:44.929678    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5xffw\" (UniqueName: \"kubernetes.io/projected/91298c38-7845-4af9-bff5-80bebb8865f4-kube-api-access-5xffw\") pod \"91298c38-7845-4af9-bff5-80bebb8865f4\" (UID: \"91298c38-7845-4af9-bff5-80bebb8865f4\") "
Jul 29 12:57:44.929781 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:44.929767    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91298c38-7845-4af9-bff5-80bebb8865f4-openshift-service-ca\") pod \"91298c38-7845-4af9-bff5-80bebb8865f4\" (UID: \"91298c38-7845-4af9-bff5-80bebb8865f4\") "
Jul 29 12:57:44.930103 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:44.930084    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91298c38-7845-4af9-bff5-80bebb8865f4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "91298c38-7845-4af9-bff5-80bebb8865f4" (UID: "91298c38-7845-4af9-bff5-80bebb8865f4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 12:57:44.931465 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:44.931439    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/91298c38-7845-4af9-bff5-80bebb8865f4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "91298c38-7845-4af9-bff5-80bebb8865f4" (UID: "91298c38-7845-4af9-bff5-80bebb8865f4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 12:57:44.931564 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:44.931546    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91298c38-7845-4af9-bff5-80bebb8865f4-kube-api-access-5xffw" (OuterVolumeSpecName: "kube-api-access-5xffw") pod "91298c38-7845-4af9-bff5-80bebb8865f4" (UID: "91298c38-7845-4af9-bff5-80bebb8865f4"). InnerVolumeSpecName "kube-api-access-5xffw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 12:57:45.031014 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:45.030988    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91298c38-7845-4af9-bff5-80bebb8865f4-maas-api-tls\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:57:45.031014 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:45.031010    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5xffw\" (UniqueName: \"kubernetes.io/projected/91298c38-7845-4af9-bff5-80bebb8865f4-kube-api-access-5xffw\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:57:45.031014 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:45.031020    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91298c38-7845-4af9-bff5-80bebb8865f4-openshift-service-ca\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:57:45.196197 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:45.196173    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-95d13da9-85c7c6d684-7k4tz"]
Jul 29 12:57:45.198629 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:45.198610    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-95d13da9-85c7c6d684-7k4tz"]
Jul 29 12:57:45.922055 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:45.922026    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="91298c38-7845-4af9-bff5-80bebb8865f4" path="/var/lib/kubelet/pods/91298c38-7845-4af9-bff5-80bebb8865f4/volumes"
Jul 29 12:57:45.922423 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:45.922351    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a6deb1ad-78da-417f-aff6-fb98787660c3" path="/var/lib/kubelet/pods/a6deb1ad-78da-417f-aff6-fb98787660c3/volumes"
Jul 29 12:57:48.326185 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:48.326146    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-329c04f9-dcbbb6d89-72qqv"]
Jul 29 12:57:48.326749 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:48.326727    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91298c38-7845-4af9-bff5-80bebb8865f4" containerName="maas-api"
Jul 29 12:57:48.326830 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:48.326751    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="91298c38-7845-4af9-bff5-80bebb8865f4" containerName="maas-api"
Jul 29 12:57:48.326830 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:48.326795    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6deb1ad-78da-417f-aff6-fb98787660c3" containerName="revoke-keys"
Jul 29 12:57:48.326830 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:48.326804    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6deb1ad-78da-417f-aff6-fb98787660c3" containerName="revoke-keys"
Jul 29 12:57:48.326971 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:48.326899    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="91298c38-7845-4af9-bff5-80bebb8865f4" containerName="maas-api"
Jul 29 12:57:48.326971 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:48.326916    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6deb1ad-78da-417f-aff6-fb98787660c3" containerName="revoke-keys"
Jul 29 12:57:48.331437 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:48.331417    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-329c04f9-dcbbb6d89-72qqv"
Jul 29 12:57:48.333759 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:48.333734    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-329c04f9-serving-cert\""
Jul 29 12:57:48.341710 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:48.341683    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-329c04f9-dcbbb6d89-72qqv"]
Jul 29 12:57:48.460340 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:48.460314    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e7441023-10a5-493f-a902-05ac2db89dca-maas-api-tls\") pod \"maas-api-e2e-ait-329c04f9-dcbbb6d89-72qqv\" (UID: \"e7441023-10a5-493f-a902-05ac2db89dca\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-329c04f9-dcbbb6d89-72qqv"
Jul 29 12:57:48.460491 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:48.460345    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfddl\" (UniqueName: \"kubernetes.io/projected/e7441023-10a5-493f-a902-05ac2db89dca-kube-api-access-kfddl\") pod \"maas-api-e2e-ait-329c04f9-dcbbb6d89-72qqv\" (UID: \"e7441023-10a5-493f-a902-05ac2db89dca\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-329c04f9-dcbbb6d89-72qqv"
Jul 29 12:57:48.460491 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:48.460468    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e7441023-10a5-493f-a902-05ac2db89dca-openshift-service-ca\") pod \"maas-api-e2e-ait-329c04f9-dcbbb6d89-72qqv\" (UID: \"e7441023-10a5-493f-a902-05ac2db89dca\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-329c04f9-dcbbb6d89-72qqv"
Jul 29 12:57:48.561357 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:48.561328    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e7441023-10a5-493f-a902-05ac2db89dca-maas-api-tls\") pod \"maas-api-e2e-ait-329c04f9-dcbbb6d89-72qqv\" (UID: \"e7441023-10a5-493f-a902-05ac2db89dca\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-329c04f9-dcbbb6d89-72qqv"
Jul 29 12:57:48.561581 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:48.561558    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfddl\" (UniqueName: \"kubernetes.io/projected/e7441023-10a5-493f-a902-05ac2db89dca-kube-api-access-kfddl\") pod \"maas-api-e2e-ait-329c04f9-dcbbb6d89-72qqv\" (UID: \"e7441023-10a5-493f-a902-05ac2db89dca\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-329c04f9-dcbbb6d89-72qqv"
Jul 29 12:57:48.561717 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:48.561699    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e7441023-10a5-493f-a902-05ac2db89dca-openshift-service-ca\") pod \"maas-api-e2e-ait-329c04f9-dcbbb6d89-72qqv\" (UID: \"e7441023-10a5-493f-a902-05ac2db89dca\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-329c04f9-dcbbb6d89-72qqv"
Jul 29 12:57:48.562338 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:48.562313    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e7441023-10a5-493f-a902-05ac2db89dca-openshift-service-ca\") pod \"maas-api-e2e-ait-329c04f9-dcbbb6d89-72qqv\" (UID: \"e7441023-10a5-493f-a902-05ac2db89dca\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-329c04f9-dcbbb6d89-72qqv"
Jul 29 12:57:48.563999 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:48.563977    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e7441023-10a5-493f-a902-05ac2db89dca-maas-api-tls\") pod \"maas-api-e2e-ait-329c04f9-dcbbb6d89-72qqv\" (UID: \"e7441023-10a5-493f-a902-05ac2db89dca\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-329c04f9-dcbbb6d89-72qqv"
Jul 29 12:57:48.569654 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:48.569637    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfddl\" (UniqueName: \"kubernetes.io/projected/e7441023-10a5-493f-a902-05ac2db89dca-kube-api-access-kfddl\") pod \"maas-api-e2e-ait-329c04f9-dcbbb6d89-72qqv\" (UID: \"e7441023-10a5-493f-a902-05ac2db89dca\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-329c04f9-dcbbb6d89-72qqv"
Jul 29 12:57:48.642755 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:48.642703    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-329c04f9-dcbbb6d89-72qqv"
Jul 29 12:57:48.987770 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:48.987740    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-329c04f9-dcbbb6d89-72qqv"]
Jul 29 12:57:48.988806 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:57:48.988780    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7441023_10a5_493f_a902_05ac2db89dca.slice/crio-b9a19324d4941056509cb74485d33ef7c46c73649abb44d405a597cac1b30d34 WatchSource:0}: Error finding container b9a19324d4941056509cb74485d33ef7c46c73649abb44d405a597cac1b30d34: Status 404 returned error can't find the container with id b9a19324d4941056509cb74485d33ef7c46c73649abb44d405a597cac1b30d34
Jul 29 12:57:49.914016 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:49.913983    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-329c04f9-dcbbb6d89-72qqv" event={"ID":"e7441023-10a5-493f-a902-05ac2db89dca","Type":"ContainerStarted","Data":"02af4905ee7249bc2467f8d9eb0879b9694de503b6255e68d56a0f13b82a7320"}
Jul 29 12:57:49.914377 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:49.914034    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-329c04f9-dcbbb6d89-72qqv" event={"ID":"e7441023-10a5-493f-a902-05ac2db89dca","Type":"ContainerStarted","Data":"b9a19324d4941056509cb74485d33ef7c46c73649abb44d405a597cac1b30d34"}
Jul 29 12:57:49.914377 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:49.914112    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-329c04f9-dcbbb6d89-72qqv"
Jul 29 12:57:49.931170 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:49.931130    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-329c04f9-dcbbb6d89-72qqv" podStartSLOduration=1.5489591630000001 podStartE2EDuration="1.931118034s" totalImagesPullingTime="382.158871ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:57:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:57:49.929750892 +0000 UTC m=+1952.623439225" watchObservedRunningTime="2026-07-29 12:57:49.931118034 +0000 UTC m=+1952.624806343"
Jul 29 12:57:55.926250 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:57:55.926217    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-329c04f9-dcbbb6d89-72qqv"
Jul 29 12:58:14.049700 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:14.049593    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-329c04f9-j9vtt"]
Jul 29 12:58:14.053777 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:14.053749    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-329c04f9-j9vtt"
Jul 29 12:58:14.056444 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:14.056392    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7gpd2\""
Jul 29 12:58:14.062471 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:14.062444    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-329c04f9-j9vtt"]
Jul 29 12:58:14.160194 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:14.160165    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a420e703-771c-4205-9628-cacb5854e67a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-329c04f9-j9vtt\" (UID: \"a420e703-771c-4205-9628-cacb5854e67a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-329c04f9-j9vtt"
Jul 29 12:58:14.260532 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:14.260504    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a420e703-771c-4205-9628-cacb5854e67a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-329c04f9-j9vtt\" (UID: \"a420e703-771c-4205-9628-cacb5854e67a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-329c04f9-j9vtt"
Jul 29 12:58:14.261208 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:14.261181    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a420e703-771c-4205-9628-cacb5854e67a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-329c04f9-j9vtt\" (UID: \"a420e703-771c-4205-9628-cacb5854e67a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-329c04f9-j9vtt"
Jul 29 12:58:14.368002 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:14.367940    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-329c04f9-j9vtt"
Jul 29 12:58:14.514126 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:14.512698    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-329c04f9-j9vtt"]
Jul 29 12:58:15.062616 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:15.062578    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-329c04f9-j9vtt" event={"ID":"a420e703-771c-4205-9628-cacb5854e67a","Type":"ContainerStarted","Data":"96f7b9c4facd0ee547d9cd378be2fc666807a06918616b20a7b9c9049e9b95fb"}
Jul 29 12:58:15.062984 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:15.062660    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-329c04f9-j9vtt" event={"ID":"a420e703-771c-4205-9628-cacb5854e67a","Type":"ContainerStarted","Data":"0f495d041eecc23d07a7cd0e0f6606fed8a2782f3838fe6a72939642d4aa8190"}
Jul 29 12:58:15.080907 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:15.080855    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-329c04f9-j9vtt" podStartSLOduration=1.080838748 podStartE2EDuration="1.080838748s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:58:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:58:15.078062182 +0000 UTC m=+1977.771750491" watchObservedRunningTime="2026-07-29 12:58:15.080838748 +0000 UTC m=+1977.774527059"
Jul 29 12:58:16.069775 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:16.069743    2567 generic.go:356] "Generic (PLEG): container finished" podID="a420e703-771c-4205-9628-cacb5854e67a" containerID="96f7b9c4facd0ee547d9cd378be2fc666807a06918616b20a7b9c9049e9b95fb" exitCode=0
Jul 29 12:58:16.070149 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:16.069824    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-329c04f9-j9vtt" event={"ID":"a420e703-771c-4205-9628-cacb5854e67a","Type":"ContainerDied","Data":"96f7b9c4facd0ee547d9cd378be2fc666807a06918616b20a7b9c9049e9b95fb"}
Jul 29 12:58:17.233279 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:17.233257    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-329c04f9-j9vtt"
Jul 29 12:58:17.285629 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:17.285605    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a420e703-771c-4205-9628-cacb5854e67a-maas-api-service-ca\") pod \"a420e703-771c-4205-9628-cacb5854e67a\" (UID: \"a420e703-771c-4205-9628-cacb5854e67a\") "
Jul 29 12:58:17.285919 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:17.285898    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a420e703-771c-4205-9628-cacb5854e67a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a420e703-771c-4205-9628-cacb5854e67a" (UID: "a420e703-771c-4205-9628-cacb5854e67a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 12:58:17.386638 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:17.386585    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a420e703-771c-4205-9628-cacb5854e67a-maas-api-service-ca\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:58:18.082976 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:18.082948    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-329c04f9-j9vtt" event={"ID":"a420e703-771c-4205-9628-cacb5854e67a","Type":"ContainerDied","Data":"0f495d041eecc23d07a7cd0e0f6606fed8a2782f3838fe6a72939642d4aa8190"}
Jul 29 12:58:18.083069 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:18.082984    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-329c04f9-j9vtt"
Jul 29 12:58:18.083069 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:18.082996    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0f495d041eecc23d07a7cd0e0f6606fed8a2782f3838fe6a72939642d4aa8190"
Jul 29 12:58:19.121839 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:19.121808    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-329c04f9-dcbbb6d89-72qqv"]
Jul 29 12:58:19.122294 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:19.122016    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-329c04f9-dcbbb6d89-72qqv" podUID="e7441023-10a5-493f-a902-05ac2db89dca" containerName="maas-api" containerID="cri-o://02af4905ee7249bc2467f8d9eb0879b9694de503b6255e68d56a0f13b82a7320" gracePeriod=30
Jul 29 12:58:19.346885 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:19.346856    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-329c04f9-j9vtt"]
Jul 29 12:58:19.350168 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:19.350140    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-329c04f9-j9vtt"]
Jul 29 12:58:19.377042 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:19.376984    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-329c04f9-dcbbb6d89-72qqv"
Jul 29 12:58:19.506970 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:19.506939    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kfddl\" (UniqueName: \"kubernetes.io/projected/e7441023-10a5-493f-a902-05ac2db89dca-kube-api-access-kfddl\") pod \"e7441023-10a5-493f-a902-05ac2db89dca\" (UID: \"e7441023-10a5-493f-a902-05ac2db89dca\") "
Jul 29 12:58:19.507110 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:19.507041    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e7441023-10a5-493f-a902-05ac2db89dca-openshift-service-ca\") pod \"e7441023-10a5-493f-a902-05ac2db89dca\" (UID: \"e7441023-10a5-493f-a902-05ac2db89dca\") "
Jul 29 12:58:19.507110 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:19.507084    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e7441023-10a5-493f-a902-05ac2db89dca-maas-api-tls\") pod \"e7441023-10a5-493f-a902-05ac2db89dca\" (UID: \"e7441023-10a5-493f-a902-05ac2db89dca\") "
Jul 29 12:58:19.507575 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:19.507540    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7441023-10a5-493f-a902-05ac2db89dca-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e7441023-10a5-493f-a902-05ac2db89dca" (UID: "e7441023-10a5-493f-a902-05ac2db89dca"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 12:58:19.509235 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:19.509204    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7441023-10a5-493f-a902-05ac2db89dca-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e7441023-10a5-493f-a902-05ac2db89dca" (UID: "e7441023-10a5-493f-a902-05ac2db89dca"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 12:58:19.509330 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:19.509288    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7441023-10a5-493f-a902-05ac2db89dca-kube-api-access-kfddl" (OuterVolumeSpecName: "kube-api-access-kfddl") pod "e7441023-10a5-493f-a902-05ac2db89dca" (UID: "e7441023-10a5-493f-a902-05ac2db89dca"). InnerVolumeSpecName "kube-api-access-kfddl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 12:58:19.607827 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:19.607805    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kfddl\" (UniqueName: \"kubernetes.io/projected/e7441023-10a5-493f-a902-05ac2db89dca-kube-api-access-kfddl\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:58:19.607827 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:19.607826    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e7441023-10a5-493f-a902-05ac2db89dca-openshift-service-ca\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:58:19.607955 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:19.607835    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e7441023-10a5-493f-a902-05ac2db89dca-maas-api-tls\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:58:19.921895 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:19.921861    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a420e703-771c-4205-9628-cacb5854e67a" path="/var/lib/kubelet/pods/a420e703-771c-4205-9628-cacb5854e67a/volumes"
Jul 29 12:58:20.095277 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:20.095245    2567 generic.go:356] "Generic (PLEG): container finished" podID="e7441023-10a5-493f-a902-05ac2db89dca" containerID="02af4905ee7249bc2467f8d9eb0879b9694de503b6255e68d56a0f13b82a7320" exitCode=0
Jul 29 12:58:20.095447 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:20.095289    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-329c04f9-dcbbb6d89-72qqv" event={"ID":"e7441023-10a5-493f-a902-05ac2db89dca","Type":"ContainerDied","Data":"02af4905ee7249bc2467f8d9eb0879b9694de503b6255e68d56a0f13b82a7320"}
Jul 29 12:58:20.095447 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:20.095309    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-329c04f9-dcbbb6d89-72qqv"
Jul 29 12:58:20.095447 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:20.095335    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-329c04f9-dcbbb6d89-72qqv" event={"ID":"e7441023-10a5-493f-a902-05ac2db89dca","Type":"ContainerDied","Data":"b9a19324d4941056509cb74485d33ef7c46c73649abb44d405a597cac1b30d34"}
Jul 29 12:58:20.095447 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:20.095359    2567 scope.go:117] "RemoveContainer" containerID="02af4905ee7249bc2467f8d9eb0879b9694de503b6255e68d56a0f13b82a7320"
Jul 29 12:58:20.104719 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:20.104701    2567 scope.go:117] "RemoveContainer" containerID="02af4905ee7249bc2467f8d9eb0879b9694de503b6255e68d56a0f13b82a7320"
Jul 29 12:58:20.104985 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:58:20.104965    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"02af4905ee7249bc2467f8d9eb0879b9694de503b6255e68d56a0f13b82a7320\": container with ID starting with 02af4905ee7249bc2467f8d9eb0879b9694de503b6255e68d56a0f13b82a7320 not found: ID does not exist" containerID="02af4905ee7249bc2467f8d9eb0879b9694de503b6255e68d56a0f13b82a7320"
Jul 29 12:58:20.105042 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:20.104996    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02af4905ee7249bc2467f8d9eb0879b9694de503b6255e68d56a0f13b82a7320"} err="failed to get container status \"02af4905ee7249bc2467f8d9eb0879b9694de503b6255e68d56a0f13b82a7320\": rpc error: code = NotFound desc = could not find container \"02af4905ee7249bc2467f8d9eb0879b9694de503b6255e68d56a0f13b82a7320\": container with ID starting with 02af4905ee7249bc2467f8d9eb0879b9694de503b6255e68d56a0f13b82a7320 not found: ID does not exist"
Jul 29 12:58:20.113813 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:20.113792    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-329c04f9-dcbbb6d89-72qqv"]
Jul 29 12:58:20.118126 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:20.118105    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-329c04f9-dcbbb6d89-72qqv"]
Jul 29 12:58:21.922044 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:21.922007    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7441023-10a5-493f-a902-05ac2db89dca" path="/var/lib/kubelet/pods/e7441023-10a5-493f-a902-05ac2db89dca/volumes"
Jul 29 12:58:23.473837 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:23.473805    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-9210d382-9d56967-ddrf8"]
Jul 29 12:58:23.474201 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:23.474188    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7441023-10a5-493f-a902-05ac2db89dca" containerName="maas-api"
Jul 29 12:58:23.474201 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:23.474202    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7441023-10a5-493f-a902-05ac2db89dca" containerName="maas-api"
Jul 29 12:58:23.474268 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:23.474216    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a420e703-771c-4205-9628-cacb5854e67a" containerName="revoke-keys"
Jul 29 12:58:23.474268 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:23.474222    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a420e703-771c-4205-9628-cacb5854e67a" containerName="revoke-keys"
Jul 29 12:58:23.474330 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:23.474284    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7441023-10a5-493f-a902-05ac2db89dca" containerName="maas-api"
Jul 29 12:58:23.474330 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:23.474294    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="a420e703-771c-4205-9628-cacb5854e67a" containerName="revoke-keys"
Jul 29 12:58:23.477385 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:23.477367    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9210d382-9d56967-ddrf8"
Jul 29 12:58:23.482355 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:23.482335    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-9210d382-serving-cert\""
Jul 29 12:58:23.490010 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:23.489991    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-9210d382-9d56967-ddrf8"]
Jul 29 12:58:23.538704 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:23.538682    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/24f19f44-6efe-4825-9dcf-2e3bc1a742cc-maas-api-tls\") pod \"maas-api-e2e-derive-9210d382-9d56967-ddrf8\" (UID: \"24f19f44-6efe-4825-9dcf-2e3bc1a742cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9210d382-9d56967-ddrf8"
Jul 29 12:58:23.538800 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:23.538719    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmcq7\" (UniqueName: \"kubernetes.io/projected/24f19f44-6efe-4825-9dcf-2e3bc1a742cc-kube-api-access-nmcq7\") pod \"maas-api-e2e-derive-9210d382-9d56967-ddrf8\" (UID: \"24f19f44-6efe-4825-9dcf-2e3bc1a742cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9210d382-9d56967-ddrf8"
Jul 29 12:58:23.538800 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:23.538762    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/24f19f44-6efe-4825-9dcf-2e3bc1a742cc-openshift-service-ca\") pod \"maas-api-e2e-derive-9210d382-9d56967-ddrf8\" (UID: \"24f19f44-6efe-4825-9dcf-2e3bc1a742cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9210d382-9d56967-ddrf8"
Jul 29 12:58:23.639649 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:23.639628    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/24f19f44-6efe-4825-9dcf-2e3bc1a742cc-maas-api-tls\") pod \"maas-api-e2e-derive-9210d382-9d56967-ddrf8\" (UID: \"24f19f44-6efe-4825-9dcf-2e3bc1a742cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9210d382-9d56967-ddrf8"
Jul 29 12:58:23.639745 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:23.639674    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmcq7\" (UniqueName: \"kubernetes.io/projected/24f19f44-6efe-4825-9dcf-2e3bc1a742cc-kube-api-access-nmcq7\") pod \"maas-api-e2e-derive-9210d382-9d56967-ddrf8\" (UID: \"24f19f44-6efe-4825-9dcf-2e3bc1a742cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9210d382-9d56967-ddrf8"
Jul 29 12:58:23.639745 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:23.639722    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/24f19f44-6efe-4825-9dcf-2e3bc1a742cc-openshift-service-ca\") pod \"maas-api-e2e-derive-9210d382-9d56967-ddrf8\" (UID: \"24f19f44-6efe-4825-9dcf-2e3bc1a742cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9210d382-9d56967-ddrf8"
Jul 29 12:58:23.640360 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:23.640341    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/24f19f44-6efe-4825-9dcf-2e3bc1a742cc-openshift-service-ca\") pod \"maas-api-e2e-derive-9210d382-9d56967-ddrf8\" (UID: \"24f19f44-6efe-4825-9dcf-2e3bc1a742cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9210d382-9d56967-ddrf8"
Jul 29 12:58:23.642090 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:23.642071    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/24f19f44-6efe-4825-9dcf-2e3bc1a742cc-maas-api-tls\") pod \"maas-api-e2e-derive-9210d382-9d56967-ddrf8\" (UID: \"24f19f44-6efe-4825-9dcf-2e3bc1a742cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9210d382-9d56967-ddrf8"
Jul 29 12:58:23.649520 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:23.649495    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmcq7\" (UniqueName: \"kubernetes.io/projected/24f19f44-6efe-4825-9dcf-2e3bc1a742cc-kube-api-access-nmcq7\") pod \"maas-api-e2e-derive-9210d382-9d56967-ddrf8\" (UID: \"24f19f44-6efe-4825-9dcf-2e3bc1a742cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9210d382-9d56967-ddrf8"
Jul 29 12:58:23.788647 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:23.788622    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9210d382-9d56967-ddrf8"
Jul 29 12:58:23.942455 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:23.942429    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-9210d382-9d56967-ddrf8"]
Jul 29 12:58:23.943179 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:58:23.943154    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24f19f44_6efe_4825_9dcf_2e3bc1a742cc.slice/crio-354b7fe634e0cd87793e7e1bcd9eed526f20fcddcf4afe68674146155c848564 WatchSource:0}: Error finding container 354b7fe634e0cd87793e7e1bcd9eed526f20fcddcf4afe68674146155c848564: Status 404 returned error can't find the container with id 354b7fe634e0cd87793e7e1bcd9eed526f20fcddcf4afe68674146155c848564
Jul 29 12:58:24.119494 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:24.119430    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9210d382-9d56967-ddrf8" event={"ID":"24f19f44-6efe-4825-9dcf-2e3bc1a742cc","Type":"ContainerStarted","Data":"354b7fe634e0cd87793e7e1bcd9eed526f20fcddcf4afe68674146155c848564"}
Jul 29 12:58:25.126801 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:25.126765    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9210d382-9d56967-ddrf8" event={"ID":"24f19f44-6efe-4825-9dcf-2e3bc1a742cc","Type":"ContainerStarted","Data":"fb1a8db221ecd33c6bb233592e8865b9194aa0720330b201f77bbcfe13f15d8e"}
Jul 29 12:58:25.127141 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:25.126901    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9210d382-9d56967-ddrf8"
Jul 29 12:58:25.146060 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:25.146017    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9210d382-9d56967-ddrf8" podStartSLOduration=1.687093624 podStartE2EDuration="2.146002691s" totalImagesPullingTime="458.909067ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:58:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:58:25.143424412 +0000 UTC m=+1987.837112729" watchObservedRunningTime="2026-07-29 12:58:25.146002691 +0000 UTC m=+1987.839691000"
Jul 29 12:58:31.137834 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:31.137793    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9210d382-9d56967-ddrf8"
Jul 29 12:58:48.284586 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:48.284549    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9210d382-4dlm5"]
Jul 29 12:58:48.288534 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:48.288514    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9210d382-4dlm5"
Jul 29 12:58:48.291414 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:48.291363    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7gpd2\""
Jul 29 12:58:48.294747 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:48.294717    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9210d382-4dlm5"]
Jul 29 12:58:48.438464 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:48.438441    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0593fafb-bd10-45da-a39f-154b20094e96-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-9210d382-4dlm5\" (UID: \"0593fafb-bd10-45da-a39f-154b20094e96\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9210d382-4dlm5"
Jul 29 12:58:48.539759 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:48.539704    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0593fafb-bd10-45da-a39f-154b20094e96-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-9210d382-4dlm5\" (UID: \"0593fafb-bd10-45da-a39f-154b20094e96\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9210d382-4dlm5"
Jul 29 12:58:48.540271 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:48.540250    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0593fafb-bd10-45da-a39f-154b20094e96-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-9210d382-4dlm5\" (UID: \"0593fafb-bd10-45da-a39f-154b20094e96\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9210d382-4dlm5"
Jul 29 12:58:48.602963 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:48.602927    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9210d382-4dlm5"
Jul 29 12:58:48.735598 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:48.735576    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9210d382-4dlm5"]
Jul 29 12:58:48.738103 ip-10-0-128-25 kubenswrapper[2567]: W0729 12:58:48.738074    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0593fafb_bd10_45da_a39f_154b20094e96.slice/crio-f2743a20613a5b9450b62473271ca5dc9f9dd755f7065650c6ed8ccef0dad620 WatchSource:0}: Error finding container f2743a20613a5b9450b62473271ca5dc9f9dd755f7065650c6ed8ccef0dad620: Status 404 returned error can't find the container with id f2743a20613a5b9450b62473271ca5dc9f9dd755f7065650c6ed8ccef0dad620
Jul 29 12:58:49.269134 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:49.269075    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9210d382-4dlm5" event={"ID":"0593fafb-bd10-45da-a39f-154b20094e96","Type":"ContainerStarted","Data":"95fd6355ec4d048522e9c591de66c4b6b59a07abf497a0a3ab702afda51b30c2"}
Jul 29 12:58:49.269134 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:49.269123    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9210d382-4dlm5" event={"ID":"0593fafb-bd10-45da-a39f-154b20094e96","Type":"ContainerStarted","Data":"f2743a20613a5b9450b62473271ca5dc9f9dd755f7065650c6ed8ccef0dad620"}
Jul 29 12:58:49.289700 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:49.289644    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9210d382-4dlm5" podStartSLOduration=1.28963045 podStartE2EDuration="1.28963045s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:58:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:58:49.287146574 +0000 UTC m=+2011.980834887" watchObservedRunningTime="2026-07-29 12:58:49.28963045 +0000 UTC m=+2011.983318817"
Jul 29 12:58:50.276865 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:50.276832    2567 generic.go:356] "Generic (PLEG): container finished" podID="0593fafb-bd10-45da-a39f-154b20094e96" containerID="95fd6355ec4d048522e9c591de66c4b6b59a07abf497a0a3ab702afda51b30c2" exitCode=0
Jul 29 12:58:50.277033 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:50.276882    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9210d382-4dlm5" event={"ID":"0593fafb-bd10-45da-a39f-154b20094e96","Type":"ContainerDied","Data":"95fd6355ec4d048522e9c591de66c4b6b59a07abf497a0a3ab702afda51b30c2"}
Jul 29 12:58:51.418570 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:51.418545    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9210d382-4dlm5"
Jul 29 12:58:51.567680 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:51.567626    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0593fafb-bd10-45da-a39f-154b20094e96-maas-api-service-ca\") pod \"0593fafb-bd10-45da-a39f-154b20094e96\" (UID: \"0593fafb-bd10-45da-a39f-154b20094e96\") "
Jul 29 12:58:51.567984 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:51.567964    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0593fafb-bd10-45da-a39f-154b20094e96-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0593fafb-bd10-45da-a39f-154b20094e96" (UID: "0593fafb-bd10-45da-a39f-154b20094e96"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 12:58:51.668935 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:51.668909    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0593fafb-bd10-45da-a39f-154b20094e96-maas-api-service-ca\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:58:52.290270 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:52.290239    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9210d382-4dlm5" event={"ID":"0593fafb-bd10-45da-a39f-154b20094e96","Type":"ContainerDied","Data":"f2743a20613a5b9450b62473271ca5dc9f9dd755f7065650c6ed8ccef0dad620"}
Jul 29 12:58:52.290437 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:52.290284    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9210d382-4dlm5"
Jul 29 12:58:52.290437 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:52.290288    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f2743a20613a5b9450b62473271ca5dc9f9dd755f7065650c6ed8ccef0dad620"
Jul 29 12:58:54.009059 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:54.009011    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-9210d382-9d56967-ddrf8"]
Jul 29 12:58:54.009538 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:54.009246    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9210d382-9d56967-ddrf8" podUID="24f19f44-6efe-4825-9dcf-2e3bc1a742cc" containerName="maas-api" containerID="cri-o://fb1a8db221ecd33c6bb233592e8865b9194aa0720330b201f77bbcfe13f15d8e" gracePeriod=30
Jul 29 12:58:54.205034 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:54.205000    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9210d382-4dlm5"]
Jul 29 12:58:54.207396 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:54.207373    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9210d382-4dlm5"]
Jul 29 12:58:54.257684 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:54.257663    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9210d382-9d56967-ddrf8"
Jul 29 12:58:54.304957 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:54.304928    2567 generic.go:356] "Generic (PLEG): container finished" podID="24f19f44-6efe-4825-9dcf-2e3bc1a742cc" containerID="fb1a8db221ecd33c6bb233592e8865b9194aa0720330b201f77bbcfe13f15d8e" exitCode=0
Jul 29 12:58:54.305118 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:54.304986    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9210d382-9d56967-ddrf8"
Jul 29 12:58:54.305118 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:54.305015    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9210d382-9d56967-ddrf8" event={"ID":"24f19f44-6efe-4825-9dcf-2e3bc1a742cc","Type":"ContainerDied","Data":"fb1a8db221ecd33c6bb233592e8865b9194aa0720330b201f77bbcfe13f15d8e"}
Jul 29 12:58:54.305118 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:54.305081    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9210d382-9d56967-ddrf8" event={"ID":"24f19f44-6efe-4825-9dcf-2e3bc1a742cc","Type":"ContainerDied","Data":"354b7fe634e0cd87793e7e1bcd9eed526f20fcddcf4afe68674146155c848564"}
Jul 29 12:58:54.305118 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:54.305110    2567 scope.go:117] "RemoveContainer" containerID="fb1a8db221ecd33c6bb233592e8865b9194aa0720330b201f77bbcfe13f15d8e"
Jul 29 12:58:54.316734 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:54.316713    2567 scope.go:117] "RemoveContainer" containerID="fb1a8db221ecd33c6bb233592e8865b9194aa0720330b201f77bbcfe13f15d8e"
Jul 29 12:58:54.317225 ip-10-0-128-25 kubenswrapper[2567]: E0729 12:58:54.317199    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb1a8db221ecd33c6bb233592e8865b9194aa0720330b201f77bbcfe13f15d8e\": container with ID starting with fb1a8db221ecd33c6bb233592e8865b9194aa0720330b201f77bbcfe13f15d8e not found: ID does not exist" containerID="fb1a8db221ecd33c6bb233592e8865b9194aa0720330b201f77bbcfe13f15d8e"
Jul 29 12:58:54.317361 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:54.317236    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb1a8db221ecd33c6bb233592e8865b9194aa0720330b201f77bbcfe13f15d8e"} err="failed to get container status \"fb1a8db221ecd33c6bb233592e8865b9194aa0720330b201f77bbcfe13f15d8e\": rpc error: code = NotFound desc = could not find container \"fb1a8db221ecd33c6bb233592e8865b9194aa0720330b201f77bbcfe13f15d8e\": container with ID starting with fb1a8db221ecd33c6bb233592e8865b9194aa0720330b201f77bbcfe13f15d8e not found: ID does not exist"
Jul 29 12:58:54.393703 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:54.393680    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/24f19f44-6efe-4825-9dcf-2e3bc1a742cc-openshift-service-ca\") pod \"24f19f44-6efe-4825-9dcf-2e3bc1a742cc\" (UID: \"24f19f44-6efe-4825-9dcf-2e3bc1a742cc\") "
Jul 29 12:58:54.393811 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:54.393742    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/24f19f44-6efe-4825-9dcf-2e3bc1a742cc-maas-api-tls\") pod \"24f19f44-6efe-4825-9dcf-2e3bc1a742cc\" (UID: \"24f19f44-6efe-4825-9dcf-2e3bc1a742cc\") "
Jul 29 12:58:54.393940 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:54.393918    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nmcq7\" (UniqueName: \"kubernetes.io/projected/24f19f44-6efe-4825-9dcf-2e3bc1a742cc-kube-api-access-nmcq7\") pod \"24f19f44-6efe-4825-9dcf-2e3bc1a742cc\" (UID: \"24f19f44-6efe-4825-9dcf-2e3bc1a742cc\") "
Jul 29 12:58:54.394080 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:54.394059    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/24f19f44-6efe-4825-9dcf-2e3bc1a742cc-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "24f19f44-6efe-4825-9dcf-2e3bc1a742cc" (UID: "24f19f44-6efe-4825-9dcf-2e3bc1a742cc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 12:58:54.394310 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:54.394292    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/24f19f44-6efe-4825-9dcf-2e3bc1a742cc-openshift-service-ca\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:58:54.395754 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:54.395724    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/24f19f44-6efe-4825-9dcf-2e3bc1a742cc-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "24f19f44-6efe-4825-9dcf-2e3bc1a742cc" (UID: "24f19f44-6efe-4825-9dcf-2e3bc1a742cc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 12:58:54.395984 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:54.395954    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24f19f44-6efe-4825-9dcf-2e3bc1a742cc-kube-api-access-nmcq7" (OuterVolumeSpecName: "kube-api-access-nmcq7") pod "24f19f44-6efe-4825-9dcf-2e3bc1a742cc" (UID: "24f19f44-6efe-4825-9dcf-2e3bc1a742cc"). InnerVolumeSpecName "kube-api-access-nmcq7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 12:58:54.494917 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:54.494897    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/24f19f44-6efe-4825-9dcf-2e3bc1a742cc-maas-api-tls\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:58:54.494917 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:54.494918    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nmcq7\" (UniqueName: \"kubernetes.io/projected/24f19f44-6efe-4825-9dcf-2e3bc1a742cc-kube-api-access-nmcq7\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 12:58:54.627460 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:54.627421    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-9210d382-9d56967-ddrf8"]
Jul 29 12:58:54.631353 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:54.631332    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-9210d382-9d56967-ddrf8"]
Jul 29 12:58:55.922059 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:55.922017    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0593fafb-bd10-45da-a39f-154b20094e96" path="/var/lib/kubelet/pods/0593fafb-bd10-45da-a39f-154b20094e96/volumes"
Jul 29 12:58:55.922472 ip-10-0-128-25 kubenswrapper[2567]: I0729 12:58:55.922396    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24f19f44-6efe-4825-9dcf-2e3bc1a742cc" path="/var/lib/kubelet/pods/24f19f44-6efe-4825-9dcf-2e3bc1a742cc/volumes"
Jul 29 13:00:00.141674 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:00:00.141637    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29755500-49hsw"]
Jul 29 13:00:00.142136 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:00:00.142066    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0593fafb-bd10-45da-a39f-154b20094e96" containerName="revoke-keys"
Jul 29 13:00:00.142136 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:00:00.142080    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="0593fafb-bd10-45da-a39f-154b20094e96" containerName="revoke-keys"
Jul 29 13:00:00.142136 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:00:00.142102    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24f19f44-6efe-4825-9dcf-2e3bc1a742cc" containerName="maas-api"
Jul 29 13:00:00.142136 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:00:00.142108    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="24f19f44-6efe-4825-9dcf-2e3bc1a742cc" containerName="maas-api"
Jul 29 13:00:00.142264 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:00:00.142176    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="0593fafb-bd10-45da-a39f-154b20094e96" containerName="revoke-keys"
Jul 29 13:00:00.142264 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:00:00.142186    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="24f19f44-6efe-4825-9dcf-2e3bc1a742cc" containerName="maas-api"
Jul 29 13:00:00.145332 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:00:00.145314    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755500-49hsw"
Jul 29 13:00:00.148050 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:00:00.148025    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7gpd2\""
Jul 29 13:00:00.154698 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:00:00.154666    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29755500-49hsw"]
Jul 29 13:00:00.156348 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:00:00.156327    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755500-49hsw"
Jul 29 13:00:00.304270 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:00:00.304243    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29755500-49hsw"]
Jul 29 13:00:00.304734 ip-10-0-128-25 kubenswrapper[2567]: W0729 13:00:00.304709    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0051e2df_1e85_40a0_9675_c233ac90ff3a.slice/crio-d3b01c41eddba8d33fef88ee79b74277e17380b2356f1ac256fbe4f6e79f9329 WatchSource:0}: Error finding container d3b01c41eddba8d33fef88ee79b74277e17380b2356f1ac256fbe4f6e79f9329: Status 404 returned error can't find the container with id d3b01c41eddba8d33fef88ee79b74277e17380b2356f1ac256fbe4f6e79f9329
Jul 29 13:00:00.712643 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:00:00.712612    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755500-49hsw" event={"ID":"0051e2df-1e85-40a0-9675-c233ac90ff3a","Type":"ContainerStarted","Data":"2a0315e1e30ad829ab6719baa8a692fcd9635fd6f2e0dc1525ac6b7b75cb5b57"}
Jul 29 13:00:00.712794 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:00:00.712673    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755500-49hsw" event={"ID":"0051e2df-1e85-40a0-9675-c233ac90ff3a","Type":"ContainerStarted","Data":"d3b01c41eddba8d33fef88ee79b74277e17380b2356f1ac256fbe4f6e79f9329"}
Jul 29 13:00:01.720062 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:00:01.720026    2567 generic.go:356] "Generic (PLEG): container finished" podID="0051e2df-1e85-40a0-9675-c233ac90ff3a" containerID="2a0315e1e30ad829ab6719baa8a692fcd9635fd6f2e0dc1525ac6b7b75cb5b57" exitCode=0
Jul 29 13:00:01.720427 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:00:01.720109    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755500-49hsw" event={"ID":"0051e2df-1e85-40a0-9675-c233ac90ff3a","Type":"ContainerDied","Data":"2a0315e1e30ad829ab6719baa8a692fcd9635fd6f2e0dc1525ac6b7b75cb5b57"}
Jul 29 13:00:02.860220 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:00:02.860198    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755500-49hsw"
Jul 29 13:00:03.740746 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:00:03.740710    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755500-49hsw" event={"ID":"0051e2df-1e85-40a0-9675-c233ac90ff3a","Type":"ContainerDied","Data":"d3b01c41eddba8d33fef88ee79b74277e17380b2356f1ac256fbe4f6e79f9329"}
Jul 29 13:00:03.740746 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:00:03.740740    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755500-49hsw"
Jul 29 13:00:03.740972 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:00:03.740762    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d3b01c41eddba8d33fef88ee79b74277e17380b2356f1ac256fbe4f6e79f9329"
Jul 29 13:00:18.040411 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:00:18.040371    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kj82f_a6d5f3a2-26cc-40a7-9841-2fbfc247661a/ovn-acl-logging/0.log"
Jul 29 13:00:18.046106 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:00:18.046087    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kj82f_a6d5f3a2-26cc-40a7-9841-2fbfc247661a/ovn-acl-logging/0.log"
Jul 29 13:03:01.473817 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:03:01.473734    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-be3b8a53-6b4544c455-dfz6n"]
Jul 29 13:03:01.474275 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:03:01.474133    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0051e2df-1e85-40a0-9675-c233ac90ff3a" containerName="cleanup"
Jul 29 13:03:01.474275 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:03:01.474146    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="0051e2df-1e85-40a0-9675-c233ac90ff3a" containerName="cleanup"
Jul 29 13:03:01.474275 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:03:01.474217    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="0051e2df-1e85-40a0-9675-c233ac90ff3a" containerName="cleanup"
Jul 29 13:03:01.477207 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:03:01.477188    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-be3b8a53-6b4544c455-dfz6n"
Jul 29 13:03:01.479881 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:03:01.479861    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-be3b8a53-serving-cert\""
Jul 29 13:03:01.494878 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:03:01.494858    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-be3b8a53-6b4544c455-dfz6n"]
Jul 29 13:03:01.509683 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:03:01.509661    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b2def39c-491f-4959-a773-a2f0e4764d34-maas-api-tls\") pod \"maas-api-e2e-mt-be3b8a53-6b4544c455-dfz6n\" (UID: \"b2def39c-491f-4959-a773-a2f0e4764d34\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-be3b8a53-6b4544c455-dfz6n"
Jul 29 13:03:01.509847 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:03:01.509825    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpcdx\" (UniqueName: \"kubernetes.io/projected/b2def39c-491f-4959-a773-a2f0e4764d34-kube-api-access-hpcdx\") pod \"maas-api-e2e-mt-be3b8a53-6b4544c455-dfz6n\" (UID: \"b2def39c-491f-4959-a773-a2f0e4764d34\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-be3b8a53-6b4544c455-dfz6n"
Jul 29 13:03:01.509975 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:03:01.509958    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2def39c-491f-4959-a773-a2f0e4764d34-openshift-service-ca\") pod \"maas-api-e2e-mt-be3b8a53-6b4544c455-dfz6n\" (UID: \"b2def39c-491f-4959-a773-a2f0e4764d34\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-be3b8a53-6b4544c455-dfz6n"
Jul 29 13:03:01.611348 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:03:01.611322    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2def39c-491f-4959-a773-a2f0e4764d34-openshift-service-ca\") pod \"maas-api-e2e-mt-be3b8a53-6b4544c455-dfz6n\" (UID: \"b2def39c-491f-4959-a773-a2f0e4764d34\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-be3b8a53-6b4544c455-dfz6n"
Jul 29 13:03:01.611461 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:03:01.611389    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b2def39c-491f-4959-a773-a2f0e4764d34-maas-api-tls\") pod \"maas-api-e2e-mt-be3b8a53-6b4544c455-dfz6n\" (UID: \"b2def39c-491f-4959-a773-a2f0e4764d34\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-be3b8a53-6b4544c455-dfz6n"
Jul 29 13:03:01.611461 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:03:01.611434    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hpcdx\" (UniqueName: \"kubernetes.io/projected/b2def39c-491f-4959-a773-a2f0e4764d34-kube-api-access-hpcdx\") pod \"maas-api-e2e-mt-be3b8a53-6b4544c455-dfz6n\" (UID: \"b2def39c-491f-4959-a773-a2f0e4764d34\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-be3b8a53-6b4544c455-dfz6n"
Jul 29 13:03:01.611566 ip-10-0-128-25 kubenswrapper[2567]: E0729 13:03:01.611549    2567 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-be3b8a53-serving-cert: secret "maas-api-e2e-mt-be3b8a53-serving-cert" not found
Jul 29 13:03:01.611633 ip-10-0-128-25 kubenswrapper[2567]: E0729 13:03:01.611623    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2def39c-491f-4959-a773-a2f0e4764d34-maas-api-tls podName:b2def39c-491f-4959-a773-a2f0e4764d34 nodeName:}" failed. No retries permitted until 2026-07-29 13:03:02.111605025 +0000 UTC m=+2264.805293326 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b2def39c-491f-4959-a773-a2f0e4764d34-maas-api-tls") pod "maas-api-e2e-mt-be3b8a53-6b4544c455-dfz6n" (UID: "b2def39c-491f-4959-a773-a2f0e4764d34") : secret "maas-api-e2e-mt-be3b8a53-serving-cert" not found
Jul 29 13:03:01.611927 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:03:01.611909    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2def39c-491f-4959-a773-a2f0e4764d34-openshift-service-ca\") pod \"maas-api-e2e-mt-be3b8a53-6b4544c455-dfz6n\" (UID: \"b2def39c-491f-4959-a773-a2f0e4764d34\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-be3b8a53-6b4544c455-dfz6n"
Jul 29 13:03:01.625708 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:03:01.625686    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpcdx\" (UniqueName: \"kubernetes.io/projected/b2def39c-491f-4959-a773-a2f0e4764d34-kube-api-access-hpcdx\") pod \"maas-api-e2e-mt-be3b8a53-6b4544c455-dfz6n\" (UID: \"b2def39c-491f-4959-a773-a2f0e4764d34\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-be3b8a53-6b4544c455-dfz6n"
Jul 29 13:03:02.116059 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:03:02.116027    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b2def39c-491f-4959-a773-a2f0e4764d34-maas-api-tls\") pod \"maas-api-e2e-mt-be3b8a53-6b4544c455-dfz6n\" (UID: \"b2def39c-491f-4959-a773-a2f0e4764d34\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-be3b8a53-6b4544c455-dfz6n"
Jul 29 13:03:02.118383 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:03:02.118352    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b2def39c-491f-4959-a773-a2f0e4764d34-maas-api-tls\") pod \"maas-api-e2e-mt-be3b8a53-6b4544c455-dfz6n\" (UID: \"b2def39c-491f-4959-a773-a2f0e4764d34\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-be3b8a53-6b4544c455-dfz6n"
Jul 29 13:03:02.386614 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:03:02.386548    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-be3b8a53-6b4544c455-dfz6n"
Jul 29 13:03:02.555448 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:03:02.555421    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-be3b8a53-6b4544c455-dfz6n"]
Jul 29 13:03:02.556864 ip-10-0-128-25 kubenswrapper[2567]: W0729 13:03:02.556839    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2def39c_491f_4959_a773_a2f0e4764d34.slice/crio-125aa401b6910c38ed2363c5e99f2886212495dfd1526294bdf65e597cf3f276 WatchSource:0}: Error finding container 125aa401b6910c38ed2363c5e99f2886212495dfd1526294bdf65e597cf3f276: Status 404 returned error can't find the container with id 125aa401b6910c38ed2363c5e99f2886212495dfd1526294bdf65e597cf3f276
Jul 29 13:03:02.558184 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:03:02.558166    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 13:03:02.816826 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:03:02.816792    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-be3b8a53-6b4544c455-dfz6n" event={"ID":"b2def39c-491f-4959-a773-a2f0e4764d34","Type":"ContainerStarted","Data":"125aa401b6910c38ed2363c5e99f2886212495dfd1526294bdf65e597cf3f276"}
Jul 29 13:03:03.823703 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:03:03.823667    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-be3b8a53-6b4544c455-dfz6n" event={"ID":"b2def39c-491f-4959-a773-a2f0e4764d34","Type":"ContainerStarted","Data":"790342191dbc118ba93678ecb8b3469170eddb1ba690b5198e4b03eb875a2f41"}
Jul 29 13:03:03.824088 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:03:03.823813    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-be3b8a53-6b4544c455-dfz6n"
Jul 29 13:03:03.841831 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:03:03.841788    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-be3b8a53-6b4544c455-dfz6n" podStartSLOduration=2.351722911 podStartE2EDuration="2.841774731s" totalImagesPullingTime="490.05182ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 13:03:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 13:03:03.84102202 +0000 UTC m=+2266.534710331" watchObservedRunningTime="2026-07-29 13:03:03.841774731 +0000 UTC m=+2266.535463041"
Jul 29 13:03:09.836036 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:03:09.835997    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-be3b8a53-6b4544c455-dfz6n"
Jul 29 13:03:18.108590 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:03:18.108560    2567 scope.go:117] "RemoveContainer" containerID="453fdf3e95a9f883da6602871f28883a01d3069fa4ae67961bf1b8b940d5d45b"
Jul 29 13:03:33.546084 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:03:33.546042    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-be3b8a53-f6t66"]
Jul 29 13:03:33.549894 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:03:33.549870    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-be3b8a53-f6t66"
Jul 29 13:03:33.552335 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:03:33.552318    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7gpd2\""
Jul 29 13:03:33.558344 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:03:33.558317    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-be3b8a53-f6t66"]
Jul 29 13:03:33.673049 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:03:33.673021    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab79463f-d124-4632-b406-5ad71bafded9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-be3b8a53-f6t66\" (UID: \"ab79463f-d124-4632-b406-5ad71bafded9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-be3b8a53-f6t66"
Jul 29 13:03:33.773959 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:03:33.773929    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab79463f-d124-4632-b406-5ad71bafded9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-be3b8a53-f6t66\" (UID: \"ab79463f-d124-4632-b406-5ad71bafded9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-be3b8a53-f6t66"
Jul 29 13:03:33.774552 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:03:33.774534    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab79463f-d124-4632-b406-5ad71bafded9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-be3b8a53-f6t66\" (UID: \"ab79463f-d124-4632-b406-5ad71bafded9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-be3b8a53-f6t66"
Jul 29 13:03:33.862192 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:03:33.862135    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-be3b8a53-f6t66"
Jul 29 13:03:33.998533 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:03:33.998503    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-be3b8a53-f6t66"]
Jul 29 13:03:33.999663 ip-10-0-128-25 kubenswrapper[2567]: W0729 13:03:33.999621    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab79463f_d124_4632_b406_5ad71bafded9.slice/crio-efbc3c9b94d9a9584c96d00006a4054d9161f22ac0abc4252fe6473d800f285a WatchSource:0}: Error finding container efbc3c9b94d9a9584c96d00006a4054d9161f22ac0abc4252fe6473d800f285a: Status 404 returned error can't find the container with id efbc3c9b94d9a9584c96d00006a4054d9161f22ac0abc4252fe6473d800f285a
Jul 29 13:03:34.010825 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:03:34.010805    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-be3b8a53-f6t66" event={"ID":"ab79463f-d124-4632-b406-5ad71bafded9","Type":"ContainerStarted","Data":"efbc3c9b94d9a9584c96d00006a4054d9161f22ac0abc4252fe6473d800f285a"}
Jul 29 13:03:35.017845 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:03:35.017772    2567 generic.go:356] "Generic (PLEG): container finished" podID="ab79463f-d124-4632-b406-5ad71bafded9" containerID="e5c37575d64956775651f398d090e1e31c5824dfefc7f96a1598b8630de1ea4f" exitCode=0
Jul 29 13:03:35.018172 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:03:35.017854    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-be3b8a53-f6t66" event={"ID":"ab79463f-d124-4632-b406-5ad71bafded9","Type":"ContainerDied","Data":"e5c37575d64956775651f398d090e1e31c5824dfefc7f96a1598b8630de1ea4f"}
Jul 29 13:03:36.162734 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:03:36.162712    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-be3b8a53-f6t66"
Jul 29 13:03:36.295927 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:03:36.295897    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab79463f-d124-4632-b406-5ad71bafded9-maas-api-service-ca\") pod \"ab79463f-d124-4632-b406-5ad71bafded9\" (UID: \"ab79463f-d124-4632-b406-5ad71bafded9\") "
Jul 29 13:03:36.296227 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:03:36.296204    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab79463f-d124-4632-b406-5ad71bafded9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ab79463f-d124-4632-b406-5ad71bafded9" (UID: "ab79463f-d124-4632-b406-5ad71bafded9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 13:03:36.396697 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:03:36.396657    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab79463f-d124-4632-b406-5ad71bafded9-maas-api-service-ca\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 13:03:37.030741 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:03:37.030705    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-be3b8a53-f6t66" event={"ID":"ab79463f-d124-4632-b406-5ad71bafded9","Type":"ContainerDied","Data":"efbc3c9b94d9a9584c96d00006a4054d9161f22ac0abc4252fe6473d800f285a"}
Jul 29 13:03:37.030907 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:03:37.030747    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-be3b8a53-f6t66"
Jul 29 13:03:37.030907 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:03:37.030758    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="efbc3c9b94d9a9584c96d00006a4054d9161f22ac0abc4252fe6473d800f285a"
Jul 29 13:03:44.057500 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:03:44.057460    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-be3b8a53-6b4544c455-dfz6n"]
Jul 29 13:03:44.058336 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:03:44.058298    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-be3b8a53-6b4544c455-dfz6n" podUID="b2def39c-491f-4959-a773-a2f0e4764d34" containerName="maas-api" containerID="cri-o://790342191dbc118ba93678ecb8b3469170eddb1ba690b5198e4b03eb875a2f41" gracePeriod=30
Jul 29 13:03:44.251977 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:03:44.251940    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-be3b8a53-f6t66"]
Jul 29 13:03:44.254857 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:03:44.254833    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-be3b8a53-f6t66"]
Jul 29 13:03:44.319474 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:03:44.319412    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-be3b8a53-6b4544c455-dfz6n"
Jul 29 13:03:44.469135 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:03:44.469108    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b2def39c-491f-4959-a773-a2f0e4764d34-maas-api-tls\") pod \"b2def39c-491f-4959-a773-a2f0e4764d34\" (UID: \"b2def39c-491f-4959-a773-a2f0e4764d34\") "
Jul 29 13:03:44.469329 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:03:44.469214    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2def39c-491f-4959-a773-a2f0e4764d34-openshift-service-ca\") pod \"b2def39c-491f-4959-a773-a2f0e4764d34\" (UID: \"b2def39c-491f-4959-a773-a2f0e4764d34\") "
Jul 29 13:03:44.469329 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:03:44.469235    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hpcdx\" (UniqueName: \"kubernetes.io/projected/b2def39c-491f-4959-a773-a2f0e4764d34-kube-api-access-hpcdx\") pod \"b2def39c-491f-4959-a773-a2f0e4764d34\" (UID: \"b2def39c-491f-4959-a773-a2f0e4764d34\") "
Jul 29 13:03:44.469598 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:03:44.469573    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2def39c-491f-4959-a773-a2f0e4764d34-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b2def39c-491f-4959-a773-a2f0e4764d34" (UID: "b2def39c-491f-4959-a773-a2f0e4764d34"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 13:03:44.471255 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:03:44.471229    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b2def39c-491f-4959-a773-a2f0e4764d34-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b2def39c-491f-4959-a773-a2f0e4764d34" (UID: "b2def39c-491f-4959-a773-a2f0e4764d34"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 13:03:44.471364 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:03:44.471345    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2def39c-491f-4959-a773-a2f0e4764d34-kube-api-access-hpcdx" (OuterVolumeSpecName: "kube-api-access-hpcdx") pod "b2def39c-491f-4959-a773-a2f0e4764d34" (UID: "b2def39c-491f-4959-a773-a2f0e4764d34"). InnerVolumeSpecName "kube-api-access-hpcdx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 13:03:44.570559 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:03:44.570468    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2def39c-491f-4959-a773-a2f0e4764d34-openshift-service-ca\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 13:03:44.570559 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:03:44.570501    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hpcdx\" (UniqueName: \"kubernetes.io/projected/b2def39c-491f-4959-a773-a2f0e4764d34-kube-api-access-hpcdx\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 13:03:44.570559 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:03:44.570512    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b2def39c-491f-4959-a773-a2f0e4764d34-maas-api-tls\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 13:03:45.076794 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:03:45.076761    2567 generic.go:356] "Generic (PLEG): container finished" podID="b2def39c-491f-4959-a773-a2f0e4764d34" containerID="790342191dbc118ba93678ecb8b3469170eddb1ba690b5198e4b03eb875a2f41" exitCode=0
Jul 29 13:03:45.077203 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:03:45.076809    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-be3b8a53-6b4544c455-dfz6n"
Jul 29 13:03:45.077203 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:03:45.076845    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-be3b8a53-6b4544c455-dfz6n" event={"ID":"b2def39c-491f-4959-a773-a2f0e4764d34","Type":"ContainerDied","Data":"790342191dbc118ba93678ecb8b3469170eddb1ba690b5198e4b03eb875a2f41"}
Jul 29 13:03:45.077203 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:03:45.076892    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-be3b8a53-6b4544c455-dfz6n" event={"ID":"b2def39c-491f-4959-a773-a2f0e4764d34","Type":"ContainerDied","Data":"125aa401b6910c38ed2363c5e99f2886212495dfd1526294bdf65e597cf3f276"}
Jul 29 13:03:45.077203 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:03:45.076914    2567 scope.go:117] "RemoveContainer" containerID="790342191dbc118ba93678ecb8b3469170eddb1ba690b5198e4b03eb875a2f41"
Jul 29 13:03:45.087153 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:03:45.087137    2567 scope.go:117] "RemoveContainer" containerID="790342191dbc118ba93678ecb8b3469170eddb1ba690b5198e4b03eb875a2f41"
Jul 29 13:03:45.087442 ip-10-0-128-25 kubenswrapper[2567]: E0729 13:03:45.087415    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"790342191dbc118ba93678ecb8b3469170eddb1ba690b5198e4b03eb875a2f41\": container with ID starting with 790342191dbc118ba93678ecb8b3469170eddb1ba690b5198e4b03eb875a2f41 not found: ID does not exist" containerID="790342191dbc118ba93678ecb8b3469170eddb1ba690b5198e4b03eb875a2f41"
Jul 29 13:03:45.087541 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:03:45.087449    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"790342191dbc118ba93678ecb8b3469170eddb1ba690b5198e4b03eb875a2f41"} err="failed to get container status \"790342191dbc118ba93678ecb8b3469170eddb1ba690b5198e4b03eb875a2f41\": rpc error: code = NotFound desc = could not find container \"790342191dbc118ba93678ecb8b3469170eddb1ba690b5198e4b03eb875a2f41\": container with ID starting with 790342191dbc118ba93678ecb8b3469170eddb1ba690b5198e4b03eb875a2f41 not found: ID does not exist"
Jul 29 13:03:45.099544 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:03:45.099506    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-be3b8a53-6b4544c455-dfz6n"]
Jul 29 13:03:45.103167 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:03:45.103145    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-be3b8a53-6b4544c455-dfz6n"]
Jul 29 13:03:45.922068 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:03:45.922037    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab79463f-d124-4632-b406-5ad71bafded9" path="/var/lib/kubelet/pods/ab79463f-d124-4632-b406-5ad71bafded9/volumes"
Jul 29 13:03:45.922453 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:03:45.922439    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2def39c-491f-4959-a773-a2f0e4764d34" path="/var/lib/kubelet/pods/b2def39c-491f-4959-a773-a2f0e4764d34/volumes"
Jul 29 13:04:18.128965 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:04:18.128887    2567 scope.go:117] "RemoveContainer" containerID="d6879196a62bd7e7cdaef21ba6bd55d49fb075c1fd6e5755da979fd3e2b6b02e"
Jul 29 13:04:18.137597 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:04:18.137573    2567 scope.go:117] "RemoveContainer" containerID="96f7b9c4facd0ee547d9cd378be2fc666807a06918616b20a7b9c9049e9b95fb"
Jul 29 13:05:18.078631 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:05:18.078523    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kj82f_a6d5f3a2-26cc-40a7-9841-2fbfc247661a/ovn-acl-logging/0.log"
Jul 29 13:05:18.084002 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:05:18.083982    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kj82f_a6d5f3a2-26cc-40a7-9841-2fbfc247661a/ovn-acl-logging/0.log"
Jul 29 13:05:18.168041 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:05:18.168011    2567 scope.go:117] "RemoveContainer" containerID="95fd6355ec4d048522e9c591de66c4b6b59a07abf497a0a3ab702afda51b30c2"
Jul 29 13:06:10.253032 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:06:10.252986    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-42b01b5c-857bc9b5d7-mlnfc"]
Jul 29 13:06:10.253515 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:06:10.253445    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2def39c-491f-4959-a773-a2f0e4764d34" containerName="maas-api"
Jul 29 13:06:10.253515 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:06:10.253458    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2def39c-491f-4959-a773-a2f0e4764d34" containerName="maas-api"
Jul 29 13:06:10.253515 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:06:10.253466    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab79463f-d124-4632-b406-5ad71bafded9" containerName="revoke-keys"
Jul 29 13:06:10.253515 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:06:10.253471    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab79463f-d124-4632-b406-5ad71bafded9" containerName="revoke-keys"
Jul 29 13:06:10.253646 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:06:10.253545    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2def39c-491f-4959-a773-a2f0e4764d34" containerName="maas-api"
Jul 29 13:06:10.253646 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:06:10.253553    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab79463f-d124-4632-b406-5ad71bafded9" containerName="revoke-keys"
Jul 29 13:06:10.256649 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:06:10.256632    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-42b01b5c-857bc9b5d7-mlnfc"
Jul 29 13:06:10.259132 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:06:10.259112    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-42b01b5c-serving-cert\""
Jul 29 13:06:10.268499 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:06:10.268475    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-42b01b5c-857bc9b5d7-mlnfc"]
Jul 29 13:06:10.385226 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:06:10.385138    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3927226b-1a27-4c65-882b-3721799ec798-maas-api-tls\") pod \"maas-api-e2e-mt-42b01b5c-857bc9b5d7-mlnfc\" (UID: \"3927226b-1a27-4c65-882b-3721799ec798\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-42b01b5c-857bc9b5d7-mlnfc"
Jul 29 13:06:10.385226 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:06:10.385177    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3927226b-1a27-4c65-882b-3721799ec798-openshift-service-ca\") pod \"maas-api-e2e-mt-42b01b5c-857bc9b5d7-mlnfc\" (UID: \"3927226b-1a27-4c65-882b-3721799ec798\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-42b01b5c-857bc9b5d7-mlnfc"
Jul 29 13:06:10.385226 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:06:10.385210    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7w88\" (UniqueName: \"kubernetes.io/projected/3927226b-1a27-4c65-882b-3721799ec798-kube-api-access-w7w88\") pod \"maas-api-e2e-mt-42b01b5c-857bc9b5d7-mlnfc\" (UID: \"3927226b-1a27-4c65-882b-3721799ec798\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-42b01b5c-857bc9b5d7-mlnfc"
Jul 29 13:06:10.485810 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:06:10.485777    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3927226b-1a27-4c65-882b-3721799ec798-maas-api-tls\") pod \"maas-api-e2e-mt-42b01b5c-857bc9b5d7-mlnfc\" (UID: \"3927226b-1a27-4c65-882b-3721799ec798\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-42b01b5c-857bc9b5d7-mlnfc"
Jul 29 13:06:10.486003 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:06:10.485815    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3927226b-1a27-4c65-882b-3721799ec798-openshift-service-ca\") pod \"maas-api-e2e-mt-42b01b5c-857bc9b5d7-mlnfc\" (UID: \"3927226b-1a27-4c65-882b-3721799ec798\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-42b01b5c-857bc9b5d7-mlnfc"
Jul 29 13:06:10.486003 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:06:10.485865    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7w88\" (UniqueName: \"kubernetes.io/projected/3927226b-1a27-4c65-882b-3721799ec798-kube-api-access-w7w88\") pod \"maas-api-e2e-mt-42b01b5c-857bc9b5d7-mlnfc\" (UID: \"3927226b-1a27-4c65-882b-3721799ec798\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-42b01b5c-857bc9b5d7-mlnfc"
Jul 29 13:06:10.486555 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:06:10.486530    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3927226b-1a27-4c65-882b-3721799ec798-openshift-service-ca\") pod \"maas-api-e2e-mt-42b01b5c-857bc9b5d7-mlnfc\" (UID: \"3927226b-1a27-4c65-882b-3721799ec798\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-42b01b5c-857bc9b5d7-mlnfc"
Jul 29 13:06:10.488262 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:06:10.488244    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3927226b-1a27-4c65-882b-3721799ec798-maas-api-tls\") pod \"maas-api-e2e-mt-42b01b5c-857bc9b5d7-mlnfc\" (UID: \"3927226b-1a27-4c65-882b-3721799ec798\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-42b01b5c-857bc9b5d7-mlnfc"
Jul 29 13:06:10.494196 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:06:10.494175    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7w88\" (UniqueName: \"kubernetes.io/projected/3927226b-1a27-4c65-882b-3721799ec798-kube-api-access-w7w88\") pod \"maas-api-e2e-mt-42b01b5c-857bc9b5d7-mlnfc\" (UID: \"3927226b-1a27-4c65-882b-3721799ec798\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-42b01b5c-857bc9b5d7-mlnfc"
Jul 29 13:06:10.567070 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:06:10.567042    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-42b01b5c-857bc9b5d7-mlnfc"
Jul 29 13:06:10.704975 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:06:10.704947    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-42b01b5c-857bc9b5d7-mlnfc"]
Jul 29 13:06:10.706830 ip-10-0-128-25 kubenswrapper[2567]: W0729 13:06:10.706806    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3927226b_1a27_4c65_882b_3721799ec798.slice/crio-22881e0233fc3c1ff49a8d4287a03fda851216630d1cb0b26d74daa7f63de0ab WatchSource:0}: Error finding container 22881e0233fc3c1ff49a8d4287a03fda851216630d1cb0b26d74daa7f63de0ab: Status 404 returned error can't find the container with id 22881e0233fc3c1ff49a8d4287a03fda851216630d1cb0b26d74daa7f63de0ab
Jul 29 13:06:10.920788 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:06:10.920683    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-42b01b5c-857bc9b5d7-mlnfc" event={"ID":"3927226b-1a27-4c65-882b-3721799ec798","Type":"ContainerStarted","Data":"22881e0233fc3c1ff49a8d4287a03fda851216630d1cb0b26d74daa7f63de0ab"}
Jul 29 13:06:12.932837 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:06:12.932794    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-42b01b5c-857bc9b5d7-mlnfc" event={"ID":"3927226b-1a27-4c65-882b-3721799ec798","Type":"ContainerStarted","Data":"9ed7146a3e94deb79341dd52cee16740c93cc971d983393c75265d093bb62e54"}
Jul 29 13:06:12.933293 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:06:12.932853    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-42b01b5c-857bc9b5d7-mlnfc"
Jul 29 13:06:12.949769 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:06:12.949714    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-42b01b5c-857bc9b5d7-mlnfc" podStartSLOduration=1.654116374 podStartE2EDuration="2.949694992s" totalImagesPullingTime="1.295578618s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 13:06:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 13:06:12.949642957 +0000 UTC m=+2455.643331279" watchObservedRunningTime="2026-07-29 13:06:12.949694992 +0000 UTC m=+2455.643383306"
Jul 29 13:06:18.943735 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:06:18.943703    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-42b01b5c-857bc9b5d7-mlnfc"
Jul 29 13:07:09.742286 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:07:09.742246    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-42b01b5c-hcrhb"]
Jul 29 13:07:09.745996 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:07:09.745975    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-42b01b5c-hcrhb"
Jul 29 13:07:09.748473 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:07:09.748444    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7gpd2\""
Jul 29 13:07:09.753039 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:07:09.753007    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-42b01b5c-hcrhb"]
Jul 29 13:07:09.887275 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:07:09.887202    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/91f44458-5435-4fc9-b57a-72717bd70b8e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-42b01b5c-hcrhb\" (UID: \"91f44458-5435-4fc9-b57a-72717bd70b8e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-42b01b5c-hcrhb"
Jul 29 13:07:09.987786 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:07:09.987756    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/91f44458-5435-4fc9-b57a-72717bd70b8e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-42b01b5c-hcrhb\" (UID: \"91f44458-5435-4fc9-b57a-72717bd70b8e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-42b01b5c-hcrhb"
Jul 29 13:07:09.988439 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:07:09.988387    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/91f44458-5435-4fc9-b57a-72717bd70b8e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-42b01b5c-hcrhb\" (UID: \"91f44458-5435-4fc9-b57a-72717bd70b8e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-42b01b5c-hcrhb"
Jul 29 13:07:10.059970 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:07:10.059946    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-42b01b5c-hcrhb"
Jul 29 13:07:10.192015 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:07:10.191992    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-42b01b5c-hcrhb"]
Jul 29 13:07:10.194283 ip-10-0-128-25 kubenswrapper[2567]: W0729 13:07:10.194251    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91f44458_5435_4fc9_b57a_72717bd70b8e.slice/crio-374355cd792d3041521abe708d1a82a55671acc1b41f20c27b7537abf7f31641 WatchSource:0}: Error finding container 374355cd792d3041521abe708d1a82a55671acc1b41f20c27b7537abf7f31641: Status 404 returned error can't find the container with id 374355cd792d3041521abe708d1a82a55671acc1b41f20c27b7537abf7f31641
Jul 29 13:07:10.270541 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:07:10.270516    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-42b01b5c-hcrhb" event={"ID":"91f44458-5435-4fc9-b57a-72717bd70b8e","Type":"ContainerStarted","Data":"374355cd792d3041521abe708d1a82a55671acc1b41f20c27b7537abf7f31641"}
Jul 29 13:07:11.279762 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:07:11.279725    2567 generic.go:356] "Generic (PLEG): container finished" podID="91f44458-5435-4fc9-b57a-72717bd70b8e" containerID="0f0fb7c085b90c15e6718710961dc3d678aafb416cd665466a3cdf36d9a420d6" exitCode=0
Jul 29 13:07:11.280140 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:07:11.279819    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-42b01b5c-hcrhb" event={"ID":"91f44458-5435-4fc9-b57a-72717bd70b8e","Type":"ContainerDied","Data":"0f0fb7c085b90c15e6718710961dc3d678aafb416cd665466a3cdf36d9a420d6"}
Jul 29 13:07:12.421223 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:07:12.421196    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-42b01b5c-hcrhb"
Jul 29 13:07:12.607815 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:07:12.607748    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/91f44458-5435-4fc9-b57a-72717bd70b8e-maas-api-service-ca\") pod \"91f44458-5435-4fc9-b57a-72717bd70b8e\" (UID: \"91f44458-5435-4fc9-b57a-72717bd70b8e\") "
Jul 29 13:07:12.608052 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:07:12.608030    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91f44458-5435-4fc9-b57a-72717bd70b8e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "91f44458-5435-4fc9-b57a-72717bd70b8e" (UID: "91f44458-5435-4fc9-b57a-72717bd70b8e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 13:07:12.708783 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:07:12.708753    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/91f44458-5435-4fc9-b57a-72717bd70b8e-maas-api-service-ca\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 13:07:13.293415 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:07:13.293365    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-42b01b5c-hcrhb" event={"ID":"91f44458-5435-4fc9-b57a-72717bd70b8e","Type":"ContainerDied","Data":"374355cd792d3041521abe708d1a82a55671acc1b41f20c27b7537abf7f31641"}
Jul 29 13:07:13.293415 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:07:13.293396    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-42b01b5c-hcrhb"
Jul 29 13:07:13.293603 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:07:13.293451    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="374355cd792d3041521abe708d1a82a55671acc1b41f20c27b7537abf7f31641"
Jul 29 13:07:25.166470 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:07:25.166435    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-42b01b5c-857bc9b5d7-mlnfc"]
Jul 29 13:07:25.166943 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:07:25.166695    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-42b01b5c-857bc9b5d7-mlnfc" podUID="3927226b-1a27-4c65-882b-3721799ec798" containerName="maas-api" containerID="cri-o://9ed7146a3e94deb79341dd52cee16740c93cc971d983393c75265d093bb62e54" gracePeriod=30
Jul 29 13:07:25.367183 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:07:25.367154    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-42b01b5c-hcrhb"]
Jul 29 13:07:25.369847 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:07:25.369815    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-42b01b5c-hcrhb"]
Jul 29 13:07:25.377054 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:07:25.377028    2567 generic.go:356] "Generic (PLEG): container finished" podID="3927226b-1a27-4c65-882b-3721799ec798" containerID="9ed7146a3e94deb79341dd52cee16740c93cc971d983393c75265d093bb62e54" exitCode=0
Jul 29 13:07:25.377172 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:07:25.377105    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-42b01b5c-857bc9b5d7-mlnfc" event={"ID":"3927226b-1a27-4c65-882b-3721799ec798","Type":"ContainerDied","Data":"9ed7146a3e94deb79341dd52cee16740c93cc971d983393c75265d093bb62e54"}
Jul 29 13:07:25.431709 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:07:25.431687    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-42b01b5c-857bc9b5d7-mlnfc"
Jul 29 13:07:25.523470 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:07:25.523444    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3927226b-1a27-4c65-882b-3721799ec798-maas-api-tls\") pod \"3927226b-1a27-4c65-882b-3721799ec798\" (UID: \"3927226b-1a27-4c65-882b-3721799ec798\") "
Jul 29 13:07:25.523611 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:07:25.523480    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3927226b-1a27-4c65-882b-3721799ec798-openshift-service-ca\") pod \"3927226b-1a27-4c65-882b-3721799ec798\" (UID: \"3927226b-1a27-4c65-882b-3721799ec798\") "
Jul 29 13:07:25.523611 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:07:25.523570    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w7w88\" (UniqueName: \"kubernetes.io/projected/3927226b-1a27-4c65-882b-3721799ec798-kube-api-access-w7w88\") pod \"3927226b-1a27-4c65-882b-3721799ec798\" (UID: \"3927226b-1a27-4c65-882b-3721799ec798\") "
Jul 29 13:07:25.523885 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:07:25.523863    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3927226b-1a27-4c65-882b-3721799ec798-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3927226b-1a27-4c65-882b-3721799ec798" (UID: "3927226b-1a27-4c65-882b-3721799ec798"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 13:07:25.525639 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:07:25.525612    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3927226b-1a27-4c65-882b-3721799ec798-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3927226b-1a27-4c65-882b-3721799ec798" (UID: "3927226b-1a27-4c65-882b-3721799ec798"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 13:07:25.525639 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:07:25.525621    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3927226b-1a27-4c65-882b-3721799ec798-kube-api-access-w7w88" (OuterVolumeSpecName: "kube-api-access-w7w88") pod "3927226b-1a27-4c65-882b-3721799ec798" (UID: "3927226b-1a27-4c65-882b-3721799ec798"). InnerVolumeSpecName "kube-api-access-w7w88". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 13:07:25.624721 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:07:25.624695    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w7w88\" (UniqueName: \"kubernetes.io/projected/3927226b-1a27-4c65-882b-3721799ec798-kube-api-access-w7w88\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 13:07:25.624721 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:07:25.624721    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3927226b-1a27-4c65-882b-3721799ec798-maas-api-tls\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 13:07:25.624857 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:07:25.624730    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3927226b-1a27-4c65-882b-3721799ec798-openshift-service-ca\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 13:07:25.922137 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:07:25.922107    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="91f44458-5435-4fc9-b57a-72717bd70b8e" path="/var/lib/kubelet/pods/91f44458-5435-4fc9-b57a-72717bd70b8e/volumes"
Jul 29 13:07:26.384433 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:07:26.384394    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-42b01b5c-857bc9b5d7-mlnfc"
Jul 29 13:07:26.384821 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:07:26.384393    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-42b01b5c-857bc9b5d7-mlnfc" event={"ID":"3927226b-1a27-4c65-882b-3721799ec798","Type":"ContainerDied","Data":"22881e0233fc3c1ff49a8d4287a03fda851216630d1cb0b26d74daa7f63de0ab"}
Jul 29 13:07:26.384821 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:07:26.384549    2567 scope.go:117] "RemoveContainer" containerID="9ed7146a3e94deb79341dd52cee16740c93cc971d983393c75265d093bb62e54"
Jul 29 13:07:26.401872 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:07:26.401848    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-42b01b5c-857bc9b5d7-mlnfc"]
Jul 29 13:07:26.404986 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:07:26.404966    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-42b01b5c-857bc9b5d7-mlnfc"]
Jul 29 13:07:27.921802 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:07:27.921768    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3927226b-1a27-4c65-882b-3721799ec798" path="/var/lib/kubelet/pods/3927226b-1a27-4c65-882b-3721799ec798/volumes"
Jul 29 13:09:52.223447 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:09:52.223393    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a7e361-78bff44497-l7hkj"]
Jul 29 13:09:52.223884 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:09:52.223787    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3927226b-1a27-4c65-882b-3721799ec798" containerName="maas-api"
Jul 29 13:09:52.223884 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:09:52.223797    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="3927226b-1a27-4c65-882b-3721799ec798" containerName="maas-api"
Jul 29 13:09:52.223884 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:09:52.223810    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91f44458-5435-4fc9-b57a-72717bd70b8e" containerName="revoke-keys"
Jul 29 13:09:52.223884 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:09:52.223816    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="91f44458-5435-4fc9-b57a-72717bd70b8e" containerName="revoke-keys"
Jul 29 13:09:52.223884 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:09:52.223881    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="91f44458-5435-4fc9-b57a-72717bd70b8e" containerName="revoke-keys"
Jul 29 13:09:52.224048 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:09:52.223893    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="3927226b-1a27-4c65-882b-3721799ec798" containerName="maas-api"
Jul 29 13:09:52.227060 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:09:52.227038    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a7e361-78bff44497-l7hkj"
Jul 29 13:09:52.229617 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:09:52.229599    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-a7e361-serving-cert\""
Jul 29 13:09:52.238756 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:09:52.238734    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a7e361-78bff44497-l7hkj"]
Jul 29 13:09:52.306139 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:09:52.306115    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/17e103df-fee9-445b-96e0-3ffcbd91d49f-openshift-service-ca\") pod \"maas-api-e2e-inf-a-a7e361-78bff44497-l7hkj\" (UID: \"17e103df-fee9-445b-96e0-3ffcbd91d49f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a7e361-78bff44497-l7hkj"
Jul 29 13:09:52.306233 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:09:52.306172    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/17e103df-fee9-445b-96e0-3ffcbd91d49f-maas-api-tls\") pod \"maas-api-e2e-inf-a-a7e361-78bff44497-l7hkj\" (UID: \"17e103df-fee9-445b-96e0-3ffcbd91d49f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a7e361-78bff44497-l7hkj"
Jul 29 13:09:52.306233 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:09:52.306224    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxq2d\" (UniqueName: \"kubernetes.io/projected/17e103df-fee9-445b-96e0-3ffcbd91d49f-kube-api-access-fxq2d\") pod \"maas-api-e2e-inf-a-a7e361-78bff44497-l7hkj\" (UID: \"17e103df-fee9-445b-96e0-3ffcbd91d49f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a7e361-78bff44497-l7hkj"
Jul 29 13:09:52.407032 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:09:52.407005    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/17e103df-fee9-445b-96e0-3ffcbd91d49f-maas-api-tls\") pod \"maas-api-e2e-inf-a-a7e361-78bff44497-l7hkj\" (UID: \"17e103df-fee9-445b-96e0-3ffcbd91d49f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a7e361-78bff44497-l7hkj"
Jul 29 13:09:52.407149 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:09:52.407059    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxq2d\" (UniqueName: \"kubernetes.io/projected/17e103df-fee9-445b-96e0-3ffcbd91d49f-kube-api-access-fxq2d\") pod \"maas-api-e2e-inf-a-a7e361-78bff44497-l7hkj\" (UID: \"17e103df-fee9-445b-96e0-3ffcbd91d49f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a7e361-78bff44497-l7hkj"
Jul 29 13:09:52.407149 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:09:52.407137    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/17e103df-fee9-445b-96e0-3ffcbd91d49f-openshift-service-ca\") pod \"maas-api-e2e-inf-a-a7e361-78bff44497-l7hkj\" (UID: \"17e103df-fee9-445b-96e0-3ffcbd91d49f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a7e361-78bff44497-l7hkj"
Jul 29 13:09:52.407234 ip-10-0-128-25 kubenswrapper[2567]: E0729 13:09:52.407156    2567 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-a7e361-serving-cert: secret "maas-api-e2e-inf-a-a7e361-serving-cert" not found
Jul 29 13:09:52.407234 ip-10-0-128-25 kubenswrapper[2567]: E0729 13:09:52.407225    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17e103df-fee9-445b-96e0-3ffcbd91d49f-maas-api-tls podName:17e103df-fee9-445b-96e0-3ffcbd91d49f nodeName:}" failed. No retries permitted until 2026-07-29 13:09:52.907206854 +0000 UTC m=+2675.600895155 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/17e103df-fee9-445b-96e0-3ffcbd91d49f-maas-api-tls") pod "maas-api-e2e-inf-a-a7e361-78bff44497-l7hkj" (UID: "17e103df-fee9-445b-96e0-3ffcbd91d49f") : secret "maas-api-e2e-inf-a-a7e361-serving-cert" not found
Jul 29 13:09:52.407736 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:09:52.407718    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/17e103df-fee9-445b-96e0-3ffcbd91d49f-openshift-service-ca\") pod \"maas-api-e2e-inf-a-a7e361-78bff44497-l7hkj\" (UID: \"17e103df-fee9-445b-96e0-3ffcbd91d49f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a7e361-78bff44497-l7hkj"
Jul 29 13:09:52.418837 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:09:52.418817    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxq2d\" (UniqueName: \"kubernetes.io/projected/17e103df-fee9-445b-96e0-3ffcbd91d49f-kube-api-access-fxq2d\") pod \"maas-api-e2e-inf-a-a7e361-78bff44497-l7hkj\" (UID: \"17e103df-fee9-445b-96e0-3ffcbd91d49f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a7e361-78bff44497-l7hkj"
Jul 29 13:09:52.910983 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:09:52.910953    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/17e103df-fee9-445b-96e0-3ffcbd91d49f-maas-api-tls\") pod \"maas-api-e2e-inf-a-a7e361-78bff44497-l7hkj\" (UID: \"17e103df-fee9-445b-96e0-3ffcbd91d49f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a7e361-78bff44497-l7hkj"
Jul 29 13:09:52.913147 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:09:52.913129    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/17e103df-fee9-445b-96e0-3ffcbd91d49f-maas-api-tls\") pod \"maas-api-e2e-inf-a-a7e361-78bff44497-l7hkj\" (UID: \"17e103df-fee9-445b-96e0-3ffcbd91d49f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a7e361-78bff44497-l7hkj"
Jul 29 13:09:53.139373 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:09:53.139345    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a7e361-78bff44497-l7hkj"
Jul 29 13:09:53.483464 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:09:53.483437    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a7e361-78bff44497-l7hkj"]
Jul 29 13:09:53.484574 ip-10-0-128-25 kubenswrapper[2567]: W0729 13:09:53.484548    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17e103df_fee9_445b_96e0_3ffcbd91d49f.slice/crio-32ca41919bb4f3b4beea58e6d12ab8e28a1cc79f507cd81bee24543e85a934fe WatchSource:0}: Error finding container 32ca41919bb4f3b4beea58e6d12ab8e28a1cc79f507cd81bee24543e85a934fe: Status 404 returned error can't find the container with id 32ca41919bb4f3b4beea58e6d12ab8e28a1cc79f507cd81bee24543e85a934fe
Jul 29 13:09:53.485776 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:09:53.485759    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 13:09:54.279533 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:09:54.279503    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a7e361-78bff44497-l7hkj" event={"ID":"17e103df-fee9-445b-96e0-3ffcbd91d49f","Type":"ContainerStarted","Data":"a29f087fc325c60e31d1f89344bd134f090bb5473eb35aa71def86d96a93f97a"}
Jul 29 13:09:54.279714 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:09:54.279549    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a7e361-78bff44497-l7hkj" event={"ID":"17e103df-fee9-445b-96e0-3ffcbd91d49f","Type":"ContainerStarted","Data":"32ca41919bb4f3b4beea58e6d12ab8e28a1cc79f507cd81bee24543e85a934fe"}
Jul 29 13:09:54.279714 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:09:54.279619    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a7e361-78bff44497-l7hkj"
Jul 29 13:09:54.295858 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:09:54.295808    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a7e361-78bff44497-l7hkj" podStartSLOduration=1.898886374 podStartE2EDuration="2.295795268s" totalImagesPullingTime="396.908894ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 13:09:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 13:09:54.295662283 +0000 UTC m=+2676.989350616" watchObservedRunningTime="2026-07-29 13:09:54.295795268 +0000 UTC m=+2676.989483591"
Jul 29 13:10:00.291542 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:10:00.291507    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a7e361-78bff44497-l7hkj"
Jul 29 13:10:18.120232 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:10:18.120123    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kj82f_a6d5f3a2-26cc-40a7-9841-2fbfc247661a/ovn-acl-logging/0.log"
Jul 29 13:10:18.137600 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:10:18.126756    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kj82f_a6d5f3a2-26cc-40a7-9841-2fbfc247661a/ovn-acl-logging/0.log"
Jul 29 13:10:18.227964 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:10:18.227935    2567 scope.go:117] "RemoveContainer" containerID="e5c37575d64956775651f398d090e1e31c5824dfefc7f96a1598b8630de1ea4f"
Jul 29 13:10:21.474160 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:10:21.474121    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ef0894-75dbdc98df-xzcl7"]
Jul 29 13:10:21.477650 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:10:21.477627    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ef0894-75dbdc98df-xzcl7"
Jul 29 13:10:21.481130 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:10:21.481104    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-ef0894-serving-cert\""
Jul 29 13:10:21.498055 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:10:21.498029    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ef0894-75dbdc98df-xzcl7"]
Jul 29 13:10:21.547370 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:10:21.547346    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4a39b6bf-1003-4636-9452-353923f29171-maas-api-tls\") pod \"maas-api-e2e-inf-b-ef0894-75dbdc98df-xzcl7\" (UID: \"4a39b6bf-1003-4636-9452-353923f29171\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ef0894-75dbdc98df-xzcl7"
Jul 29 13:10:21.547495 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:10:21.547430    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a39b6bf-1003-4636-9452-353923f29171-openshift-service-ca\") pod \"maas-api-e2e-inf-b-ef0894-75dbdc98df-xzcl7\" (UID: \"4a39b6bf-1003-4636-9452-353923f29171\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ef0894-75dbdc98df-xzcl7"
Jul 29 13:10:21.547495 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:10:21.547454    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4pcv\" (UniqueName: \"kubernetes.io/projected/4a39b6bf-1003-4636-9452-353923f29171-kube-api-access-m4pcv\") pod \"maas-api-e2e-inf-b-ef0894-75dbdc98df-xzcl7\" (UID: \"4a39b6bf-1003-4636-9452-353923f29171\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ef0894-75dbdc98df-xzcl7"
Jul 29 13:10:21.647853 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:10:21.647827    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4a39b6bf-1003-4636-9452-353923f29171-maas-api-tls\") pod \"maas-api-e2e-inf-b-ef0894-75dbdc98df-xzcl7\" (UID: \"4a39b6bf-1003-4636-9452-353923f29171\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ef0894-75dbdc98df-xzcl7"
Jul 29 13:10:21.647971 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:10:21.647865    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a39b6bf-1003-4636-9452-353923f29171-openshift-service-ca\") pod \"maas-api-e2e-inf-b-ef0894-75dbdc98df-xzcl7\" (UID: \"4a39b6bf-1003-4636-9452-353923f29171\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ef0894-75dbdc98df-xzcl7"
Jul 29 13:10:21.647971 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:10:21.647883    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4pcv\" (UniqueName: \"kubernetes.io/projected/4a39b6bf-1003-4636-9452-353923f29171-kube-api-access-m4pcv\") pod \"maas-api-e2e-inf-b-ef0894-75dbdc98df-xzcl7\" (UID: \"4a39b6bf-1003-4636-9452-353923f29171\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ef0894-75dbdc98df-xzcl7"
Jul 29 13:10:21.647971 ip-10-0-128-25 kubenswrapper[2567]: E0729 13:10:21.647964    2567 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-ef0894-serving-cert: secret "maas-api-e2e-inf-b-ef0894-serving-cert" not found
Jul 29 13:10:21.648137 ip-10-0-128-25 kubenswrapper[2567]: E0729 13:10:21.648034    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a39b6bf-1003-4636-9452-353923f29171-maas-api-tls podName:4a39b6bf-1003-4636-9452-353923f29171 nodeName:}" failed. No retries permitted until 2026-07-29 13:10:22.148018311 +0000 UTC m=+2704.841706601 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4a39b6bf-1003-4636-9452-353923f29171-maas-api-tls") pod "maas-api-e2e-inf-b-ef0894-75dbdc98df-xzcl7" (UID: "4a39b6bf-1003-4636-9452-353923f29171") : secret "maas-api-e2e-inf-b-ef0894-serving-cert" not found
Jul 29 13:10:21.648551 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:10:21.648530    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a39b6bf-1003-4636-9452-353923f29171-openshift-service-ca\") pod \"maas-api-e2e-inf-b-ef0894-75dbdc98df-xzcl7\" (UID: \"4a39b6bf-1003-4636-9452-353923f29171\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ef0894-75dbdc98df-xzcl7"
Jul 29 13:10:21.658704 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:10:21.658682    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4pcv\" (UniqueName: \"kubernetes.io/projected/4a39b6bf-1003-4636-9452-353923f29171-kube-api-access-m4pcv\") pod \"maas-api-e2e-inf-b-ef0894-75dbdc98df-xzcl7\" (UID: \"4a39b6bf-1003-4636-9452-353923f29171\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ef0894-75dbdc98df-xzcl7"
Jul 29 13:10:22.151206 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:10:22.151169    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4a39b6bf-1003-4636-9452-353923f29171-maas-api-tls\") pod \"maas-api-e2e-inf-b-ef0894-75dbdc98df-xzcl7\" (UID: \"4a39b6bf-1003-4636-9452-353923f29171\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ef0894-75dbdc98df-xzcl7"
Jul 29 13:10:22.153438 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:10:22.153420    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4a39b6bf-1003-4636-9452-353923f29171-maas-api-tls\") pod \"maas-api-e2e-inf-b-ef0894-75dbdc98df-xzcl7\" (UID: \"4a39b6bf-1003-4636-9452-353923f29171\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ef0894-75dbdc98df-xzcl7"
Jul 29 13:10:22.235063 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:10:22.235035    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-ef0894-667f4d77b8-lkx8f"]
Jul 29 13:10:22.237706 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:10:22.237681    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-ef0894-667f4d77b8-lkx8f"
Jul 29 13:10:22.240717 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:10:22.240694    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-ef0894\""
Jul 29 13:10:22.240842 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:10:22.240804    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-ef0894-dockercfg-jkfjf\""
Jul 29 13:10:22.249946 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:10:22.249926    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-ef0894-667f4d77b8-lkx8f"]
Jul 29 13:10:22.252351 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:10:22.252328    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ltdr\" (UniqueName: \"kubernetes.io/projected/3b2bef17-4396-4a4c-9d83-2b845db4ae73-kube-api-access-4ltdr\") pod \"payload-processing-e2e-inf-b-ef0894-667f4d77b8-lkx8f\" (UID: \"3b2bef17-4396-4a4c-9d83-2b845db4ae73\") " pod="openshift-ingress/payload-processing-e2e-inf-b-ef0894-667f4d77b8-lkx8f"
Jul 29 13:10:22.252462 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:10:22.252383    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b2bef17-4396-4a4c-9d83-2b845db4ae73-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-ef0894-667f4d77b8-lkx8f\" (UID: \"3b2bef17-4396-4a4c-9d83-2b845db4ae73\") " pod="openshift-ingress/payload-processing-e2e-inf-b-ef0894-667f4d77b8-lkx8f"
Jul 29 13:10:22.352827 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:10:22.352797    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b2bef17-4396-4a4c-9d83-2b845db4ae73-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-ef0894-667f4d77b8-lkx8f\" (UID: \"3b2bef17-4396-4a4c-9d83-2b845db4ae73\") " pod="openshift-ingress/payload-processing-e2e-inf-b-ef0894-667f4d77b8-lkx8f"
Jul 29 13:10:22.352969 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:10:22.352899    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ltdr\" (UniqueName: \"kubernetes.io/projected/3b2bef17-4396-4a4c-9d83-2b845db4ae73-kube-api-access-4ltdr\") pod \"payload-processing-e2e-inf-b-ef0894-667f4d77b8-lkx8f\" (UID: \"3b2bef17-4396-4a4c-9d83-2b845db4ae73\") " pod="openshift-ingress/payload-processing-e2e-inf-b-ef0894-667f4d77b8-lkx8f"
Jul 29 13:10:22.353481 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:10:22.353463    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b2bef17-4396-4a4c-9d83-2b845db4ae73-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-ef0894-667f4d77b8-lkx8f\" (UID: \"3b2bef17-4396-4a4c-9d83-2b845db4ae73\") " pod="openshift-ingress/payload-processing-e2e-inf-b-ef0894-667f4d77b8-lkx8f"
Jul 29 13:10:22.360011 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:10:22.359984    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ltdr\" (UniqueName: \"kubernetes.io/projected/3b2bef17-4396-4a4c-9d83-2b845db4ae73-kube-api-access-4ltdr\") pod \"payload-processing-e2e-inf-b-ef0894-667f4d77b8-lkx8f\" (UID: \"3b2bef17-4396-4a4c-9d83-2b845db4ae73\") " pod="openshift-ingress/payload-processing-e2e-inf-b-ef0894-667f4d77b8-lkx8f"
Jul 29 13:10:22.387666 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:10:22.387641    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ef0894-75dbdc98df-xzcl7"
Jul 29 13:10:22.524913 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:10:22.524888    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ef0894-75dbdc98df-xzcl7"]
Jul 29 13:10:22.525578 ip-10-0-128-25 kubenswrapper[2567]: W0729 13:10:22.525548    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a39b6bf_1003_4636_9452_353923f29171.slice/crio-bb12e5386e35908bcd3aa2fa4210b4fc0a95238678c6c21d5bca8b65e78b5a67 WatchSource:0}: Error finding container bb12e5386e35908bcd3aa2fa4210b4fc0a95238678c6c21d5bca8b65e78b5a67: Status 404 returned error can't find the container with id bb12e5386e35908bcd3aa2fa4210b4fc0a95238678c6c21d5bca8b65e78b5a67
Jul 29 13:10:22.550564 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:10:22.550540    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-ef0894-667f4d77b8-lkx8f"
Jul 29 13:10:22.681532 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:10:22.681504    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-ef0894-667f4d77b8-lkx8f"]
Jul 29 13:10:22.682513 ip-10-0-128-25 kubenswrapper[2567]: W0729 13:10:22.682487    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b2bef17_4396_4a4c_9d83_2b845db4ae73.slice/crio-5b508e7b12b50f210f6828126530299c663005563f22cfee13149901bff7abba WatchSource:0}: Error finding container 5b508e7b12b50f210f6828126530299c663005563f22cfee13149901bff7abba: Status 404 returned error can't find the container with id 5b508e7b12b50f210f6828126530299c663005563f22cfee13149901bff7abba
Jul 29 13:10:23.457202 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:10:23.457171    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-ef0894-667f4d77b8-lkx8f" event={"ID":"3b2bef17-4396-4a4c-9d83-2b845db4ae73","Type":"ContainerStarted","Data":"2c2273a2057a0b61fe87e2484e8555dd8645f1f7324149b7891162864c2808d8"}
Jul 29 13:10:23.457629 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:10:23.457230    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-ef0894-667f4d77b8-lkx8f"
Jul 29 13:10:23.457629 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:10:23.457248    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-ef0894-667f4d77b8-lkx8f" event={"ID":"3b2bef17-4396-4a4c-9d83-2b845db4ae73","Type":"ContainerStarted","Data":"5b508e7b12b50f210f6828126530299c663005563f22cfee13149901bff7abba"}
Jul 29 13:10:23.458686 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:10:23.458665    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ef0894-75dbdc98df-xzcl7" event={"ID":"4a39b6bf-1003-4636-9452-353923f29171","Type":"ContainerStarted","Data":"bb533f360fa4a9ecf2c950146a31fc8ebbf381816f3b096031df2d4c948eacd2"}
Jul 29 13:10:23.458790 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:10:23.458707    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ef0894-75dbdc98df-xzcl7"
Jul 29 13:10:23.458790 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:10:23.458722    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ef0894-75dbdc98df-xzcl7" event={"ID":"4a39b6bf-1003-4636-9452-353923f29171","Type":"ContainerStarted","Data":"bb12e5386e35908bcd3aa2fa4210b4fc0a95238678c6c21d5bca8b65e78b5a67"}
Jul 29 13:10:23.477146 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:10:23.476956    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-ef0894-667f4d77b8-lkx8f" podStartSLOduration=1.476939645 podStartE2EDuration="1.476939645s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 13:10:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 13:10:23.473839161 +0000 UTC m=+2706.167527476" watchObservedRunningTime="2026-07-29 13:10:23.476939645 +0000 UTC m=+2706.170627957"
Jul 29 13:10:23.495962 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:10:23.495914    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ef0894-75dbdc98df-xzcl7" podStartSLOduration=1.691519153 podStartE2EDuration="2.495900266s" totalImagesPullingTime="804.381113ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 13:10:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 13:10:23.493349468 +0000 UTC m=+2706.187037781" watchObservedRunningTime="2026-07-29 13:10:23.495900266 +0000 UTC m=+2706.189588614"
Jul 29 13:10:29.471007 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:10:29.470977    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ef0894-75dbdc98df-xzcl7"
Jul 29 13:10:34.466951 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:10:34.466921    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-ef0894-667f4d77b8-lkx8f"
Jul 29 13:12:07.584976 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:07.584940    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a7e361-sxll2"]
Jul 29 13:12:07.588517 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:07.588499    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a7e361-sxll2"
Jul 29 13:12:07.592018 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:07.591997    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7gpd2\""
Jul 29 13:12:07.600373 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:07.600349    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a7e361-sxll2"]
Jul 29 13:12:07.678525 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:07.678498    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c2c9ea6-3db5-4764-a654-77c16119ce68-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-a7e361-sxll2\" (UID: \"5c2c9ea6-3db5-4764-a654-77c16119ce68\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a7e361-sxll2"
Jul 29 13:12:07.779805 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:07.779777    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c2c9ea6-3db5-4764-a654-77c16119ce68-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-a7e361-sxll2\" (UID: \"5c2c9ea6-3db5-4764-a654-77c16119ce68\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a7e361-sxll2"
Jul 29 13:12:07.780334 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:07.780317    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c2c9ea6-3db5-4764-a654-77c16119ce68-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-a7e361-sxll2\" (UID: \"5c2c9ea6-3db5-4764-a654-77c16119ce68\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a7e361-sxll2"
Jul 29 13:12:07.899585 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:07.899524    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a7e361-sxll2"
Jul 29 13:12:08.043050 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:08.043020    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a7e361-sxll2"]
Jul 29 13:12:08.043665 ip-10-0-128-25 kubenswrapper[2567]: W0729 13:12:08.043640    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c2c9ea6_3db5_4764_a654_77c16119ce68.slice/crio-a5a1b527320511259600d66a93af52d5a87e9cc0098ae7a62f5b968e3493abd3 WatchSource:0}: Error finding container a5a1b527320511259600d66a93af52d5a87e9cc0098ae7a62f5b968e3493abd3: Status 404 returned error can't find the container with id a5a1b527320511259600d66a93af52d5a87e9cc0098ae7a62f5b968e3493abd3
Jul 29 13:12:08.108966 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:08.108940    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a7e361-sxll2" event={"ID":"5c2c9ea6-3db5-4764-a654-77c16119ce68","Type":"ContainerStarted","Data":"a5a1b527320511259600d66a93af52d5a87e9cc0098ae7a62f5b968e3493abd3"}
Jul 29 13:12:09.121642 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:09.121606    2567 generic.go:356] "Generic (PLEG): container finished" podID="5c2c9ea6-3db5-4764-a654-77c16119ce68" containerID="3ff7cc42d758755e4233dd7f055f4d2ca6b255d906cb283ded5988751129f7ba" exitCode=0
Jul 29 13:12:09.121984 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:09.121692    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a7e361-sxll2" event={"ID":"5c2c9ea6-3db5-4764-a654-77c16119ce68","Type":"ContainerDied","Data":"3ff7cc42d758755e4233dd7f055f4d2ca6b255d906cb283ded5988751129f7ba"}
Jul 29 13:12:10.268432 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:10.268396    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a7e361-sxll2"
Jul 29 13:12:10.408060 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:10.407987    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c2c9ea6-3db5-4764-a654-77c16119ce68-maas-api-service-ca\") pod \"5c2c9ea6-3db5-4764-a654-77c16119ce68\" (UID: \"5c2c9ea6-3db5-4764-a654-77c16119ce68\") "
Jul 29 13:12:10.408312 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:10.408290    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c2c9ea6-3db5-4764-a654-77c16119ce68-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5c2c9ea6-3db5-4764-a654-77c16119ce68" (UID: "5c2c9ea6-3db5-4764-a654-77c16119ce68"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 13:12:10.509262 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:10.509234    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c2c9ea6-3db5-4764-a654-77c16119ce68-maas-api-service-ca\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 13:12:11.135946 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:11.135921    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a7e361-sxll2"
Jul 29 13:12:11.136112 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:11.135957    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a7e361-sxll2" event={"ID":"5c2c9ea6-3db5-4764-a654-77c16119ce68","Type":"ContainerDied","Data":"a5a1b527320511259600d66a93af52d5a87e9cc0098ae7a62f5b968e3493abd3"}
Jul 29 13:12:11.136112 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:11.136001    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a5a1b527320511259600d66a93af52d5a87e9cc0098ae7a62f5b968e3493abd3"
Jul 29 13:12:18.126354 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:18.126322    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a7e361-78bff44497-l7hkj"]
Jul 29 13:12:18.126786 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:18.126586    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a7e361-78bff44497-l7hkj" podUID="17e103df-fee9-445b-96e0-3ffcbd91d49f" containerName="maas-api" containerID="cri-o://a29f087fc325c60e31d1f89344bd134f090bb5473eb35aa71def86d96a93f97a" gracePeriod=30
Jul 29 13:12:18.388007 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:18.387952    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a7e361-78bff44497-l7hkj"
Jul 29 13:12:18.394962 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:18.394935    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a7e361-sxll2"]
Jul 29 13:12:18.398573 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:18.398550    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a7e361-sxll2"]
Jul 29 13:12:18.473807 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:18.473784    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/17e103df-fee9-445b-96e0-3ffcbd91d49f-maas-api-tls\") pod \"17e103df-fee9-445b-96e0-3ffcbd91d49f\" (UID: \"17e103df-fee9-445b-96e0-3ffcbd91d49f\") "
Jul 29 13:12:18.473918 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:18.473821    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fxq2d\" (UniqueName: \"kubernetes.io/projected/17e103df-fee9-445b-96e0-3ffcbd91d49f-kube-api-access-fxq2d\") pod \"17e103df-fee9-445b-96e0-3ffcbd91d49f\" (UID: \"17e103df-fee9-445b-96e0-3ffcbd91d49f\") "
Jul 29 13:12:18.473918 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:18.473899    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/17e103df-fee9-445b-96e0-3ffcbd91d49f-openshift-service-ca\") pod \"17e103df-fee9-445b-96e0-3ffcbd91d49f\" (UID: \"17e103df-fee9-445b-96e0-3ffcbd91d49f\") "
Jul 29 13:12:18.474288 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:18.474257    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17e103df-fee9-445b-96e0-3ffcbd91d49f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "17e103df-fee9-445b-96e0-3ffcbd91d49f" (UID: "17e103df-fee9-445b-96e0-3ffcbd91d49f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 13:12:18.475881 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:18.475854    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17e103df-fee9-445b-96e0-3ffcbd91d49f-kube-api-access-fxq2d" (OuterVolumeSpecName: "kube-api-access-fxq2d") pod "17e103df-fee9-445b-96e0-3ffcbd91d49f" (UID: "17e103df-fee9-445b-96e0-3ffcbd91d49f"). InnerVolumeSpecName "kube-api-access-fxq2d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 13:12:18.475986 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:18.475917    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/17e103df-fee9-445b-96e0-3ffcbd91d49f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "17e103df-fee9-445b-96e0-3ffcbd91d49f" (UID: "17e103df-fee9-445b-96e0-3ffcbd91d49f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 13:12:18.575287 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:18.575264    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/17e103df-fee9-445b-96e0-3ffcbd91d49f-maas-api-tls\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 13:12:18.575287 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:18.575286    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fxq2d\" (UniqueName: \"kubernetes.io/projected/17e103df-fee9-445b-96e0-3ffcbd91d49f-kube-api-access-fxq2d\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 13:12:18.575486 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:18.575297    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/17e103df-fee9-445b-96e0-3ffcbd91d49f-openshift-service-ca\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 13:12:19.183746 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:19.183714    2567 generic.go:356] "Generic (PLEG): container finished" podID="17e103df-fee9-445b-96e0-3ffcbd91d49f" containerID="a29f087fc325c60e31d1f89344bd134f090bb5473eb35aa71def86d96a93f97a" exitCode=0
Jul 29 13:12:19.184119 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:19.183757    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a7e361-78bff44497-l7hkj"
Jul 29 13:12:19.184119 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:19.183760    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a7e361-78bff44497-l7hkj" event={"ID":"17e103df-fee9-445b-96e0-3ffcbd91d49f","Type":"ContainerDied","Data":"a29f087fc325c60e31d1f89344bd134f090bb5473eb35aa71def86d96a93f97a"}
Jul 29 13:12:19.184119 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:19.183812    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a7e361-78bff44497-l7hkj" event={"ID":"17e103df-fee9-445b-96e0-3ffcbd91d49f","Type":"ContainerDied","Data":"32ca41919bb4f3b4beea58e6d12ab8e28a1cc79f507cd81bee24543e85a934fe"}
Jul 29 13:12:19.184119 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:19.183834    2567 scope.go:117] "RemoveContainer" containerID="a29f087fc325c60e31d1f89344bd134f090bb5473eb35aa71def86d96a93f97a"
Jul 29 13:12:19.193853 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:19.193834    2567 scope.go:117] "RemoveContainer" containerID="a29f087fc325c60e31d1f89344bd134f090bb5473eb35aa71def86d96a93f97a"
Jul 29 13:12:19.194085 ip-10-0-128-25 kubenswrapper[2567]: E0729 13:12:19.194067    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a29f087fc325c60e31d1f89344bd134f090bb5473eb35aa71def86d96a93f97a\": container with ID starting with a29f087fc325c60e31d1f89344bd134f090bb5473eb35aa71def86d96a93f97a not found: ID does not exist" containerID="a29f087fc325c60e31d1f89344bd134f090bb5473eb35aa71def86d96a93f97a"
Jul 29 13:12:19.194127 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:19.194093    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a29f087fc325c60e31d1f89344bd134f090bb5473eb35aa71def86d96a93f97a"} err="failed to get container status \"a29f087fc325c60e31d1f89344bd134f090bb5473eb35aa71def86d96a93f97a\": rpc error: code = NotFound desc = could not find container \"a29f087fc325c60e31d1f89344bd134f090bb5473eb35aa71def86d96a93f97a\": container with ID starting with a29f087fc325c60e31d1f89344bd134f090bb5473eb35aa71def86d96a93f97a not found: ID does not exist"
Jul 29 13:12:19.206851 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:19.206828    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a7e361-78bff44497-l7hkj"]
Jul 29 13:12:19.210930 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:19.210910    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a7e361-78bff44497-l7hkj"]
Jul 29 13:12:19.923352 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:19.923308    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="17e103df-fee9-445b-96e0-3ffcbd91d49f" path="/var/lib/kubelet/pods/17e103df-fee9-445b-96e0-3ffcbd91d49f/volumes"
Jul 29 13:12:19.923693 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:19.923675    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c2c9ea6-3db5-4764-a654-77c16119ce68" path="/var/lib/kubelet/pods/5c2c9ea6-3db5-4764-a654-77c16119ce68/volumes"
Jul 29 13:12:25.333584 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:25.333547    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ef0894-8bslw"]
Jul 29 13:12:25.333966 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:25.333953    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17e103df-fee9-445b-96e0-3ffcbd91d49f" containerName="maas-api"
Jul 29 13:12:25.333966 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:25.333966    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="17e103df-fee9-445b-96e0-3ffcbd91d49f" containerName="maas-api"
Jul 29 13:12:25.334044 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:25.333982    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c2c9ea6-3db5-4764-a654-77c16119ce68" containerName="revoke-keys"
Jul 29 13:12:25.334044 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:25.333987    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c2c9ea6-3db5-4764-a654-77c16119ce68" containerName="revoke-keys"
Jul 29 13:12:25.334106 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:25.334060    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="17e103df-fee9-445b-96e0-3ffcbd91d49f" containerName="maas-api"
Jul 29 13:12:25.334106 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:25.334072    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c2c9ea6-3db5-4764-a654-77c16119ce68" containerName="revoke-keys"
Jul 29 13:12:25.338818 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:25.338799    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ef0894-8bslw"
Jul 29 13:12:25.341262 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:25.341238    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7gpd2\""
Jul 29 13:12:25.345536 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:25.345508    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ef0894-8bslw"]
Jul 29 13:12:25.428081 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:25.428058    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f579251f-b38f-4585-a27e-1a110339c5fa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-ef0894-8bslw\" (UID: \"f579251f-b38f-4585-a27e-1a110339c5fa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ef0894-8bslw"
Jul 29 13:12:25.528700 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:25.528679    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f579251f-b38f-4585-a27e-1a110339c5fa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-ef0894-8bslw\" (UID: \"f579251f-b38f-4585-a27e-1a110339c5fa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ef0894-8bslw"
Jul 29 13:12:25.529300 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:25.529280    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f579251f-b38f-4585-a27e-1a110339c5fa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-ef0894-8bslw\" (UID: \"f579251f-b38f-4585-a27e-1a110339c5fa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ef0894-8bslw"
Jul 29 13:12:25.651393 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:25.651328    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ef0894-8bslw"
Jul 29 13:12:25.793795 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:25.793766    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ef0894-8bslw"]
Jul 29 13:12:25.794979 ip-10-0-128-25 kubenswrapper[2567]: W0729 13:12:25.794954    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf579251f_b38f_4585_a27e_1a110339c5fa.slice/crio-d87315563194d5cc9e24ec22d909b9f81cf275690e50457325ea10a5eb516f02 WatchSource:0}: Error finding container d87315563194d5cc9e24ec22d909b9f81cf275690e50457325ea10a5eb516f02: Status 404 returned error can't find the container with id d87315563194d5cc9e24ec22d909b9f81cf275690e50457325ea10a5eb516f02
Jul 29 13:12:26.229507 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:26.229475    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ef0894-8bslw" event={"ID":"f579251f-b38f-4585-a27e-1a110339c5fa","Type":"ContainerStarted","Data":"e620e69ef5871a07cc4b4cc066039ae9f8e1c1308cd629ca55c1ded6cf2bdd3d"}
Jul 29 13:12:26.229624 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:26.229558    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ef0894-8bslw" event={"ID":"f579251f-b38f-4585-a27e-1a110339c5fa","Type":"ContainerStarted","Data":"d87315563194d5cc9e24ec22d909b9f81cf275690e50457325ea10a5eb516f02"}
Jul 29 13:12:26.250979 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:26.250938    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ef0894-8bslw" podStartSLOduration=1.250925142 podStartE2EDuration="1.250925142s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 13:12:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 13:12:26.25016866 +0000 UTC m=+2828.943856998" watchObservedRunningTime="2026-07-29 13:12:26.250925142 +0000 UTC m=+2828.944613452"
Jul 29 13:12:27.237535 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:27.237498    2567 generic.go:356] "Generic (PLEG): container finished" podID="f579251f-b38f-4585-a27e-1a110339c5fa" containerID="e620e69ef5871a07cc4b4cc066039ae9f8e1c1308cd629ca55c1ded6cf2bdd3d" exitCode=0
Jul 29 13:12:27.237952 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:27.237589    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ef0894-8bslw" event={"ID":"f579251f-b38f-4585-a27e-1a110339c5fa","Type":"ContainerDied","Data":"e620e69ef5871a07cc4b4cc066039ae9f8e1c1308cd629ca55c1ded6cf2bdd3d"}
Jul 29 13:12:28.386578 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:28.386556    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ef0894-8bslw"
Jul 29 13:12:28.551889 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:28.551863    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f579251f-b38f-4585-a27e-1a110339c5fa-maas-api-service-ca\") pod \"f579251f-b38f-4585-a27e-1a110339c5fa\" (UID: \"f579251f-b38f-4585-a27e-1a110339c5fa\") "
Jul 29 13:12:28.552244 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:28.552221    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f579251f-b38f-4585-a27e-1a110339c5fa-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f579251f-b38f-4585-a27e-1a110339c5fa" (UID: "f579251f-b38f-4585-a27e-1a110339c5fa"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 13:12:28.652724 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:28.652698    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f579251f-b38f-4585-a27e-1a110339c5fa-maas-api-service-ca\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 13:12:29.251739 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:29.251707    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ef0894-8bslw"
Jul 29 13:12:29.251929 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:29.251743    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ef0894-8bslw" event={"ID":"f579251f-b38f-4585-a27e-1a110339c5fa","Type":"ContainerDied","Data":"d87315563194d5cc9e24ec22d909b9f81cf275690e50457325ea10a5eb516f02"}
Jul 29 13:12:29.251929 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:29.251793    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d87315563194d5cc9e24ec22d909b9f81cf275690e50457325ea10a5eb516f02"
Jul 29 13:12:35.808446 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:35.808387    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ef0894-75dbdc98df-xzcl7"]
Jul 29 13:12:35.808992 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:35.808677    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ef0894-75dbdc98df-xzcl7" podUID="4a39b6bf-1003-4636-9452-353923f29171" containerName="maas-api" containerID="cri-o://bb533f360fa4a9ecf2c950146a31fc8ebbf381816f3b096031df2d4c948eacd2" gracePeriod=30
Jul 29 13:12:35.880634 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:35.880605    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-ef0894-667f4d77b8-lkx8f"]
Jul 29 13:12:35.880847 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:35.880820    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-ef0894-667f4d77b8-lkx8f" podUID="3b2bef17-4396-4a4c-9d83-2b845db4ae73" containerName="payload-processing" containerID="cri-o://2c2273a2057a0b61fe87e2484e8555dd8645f1f7324149b7891162864c2808d8" gracePeriod=30
Jul 29 13:12:36.054078 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:36.054037    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ef0894-8bslw"]
Jul 29 13:12:36.058961 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:36.058863    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ef0894-8bslw"]
Jul 29 13:12:36.129621 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:36.129601    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ef0894-75dbdc98df-xzcl7"
Jul 29 13:12:36.163123 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:36.163102    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-ef0894-667f4d77b8-lkx8f"
Jul 29 13:12:36.305699 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:36.305661    2567 generic.go:356] "Generic (PLEG): container finished" podID="3b2bef17-4396-4a4c-9d83-2b845db4ae73" containerID="2c2273a2057a0b61fe87e2484e8555dd8645f1f7324149b7891162864c2808d8" exitCode=0
Jul 29 13:12:36.305872 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:36.305811    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-ef0894-667f4d77b8-lkx8f" event={"ID":"3b2bef17-4396-4a4c-9d83-2b845db4ae73","Type":"ContainerDied","Data":"2c2273a2057a0b61fe87e2484e8555dd8645f1f7324149b7891162864c2808d8"}
Jul 29 13:12:36.305928 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:36.305897    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-ef0894-667f4d77b8-lkx8f" event={"ID":"3b2bef17-4396-4a4c-9d83-2b845db4ae73","Type":"ContainerDied","Data":"5b508e7b12b50f210f6828126530299c663005563f22cfee13149901bff7abba"}
Jul 29 13:12:36.305977 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:36.305932    2567 scope.go:117] "RemoveContainer" containerID="2c2273a2057a0b61fe87e2484e8555dd8645f1f7324149b7891162864c2808d8"
Jul 29 13:12:36.306422 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:36.306117    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-ef0894-667f4d77b8-lkx8f"
Jul 29 13:12:36.307957 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:36.307930    2567 generic.go:356] "Generic (PLEG): container finished" podID="4a39b6bf-1003-4636-9452-353923f29171" containerID="bb533f360fa4a9ecf2c950146a31fc8ebbf381816f3b096031df2d4c948eacd2" exitCode=0
Jul 29 13:12:36.308058 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:36.307991    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ef0894-75dbdc98df-xzcl7"
Jul 29 13:12:36.308058 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:36.308019    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ef0894-75dbdc98df-xzcl7" event={"ID":"4a39b6bf-1003-4636-9452-353923f29171","Type":"ContainerDied","Data":"bb533f360fa4a9ecf2c950146a31fc8ebbf381816f3b096031df2d4c948eacd2"}
Jul 29 13:12:36.308134 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:36.308072    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ef0894-75dbdc98df-xzcl7" event={"ID":"4a39b6bf-1003-4636-9452-353923f29171","Type":"ContainerDied","Data":"bb12e5386e35908bcd3aa2fa4210b4fc0a95238678c6c21d5bca8b65e78b5a67"}
Jul 29 13:12:36.315387 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:36.315368    2567 scope.go:117] "RemoveContainer" containerID="2c2273a2057a0b61fe87e2484e8555dd8645f1f7324149b7891162864c2808d8"
Jul 29 13:12:36.315664 ip-10-0-128-25 kubenswrapper[2567]: E0729 13:12:36.315635    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c2273a2057a0b61fe87e2484e8555dd8645f1f7324149b7891162864c2808d8\": container with ID starting with 2c2273a2057a0b61fe87e2484e8555dd8645f1f7324149b7891162864c2808d8 not found: ID does not exist" containerID="2c2273a2057a0b61fe87e2484e8555dd8645f1f7324149b7891162864c2808d8"
Jul 29 13:12:36.315762 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:36.315669    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c2273a2057a0b61fe87e2484e8555dd8645f1f7324149b7891162864c2808d8"} err="failed to get container status \"2c2273a2057a0b61fe87e2484e8555dd8645f1f7324149b7891162864c2808d8\": rpc error: code = NotFound desc = could not find container \"2c2273a2057a0b61fe87e2484e8555dd8645f1f7324149b7891162864c2808d8\": container with ID starting with 2c2273a2057a0b61fe87e2484e8555dd8645f1f7324149b7891162864c2808d8 not found: ID does not exist"
Jul 29 13:12:36.315762 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:36.315684    2567 scope.go:117] "RemoveContainer" containerID="bb533f360fa4a9ecf2c950146a31fc8ebbf381816f3b096031df2d4c948eacd2"
Jul 29 13:12:36.317717 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:36.317691    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4a39b6bf-1003-4636-9452-353923f29171-maas-api-tls\") pod \"4a39b6bf-1003-4636-9452-353923f29171\" (UID: \"4a39b6bf-1003-4636-9452-353923f29171\") "
Jul 29 13:12:36.317803 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:36.317769    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b2bef17-4396-4a4c-9d83-2b845db4ae73-plugins-config-volume\") pod \"3b2bef17-4396-4a4c-9d83-2b845db4ae73\" (UID: \"3b2bef17-4396-4a4c-9d83-2b845db4ae73\") "
Jul 29 13:12:36.317843 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:36.317816    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4ltdr\" (UniqueName: \"kubernetes.io/projected/3b2bef17-4396-4a4c-9d83-2b845db4ae73-kube-api-access-4ltdr\") pod \"3b2bef17-4396-4a4c-9d83-2b845db4ae73\" (UID: \"3b2bef17-4396-4a4c-9d83-2b845db4ae73\") "
Jul 29 13:12:36.317877 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:36.317852    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m4pcv\" (UniqueName: \"kubernetes.io/projected/4a39b6bf-1003-4636-9452-353923f29171-kube-api-access-m4pcv\") pod \"4a39b6bf-1003-4636-9452-353923f29171\" (UID: \"4a39b6bf-1003-4636-9452-353923f29171\") "
Jul 29 13:12:36.317912 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:36.317884    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a39b6bf-1003-4636-9452-353923f29171-openshift-service-ca\") pod \"4a39b6bf-1003-4636-9452-353923f29171\" (UID: \"4a39b6bf-1003-4636-9452-353923f29171\") "
Jul 29 13:12:36.318286 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:36.318148    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b2bef17-4396-4a4c-9d83-2b845db4ae73-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3b2bef17-4396-4a4c-9d83-2b845db4ae73" (UID: "3b2bef17-4396-4a4c-9d83-2b845db4ae73"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 13:12:36.318391 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:36.318307    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a39b6bf-1003-4636-9452-353923f29171-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4a39b6bf-1003-4636-9452-353923f29171" (UID: "4a39b6bf-1003-4636-9452-353923f29171"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 13:12:36.319702 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:36.319677    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a39b6bf-1003-4636-9452-353923f29171-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4a39b6bf-1003-4636-9452-353923f29171" (UID: "4a39b6bf-1003-4636-9452-353923f29171"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 13:12:36.319834 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:36.319811    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a39b6bf-1003-4636-9452-353923f29171-kube-api-access-m4pcv" (OuterVolumeSpecName: "kube-api-access-m4pcv") pod "4a39b6bf-1003-4636-9452-353923f29171" (UID: "4a39b6bf-1003-4636-9452-353923f29171"). InnerVolumeSpecName "kube-api-access-m4pcv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 13:12:36.320155 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:36.320130    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b2bef17-4396-4a4c-9d83-2b845db4ae73-kube-api-access-4ltdr" (OuterVolumeSpecName: "kube-api-access-4ltdr") pod "3b2bef17-4396-4a4c-9d83-2b845db4ae73" (UID: "3b2bef17-4396-4a4c-9d83-2b845db4ae73"). InnerVolumeSpecName "kube-api-access-4ltdr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 13:12:36.323932 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:36.323905    2567 scope.go:117] "RemoveContainer" containerID="bb533f360fa4a9ecf2c950146a31fc8ebbf381816f3b096031df2d4c948eacd2"
Jul 29 13:12:36.324174 ip-10-0-128-25 kubenswrapper[2567]: E0729 13:12:36.324153    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb533f360fa4a9ecf2c950146a31fc8ebbf381816f3b096031df2d4c948eacd2\": container with ID starting with bb533f360fa4a9ecf2c950146a31fc8ebbf381816f3b096031df2d4c948eacd2 not found: ID does not exist" containerID="bb533f360fa4a9ecf2c950146a31fc8ebbf381816f3b096031df2d4c948eacd2"
Jul 29 13:12:36.324223 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:36.324182    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb533f360fa4a9ecf2c950146a31fc8ebbf381816f3b096031df2d4c948eacd2"} err="failed to get container status \"bb533f360fa4a9ecf2c950146a31fc8ebbf381816f3b096031df2d4c948eacd2\": rpc error: code = NotFound desc = could not find container \"bb533f360fa4a9ecf2c950146a31fc8ebbf381816f3b096031df2d4c948eacd2\": container with ID starting with bb533f360fa4a9ecf2c950146a31fc8ebbf381816f3b096031df2d4c948eacd2 not found: ID does not exist"
Jul 29 13:12:36.419175 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:36.419149    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b2bef17-4396-4a4c-9d83-2b845db4ae73-plugins-config-volume\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 13:12:36.419175 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:36.419169    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4ltdr\" (UniqueName: \"kubernetes.io/projected/3b2bef17-4396-4a4c-9d83-2b845db4ae73-kube-api-access-4ltdr\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 13:12:36.419291 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:36.419179    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m4pcv\" (UniqueName: \"kubernetes.io/projected/4a39b6bf-1003-4636-9452-353923f29171-kube-api-access-m4pcv\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 13:12:36.419291 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:36.419187    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a39b6bf-1003-4636-9452-353923f29171-openshift-service-ca\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 13:12:36.419291 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:36.419197    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4a39b6bf-1003-4636-9452-353923f29171-maas-api-tls\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 13:12:36.633683 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:36.633655    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-ef0894-667f4d77b8-lkx8f"]
Jul 29 13:12:36.642697 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:36.642675    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-ef0894-667f4d77b8-lkx8f"]
Jul 29 13:12:36.653627 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:36.653606    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ef0894-75dbdc98df-xzcl7"]
Jul 29 13:12:36.658280 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:36.658256    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ef0894-75dbdc98df-xzcl7"]
Jul 29 13:12:37.923182 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:37.923141    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b2bef17-4396-4a4c-9d83-2b845db4ae73" path="/var/lib/kubelet/pods/3b2bef17-4396-4a4c-9d83-2b845db4ae73/volumes"
Jul 29 13:12:37.923738 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:37.923715    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a39b6bf-1003-4636-9452-353923f29171" path="/var/lib/kubelet/pods/4a39b6bf-1003-4636-9452-353923f29171/volumes"
Jul 29 13:12:37.924077 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:37.924062    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f579251f-b38f-4585-a27e-1a110339c5fa" path="/var/lib/kubelet/pods/f579251f-b38f-4585-a27e-1a110339c5fa/volumes"
Jul 29 13:12:47.137358 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:47.137326    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-e8f960-5c4788557f-d5z9z"]
Jul 29 13:12:47.137849 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:47.137728    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a39b6bf-1003-4636-9452-353923f29171" containerName="maas-api"
Jul 29 13:12:47.137849 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:47.137739    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a39b6bf-1003-4636-9452-353923f29171" containerName="maas-api"
Jul 29 13:12:47.137849 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:47.137762    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b2bef17-4396-4a4c-9d83-2b845db4ae73" containerName="payload-processing"
Jul 29 13:12:47.137849 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:47.137767    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b2bef17-4396-4a4c-9d83-2b845db4ae73" containerName="payload-processing"
Jul 29 13:12:47.137849 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:47.137774    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f579251f-b38f-4585-a27e-1a110339c5fa" containerName="revoke-keys"
Jul 29 13:12:47.137849 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:47.137779    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="f579251f-b38f-4585-a27e-1a110339c5fa" containerName="revoke-keys"
Jul 29 13:12:47.137849 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:47.137852    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b2bef17-4396-4a4c-9d83-2b845db4ae73" containerName="payload-processing"
Jul 29 13:12:47.138092 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:47.137865    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a39b6bf-1003-4636-9452-353923f29171" containerName="maas-api"
Jul 29 13:12:47.138092 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:47.137875    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="f579251f-b38f-4585-a27e-1a110339c5fa" containerName="revoke-keys"
Jul 29 13:12:47.142665 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:47.142647    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e8f960-5c4788557f-d5z9z"
Jul 29 13:12:47.145395 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:47.145366    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-e8f960-serving-cert\""
Jul 29 13:12:47.155297 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:47.155275    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-e8f960-5c4788557f-d5z9z"]
Jul 29 13:12:47.203826 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:47.203802    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99e95f38-c219-4ead-80c6-051c2d7c8af3-maas-api-tls\") pod \"maas-api-e2e-api-a-e8f960-5c4788557f-d5z9z\" (UID: \"99e95f38-c219-4ead-80c6-051c2d7c8af3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e8f960-5c4788557f-d5z9z"
Jul 29 13:12:47.203942 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:47.203874    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9z9vg\" (UniqueName: \"kubernetes.io/projected/99e95f38-c219-4ead-80c6-051c2d7c8af3-kube-api-access-9z9vg\") pod \"maas-api-e2e-api-a-e8f960-5c4788557f-d5z9z\" (UID: \"99e95f38-c219-4ead-80c6-051c2d7c8af3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e8f960-5c4788557f-d5z9z"
Jul 29 13:12:47.204008 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:47.203950    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/99e95f38-c219-4ead-80c6-051c2d7c8af3-openshift-service-ca\") pod \"maas-api-e2e-api-a-e8f960-5c4788557f-d5z9z\" (UID: \"99e95f38-c219-4ead-80c6-051c2d7c8af3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e8f960-5c4788557f-d5z9z"
Jul 29 13:12:47.305015 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:47.304985    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/99e95f38-c219-4ead-80c6-051c2d7c8af3-openshift-service-ca\") pod \"maas-api-e2e-api-a-e8f960-5c4788557f-d5z9z\" (UID: \"99e95f38-c219-4ead-80c6-051c2d7c8af3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e8f960-5c4788557f-d5z9z"
Jul 29 13:12:47.305148 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:47.305034    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99e95f38-c219-4ead-80c6-051c2d7c8af3-maas-api-tls\") pod \"maas-api-e2e-api-a-e8f960-5c4788557f-d5z9z\" (UID: \"99e95f38-c219-4ead-80c6-051c2d7c8af3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e8f960-5c4788557f-d5z9z"
Jul 29 13:12:47.305148 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:47.305072    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9z9vg\" (UniqueName: \"kubernetes.io/projected/99e95f38-c219-4ead-80c6-051c2d7c8af3-kube-api-access-9z9vg\") pod \"maas-api-e2e-api-a-e8f960-5c4788557f-d5z9z\" (UID: \"99e95f38-c219-4ead-80c6-051c2d7c8af3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e8f960-5c4788557f-d5z9z"
Jul 29 13:12:47.305655 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:47.305629    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/99e95f38-c219-4ead-80c6-051c2d7c8af3-openshift-service-ca\") pod \"maas-api-e2e-api-a-e8f960-5c4788557f-d5z9z\" (UID: \"99e95f38-c219-4ead-80c6-051c2d7c8af3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e8f960-5c4788557f-d5z9z"
Jul 29 13:12:47.307378 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:47.307357    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99e95f38-c219-4ead-80c6-051c2d7c8af3-maas-api-tls\") pod \"maas-api-e2e-api-a-e8f960-5c4788557f-d5z9z\" (UID: \"99e95f38-c219-4ead-80c6-051c2d7c8af3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e8f960-5c4788557f-d5z9z"
Jul 29 13:12:47.313830 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:47.313805    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9z9vg\" (UniqueName: \"kubernetes.io/projected/99e95f38-c219-4ead-80c6-051c2d7c8af3-kube-api-access-9z9vg\") pod \"maas-api-e2e-api-a-e8f960-5c4788557f-d5z9z\" (UID: \"99e95f38-c219-4ead-80c6-051c2d7c8af3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e8f960-5c4788557f-d5z9z"
Jul 29 13:12:47.453176 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:47.453107    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e8f960-5c4788557f-d5z9z"
Jul 29 13:12:47.592572 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:47.592543    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-e8f960-5c4788557f-d5z9z"]
Jul 29 13:12:47.593671 ip-10-0-128-25 kubenswrapper[2567]: W0729 13:12:47.593646    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99e95f38_c219_4ead_80c6_051c2d7c8af3.slice/crio-c80356ab58691512aff6cdb6d9e1e62d36d39cc352d154cb01d51f4e7500395d WatchSource:0}: Error finding container c80356ab58691512aff6cdb6d9e1e62d36d39cc352d154cb01d51f4e7500395d: Status 404 returned error can't find the container with id c80356ab58691512aff6cdb6d9e1e62d36d39cc352d154cb01d51f4e7500395d
Jul 29 13:12:48.384691 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:48.384661    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e8f960-5c4788557f-d5z9z" event={"ID":"99e95f38-c219-4ead-80c6-051c2d7c8af3","Type":"ContainerStarted","Data":"4884620c365d4f7b31dd46ee42384dc557ee760639d0fb53475797ae36f2937e"}
Jul 29 13:12:48.385028 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:48.384728    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e8f960-5c4788557f-d5z9z"
Jul 29 13:12:48.385028 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:48.384754    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e8f960-5c4788557f-d5z9z" event={"ID":"99e95f38-c219-4ead-80c6-051c2d7c8af3","Type":"ContainerStarted","Data":"c80356ab58691512aff6cdb6d9e1e62d36d39cc352d154cb01d51f4e7500395d"}
Jul 29 13:12:48.401857 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:48.401811    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e8f960-5c4788557f-d5z9z" podStartSLOduration=0.792776429 podStartE2EDuration="1.401796262s" totalImagesPullingTime="609.019833ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 13:12:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 13:12:48.401082988 +0000 UTC m=+2851.094771321" watchObservedRunningTime="2026-07-29 13:12:48.401796262 +0000 UTC m=+2851.095484573"
Jul 29 13:12:54.397172 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:12:54.397133    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e8f960-5c4788557f-d5z9z"
Jul 29 13:13:16.244615 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:13:16.244573    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-2f4f5e-55498bd8c4-sp6jn"]
Jul 29 13:13:16.248272 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:13:16.248254    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2f4f5e-55498bd8c4-sp6jn"
Jul 29 13:13:16.250842 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:13:16.250815    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-2f4f5e-serving-cert\""
Jul 29 13:13:16.258349 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:13:16.258328    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-2f4f5e-55498bd8c4-sp6jn"]
Jul 29 13:13:16.348713 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:13:16.348687    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cdsr\" (UniqueName: \"kubernetes.io/projected/a8255ac9-4946-43e3-b9cd-de66abc45ecb-kube-api-access-2cdsr\") pod \"maas-api-e2e-api-b-2f4f5e-55498bd8c4-sp6jn\" (UID: \"a8255ac9-4946-43e3-b9cd-de66abc45ecb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2f4f5e-55498bd8c4-sp6jn"
Jul 29 13:13:16.348861 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:13:16.348720    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8255ac9-4946-43e3-b9cd-de66abc45ecb-openshift-service-ca\") pod \"maas-api-e2e-api-b-2f4f5e-55498bd8c4-sp6jn\" (UID: \"a8255ac9-4946-43e3-b9cd-de66abc45ecb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2f4f5e-55498bd8c4-sp6jn"
Jul 29 13:13:16.348861 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:13:16.348764    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8255ac9-4946-43e3-b9cd-de66abc45ecb-maas-api-tls\") pod \"maas-api-e2e-api-b-2f4f5e-55498bd8c4-sp6jn\" (UID: \"a8255ac9-4946-43e3-b9cd-de66abc45ecb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2f4f5e-55498bd8c4-sp6jn"
Jul 29 13:13:16.450160 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:13:16.450131    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cdsr\" (UniqueName: \"kubernetes.io/projected/a8255ac9-4946-43e3-b9cd-de66abc45ecb-kube-api-access-2cdsr\") pod \"maas-api-e2e-api-b-2f4f5e-55498bd8c4-sp6jn\" (UID: \"a8255ac9-4946-43e3-b9cd-de66abc45ecb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2f4f5e-55498bd8c4-sp6jn"
Jul 29 13:13:16.450282 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:13:16.450165    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8255ac9-4946-43e3-b9cd-de66abc45ecb-openshift-service-ca\") pod \"maas-api-e2e-api-b-2f4f5e-55498bd8c4-sp6jn\" (UID: \"a8255ac9-4946-43e3-b9cd-de66abc45ecb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2f4f5e-55498bd8c4-sp6jn"
Jul 29 13:13:16.450282 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:13:16.450214    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8255ac9-4946-43e3-b9cd-de66abc45ecb-maas-api-tls\") pod \"maas-api-e2e-api-b-2f4f5e-55498bd8c4-sp6jn\" (UID: \"a8255ac9-4946-43e3-b9cd-de66abc45ecb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2f4f5e-55498bd8c4-sp6jn"
Jul 29 13:13:16.450441 ip-10-0-128-25 kubenswrapper[2567]: E0729 13:13:16.450359    2567 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-2f4f5e-serving-cert: secret "maas-api-e2e-api-b-2f4f5e-serving-cert" not found
Jul 29 13:13:16.450522 ip-10-0-128-25 kubenswrapper[2567]: E0729 13:13:16.450465    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8255ac9-4946-43e3-b9cd-de66abc45ecb-maas-api-tls podName:a8255ac9-4946-43e3-b9cd-de66abc45ecb nodeName:}" failed. No retries permitted until 2026-07-29 13:13:16.950443812 +0000 UTC m=+2879.644132103 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a8255ac9-4946-43e3-b9cd-de66abc45ecb-maas-api-tls") pod "maas-api-e2e-api-b-2f4f5e-55498bd8c4-sp6jn" (UID: "a8255ac9-4946-43e3-b9cd-de66abc45ecb") : secret "maas-api-e2e-api-b-2f4f5e-serving-cert" not found
Jul 29 13:13:16.450905 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:13:16.450886    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8255ac9-4946-43e3-b9cd-de66abc45ecb-openshift-service-ca\") pod \"maas-api-e2e-api-b-2f4f5e-55498bd8c4-sp6jn\" (UID: \"a8255ac9-4946-43e3-b9cd-de66abc45ecb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2f4f5e-55498bd8c4-sp6jn"
Jul 29 13:13:16.465427 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:13:16.465394    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cdsr\" (UniqueName: \"kubernetes.io/projected/a8255ac9-4946-43e3-b9cd-de66abc45ecb-kube-api-access-2cdsr\") pod \"maas-api-e2e-api-b-2f4f5e-55498bd8c4-sp6jn\" (UID: \"a8255ac9-4946-43e3-b9cd-de66abc45ecb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2f4f5e-55498bd8c4-sp6jn"
Jul 29 13:13:16.954575 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:13:16.954541    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8255ac9-4946-43e3-b9cd-de66abc45ecb-maas-api-tls\") pod \"maas-api-e2e-api-b-2f4f5e-55498bd8c4-sp6jn\" (UID: \"a8255ac9-4946-43e3-b9cd-de66abc45ecb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2f4f5e-55498bd8c4-sp6jn"
Jul 29 13:13:16.956806 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:13:16.956787    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8255ac9-4946-43e3-b9cd-de66abc45ecb-maas-api-tls\") pod \"maas-api-e2e-api-b-2f4f5e-55498bd8c4-sp6jn\" (UID: \"a8255ac9-4946-43e3-b9cd-de66abc45ecb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2f4f5e-55498bd8c4-sp6jn"
Jul 29 13:13:17.135377 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:13:17.135344    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-2f4f5e-676676d78-xgvzm"]
Jul 29 13:13:17.139418 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:13:17.139391    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-2f4f5e-676676d78-xgvzm"
Jul 29 13:13:17.141835 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:13:17.141807    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-2f4f5e\""
Jul 29 13:13:17.141952 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:13:17.141859    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-2f4f5e-dockercfg-ncqp6\""
Jul 29 13:13:17.149924 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:13:17.149841    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-2f4f5e-676676d78-xgvzm"]
Jul 29 13:13:17.160254 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:13:17.160227    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2f4f5e-55498bd8c4-sp6jn"
Jul 29 13:13:17.259163 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:13:17.259124    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrmn5\" (UniqueName: \"kubernetes.io/projected/36b786f9-d92d-4aad-9b1b-979874fd5242-kube-api-access-jrmn5\") pod \"payload-processing-e2e-api-b-2f4f5e-676676d78-xgvzm\" (UID: \"36b786f9-d92d-4aad-9b1b-979874fd5242\") " pod="openshift-ingress/payload-processing-e2e-api-b-2f4f5e-676676d78-xgvzm"
Jul 29 13:13:17.259646 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:13:17.259180    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36b786f9-d92d-4aad-9b1b-979874fd5242-plugins-config-volume\") pod \"payload-processing-e2e-api-b-2f4f5e-676676d78-xgvzm\" (UID: \"36b786f9-d92d-4aad-9b1b-979874fd5242\") " pod="openshift-ingress/payload-processing-e2e-api-b-2f4f5e-676676d78-xgvzm"
Jul 29 13:13:17.295170 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:13:17.295140    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-2f4f5e-55498bd8c4-sp6jn"]
Jul 29 13:13:17.296351 ip-10-0-128-25 kubenswrapper[2567]: W0729 13:13:17.296326    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8255ac9_4946_43e3_b9cd_de66abc45ecb.slice/crio-eda76690a5da1b6fd7c31a76e26b88549cb00658dc66e60abc0d55c567876dfd WatchSource:0}: Error finding container eda76690a5da1b6fd7c31a76e26b88549cb00658dc66e60abc0d55c567876dfd: Status 404 returned error can't find the container with id eda76690a5da1b6fd7c31a76e26b88549cb00658dc66e60abc0d55c567876dfd
Jul 29 13:13:17.360656 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:13:17.360628    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrmn5\" (UniqueName: \"kubernetes.io/projected/36b786f9-d92d-4aad-9b1b-979874fd5242-kube-api-access-jrmn5\") pod \"payload-processing-e2e-api-b-2f4f5e-676676d78-xgvzm\" (UID: \"36b786f9-d92d-4aad-9b1b-979874fd5242\") " pod="openshift-ingress/payload-processing-e2e-api-b-2f4f5e-676676d78-xgvzm"
Jul 29 13:13:17.360770 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:13:17.360665    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36b786f9-d92d-4aad-9b1b-979874fd5242-plugins-config-volume\") pod \"payload-processing-e2e-api-b-2f4f5e-676676d78-xgvzm\" (UID: \"36b786f9-d92d-4aad-9b1b-979874fd5242\") " pod="openshift-ingress/payload-processing-e2e-api-b-2f4f5e-676676d78-xgvzm"
Jul 29 13:13:17.361376 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:13:17.361354    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36b786f9-d92d-4aad-9b1b-979874fd5242-plugins-config-volume\") pod \"payload-processing-e2e-api-b-2f4f5e-676676d78-xgvzm\" (UID: \"36b786f9-d92d-4aad-9b1b-979874fd5242\") " pod="openshift-ingress/payload-processing-e2e-api-b-2f4f5e-676676d78-xgvzm"
Jul 29 13:13:17.371038 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:13:17.371014    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrmn5\" (UniqueName: \"kubernetes.io/projected/36b786f9-d92d-4aad-9b1b-979874fd5242-kube-api-access-jrmn5\") pod \"payload-processing-e2e-api-b-2f4f5e-676676d78-xgvzm\" (UID: \"36b786f9-d92d-4aad-9b1b-979874fd5242\") " pod="openshift-ingress/payload-processing-e2e-api-b-2f4f5e-676676d78-xgvzm"
Jul 29 13:13:17.454469 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:13:17.454448    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-2f4f5e-676676d78-xgvzm"
Jul 29 13:13:17.565197 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:13:17.565164    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2f4f5e-55498bd8c4-sp6jn" event={"ID":"a8255ac9-4946-43e3-b9cd-de66abc45ecb","Type":"ContainerStarted","Data":"eda76690a5da1b6fd7c31a76e26b88549cb00658dc66e60abc0d55c567876dfd"}
Jul 29 13:13:17.592847 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:13:17.592819    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-2f4f5e-676676d78-xgvzm"]
Jul 29 13:13:17.594330 ip-10-0-128-25 kubenswrapper[2567]: W0729 13:13:17.594307    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36b786f9_d92d_4aad_9b1b_979874fd5242.slice/crio-1da864fe0a2761a1a53ae6048d2834622a43a265fc19f4711618949c20902464 WatchSource:0}: Error finding container 1da864fe0a2761a1a53ae6048d2834622a43a265fc19f4711618949c20902464: Status 404 returned error can't find the container with id 1da864fe0a2761a1a53ae6048d2834622a43a265fc19f4711618949c20902464
Jul 29 13:13:18.266548 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:13:18.266517    2567 scope.go:117] "RemoveContainer" containerID="0f0fb7c085b90c15e6718710961dc3d678aafb416cd665466a3cdf36d9a420d6"
Jul 29 13:13:18.572526 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:13:18.572493    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-2f4f5e-676676d78-xgvzm" event={"ID":"36b786f9-d92d-4aad-9b1b-979874fd5242","Type":"ContainerStarted","Data":"250149d4ac6c9927a1f220a5b89269578957e15d3a4866f249ab28ef9b56f8bd"}
Jul 29 13:13:18.572675 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:13:18.572548    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-2f4f5e-676676d78-xgvzm" event={"ID":"36b786f9-d92d-4aad-9b1b-979874fd5242","Type":"ContainerStarted","Data":"1da864fe0a2761a1a53ae6048d2834622a43a265fc19f4711618949c20902464"}
Jul 29 13:13:18.572675 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:13:18.572594    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-2f4f5e-676676d78-xgvzm"
Jul 29 13:13:18.574087 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:13:18.574065    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2f4f5e-55498bd8c4-sp6jn" event={"ID":"a8255ac9-4946-43e3-b9cd-de66abc45ecb","Type":"ContainerStarted","Data":"53f03c83017ff119e5f59b37728092095fbade296e4ef2594040918cb826ce86"}
Jul 29 13:13:18.574190 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:13:18.574119    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2f4f5e-55498bd8c4-sp6jn"
Jul 29 13:13:18.590181 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:13:18.590133    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-2f4f5e-676676d78-xgvzm" podStartSLOduration=1.590120958 podStartE2EDuration="1.590120958s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 13:13:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 13:13:18.589562276 +0000 UTC m=+2881.283250606" watchObservedRunningTime="2026-07-29 13:13:18.590120958 +0000 UTC m=+2881.283809268"
Jul 29 13:13:18.609633 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:13:18.609595    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2f4f5e-55498bd8c4-sp6jn" podStartSLOduration=2.212095921 podStartE2EDuration="2.609584776s" totalImagesPullingTime="397.488855ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 13:13:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 13:13:18.607452222 +0000 UTC m=+2881.301140555" watchObservedRunningTime="2026-07-29 13:13:18.609584776 +0000 UTC m=+2881.303273086"
Jul 29 13:13:24.585489 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:13:24.585461    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2f4f5e-55498bd8c4-sp6jn"
Jul 29 13:13:29.582622 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:13:29.582583    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-2f4f5e-676676d78-xgvzm"
Jul 29 13:13:44.511547 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:13:44.511515    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e8f960-ftp6q"]
Jul 29 13:13:44.524135 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:13:44.524102    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e8f960-ftp6q"]
Jul 29 13:13:44.524376 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:13:44.524164    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e8f960-ftp6q"
Jul 29 13:13:44.526995 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:13:44.526972    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7gpd2\""
Jul 29 13:13:44.584991 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:13:44.584968    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7be8cb92-7e30-4ae7-b92f-f7552f57e89d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-e8f960-ftp6q\" (UID: \"7be8cb92-7e30-4ae7-b92f-f7552f57e89d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e8f960-ftp6q"
Jul 29 13:13:44.685988 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:13:44.685966    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7be8cb92-7e30-4ae7-b92f-f7552f57e89d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-e8f960-ftp6q\" (UID: \"7be8cb92-7e30-4ae7-b92f-f7552f57e89d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e8f960-ftp6q"
Jul 29 13:13:44.686547 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:13:44.686529    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7be8cb92-7e30-4ae7-b92f-f7552f57e89d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-e8f960-ftp6q\" (UID: \"7be8cb92-7e30-4ae7-b92f-f7552f57e89d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e8f960-ftp6q"
Jul 29 13:13:44.835315 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:13:44.835297    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e8f960-ftp6q"
Jul 29 13:13:44.967793 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:13:44.967755    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e8f960-ftp6q"]
Jul 29 13:13:44.971037 ip-10-0-128-25 kubenswrapper[2567]: W0729 13:13:44.971011    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7be8cb92_7e30_4ae7_b92f_f7552f57e89d.slice/crio-08ae20e8e4e15caad269fceb958e007cb8a0c5280ba7fbfc5411452b18e29b41 WatchSource:0}: Error finding container 08ae20e8e4e15caad269fceb958e007cb8a0c5280ba7fbfc5411452b18e29b41: Status 404 returned error can't find the container with id 08ae20e8e4e15caad269fceb958e007cb8a0c5280ba7fbfc5411452b18e29b41
Jul 29 13:13:45.750994 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:13:45.750964    2567 generic.go:356] "Generic (PLEG): container finished" podID="7be8cb92-7e30-4ae7-b92f-f7552f57e89d" containerID="1ef1aa5dd02a633cc13227a25a68cdc96d0827e038d3cefdc6187374be115eaa" exitCode=0
Jul 29 13:13:45.751302 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:13:45.751057    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e8f960-ftp6q" event={"ID":"7be8cb92-7e30-4ae7-b92f-f7552f57e89d","Type":"ContainerDied","Data":"1ef1aa5dd02a633cc13227a25a68cdc96d0827e038d3cefdc6187374be115eaa"}
Jul 29 13:13:45.751302 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:13:45.751116    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e8f960-ftp6q" event={"ID":"7be8cb92-7e30-4ae7-b92f-f7552f57e89d","Type":"ContainerStarted","Data":"08ae20e8e4e15caad269fceb958e007cb8a0c5280ba7fbfc5411452b18e29b41"}
Jul 29 13:13:46.891095 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:13:46.891073    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e8f960-ftp6q"
Jul 29 13:13:46.903957 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:13:46.903932    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7be8cb92-7e30-4ae7-b92f-f7552f57e89d-maas-api-service-ca\") pod \"7be8cb92-7e30-4ae7-b92f-f7552f57e89d\" (UID: \"7be8cb92-7e30-4ae7-b92f-f7552f57e89d\") "
Jul 29 13:13:46.904259 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:13:46.904238    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7be8cb92-7e30-4ae7-b92f-f7552f57e89d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7be8cb92-7e30-4ae7-b92f-f7552f57e89d" (UID: "7be8cb92-7e30-4ae7-b92f-f7552f57e89d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 13:13:47.004886 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:13:47.004862    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7be8cb92-7e30-4ae7-b92f-f7552f57e89d-maas-api-service-ca\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 13:13:47.764945 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:13:47.764917    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e8f960-ftp6q"
Jul 29 13:13:47.765128 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:13:47.764942    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e8f960-ftp6q" event={"ID":"7be8cb92-7e30-4ae7-b92f-f7552f57e89d","Type":"ContainerDied","Data":"08ae20e8e4e15caad269fceb958e007cb8a0c5280ba7fbfc5411452b18e29b41"}
Jul 29 13:13:47.765128 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:13:47.764994    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="08ae20e8e4e15caad269fceb958e007cb8a0c5280ba7fbfc5411452b18e29b41"
Jul 29 13:13:59.927974 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:13:59.927940    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-e8f960-5c4788557f-d5z9z"]
Jul 29 13:13:59.928765 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:13:59.928175    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e8f960-5c4788557f-d5z9z" podUID="99e95f38-c219-4ead-80c6-051c2d7c8af3" containerName="maas-api" containerID="cri-o://4884620c365d4f7b31dd46ee42384dc557ee760639d0fb53475797ae36f2937e" gracePeriod=30
Jul 29 13:14:00.138506 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:00.138476    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e8f960-ftp6q"]
Jul 29 13:14:00.142626 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:00.142593    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e8f960-ftp6q"]
Jul 29 13:14:00.185667 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:00.185611    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e8f960-5c4788557f-d5z9z"
Jul 29 13:14:00.207940 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:00.207915    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9z9vg\" (UniqueName: \"kubernetes.io/projected/99e95f38-c219-4ead-80c6-051c2d7c8af3-kube-api-access-9z9vg\") pod \"99e95f38-c219-4ead-80c6-051c2d7c8af3\" (UID: \"99e95f38-c219-4ead-80c6-051c2d7c8af3\") "
Jul 29 13:14:00.208072 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:00.207978    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99e95f38-c219-4ead-80c6-051c2d7c8af3-maas-api-tls\") pod \"99e95f38-c219-4ead-80c6-051c2d7c8af3\" (UID: \"99e95f38-c219-4ead-80c6-051c2d7c8af3\") "
Jul 29 13:14:00.208072 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:00.208038    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/99e95f38-c219-4ead-80c6-051c2d7c8af3-openshift-service-ca\") pod \"99e95f38-c219-4ead-80c6-051c2d7c8af3\" (UID: \"99e95f38-c219-4ead-80c6-051c2d7c8af3\") "
Jul 29 13:14:00.208547 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:00.208503    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99e95f38-c219-4ead-80c6-051c2d7c8af3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "99e95f38-c219-4ead-80c6-051c2d7c8af3" (UID: "99e95f38-c219-4ead-80c6-051c2d7c8af3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 13:14:00.210486 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:00.210455    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99e95f38-c219-4ead-80c6-051c2d7c8af3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "99e95f38-c219-4ead-80c6-051c2d7c8af3" (UID: "99e95f38-c219-4ead-80c6-051c2d7c8af3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 13:14:00.210486 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:00.210466    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99e95f38-c219-4ead-80c6-051c2d7c8af3-kube-api-access-9z9vg" (OuterVolumeSpecName: "kube-api-access-9z9vg") pod "99e95f38-c219-4ead-80c6-051c2d7c8af3" (UID: "99e95f38-c219-4ead-80c6-051c2d7c8af3"). InnerVolumeSpecName "kube-api-access-9z9vg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 13:14:00.309583 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:00.309554    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9z9vg\" (UniqueName: \"kubernetes.io/projected/99e95f38-c219-4ead-80c6-051c2d7c8af3-kube-api-access-9z9vg\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 13:14:00.309583 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:00.309577    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99e95f38-c219-4ead-80c6-051c2d7c8af3-maas-api-tls\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 13:14:00.309583 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:00.309587    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/99e95f38-c219-4ead-80c6-051c2d7c8af3-openshift-service-ca\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 13:14:00.843352 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:00.843319    2567 generic.go:356] "Generic (PLEG): container finished" podID="99e95f38-c219-4ead-80c6-051c2d7c8af3" containerID="4884620c365d4f7b31dd46ee42384dc557ee760639d0fb53475797ae36f2937e" exitCode=0
Jul 29 13:14:00.843527 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:00.843363    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e8f960-5c4788557f-d5z9z"
Jul 29 13:14:00.843527 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:00.843416    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e8f960-5c4788557f-d5z9z" event={"ID":"99e95f38-c219-4ead-80c6-051c2d7c8af3","Type":"ContainerDied","Data":"4884620c365d4f7b31dd46ee42384dc557ee760639d0fb53475797ae36f2937e"}
Jul 29 13:14:00.843527 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:00.843468    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e8f960-5c4788557f-d5z9z" event={"ID":"99e95f38-c219-4ead-80c6-051c2d7c8af3","Type":"ContainerDied","Data":"c80356ab58691512aff6cdb6d9e1e62d36d39cc352d154cb01d51f4e7500395d"}
Jul 29 13:14:00.843527 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:00.843492    2567 scope.go:117] "RemoveContainer" containerID="4884620c365d4f7b31dd46ee42384dc557ee760639d0fb53475797ae36f2937e"
Jul 29 13:14:00.853479 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:00.853461    2567 scope.go:117] "RemoveContainer" containerID="4884620c365d4f7b31dd46ee42384dc557ee760639d0fb53475797ae36f2937e"
Jul 29 13:14:00.853733 ip-10-0-128-25 kubenswrapper[2567]: E0729 13:14:00.853713    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4884620c365d4f7b31dd46ee42384dc557ee760639d0fb53475797ae36f2937e\": container with ID starting with 4884620c365d4f7b31dd46ee42384dc557ee760639d0fb53475797ae36f2937e not found: ID does not exist" containerID="4884620c365d4f7b31dd46ee42384dc557ee760639d0fb53475797ae36f2937e"
Jul 29 13:14:00.853810 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:00.853745    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4884620c365d4f7b31dd46ee42384dc557ee760639d0fb53475797ae36f2937e"} err="failed to get container status \"4884620c365d4f7b31dd46ee42384dc557ee760639d0fb53475797ae36f2937e\": rpc error: code = NotFound desc = could not find container \"4884620c365d4f7b31dd46ee42384dc557ee760639d0fb53475797ae36f2937e\": container with ID starting with 4884620c365d4f7b31dd46ee42384dc557ee760639d0fb53475797ae36f2937e not found: ID does not exist"
Jul 29 13:14:00.866080 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:00.866059    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-e8f960-5c4788557f-d5z9z"]
Jul 29 13:14:00.870221 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:00.870200    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-e8f960-5c4788557f-d5z9z"]
Jul 29 13:14:01.921967 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:01.921927    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7be8cb92-7e30-4ae7-b92f-f7552f57e89d" path="/var/lib/kubelet/pods/7be8cb92-7e30-4ae7-b92f-f7552f57e89d/volumes"
Jul 29 13:14:01.922345 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:01.922319    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99e95f38-c219-4ead-80c6-051c2d7c8af3" path="/var/lib/kubelet/pods/99e95f38-c219-4ead-80c6-051c2d7c8af3/volumes"
Jul 29 13:14:07.023533 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:07.023500    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2f4f5e-hhclg"]
Jul 29 13:14:07.023944 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:07.023930    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7be8cb92-7e30-4ae7-b92f-f7552f57e89d" containerName="revoke-keys"
Jul 29 13:14:07.023944 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:07.023945    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="7be8cb92-7e30-4ae7-b92f-f7552f57e89d" containerName="revoke-keys"
Jul 29 13:14:07.024026 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:07.023953    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99e95f38-c219-4ead-80c6-051c2d7c8af3" containerName="maas-api"
Jul 29 13:14:07.024026 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:07.023959    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="99e95f38-c219-4ead-80c6-051c2d7c8af3" containerName="maas-api"
Jul 29 13:14:07.024026 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:07.024021    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="99e95f38-c219-4ead-80c6-051c2d7c8af3" containerName="maas-api"
Jul 29 13:14:07.024118 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:07.024032    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="7be8cb92-7e30-4ae7-b92f-f7552f57e89d" containerName="revoke-keys"
Jul 29 13:14:07.028335 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:07.028316    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2f4f5e-hhclg"
Jul 29 13:14:07.030980 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:07.030958    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7gpd2\""
Jul 29 13:14:07.037597 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:07.037577    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2f4f5e-hhclg"]
Jul 29 13:14:07.067790 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:07.067759    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f04416dd-0898-4692-a240-78955f6ffa2d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-2f4f5e-hhclg\" (UID: \"f04416dd-0898-4692-a240-78955f6ffa2d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2f4f5e-hhclg"
Jul 29 13:14:07.168744 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:07.168722    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f04416dd-0898-4692-a240-78955f6ffa2d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-2f4f5e-hhclg\" (UID: \"f04416dd-0898-4692-a240-78955f6ffa2d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2f4f5e-hhclg"
Jul 29 13:14:07.169263 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:07.169245    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f04416dd-0898-4692-a240-78955f6ffa2d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-2f4f5e-hhclg\" (UID: \"f04416dd-0898-4692-a240-78955f6ffa2d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2f4f5e-hhclg"
Jul 29 13:14:07.340558 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:07.340524    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2f4f5e-hhclg"
Jul 29 13:14:07.488910 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:07.488878    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2f4f5e-hhclg"]
Jul 29 13:14:07.489969 ip-10-0-128-25 kubenswrapper[2567]: W0729 13:14:07.489945    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf04416dd_0898_4692_a240_78955f6ffa2d.slice/crio-42fc74f9f79d489da1dddd4d0d8b70159fbec2a65ddb8c304c617818006b8ae4 WatchSource:0}: Error finding container 42fc74f9f79d489da1dddd4d0d8b70159fbec2a65ddb8c304c617818006b8ae4: Status 404 returned error can't find the container with id 42fc74f9f79d489da1dddd4d0d8b70159fbec2a65ddb8c304c617818006b8ae4
Jul 29 13:14:07.890118 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:07.890031    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2f4f5e-hhclg" event={"ID":"f04416dd-0898-4692-a240-78955f6ffa2d","Type":"ContainerStarted","Data":"075b79020cf1bcd508b352c8b6f0d141766d25a24b6b4bdd3a067474f06cd44b"}
Jul 29 13:14:07.890118 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:07.890081    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2f4f5e-hhclg" event={"ID":"f04416dd-0898-4692-a240-78955f6ffa2d","Type":"ContainerStarted","Data":"42fc74f9f79d489da1dddd4d0d8b70159fbec2a65ddb8c304c617818006b8ae4"}
Jul 29 13:14:07.908043 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:07.907990    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2f4f5e-hhclg" podStartSLOduration=0.907974315 podStartE2EDuration="907.974315ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 13:14:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 13:14:07.906050789 +0000 UTC m=+2930.599739099" watchObservedRunningTime="2026-07-29 13:14:07.907974315 +0000 UTC m=+2930.601662640"
Jul 29 13:14:08.896896 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:08.896862    2567 generic.go:356] "Generic (PLEG): container finished" podID="f04416dd-0898-4692-a240-78955f6ffa2d" containerID="075b79020cf1bcd508b352c8b6f0d141766d25a24b6b4bdd3a067474f06cd44b" exitCode=0
Jul 29 13:14:08.897251 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:08.896913    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2f4f5e-hhclg" event={"ID":"f04416dd-0898-4692-a240-78955f6ffa2d","Type":"ContainerDied","Data":"075b79020cf1bcd508b352c8b6f0d141766d25a24b6b4bdd3a067474f06cd44b"}
Jul 29 13:14:10.038986 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:10.038961    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2f4f5e-hhclg"
Jul 29 13:14:10.094727 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:10.094707    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f04416dd-0898-4692-a240-78955f6ffa2d-maas-api-service-ca\") pod \"f04416dd-0898-4692-a240-78955f6ffa2d\" (UID: \"f04416dd-0898-4692-a240-78955f6ffa2d\") "
Jul 29 13:14:10.095012 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:10.094991    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f04416dd-0898-4692-a240-78955f6ffa2d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f04416dd-0898-4692-a240-78955f6ffa2d" (UID: "f04416dd-0898-4692-a240-78955f6ffa2d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 13:14:10.196227 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:10.196173    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f04416dd-0898-4692-a240-78955f6ffa2d-maas-api-service-ca\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 13:14:10.911667 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:10.911640    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2f4f5e-hhclg"
Jul 29 13:14:10.911807 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:10.911679    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2f4f5e-hhclg" event={"ID":"f04416dd-0898-4692-a240-78955f6ffa2d","Type":"ContainerDied","Data":"42fc74f9f79d489da1dddd4d0d8b70159fbec2a65ddb8c304c617818006b8ae4"}
Jul 29 13:14:10.911807 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:10.911745    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="42fc74f9f79d489da1dddd4d0d8b70159fbec2a65ddb8c304c617818006b8ae4"
Jul 29 13:14:22.543998 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:22.543951    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-2f4f5e-55498bd8c4-sp6jn"]
Jul 29 13:14:22.544540 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:22.544204    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2f4f5e-55498bd8c4-sp6jn" podUID="a8255ac9-4946-43e3-b9cd-de66abc45ecb" containerName="maas-api" containerID="cri-o://53f03c83017ff119e5f59b37728092095fbade296e4ef2594040918cb826ce86" gracePeriod=30
Jul 29 13:14:22.601906 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:22.601875    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-2f4f5e-676676d78-xgvzm"]
Jul 29 13:14:22.602174 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:22.602126    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-2f4f5e-676676d78-xgvzm" podUID="36b786f9-d92d-4aad-9b1b-979874fd5242" containerName="payload-processing" containerID="cri-o://250149d4ac6c9927a1f220a5b89269578957e15d3a4866f249ab28ef9b56f8bd" gracePeriod=30
Jul 29 13:14:22.738855 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:22.738797    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2f4f5e-hhclg"]
Jul 29 13:14:22.742244 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:22.742217    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2f4f5e-hhclg"]
Jul 29 13:14:22.856714 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:22.856693    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2f4f5e-55498bd8c4-sp6jn"
Jul 29 13:14:22.883271 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:22.883252    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-2f4f5e-676676d78-xgvzm"
Jul 29 13:14:22.995956 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:22.995927    2567 generic.go:356] "Generic (PLEG): container finished" podID="36b786f9-d92d-4aad-9b1b-979874fd5242" containerID="250149d4ac6c9927a1f220a5b89269578957e15d3a4866f249ab28ef9b56f8bd" exitCode=0
Jul 29 13:14:22.996078 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:22.995974    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-2f4f5e-676676d78-xgvzm"
Jul 29 13:14:22.996078 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:22.996021    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-2f4f5e-676676d78-xgvzm" event={"ID":"36b786f9-d92d-4aad-9b1b-979874fd5242","Type":"ContainerDied","Data":"250149d4ac6c9927a1f220a5b89269578957e15d3a4866f249ab28ef9b56f8bd"}
Jul 29 13:14:22.996219 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:22.996081    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-2f4f5e-676676d78-xgvzm" event={"ID":"36b786f9-d92d-4aad-9b1b-979874fd5242","Type":"ContainerDied","Data":"1da864fe0a2761a1a53ae6048d2834622a43a265fc19f4711618949c20902464"}
Jul 29 13:14:22.996219 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:22.996115    2567 scope.go:117] "RemoveContainer" containerID="250149d4ac6c9927a1f220a5b89269578957e15d3a4866f249ab28ef9b56f8bd"
Jul 29 13:14:22.997514 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:22.997490    2567 generic.go:356] "Generic (PLEG): container finished" podID="a8255ac9-4946-43e3-b9cd-de66abc45ecb" containerID="53f03c83017ff119e5f59b37728092095fbade296e4ef2594040918cb826ce86" exitCode=0
Jul 29 13:14:22.997645 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:22.997530    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2f4f5e-55498bd8c4-sp6jn"
Jul 29 13:14:22.997645 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:22.997576    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2f4f5e-55498bd8c4-sp6jn" event={"ID":"a8255ac9-4946-43e3-b9cd-de66abc45ecb","Type":"ContainerDied","Data":"53f03c83017ff119e5f59b37728092095fbade296e4ef2594040918cb826ce86"}
Jul 29 13:14:22.997645 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:22.997641    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2f4f5e-55498bd8c4-sp6jn" event={"ID":"a8255ac9-4946-43e3-b9cd-de66abc45ecb","Type":"ContainerDied","Data":"eda76690a5da1b6fd7c31a76e26b88549cb00658dc66e60abc0d55c567876dfd"}
Jul 29 13:14:23.002456 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:23.002435    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2cdsr\" (UniqueName: \"kubernetes.io/projected/a8255ac9-4946-43e3-b9cd-de66abc45ecb-kube-api-access-2cdsr\") pod \"a8255ac9-4946-43e3-b9cd-de66abc45ecb\" (UID: \"a8255ac9-4946-43e3-b9cd-de66abc45ecb\") "
Jul 29 13:14:23.002579 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:23.002483    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jrmn5\" (UniqueName: \"kubernetes.io/projected/36b786f9-d92d-4aad-9b1b-979874fd5242-kube-api-access-jrmn5\") pod \"36b786f9-d92d-4aad-9b1b-979874fd5242\" (UID: \"36b786f9-d92d-4aad-9b1b-979874fd5242\") "
Jul 29 13:14:23.002579 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:23.002503    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8255ac9-4946-43e3-b9cd-de66abc45ecb-maas-api-tls\") pod \"a8255ac9-4946-43e3-b9cd-de66abc45ecb\" (UID: \"a8255ac9-4946-43e3-b9cd-de66abc45ecb\") "
Jul 29 13:14:23.002579 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:23.002525    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8255ac9-4946-43e3-b9cd-de66abc45ecb-openshift-service-ca\") pod \"a8255ac9-4946-43e3-b9cd-de66abc45ecb\" (UID: \"a8255ac9-4946-43e3-b9cd-de66abc45ecb\") "
Jul 29 13:14:23.002579 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:23.002543    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36b786f9-d92d-4aad-9b1b-979874fd5242-plugins-config-volume\") pod \"36b786f9-d92d-4aad-9b1b-979874fd5242\" (UID: \"36b786f9-d92d-4aad-9b1b-979874fd5242\") "
Jul 29 13:14:23.003182 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:23.002921    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8255ac9-4946-43e3-b9cd-de66abc45ecb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a8255ac9-4946-43e3-b9cd-de66abc45ecb" (UID: "a8255ac9-4946-43e3-b9cd-de66abc45ecb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 13:14:23.003291 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:23.003256    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36b786f9-d92d-4aad-9b1b-979874fd5242-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "36b786f9-d92d-4aad-9b1b-979874fd5242" (UID: "36b786f9-d92d-4aad-9b1b-979874fd5242"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 13:14:23.004554 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:23.004532    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8255ac9-4946-43e3-b9cd-de66abc45ecb-kube-api-access-2cdsr" (OuterVolumeSpecName: "kube-api-access-2cdsr") pod "a8255ac9-4946-43e3-b9cd-de66abc45ecb" (UID: "a8255ac9-4946-43e3-b9cd-de66abc45ecb"). InnerVolumeSpecName "kube-api-access-2cdsr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 13:14:23.004737 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:23.004715    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36b786f9-d92d-4aad-9b1b-979874fd5242-kube-api-access-jrmn5" (OuterVolumeSpecName: "kube-api-access-jrmn5") pod "36b786f9-d92d-4aad-9b1b-979874fd5242" (UID: "36b786f9-d92d-4aad-9b1b-979874fd5242"). InnerVolumeSpecName "kube-api-access-jrmn5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 13:14:23.004737 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:23.004723    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8255ac9-4946-43e3-b9cd-de66abc45ecb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a8255ac9-4946-43e3-b9cd-de66abc45ecb" (UID: "a8255ac9-4946-43e3-b9cd-de66abc45ecb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 13:14:23.006390 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:23.006374    2567 scope.go:117] "RemoveContainer" containerID="250149d4ac6c9927a1f220a5b89269578957e15d3a4866f249ab28ef9b56f8bd"
Jul 29 13:14:23.006738 ip-10-0-128-25 kubenswrapper[2567]: E0729 13:14:23.006720    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"250149d4ac6c9927a1f220a5b89269578957e15d3a4866f249ab28ef9b56f8bd\": container with ID starting with 250149d4ac6c9927a1f220a5b89269578957e15d3a4866f249ab28ef9b56f8bd not found: ID does not exist" containerID="250149d4ac6c9927a1f220a5b89269578957e15d3a4866f249ab28ef9b56f8bd"
Jul 29 13:14:23.006813 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:23.006745    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"250149d4ac6c9927a1f220a5b89269578957e15d3a4866f249ab28ef9b56f8bd"} err="failed to get container status \"250149d4ac6c9927a1f220a5b89269578957e15d3a4866f249ab28ef9b56f8bd\": rpc error: code = NotFound desc = could not find container \"250149d4ac6c9927a1f220a5b89269578957e15d3a4866f249ab28ef9b56f8bd\": container with ID starting with 250149d4ac6c9927a1f220a5b89269578957e15d3a4866f249ab28ef9b56f8bd not found: ID does not exist"
Jul 29 13:14:23.006813 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:23.006765    2567 scope.go:117] "RemoveContainer" containerID="53f03c83017ff119e5f59b37728092095fbade296e4ef2594040918cb826ce86"
Jul 29 13:14:23.014937 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:23.014914    2567 scope.go:117] "RemoveContainer" containerID="53f03c83017ff119e5f59b37728092095fbade296e4ef2594040918cb826ce86"
Jul 29 13:14:23.015188 ip-10-0-128-25 kubenswrapper[2567]: E0729 13:14:23.015168    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53f03c83017ff119e5f59b37728092095fbade296e4ef2594040918cb826ce86\": container with ID starting with 53f03c83017ff119e5f59b37728092095fbade296e4ef2594040918cb826ce86 not found: ID does not exist" containerID="53f03c83017ff119e5f59b37728092095fbade296e4ef2594040918cb826ce86"
Jul 29 13:14:23.015238 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:23.015194    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53f03c83017ff119e5f59b37728092095fbade296e4ef2594040918cb826ce86"} err="failed to get container status \"53f03c83017ff119e5f59b37728092095fbade296e4ef2594040918cb826ce86\": rpc error: code = NotFound desc = could not find container \"53f03c83017ff119e5f59b37728092095fbade296e4ef2594040918cb826ce86\": container with ID starting with 53f03c83017ff119e5f59b37728092095fbade296e4ef2594040918cb826ce86 not found: ID does not exist"
Jul 29 13:14:23.103826 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:23.103767    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2cdsr\" (UniqueName: \"kubernetes.io/projected/a8255ac9-4946-43e3-b9cd-de66abc45ecb-kube-api-access-2cdsr\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 13:14:23.103826 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:23.103789    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jrmn5\" (UniqueName: \"kubernetes.io/projected/36b786f9-d92d-4aad-9b1b-979874fd5242-kube-api-access-jrmn5\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 13:14:23.103826 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:23.103799    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8255ac9-4946-43e3-b9cd-de66abc45ecb-maas-api-tls\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 13:14:23.103826 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:23.103807    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8255ac9-4946-43e3-b9cd-de66abc45ecb-openshift-service-ca\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 13:14:23.103826 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:23.103814    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36b786f9-d92d-4aad-9b1b-979874fd5242-plugins-config-volume\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 13:14:23.338028 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:23.338000    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-2f4f5e-676676d78-xgvzm"]
Jul 29 13:14:23.347813 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:23.347792    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-2f4f5e-676676d78-xgvzm"]
Jul 29 13:14:23.366534 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:23.366482    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-2f4f5e-55498bd8c4-sp6jn"]
Jul 29 13:14:23.383325 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:23.383299    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-2f4f5e-55498bd8c4-sp6jn"]
Jul 29 13:14:23.922348 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:23.922315    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36b786f9-d92d-4aad-9b1b-979874fd5242" path="/var/lib/kubelet/pods/36b786f9-d92d-4aad-9b1b-979874fd5242/volumes"
Jul 29 13:14:23.922731 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:23.922715    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8255ac9-4946-43e3-b9cd-de66abc45ecb" path="/var/lib/kubelet/pods/a8255ac9-4946-43e3-b9cd-de66abc45ecb/volumes"
Jul 29 13:14:23.923014 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:23.923002    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f04416dd-0898-4692-a240-78955f6ffa2d" path="/var/lib/kubelet/pods/f04416dd-0898-4692-a240-78955f6ffa2d/volumes"
Jul 29 13:14:33.638557 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:33.638524    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-f0ad55-5c8fcc6b55-gb5dw"]
Jul 29 13:14:33.639146 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:33.639123    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8255ac9-4946-43e3-b9cd-de66abc45ecb" containerName="maas-api"
Jul 29 13:14:33.639229 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:33.639149    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8255ac9-4946-43e3-b9cd-de66abc45ecb" containerName="maas-api"
Jul 29 13:14:33.639229 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:33.639166    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36b786f9-d92d-4aad-9b1b-979874fd5242" containerName="payload-processing"
Jul 29 13:14:33.639229 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:33.639175    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="36b786f9-d92d-4aad-9b1b-979874fd5242" containerName="payload-processing"
Jul 29 13:14:33.639229 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:33.639192    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f04416dd-0898-4692-a240-78955f6ffa2d" containerName="revoke-keys"
Jul 29 13:14:33.639229 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:33.639201    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="f04416dd-0898-4692-a240-78955f6ffa2d" containerName="revoke-keys"
Jul 29 13:14:33.639500 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:33.639312    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="f04416dd-0898-4692-a240-78955f6ffa2d" containerName="revoke-keys"
Jul 29 13:14:33.639500 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:33.639326    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8255ac9-4946-43e3-b9cd-de66abc45ecb" containerName="maas-api"
Jul 29 13:14:33.639500 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:33.639335    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="36b786f9-d92d-4aad-9b1b-979874fd5242" containerName="payload-processing"
Jul 29 13:14:33.642725 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:33.642703    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f0ad55-5c8fcc6b55-gb5dw"
Jul 29 13:14:33.645108 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:33.645087    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-f0ad55-serving-cert\""
Jul 29 13:14:33.654075 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:33.654053    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-f0ad55-5c8fcc6b55-gb5dw"]
Jul 29 13:14:33.787477 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:33.787450    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8e9938b-04b9-483f-88de-606e938626bc-maas-api-tls\") pod \"maas-api-e2e-shared-a-f0ad55-5c8fcc6b55-gb5dw\" (UID: \"a8e9938b-04b9-483f-88de-606e938626bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f0ad55-5c8fcc6b55-gb5dw"
Jul 29 13:14:33.787620 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:33.787551    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8e9938b-04b9-483f-88de-606e938626bc-openshift-service-ca\") pod \"maas-api-e2e-shared-a-f0ad55-5c8fcc6b55-gb5dw\" (UID: \"a8e9938b-04b9-483f-88de-606e938626bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f0ad55-5c8fcc6b55-gb5dw"
Jul 29 13:14:33.787673 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:33.787623    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27cmw\" (UniqueName: \"kubernetes.io/projected/a8e9938b-04b9-483f-88de-606e938626bc-kube-api-access-27cmw\") pod \"maas-api-e2e-shared-a-f0ad55-5c8fcc6b55-gb5dw\" (UID: \"a8e9938b-04b9-483f-88de-606e938626bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f0ad55-5c8fcc6b55-gb5dw"
Jul 29 13:14:33.888676 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:33.888605    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8e9938b-04b9-483f-88de-606e938626bc-openshift-service-ca\") pod \"maas-api-e2e-shared-a-f0ad55-5c8fcc6b55-gb5dw\" (UID: \"a8e9938b-04b9-483f-88de-606e938626bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f0ad55-5c8fcc6b55-gb5dw"
Jul 29 13:14:33.888676 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:33.888657    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27cmw\" (UniqueName: \"kubernetes.io/projected/a8e9938b-04b9-483f-88de-606e938626bc-kube-api-access-27cmw\") pod \"maas-api-e2e-shared-a-f0ad55-5c8fcc6b55-gb5dw\" (UID: \"a8e9938b-04b9-483f-88de-606e938626bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f0ad55-5c8fcc6b55-gb5dw"
Jul 29 13:14:33.888827 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:33.888775    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8e9938b-04b9-483f-88de-606e938626bc-maas-api-tls\") pod \"maas-api-e2e-shared-a-f0ad55-5c8fcc6b55-gb5dw\" (UID: \"a8e9938b-04b9-483f-88de-606e938626bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f0ad55-5c8fcc6b55-gb5dw"
Jul 29 13:14:33.889263 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:33.889237    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8e9938b-04b9-483f-88de-606e938626bc-openshift-service-ca\") pod \"maas-api-e2e-shared-a-f0ad55-5c8fcc6b55-gb5dw\" (UID: \"a8e9938b-04b9-483f-88de-606e938626bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f0ad55-5c8fcc6b55-gb5dw"
Jul 29 13:14:33.891100 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:33.891073    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8e9938b-04b9-483f-88de-606e938626bc-maas-api-tls\") pod \"maas-api-e2e-shared-a-f0ad55-5c8fcc6b55-gb5dw\" (UID: \"a8e9938b-04b9-483f-88de-606e938626bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f0ad55-5c8fcc6b55-gb5dw"
Jul 29 13:14:33.928268 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:33.928250    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-27cmw\" (UniqueName: \"kubernetes.io/projected/a8e9938b-04b9-483f-88de-606e938626bc-kube-api-access-27cmw\") pod \"maas-api-e2e-shared-a-f0ad55-5c8fcc6b55-gb5dw\" (UID: \"a8e9938b-04b9-483f-88de-606e938626bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f0ad55-5c8fcc6b55-gb5dw"
Jul 29 13:14:33.954105 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:33.954084    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f0ad55-5c8fcc6b55-gb5dw"
Jul 29 13:14:34.117646 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:34.117624    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-f0ad55-5c8fcc6b55-gb5dw"]
Jul 29 13:14:34.118814 ip-10-0-128-25 kubenswrapper[2567]: W0729 13:14:34.118787    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8e9938b_04b9_483f_88de_606e938626bc.slice/crio-58c5a0a294e187a2319081253224785db2bb56f4373129cbae311cbeda1c73e1 WatchSource:0}: Error finding container 58c5a0a294e187a2319081253224785db2bb56f4373129cbae311cbeda1c73e1: Status 404 returned error can't find the container with id 58c5a0a294e187a2319081253224785db2bb56f4373129cbae311cbeda1c73e1
Jul 29 13:14:35.077192 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:35.077153    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f0ad55-5c8fcc6b55-gb5dw" event={"ID":"a8e9938b-04b9-483f-88de-606e938626bc","Type":"ContainerStarted","Data":"58c5a0a294e187a2319081253224785db2bb56f4373129cbae311cbeda1c73e1"}
Jul 29 13:14:36.084499 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:36.084465    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f0ad55-5c8fcc6b55-gb5dw" event={"ID":"a8e9938b-04b9-483f-88de-606e938626bc","Type":"ContainerStarted","Data":"977bf2dd6b045be54b55b2f1c59520150baf587a213dd6262b247e5f2a0d2225"}
Jul 29 13:14:36.084907 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:36.084596    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f0ad55-5c8fcc6b55-gb5dw"
Jul 29 13:14:36.102093 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:36.102035    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f0ad55-5c8fcc6b55-gb5dw" podStartSLOduration=1.898455841 podStartE2EDuration="3.10201574s" totalImagesPullingTime="1.203559899s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 13:14:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 13:14:36.10155815 +0000 UTC m=+2958.795246472" watchObservedRunningTime="2026-07-29 13:14:36.10201574 +0000 UTC m=+2958.795704057"
Jul 29 13:14:42.096835 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:14:42.096753    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f0ad55-5c8fcc6b55-gb5dw"
Jul 29 13:15:00.146528 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:15:00.146495    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-f0ad55-29755515-jhpb7"]
Jul 29 13:15:00.150026 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:15:00.150006    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29755515-txk4l"]
Jul 29 13:15:00.150144 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:15:00.150120    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-f0ad55-29755515-jhpb7"
Jul 29 13:15:00.152532 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:15:00.152513    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7gpd2\""
Jul 29 13:15:00.153327 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:15:00.153311    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755515-txk4l"
Jul 29 13:15:00.158050 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:15:00.157677    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29755515-txk4l"]
Jul 29 13:15:00.162334 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:15:00.162312    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-f0ad55-29755515-jhpb7"]
Jul 29 13:15:00.164216 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:15:00.164195    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-f0ad55-29755515-jhpb7"
Jul 29 13:15:00.170893 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:15:00.170876    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755515-txk4l"
Jul 29 13:15:00.308914 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:15:00.308841    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-f0ad55-29755515-jhpb7"]
Jul 29 13:15:00.310257 ip-10-0-128-25 kubenswrapper[2567]: W0729 13:15:00.310226    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c509298_80f9_432e_b271_e4fdb352f1fc.slice/crio-89534e299b2629c47db2647724a1af436848361ba357b0afe033c51d03a8e334 WatchSource:0}: Error finding container 89534e299b2629c47db2647724a1af436848361ba357b0afe033c51d03a8e334: Status 404 returned error can't find the container with id 89534e299b2629c47db2647724a1af436848361ba357b0afe033c51d03a8e334
Jul 29 13:15:00.542421 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:15:00.542387    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29755515-txk4l"]
Jul 29 13:15:00.543446 ip-10-0-128-25 kubenswrapper[2567]: W0729 13:15:00.543396    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc279df75_8a11_45a6_b915_6ab723f191b0.slice/crio-820256379ef0e9c44063ef989656c8758bcda78fb8de4989c710d46f309c3c10 WatchSource:0}: Error finding container 820256379ef0e9c44063ef989656c8758bcda78fb8de4989c710d46f309c3c10: Status 404 returned error can't find the container with id 820256379ef0e9c44063ef989656c8758bcda78fb8de4989c710d46f309c3c10
Jul 29 13:15:01.239076 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:15:01.239047    2567 generic.go:356] "Generic (PLEG): container finished" podID="7c509298-80f9-432e-b271-e4fdb352f1fc" containerID="6853c8792bd342cd20b25f5a268f2e610599a4e07f6f4e45e8887587491a2ac3" exitCode=0
Jul 29 13:15:01.239379 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:15:01.239126    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-f0ad55-29755515-jhpb7" event={"ID":"7c509298-80f9-432e-b271-e4fdb352f1fc","Type":"ContainerDied","Data":"6853c8792bd342cd20b25f5a268f2e610599a4e07f6f4e45e8887587491a2ac3"}
Jul 29 13:15:01.239379 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:15:01.239173    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-f0ad55-29755515-jhpb7" event={"ID":"7c509298-80f9-432e-b271-e4fdb352f1fc","Type":"ContainerStarted","Data":"89534e299b2629c47db2647724a1af436848361ba357b0afe033c51d03a8e334"}
Jul 29 13:15:01.240716 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:15:01.240693    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755515-txk4l" event={"ID":"c279df75-8a11-45a6-b915-6ab723f191b0","Type":"ContainerStarted","Data":"27b208dff14a13dfd47bbc5bcabcb0a10fc042594d9a15e5e5f69f1470aafeb3"}
Jul 29 13:15:01.240806 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:15:01.240732    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755515-txk4l" event={"ID":"c279df75-8a11-45a6-b915-6ab723f191b0","Type":"ContainerStarted","Data":"820256379ef0e9c44063ef989656c8758bcda78fb8de4989c710d46f309c3c10"}
Jul 29 13:15:01.273269 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:15:01.273226    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755515-txk4l" podStartSLOduration=1.273212966 podStartE2EDuration="1.273212966s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 13:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 13:15:01.271089433 +0000 UTC m=+2983.964777764" watchObservedRunningTime="2026-07-29 13:15:01.273212966 +0000 UTC m=+2983.966901276"
Jul 29 13:15:02.249743 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:15:02.249698    2567 generic.go:356] "Generic (PLEG): container finished" podID="c279df75-8a11-45a6-b915-6ab723f191b0" containerID="27b208dff14a13dfd47bbc5bcabcb0a10fc042594d9a15e5e5f69f1470aafeb3" exitCode=0
Jul 29 13:15:02.250204 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:15:02.249801    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755515-txk4l" event={"ID":"c279df75-8a11-45a6-b915-6ab723f191b0","Type":"ContainerDied","Data":"27b208dff14a13dfd47bbc5bcabcb0a10fc042594d9a15e5e5f69f1470aafeb3"}
Jul 29 13:15:02.390036 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:15:02.390012    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-f0ad55-29755515-jhpb7"
Jul 29 13:15:02.844322 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:15:02.844285    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-11a904-7c9668d9d9-msxz6"]
Jul 29 13:15:02.844767 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:15:02.844753    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c509298-80f9-432e-b271-e4fdb352f1fc" containerName="cleanup"
Jul 29 13:15:02.844821 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:15:02.844769    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c509298-80f9-432e-b271-e4fdb352f1fc" containerName="cleanup"
Jul 29 13:15:02.844863 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:15:02.844854    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c509298-80f9-432e-b271-e4fdb352f1fc" containerName="cleanup"
Jul 29 13:15:02.847910 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:15:02.847892    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-11a904-7c9668d9d9-msxz6"
Jul 29 13:15:02.850964 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:15:02.850943    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-11a904-serving-cert\""
Jul 29 13:15:02.861054 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:15:02.861032    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-11a904-7c9668d9d9-msxz6"]
Jul 29 13:15:02.918904 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:15:02.918880    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f2232e07-a88e-4a23-8bf2-f290c0bd7116-maas-api-tls\") pod \"maas-api-e2e-shared-b-11a904-7c9668d9d9-msxz6\" (UID: \"f2232e07-a88e-4a23-8bf2-f290c0bd7116\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-11a904-7c9668d9d9-msxz6"
Jul 29 13:15:02.919006 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:15:02.918912    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2232e07-a88e-4a23-8bf2-f290c0bd7116-openshift-service-ca\") pod \"maas-api-e2e-shared-b-11a904-7c9668d9d9-msxz6\" (UID: \"f2232e07-a88e-4a23-8bf2-f290c0bd7116\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-11a904-7c9668d9d9-msxz6"
Jul 29 13:15:02.919006 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:15:02.918946    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkhz5\" (UniqueName: \"kubernetes.io/projected/f2232e07-a88e-4a23-8bf2-f290c0bd7116-kube-api-access-nkhz5\") pod \"maas-api-e2e-shared-b-11a904-7c9668d9d9-msxz6\" (UID: \"f2232e07-a88e-4a23-8bf2-f290c0bd7116\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-11a904-7c9668d9d9-msxz6"
Jul 29 13:15:03.020292 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:15:03.020265    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f2232e07-a88e-4a23-8bf2-f290c0bd7116-maas-api-tls\") pod \"maas-api-e2e-shared-b-11a904-7c9668d9d9-msxz6\" (UID: \"f2232e07-a88e-4a23-8bf2-f290c0bd7116\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-11a904-7c9668d9d9-msxz6"
Jul 29 13:15:03.020419 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:15:03.020302    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2232e07-a88e-4a23-8bf2-f290c0bd7116-openshift-service-ca\") pod \"maas-api-e2e-shared-b-11a904-7c9668d9d9-msxz6\" (UID: \"f2232e07-a88e-4a23-8bf2-f290c0bd7116\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-11a904-7c9668d9d9-msxz6"
Jul 29 13:15:03.020419 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:15:03.020340    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkhz5\" (UniqueName: \"kubernetes.io/projected/f2232e07-a88e-4a23-8bf2-f290c0bd7116-kube-api-access-nkhz5\") pod \"maas-api-e2e-shared-b-11a904-7c9668d9d9-msxz6\" (UID: \"f2232e07-a88e-4a23-8bf2-f290c0bd7116\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-11a904-7c9668d9d9-msxz6"
Jul 29 13:15:03.020933 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:15:03.020909    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2232e07-a88e-4a23-8bf2-f290c0bd7116-openshift-service-ca\") pod \"maas-api-e2e-shared-b-11a904-7c9668d9d9-msxz6\" (UID: \"f2232e07-a88e-4a23-8bf2-f290c0bd7116\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-11a904-7c9668d9d9-msxz6"
Jul 29 13:15:03.022983 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:15:03.022886    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f2232e07-a88e-4a23-8bf2-f290c0bd7116-maas-api-tls\") pod \"maas-api-e2e-shared-b-11a904-7c9668d9d9-msxz6\" (UID: \"f2232e07-a88e-4a23-8bf2-f290c0bd7116\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-11a904-7c9668d9d9-msxz6"
Jul 29 13:15:03.028303 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:15:03.028284    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkhz5\" (UniqueName: \"kubernetes.io/projected/f2232e07-a88e-4a23-8bf2-f290c0bd7116-kube-api-access-nkhz5\") pod \"maas-api-e2e-shared-b-11a904-7c9668d9d9-msxz6\" (UID: \"f2232e07-a88e-4a23-8bf2-f290c0bd7116\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-11a904-7c9668d9d9-msxz6"
Jul 29 13:15:03.159533 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:15:03.159480    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-11a904-7c9668d9d9-msxz6"
Jul 29 13:15:03.260181 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:15:03.260158    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-f0ad55-29755515-jhpb7"
Jul 29 13:15:03.260586 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:15:03.260339    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-f0ad55-29755515-jhpb7" event={"ID":"7c509298-80f9-432e-b271-e4fdb352f1fc","Type":"ContainerDied","Data":"89534e299b2629c47db2647724a1af436848361ba357b0afe033c51d03a8e334"}
Jul 29 13:15:03.260586 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:15:03.260417    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="89534e299b2629c47db2647724a1af436848361ba357b0afe033c51d03a8e334"
Jul 29 13:15:03.307605 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:15:03.307572    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-11a904-7c9668d9d9-msxz6"]
Jul 29 13:15:03.309341 ip-10-0-128-25 kubenswrapper[2567]: W0729 13:15:03.309310    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2232e07_a88e_4a23_8bf2_f290c0bd7116.slice/crio-9c45389c81f41c598da2b02348f2692dd9a39cfa26e85b49761d9ba40dd97f40 WatchSource:0}: Error finding container 9c45389c81f41c598da2b02348f2692dd9a39cfa26e85b49761d9ba40dd97f40: Status 404 returned error can't find the container with id 9c45389c81f41c598da2b02348f2692dd9a39cfa26e85b49761d9ba40dd97f40
Jul 29 13:15:03.310931 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:15:03.310912    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 13:15:03.411013 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:15:03.410961    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755515-txk4l"
Jul 29 13:15:04.267995 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:15:04.267938    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755515-txk4l"
Jul 29 13:15:04.267995 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:15:04.267950    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755515-txk4l" event={"ID":"c279df75-8a11-45a6-b915-6ab723f191b0","Type":"ContainerDied","Data":"820256379ef0e9c44063ef989656c8758bcda78fb8de4989c710d46f309c3c10"}
Jul 29 13:15:04.268389 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:15:04.267993    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="820256379ef0e9c44063ef989656c8758bcda78fb8de4989c710d46f309c3c10"
Jul 29 13:15:04.269391 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:15:04.269320    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-11a904-7c9668d9d9-msxz6" event={"ID":"f2232e07-a88e-4a23-8bf2-f290c0bd7116","Type":"ContainerStarted","Data":"9c45389c81f41c598da2b02348f2692dd9a39cfa26e85b49761d9ba40dd97f40"}
Jul 29 13:15:05.283853 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:15:05.283816    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-11a904-7c9668d9d9-msxz6" event={"ID":"f2232e07-a88e-4a23-8bf2-f290c0bd7116","Type":"ContainerStarted","Data":"c7040e149a75e8385cdc0488e2d30f79195874334ed44ffdf7e9879795331b43"}
Jul 29 13:15:05.284225 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:15:05.283994    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-11a904-7c9668d9d9-msxz6"
Jul 29 13:15:05.305757 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:15:05.305712    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-11a904-7c9668d9d9-msxz6" podStartSLOduration=1.7592702180000002 podStartE2EDuration="3.30569915s" totalImagesPullingTime="1.546428932s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 13:15:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 13:15:05.303615498 +0000 UTC m=+2987.997303819" watchObservedRunningTime="2026-07-29 13:15:05.30569915 +0000 UTC m=+2987.999387461"
Jul 29 13:15:11.294532 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:15:11.294502    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-11a904-7c9668d9d9-msxz6"
Jul 29 13:15:18.160331 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:15:18.160235    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kj82f_a6d5f3a2-26cc-40a7-9841-2fbfc247661a/ovn-acl-logging/0.log"
Jul 29 13:15:18.185038 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:15:18.168102    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kj82f_a6d5f3a2-26cc-40a7-9841-2fbfc247661a/ovn-acl-logging/0.log"
Jul 29 13:17:13.335803 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:17:13.335773    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5lknx"]
Jul 29 13:17:13.336251 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:17:13.336157    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c279df75-8a11-45a6-b915-6ab723f191b0" containerName="cleanup"
Jul 29 13:17:13.336251 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:17:13.336168    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c279df75-8a11-45a6-b915-6ab723f191b0" containerName="cleanup"
Jul 29 13:17:13.336251 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:17:13.336239    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="c279df75-8a11-45a6-b915-6ab723f191b0" containerName="cleanup"
Jul 29 13:17:13.339184 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:17:13.339167    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5lknx"
Jul 29 13:17:13.341729 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:17:13.341709    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-zngpf\""
Jul 29 13:17:13.349511 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:17:13.349491    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5lknx"]
Jul 29 13:17:13.430218 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:17:13.430196    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flnsh\" (UniqueName: \"kubernetes.io/projected/7a0c93d3-0fcd-4f48-ba67-13f38a885184-kube-api-access-flnsh\") pod \"kuadrant-operator-catalog-5lknx\" (UID: \"7a0c93d3-0fcd-4f48-ba67-13f38a885184\") " pod="kuadrant-system/kuadrant-operator-catalog-5lknx"
Jul 29 13:17:13.531090 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:17:13.531060    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flnsh\" (UniqueName: \"kubernetes.io/projected/7a0c93d3-0fcd-4f48-ba67-13f38a885184-kube-api-access-flnsh\") pod \"kuadrant-operator-catalog-5lknx\" (UID: \"7a0c93d3-0fcd-4f48-ba67-13f38a885184\") " pod="kuadrant-system/kuadrant-operator-catalog-5lknx"
Jul 29 13:17:13.543579 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:17:13.543551    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-flnsh\" (UniqueName: \"kubernetes.io/projected/7a0c93d3-0fcd-4f48-ba67-13f38a885184-kube-api-access-flnsh\") pod \"kuadrant-operator-catalog-5lknx\" (UID: \"7a0c93d3-0fcd-4f48-ba67-13f38a885184\") " pod="kuadrant-system/kuadrant-operator-catalog-5lknx"
Jul 29 13:17:13.648898 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:17:13.648844    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5lknx"
Jul 29 13:17:13.806314 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:17:13.806280    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5lknx"]
Jul 29 13:17:13.807240 ip-10-0-128-25 kubenswrapper[2567]: W0729 13:17:13.807209    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a0c93d3_0fcd_4f48_ba67_13f38a885184.slice/crio-1ca1590a8f60511176d1f76c81590887967266a492e9bb270b77d23fd2cc28c1 WatchSource:0}: Error finding container 1ca1590a8f60511176d1f76c81590887967266a492e9bb270b77d23fd2cc28c1: Status 404 returned error can't find the container with id 1ca1590a8f60511176d1f76c81590887967266a492e9bb270b77d23fd2cc28c1
Jul 29 13:17:14.054687 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:17:14.054661    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5lknx" event={"ID":"7a0c93d3-0fcd-4f48-ba67-13f38a885184","Type":"ContainerStarted","Data":"1ca1590a8f60511176d1f76c81590887967266a492e9bb270b77d23fd2cc28c1"}
Jul 29 13:17:16.069246 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:17:16.069195    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5lknx" event={"ID":"7a0c93d3-0fcd-4f48-ba67-13f38a885184","Type":"ContainerStarted","Data":"6c5e1bb69b3eb6b385a37c74beaae702c19494885dac50e32121ec1f8e724f7b"}
Jul 29 13:17:16.088346 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:17:16.088300    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-5lknx" podStartSLOduration=1.037407436 podStartE2EDuration="3.088286832s" totalImagesPullingTime="2.050879396s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 13:17:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 13:17:16.086663585 +0000 UTC m=+3118.780351897" watchObservedRunningTime="2026-07-29 13:17:16.088286832 +0000 UTC m=+3118.781975142"
Jul 29 13:17:23.649519 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:17:23.649443    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-5lknx"
Jul 29 13:17:23.649519 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:17:23.649499    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-5lknx"
Jul 29 13:17:23.671167 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:17:23.671143    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-5lknx"
Jul 29 13:17:24.136854 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:17:24.136828    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-5lknx"
Jul 29 13:17:24.175730 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:17:24.175701    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5lknx"]
Jul 29 13:17:26.128720 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:17:26.128650    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-5lknx" podUID="7a0c93d3-0fcd-4f48-ba67-13f38a885184" containerName="registry-server" containerID="cri-o://6c5e1bb69b3eb6b385a37c74beaae702c19494885dac50e32121ec1f8e724f7b" gracePeriod=2
Jul 29 13:17:26.384875 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:17:26.384818    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5lknx"
Jul 29 13:17:26.445433 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:17:26.445393    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-flnsh\" (UniqueName: \"kubernetes.io/projected/7a0c93d3-0fcd-4f48-ba67-13f38a885184-kube-api-access-flnsh\") pod \"7a0c93d3-0fcd-4f48-ba67-13f38a885184\" (UID: \"7a0c93d3-0fcd-4f48-ba67-13f38a885184\") "
Jul 29 13:17:26.447292 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:17:26.447267    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a0c93d3-0fcd-4f48-ba67-13f38a885184-kube-api-access-flnsh" (OuterVolumeSpecName: "kube-api-access-flnsh") pod "7a0c93d3-0fcd-4f48-ba67-13f38a885184" (UID: "7a0c93d3-0fcd-4f48-ba67-13f38a885184"). InnerVolumeSpecName "kube-api-access-flnsh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 13:17:26.546046 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:17:26.546018    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-flnsh\" (UniqueName: \"kubernetes.io/projected/7a0c93d3-0fcd-4f48-ba67-13f38a885184-kube-api-access-flnsh\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 13:17:27.136182 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:17:27.136152    2567 generic.go:356] "Generic (PLEG): container finished" podID="7a0c93d3-0fcd-4f48-ba67-13f38a885184" containerID="6c5e1bb69b3eb6b385a37c74beaae702c19494885dac50e32121ec1f8e724f7b" exitCode=0
Jul 29 13:17:27.136641 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:17:27.136198    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5lknx"
Jul 29 13:17:27.136641 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:17:27.136236    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5lknx" event={"ID":"7a0c93d3-0fcd-4f48-ba67-13f38a885184","Type":"ContainerDied","Data":"6c5e1bb69b3eb6b385a37c74beaae702c19494885dac50e32121ec1f8e724f7b"}
Jul 29 13:17:27.136641 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:17:27.136283    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5lknx" event={"ID":"7a0c93d3-0fcd-4f48-ba67-13f38a885184","Type":"ContainerDied","Data":"1ca1590a8f60511176d1f76c81590887967266a492e9bb270b77d23fd2cc28c1"}
Jul 29 13:17:27.136641 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:17:27.136308    2567 scope.go:117] "RemoveContainer" containerID="6c5e1bb69b3eb6b385a37c74beaae702c19494885dac50e32121ec1f8e724f7b"
Jul 29 13:17:27.146857 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:17:27.146837    2567 scope.go:117] "RemoveContainer" containerID="6c5e1bb69b3eb6b385a37c74beaae702c19494885dac50e32121ec1f8e724f7b"
Jul 29 13:17:27.147113 ip-10-0-128-25 kubenswrapper[2567]: E0729 13:17:27.147096    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c5e1bb69b3eb6b385a37c74beaae702c19494885dac50e32121ec1f8e724f7b\": container with ID starting with 6c5e1bb69b3eb6b385a37c74beaae702c19494885dac50e32121ec1f8e724f7b not found: ID does not exist" containerID="6c5e1bb69b3eb6b385a37c74beaae702c19494885dac50e32121ec1f8e724f7b"
Jul 29 13:17:27.147181 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:17:27.147120    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c5e1bb69b3eb6b385a37c74beaae702c19494885dac50e32121ec1f8e724f7b"} err="failed to get container status \"6c5e1bb69b3eb6b385a37c74beaae702c19494885dac50e32121ec1f8e724f7b\": rpc error: code = NotFound desc = could not find container \"6c5e1bb69b3eb6b385a37c74beaae702c19494885dac50e32121ec1f8e724f7b\": container with ID starting with 6c5e1bb69b3eb6b385a37c74beaae702c19494885dac50e32121ec1f8e724f7b not found: ID does not exist"
Jul 29 13:17:27.161267 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:17:27.161237    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5lknx"]
Jul 29 13:17:27.164201 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:17:27.164181    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5lknx"]
Jul 29 13:17:27.921781 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:17:27.921747    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a0c93d3-0fcd-4f48-ba67-13f38a885184" path="/var/lib/kubelet/pods/7a0c93d3-0fcd-4f48-ba67-13f38a885184/volumes"
Jul 29 13:18:18.352237 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:18:18.352202    2567 scope.go:117] "RemoveContainer" containerID="3ff7cc42d758755e4233dd7f055f4d2ca6b255d906cb283ded5988751129f7ba"
Jul 29 13:19:18.379457 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:19:18.379363    2567 scope.go:117] "RemoveContainer" containerID="e620e69ef5871a07cc4b4cc066039ae9f8e1c1308cd629ca55c1ded6cf2bdd3d"
Jul 29 13:20:18.201328 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:20:18.201215    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kj82f_a6d5f3a2-26cc-40a7-9841-2fbfc247661a/ovn-acl-logging/0.log"
Jul 29 13:20:18.210201 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:20:18.210183    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kj82f_a6d5f3a2-26cc-40a7-9841-2fbfc247661a/ovn-acl-logging/0.log"
Jul 29 13:20:18.402416 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:20:18.402381    2567 scope.go:117] "RemoveContainer" containerID="075b79020cf1bcd508b352c8b6f0d141766d25a24b6b4bdd3a067474f06cd44b"
Jul 29 13:20:18.411147 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:20:18.411126    2567 scope.go:117] "RemoveContainer" containerID="1ef1aa5dd02a633cc13227a25a68cdc96d0827e038d3cefdc6187374be115eaa"
Jul 29 13:25:14.606805 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:14.606772    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-86053a-5cd6f65489-qmx7g"]
Jul 29 13:25:14.607225 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:14.607160    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a0c93d3-0fcd-4f48-ba67-13f38a885184" containerName="registry-server"
Jul 29 13:25:14.607225 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:14.607171    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a0c93d3-0fcd-4f48-ba67-13f38a885184" containerName="registry-server"
Jul 29 13:25:14.607300 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:14.607250    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a0c93d3-0fcd-4f48-ba67-13f38a885184" containerName="registry-server"
Jul 29 13:25:14.610345 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:14.610329    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-86053a-5cd6f65489-qmx7g"
Jul 29 13:25:14.612762 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:14.612742    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-86053a-serving-cert\""
Jul 29 13:25:14.620284 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:14.620259    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-86053a-5cd6f65489-qmx7g"]
Jul 29 13:25:14.746731 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:14.746704    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcz46\" (UniqueName: \"kubernetes.io/projected/78444b1d-6636-4495-b9ad-f35013e35690-kube-api-access-tcz46\") pod \"maas-api-e2e-ipp-a-86053a-5cd6f65489-qmx7g\" (UID: \"78444b1d-6636-4495-b9ad-f35013e35690\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-86053a-5cd6f65489-qmx7g"
Jul 29 13:25:14.746876 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:14.746776    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/78444b1d-6636-4495-b9ad-f35013e35690-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-86053a-5cd6f65489-qmx7g\" (UID: \"78444b1d-6636-4495-b9ad-f35013e35690\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-86053a-5cd6f65489-qmx7g"
Jul 29 13:25:14.746876 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:14.746815    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/78444b1d-6636-4495-b9ad-f35013e35690-maas-api-tls\") pod \"maas-api-e2e-ipp-a-86053a-5cd6f65489-qmx7g\" (UID: \"78444b1d-6636-4495-b9ad-f35013e35690\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-86053a-5cd6f65489-qmx7g"
Jul 29 13:25:14.847675 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:14.847649    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/78444b1d-6636-4495-b9ad-f35013e35690-maas-api-tls\") pod \"maas-api-e2e-ipp-a-86053a-5cd6f65489-qmx7g\" (UID: \"78444b1d-6636-4495-b9ad-f35013e35690\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-86053a-5cd6f65489-qmx7g"
Jul 29 13:25:14.847792 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:14.847712    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcz46\" (UniqueName: \"kubernetes.io/projected/78444b1d-6636-4495-b9ad-f35013e35690-kube-api-access-tcz46\") pod \"maas-api-e2e-ipp-a-86053a-5cd6f65489-qmx7g\" (UID: \"78444b1d-6636-4495-b9ad-f35013e35690\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-86053a-5cd6f65489-qmx7g"
Jul 29 13:25:14.847792 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:14.847779    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/78444b1d-6636-4495-b9ad-f35013e35690-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-86053a-5cd6f65489-qmx7g\" (UID: \"78444b1d-6636-4495-b9ad-f35013e35690\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-86053a-5cd6f65489-qmx7g"
Jul 29 13:25:14.848474 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:14.848456    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/78444b1d-6636-4495-b9ad-f35013e35690-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-86053a-5cd6f65489-qmx7g\" (UID: \"78444b1d-6636-4495-b9ad-f35013e35690\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-86053a-5cd6f65489-qmx7g"
Jul 29 13:25:14.850210 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:14.850189    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/78444b1d-6636-4495-b9ad-f35013e35690-maas-api-tls\") pod \"maas-api-e2e-ipp-a-86053a-5cd6f65489-qmx7g\" (UID: \"78444b1d-6636-4495-b9ad-f35013e35690\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-86053a-5cd6f65489-qmx7g"
Jul 29 13:25:14.856089 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:14.856068    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcz46\" (UniqueName: \"kubernetes.io/projected/78444b1d-6636-4495-b9ad-f35013e35690-kube-api-access-tcz46\") pod \"maas-api-e2e-ipp-a-86053a-5cd6f65489-qmx7g\" (UID: \"78444b1d-6636-4495-b9ad-f35013e35690\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-86053a-5cd6f65489-qmx7g"
Jul 29 13:25:14.921568 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:14.921519    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-86053a-5cd6f65489-qmx7g"
Jul 29 13:25:15.061564 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:15.061538    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-86053a-5cd6f65489-qmx7g"]
Jul 29 13:25:15.063522 ip-10-0-128-25 kubenswrapper[2567]: W0729 13:25:15.063491    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78444b1d_6636_4495_b9ad_f35013e35690.slice/crio-06e7ba57c0dd25470abbefef761fdb9999ec46533aff09c9392448284944c4a9 WatchSource:0}: Error finding container 06e7ba57c0dd25470abbefef761fdb9999ec46533aff09c9392448284944c4a9: Status 404 returned error can't find the container with id 06e7ba57c0dd25470abbefef761fdb9999ec46533aff09c9392448284944c4a9
Jul 29 13:25:15.064797 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:15.064781    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 13:25:16.002979 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:16.002941    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-86053a-5cd6f65489-qmx7g" event={"ID":"78444b1d-6636-4495-b9ad-f35013e35690","Type":"ContainerStarted","Data":"2f95fd5b11df10b62f66764a273c2938e370d7b17ec48ac117dc80d78de8abe1"}
Jul 29 13:25:16.003391 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:16.003017    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-86053a-5cd6f65489-qmx7g" event={"ID":"78444b1d-6636-4495-b9ad-f35013e35690","Type":"ContainerStarted","Data":"06e7ba57c0dd25470abbefef761fdb9999ec46533aff09c9392448284944c4a9"}
Jul 29 13:25:16.003391 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:16.003145    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-86053a-5cd6f65489-qmx7g"
Jul 29 13:25:16.021357 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:16.021314    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-86053a-5cd6f65489-qmx7g" podStartSLOduration=1.459059082 podStartE2EDuration="2.021301377s" totalImagesPullingTime="562.242295ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 13:25:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 13:25:16.019839683 +0000 UTC m=+3598.713528016" watchObservedRunningTime="2026-07-29 13:25:16.021301377 +0000 UTC m=+3598.714989690"
Jul 29 13:25:18.244985 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:18.244874    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kj82f_a6d5f3a2-26cc-40a7-9841-2fbfc247661a/ovn-acl-logging/0.log"
Jul 29 13:25:18.260224 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:18.255019    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kj82f_a6d5f3a2-26cc-40a7-9841-2fbfc247661a/ovn-acl-logging/0.log"
Jul 29 13:25:22.014344 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:22.014309    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-86053a-5cd6f65489-qmx7g"
Jul 29 13:25:40.523502 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:40.523469    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n"]
Jul 29 13:25:40.527125 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:40.527100    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n"
Jul 29 13:25:40.530470 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:40.530445    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-bb0da0-openshift-default-dockercfg-s9clg\""
Jul 29 13:25:40.539738 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:40.539712    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n"]
Jul 29 13:25:40.544191 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:40.544163    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe280394-648b-4c05-8fd4-30535c2fc262-istiod-ca-cert\") pod \"e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n\" (UID: \"fe280394-648b-4c05-8fd4-30535c2fc262\") " pod="openshift-ingress/e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n"
Jul 29 13:25:40.544295 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:40.544216    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe280394-648b-4c05-8fd4-30535c2fc262-credential-socket\") pod \"e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n\" (UID: \"fe280394-648b-4c05-8fd4-30535c2fc262\") " pod="openshift-ingress/e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n"
Jul 29 13:25:40.544360 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:40.544340    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe280394-648b-4c05-8fd4-30535c2fc262-istio-envoy\") pod \"e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n\" (UID: \"fe280394-648b-4c05-8fd4-30535c2fc262\") " pod="openshift-ingress/e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n"
Jul 29 13:25:40.544411 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:40.544368    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe280394-648b-4c05-8fd4-30535c2fc262-istio-podinfo\") pod \"e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n\" (UID: \"fe280394-648b-4c05-8fd4-30535c2fc262\") " pod="openshift-ingress/e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n"
Jul 29 13:25:40.544411 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:40.544391    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2smpn\" (UniqueName: \"kubernetes.io/projected/fe280394-648b-4c05-8fd4-30535c2fc262-kube-api-access-2smpn\") pod \"e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n\" (UID: \"fe280394-648b-4c05-8fd4-30535c2fc262\") " pod="openshift-ingress/e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n"
Jul 29 13:25:40.544493 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:40.544429    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe280394-648b-4c05-8fd4-30535c2fc262-istio-token\") pod \"e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n\" (UID: \"fe280394-648b-4c05-8fd4-30535c2fc262\") " pod="openshift-ingress/e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n"
Jul 29 13:25:40.544493 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:40.544478    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe280394-648b-4c05-8fd4-30535c2fc262-workload-socket\") pod \"e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n\" (UID: \"fe280394-648b-4c05-8fd4-30535c2fc262\") " pod="openshift-ingress/e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n"
Jul 29 13:25:40.544557 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:40.544511    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe280394-648b-4c05-8fd4-30535c2fc262-workload-certs\") pod \"e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n\" (UID: \"fe280394-648b-4c05-8fd4-30535c2fc262\") " pod="openshift-ingress/e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n"
Jul 29 13:25:40.544557 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:40.544539    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe280394-648b-4c05-8fd4-30535c2fc262-istio-data\") pod \"e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n\" (UID: \"fe280394-648b-4c05-8fd4-30535c2fc262\") " pod="openshift-ingress/e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n"
Jul 29 13:25:40.645795 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:40.645756    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe280394-648b-4c05-8fd4-30535c2fc262-istio-envoy\") pod \"e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n\" (UID: \"fe280394-648b-4c05-8fd4-30535c2fc262\") " pod="openshift-ingress/e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n"
Jul 29 13:25:40.645951 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:40.645803    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe280394-648b-4c05-8fd4-30535c2fc262-istio-podinfo\") pod \"e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n\" (UID: \"fe280394-648b-4c05-8fd4-30535c2fc262\") " pod="openshift-ingress/e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n"
Jul 29 13:25:40.645951 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:40.645825    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2smpn\" (UniqueName: \"kubernetes.io/projected/fe280394-648b-4c05-8fd4-30535c2fc262-kube-api-access-2smpn\") pod \"e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n\" (UID: \"fe280394-648b-4c05-8fd4-30535c2fc262\") " pod="openshift-ingress/e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n"
Jul 29 13:25:40.645951 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:40.645853    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe280394-648b-4c05-8fd4-30535c2fc262-istio-token\") pod \"e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n\" (UID: \"fe280394-648b-4c05-8fd4-30535c2fc262\") " pod="openshift-ingress/e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n"
Jul 29 13:25:40.645951 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:40.645878    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe280394-648b-4c05-8fd4-30535c2fc262-workload-socket\") pod \"e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n\" (UID: \"fe280394-648b-4c05-8fd4-30535c2fc262\") " pod="openshift-ingress/e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n"
Jul 29 13:25:40.645951 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:40.645897    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe280394-648b-4c05-8fd4-30535c2fc262-workload-certs\") pod \"e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n\" (UID: \"fe280394-648b-4c05-8fd4-30535c2fc262\") " pod="openshift-ingress/e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n"
Jul 29 13:25:40.645951 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:40.645951    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe280394-648b-4c05-8fd4-30535c2fc262-istio-data\") pod \"e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n\" (UID: \"fe280394-648b-4c05-8fd4-30535c2fc262\") " pod="openshift-ingress/e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n"
Jul 29 13:25:40.646273 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:40.646127    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe280394-648b-4c05-8fd4-30535c2fc262-istiod-ca-cert\") pod \"e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n\" (UID: \"fe280394-648b-4c05-8fd4-30535c2fc262\") " pod="openshift-ingress/e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n"
Jul 29 13:25:40.646273 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:40.646196    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe280394-648b-4c05-8fd4-30535c2fc262-credential-socket\") pod \"e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n\" (UID: \"fe280394-648b-4c05-8fd4-30535c2fc262\") " pod="openshift-ingress/e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n"
Jul 29 13:25:40.646379 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:40.646321    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe280394-648b-4c05-8fd4-30535c2fc262-workload-certs\") pod \"e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n\" (UID: \"fe280394-648b-4c05-8fd4-30535c2fc262\") " pod="openshift-ingress/e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n"
Jul 29 13:25:40.646379 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:40.646341    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe280394-648b-4c05-8fd4-30535c2fc262-istio-data\") pod \"e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n\" (UID: \"fe280394-648b-4c05-8fd4-30535c2fc262\") " pod="openshift-ingress/e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n"
Jul 29 13:25:40.646611 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:40.646588    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe280394-648b-4c05-8fd4-30535c2fc262-credential-socket\") pod \"e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n\" (UID: \"fe280394-648b-4c05-8fd4-30535c2fc262\") " pod="openshift-ingress/e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n"
Jul 29 13:25:40.646845 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:40.646652    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe280394-648b-4c05-8fd4-30535c2fc262-workload-socket\") pod \"e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n\" (UID: \"fe280394-648b-4c05-8fd4-30535c2fc262\") " pod="openshift-ingress/e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n"
Jul 29 13:25:40.646962 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:40.646855    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe280394-648b-4c05-8fd4-30535c2fc262-istiod-ca-cert\") pod \"e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n\" (UID: \"fe280394-648b-4c05-8fd4-30535c2fc262\") " pod="openshift-ingress/e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n"
Jul 29 13:25:40.648295 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:40.648274    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe280394-648b-4c05-8fd4-30535c2fc262-istio-envoy\") pod \"e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n\" (UID: \"fe280394-648b-4c05-8fd4-30535c2fc262\") " pod="openshift-ingress/e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n"
Jul 29 13:25:40.648442 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:40.648424    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe280394-648b-4c05-8fd4-30535c2fc262-istio-podinfo\") pod \"e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n\" (UID: \"fe280394-648b-4c05-8fd4-30535c2fc262\") " pod="openshift-ingress/e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n"
Jul 29 13:25:40.654915 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:40.654893    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe280394-648b-4c05-8fd4-30535c2fc262-istio-token\") pod \"e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n\" (UID: \"fe280394-648b-4c05-8fd4-30535c2fc262\") " pod="openshift-ingress/e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n"
Jul 29 13:25:40.655332 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:40.655313    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2smpn\" (UniqueName: \"kubernetes.io/projected/fe280394-648b-4c05-8fd4-30535c2fc262-kube-api-access-2smpn\") pod \"e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n\" (UID: \"fe280394-648b-4c05-8fd4-30535c2fc262\") " pod="openshift-ingress/e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n"
Jul 29 13:25:40.840008 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:40.839981    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n"
Jul 29 13:25:40.976395 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:40.976369    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n"]
Jul 29 13:25:40.976930 ip-10-0-128-25 kubenswrapper[2567]: W0729 13:25:40.976909    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe280394_648b_4c05_8fd4_30535c2fc262.slice/crio-b1c0f8b7e71afe15e7116b7c39eaf592be02fd5bbed50976b7d29c834a4c8be8 WatchSource:0}: Error finding container b1c0f8b7e71afe15e7116b7c39eaf592be02fd5bbed50976b7d29c834a4c8be8: Status 404 returned error can't find the container with id b1c0f8b7e71afe15e7116b7c39eaf592be02fd5bbed50976b7d29c834a4c8be8
Jul 29 13:25:41.161303 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:41.161228    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n" event={"ID":"fe280394-648b-4c05-8fd4-30535c2fc262","Type":"ContainerStarted","Data":"b1c0f8b7e71afe15e7116b7c39eaf592be02fd5bbed50976b7d29c834a4c8be8"}
Jul 29 13:25:44.123289 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:44.123258    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 13:25:44.123555 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:44.123325    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 13:25:44.123555 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:44.123353    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 13:25:45.191378 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:45.191334    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n" event={"ID":"fe280394-648b-4c05-8fd4-30535c2fc262","Type":"ContainerStarted","Data":"477d3d6af084aba2e8590db70e57ce37c823779e467eaad9417558952efcbc9b"}
Jul 29 13:25:45.213135 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:45.213076    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n" podStartSLOduration=2.06892977 podStartE2EDuration="5.2130558s" totalImagesPullingTime="3.14412603s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 13:25:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 13:25:45.212873172 +0000 UTC m=+3627.906561554" watchObservedRunningTime="2026-07-29 13:25:45.2130558 +0000 UTC m=+3627.906744134"
Jul 29 13:25:45.840483 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:45.840450    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n"
Jul 29 13:25:45.845986 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:45.845963    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n"
Jul 29 13:25:46.198164 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:46.198092    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n"
Jul 29 13:25:46.199113 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:46.199091    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n"
Jul 29 13:25:47.724116 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:47.724085    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-bb0da0-75bb5b6bb6-qbbhz"]
Jul 29 13:25:47.728051 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:47.728036    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bb0da0-75bb5b6bb6-qbbhz"
Jul 29 13:25:47.732888 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:47.732863    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-bb0da0-serving-cert\""
Jul 29 13:25:47.750500 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:47.750469    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-bb0da0-75bb5b6bb6-qbbhz"]
Jul 29 13:25:47.810544 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:47.810515    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c6dbe2f0-4e83-4f60-b80f-90fe7f05b189-maas-api-tls\") pod \"maas-api-e2e-ipp-b-bb0da0-75bb5b6bb6-qbbhz\" (UID: \"c6dbe2f0-4e83-4f60-b80f-90fe7f05b189\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bb0da0-75bb5b6bb6-qbbhz"
Jul 29 13:25:47.810677 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:47.810559    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6dbe2f0-4e83-4f60-b80f-90fe7f05b189-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-bb0da0-75bb5b6bb6-qbbhz\" (UID: \"c6dbe2f0-4e83-4f60-b80f-90fe7f05b189\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bb0da0-75bb5b6bb6-qbbhz"
Jul 29 13:25:47.810723 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:47.810688    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ztmd\" (UniqueName: \"kubernetes.io/projected/c6dbe2f0-4e83-4f60-b80f-90fe7f05b189-kube-api-access-9ztmd\") pod \"maas-api-e2e-ipp-b-bb0da0-75bb5b6bb6-qbbhz\" (UID: \"c6dbe2f0-4e83-4f60-b80f-90fe7f05b189\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bb0da0-75bb5b6bb6-qbbhz"
Jul 29 13:25:47.911463 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:47.911438    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ztmd\" (UniqueName: \"kubernetes.io/projected/c6dbe2f0-4e83-4f60-b80f-90fe7f05b189-kube-api-access-9ztmd\") pod \"maas-api-e2e-ipp-b-bb0da0-75bb5b6bb6-qbbhz\" (UID: \"c6dbe2f0-4e83-4f60-b80f-90fe7f05b189\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bb0da0-75bb5b6bb6-qbbhz"
Jul 29 13:25:47.911581 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:47.911480    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c6dbe2f0-4e83-4f60-b80f-90fe7f05b189-maas-api-tls\") pod \"maas-api-e2e-ipp-b-bb0da0-75bb5b6bb6-qbbhz\" (UID: \"c6dbe2f0-4e83-4f60-b80f-90fe7f05b189\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bb0da0-75bb5b6bb6-qbbhz"
Jul 29 13:25:47.911581 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:47.911520    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6dbe2f0-4e83-4f60-b80f-90fe7f05b189-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-bb0da0-75bb5b6bb6-qbbhz\" (UID: \"c6dbe2f0-4e83-4f60-b80f-90fe7f05b189\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bb0da0-75bb5b6bb6-qbbhz"
Jul 29 13:25:47.912209 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:47.912190    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6dbe2f0-4e83-4f60-b80f-90fe7f05b189-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-bb0da0-75bb5b6bb6-qbbhz\" (UID: \"c6dbe2f0-4e83-4f60-b80f-90fe7f05b189\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bb0da0-75bb5b6bb6-qbbhz"
Jul 29 13:25:47.913781 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:47.913764    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c6dbe2f0-4e83-4f60-b80f-90fe7f05b189-maas-api-tls\") pod \"maas-api-e2e-ipp-b-bb0da0-75bb5b6bb6-qbbhz\" (UID: \"c6dbe2f0-4e83-4f60-b80f-90fe7f05b189\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bb0da0-75bb5b6bb6-qbbhz"
Jul 29 13:25:47.922815 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:47.922775    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ztmd\" (UniqueName: \"kubernetes.io/projected/c6dbe2f0-4e83-4f60-b80f-90fe7f05b189-kube-api-access-9ztmd\") pod \"maas-api-e2e-ipp-b-bb0da0-75bb5b6bb6-qbbhz\" (UID: \"c6dbe2f0-4e83-4f60-b80f-90fe7f05b189\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bb0da0-75bb5b6bb6-qbbhz"
Jul 29 13:25:48.038727 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:48.038706    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bb0da0-75bb5b6bb6-qbbhz"
Jul 29 13:25:48.185826 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:48.185800    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-bb0da0-75bb5b6bb6-qbbhz"]
Jul 29 13:25:48.186712 ip-10-0-128-25 kubenswrapper[2567]: W0729 13:25:48.186684    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6dbe2f0_4e83_4f60_b80f_90fe7f05b189.slice/crio-4bf256dcc0f3587b76e49aefa1ba592f0446e71a008915eed1fd486f95679c61 WatchSource:0}: Error finding container 4bf256dcc0f3587b76e49aefa1ba592f0446e71a008915eed1fd486f95679c61: Status 404 returned error can't find the container with id 4bf256dcc0f3587b76e49aefa1ba592f0446e71a008915eed1fd486f95679c61
Jul 29 13:25:48.211423 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:48.211383    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bb0da0-75bb5b6bb6-qbbhz" event={"ID":"c6dbe2f0-4e83-4f60-b80f-90fe7f05b189","Type":"ContainerStarted","Data":"4bf256dcc0f3587b76e49aefa1ba592f0446e71a008915eed1fd486f95679c61"}
Jul 29 13:25:49.218798 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:49.218762    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bb0da0-75bb5b6bb6-qbbhz" event={"ID":"c6dbe2f0-4e83-4f60-b80f-90fe7f05b189","Type":"ContainerStarted","Data":"3cf890b16d0f81881a9b5a1132555460c395a78f063f963758f6592f5812bd65"}
Jul 29 13:25:49.219227 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:49.218883    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bb0da0-75bb5b6bb6-qbbhz"
Jul 29 13:25:49.239532 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:49.239489    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bb0da0-75bb5b6bb6-qbbhz" podStartSLOduration=1.897032847 podStartE2EDuration="2.239474929s" totalImagesPullingTime="342.442082ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 13:25:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 13:25:49.236617004 +0000 UTC m=+3631.930305345" watchObservedRunningTime="2026-07-29 13:25:49.239474929 +0000 UTC m=+3631.933163241"
Jul 29 13:25:55.231300 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:55.231271    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bb0da0-75bb5b6bb6-qbbhz"
Jul 29 13:25:59.288365 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:59.288334    2567 generic.go:356] "Generic (PLEG): container finished" podID="fe280394-648b-4c05-8fd4-30535c2fc262" containerID="477d3d6af084aba2e8590db70e57ce37c823779e467eaad9417558952efcbc9b" exitCode=0
Jul 29 13:25:59.288780 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:59.288428    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n" event={"ID":"fe280394-648b-4c05-8fd4-30535c2fc262","Type":"ContainerDied","Data":"477d3d6af084aba2e8590db70e57ce37c823779e467eaad9417558952efcbc9b"}
Jul 29 13:25:59.288840 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:59.288822    2567 scope.go:117] "RemoveContainer" containerID="477d3d6af084aba2e8590db70e57ce37c823779e467eaad9417558952efcbc9b"
Jul 29 13:25:59.289877 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:59.289844    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 13:25:59.289930 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:59.289918    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 13:25:59.289965 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:25:59.289954    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 13:26:00.297682 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:26:00.297634    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n" event={"ID":"fe280394-648b-4c05-8fd4-30535c2fc262","Type":"ContainerStarted","Data":"e7292479f3fcb7af0c563de4f4fe1fd51bc70e61d42154f13513293eeda0d92b"}
Jul 29 13:26:00.840324 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:26:00.840289    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n"
Jul 29 13:26:00.841320 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:26:00.841293    2567 patch_prober.go:28] interesting pod/e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Jul 29 13:26:00.841459 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:26:00.841344    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n" podUID="fe280394-648b-4c05-8fd4-30535c2fc262" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Jul 29 13:26:01.198379 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:26:01.198344    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n"
Jul 29 13:26:01.841088 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:26:01.841055    2567 patch_prober.go:28] interesting pod/e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Jul 29 13:26:01.841513 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:26:01.841117    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n" podUID="fe280394-648b-4c05-8fd4-30535c2fc262" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Jul 29 13:26:02.840464 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:26:02.840429    2567 patch_prober.go:28] interesting pod/e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Jul 29 13:26:02.840650 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:26:02.840497    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n" podUID="fe280394-648b-4c05-8fd4-30535c2fc262" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Jul 29 13:26:03.840703 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:26:03.840668    2567 patch_prober.go:28] interesting pod/e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Jul 29 13:26:03.841121 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:26:03.840734    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n" podUID="fe280394-648b-4c05-8fd4-30535c2fc262" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Jul 29 13:26:04.845441 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:26:04.845366    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n"
Jul 29 13:26:04.846734 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:26:04.846711    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n"
Jul 29 13:26:19.276391 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:26:19.276363    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-k7kgk_c261451c-033e-4484-ae79-c1a2ebb7d81b/payload-processing/0.log"
Jul 29 13:27:06.519921 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:06.519850    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-k7kgk_c261451c-033e-4484-ae79-c1a2ebb7d81b/payload-processing/0.log"
Jul 29 13:27:10.150618 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:10.150588    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-08bb55-7f5b9f8655-k9vlw"]
Jul 29 13:27:10.154279 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:10.154256    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-08bb55-7f5b9f8655-k9vlw"
Jul 29 13:27:10.157121 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:10.157096    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-08bb55-serving-cert\""
Jul 29 13:27:10.170238 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:10.170215    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-08bb55-7f5b9f8655-k9vlw"]
Jul 29 13:27:10.203213 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:10.203191    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3cc1d73f-77ea-4915-9095-b4357797c198-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-08bb55-7f5b9f8655-k9vlw\" (UID: \"3cc1d73f-77ea-4915-9095-b4357797c198\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-08bb55-7f5b9f8655-k9vlw"
Jul 29 13:27:10.203327 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:10.203236    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cc1d73f-77ea-4915-9095-b4357797c198-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-08bb55-7f5b9f8655-k9vlw\" (UID: \"3cc1d73f-77ea-4915-9095-b4357797c198\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-08bb55-7f5b9f8655-k9vlw"
Jul 29 13:27:10.203327 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:10.203271    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2kdm\" (UniqueName: \"kubernetes.io/projected/3cc1d73f-77ea-4915-9095-b4357797c198-kube-api-access-k2kdm\") pod \"maas-api-e2e-ipp-cleanup-08bb55-7f5b9f8655-k9vlw\" (UID: \"3cc1d73f-77ea-4915-9095-b4357797c198\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-08bb55-7f5b9f8655-k9vlw"
Jul 29 13:27:10.304311 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:10.304275    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3cc1d73f-77ea-4915-9095-b4357797c198-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-08bb55-7f5b9f8655-k9vlw\" (UID: \"3cc1d73f-77ea-4915-9095-b4357797c198\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-08bb55-7f5b9f8655-k9vlw"
Jul 29 13:27:10.304481 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:10.304324    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cc1d73f-77ea-4915-9095-b4357797c198-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-08bb55-7f5b9f8655-k9vlw\" (UID: \"3cc1d73f-77ea-4915-9095-b4357797c198\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-08bb55-7f5b9f8655-k9vlw"
Jul 29 13:27:10.304481 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:10.304357    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2kdm\" (UniqueName: \"kubernetes.io/projected/3cc1d73f-77ea-4915-9095-b4357797c198-kube-api-access-k2kdm\") pod \"maas-api-e2e-ipp-cleanup-08bb55-7f5b9f8655-k9vlw\" (UID: \"3cc1d73f-77ea-4915-9095-b4357797c198\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-08bb55-7f5b9f8655-k9vlw"
Jul 29 13:27:10.304481 ip-10-0-128-25 kubenswrapper[2567]: E0729 13:27:10.304455    2567 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-08bb55-serving-cert: secret "maas-api-e2e-ipp-cleanup-08bb55-serving-cert" not found
Jul 29 13:27:10.304597 ip-10-0-128-25 kubenswrapper[2567]: E0729 13:27:10.304532    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3cc1d73f-77ea-4915-9095-b4357797c198-maas-api-tls podName:3cc1d73f-77ea-4915-9095-b4357797c198 nodeName:}" failed. No retries permitted until 2026-07-29 13:27:10.804511611 +0000 UTC m=+3713.498199905 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3cc1d73f-77ea-4915-9095-b4357797c198-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-08bb55-7f5b9f8655-k9vlw" (UID: "3cc1d73f-77ea-4915-9095-b4357797c198") : secret "maas-api-e2e-ipp-cleanup-08bb55-serving-cert" not found
Jul 29 13:27:10.305065 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:10.305046    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cc1d73f-77ea-4915-9095-b4357797c198-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-08bb55-7f5b9f8655-k9vlw\" (UID: \"3cc1d73f-77ea-4915-9095-b4357797c198\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-08bb55-7f5b9f8655-k9vlw"
Jul 29 13:27:10.313010 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:10.312989    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2kdm\" (UniqueName: \"kubernetes.io/projected/3cc1d73f-77ea-4915-9095-b4357797c198-kube-api-access-k2kdm\") pod \"maas-api-e2e-ipp-cleanup-08bb55-7f5b9f8655-k9vlw\" (UID: \"3cc1d73f-77ea-4915-9095-b4357797c198\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-08bb55-7f5b9f8655-k9vlw"
Jul 29 13:27:10.808885 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:10.808862    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3cc1d73f-77ea-4915-9095-b4357797c198-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-08bb55-7f5b9f8655-k9vlw\" (UID: \"3cc1d73f-77ea-4915-9095-b4357797c198\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-08bb55-7f5b9f8655-k9vlw"
Jul 29 13:27:10.811173 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:10.811152    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3cc1d73f-77ea-4915-9095-b4357797c198-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-08bb55-7f5b9f8655-k9vlw\" (UID: \"3cc1d73f-77ea-4915-9095-b4357797c198\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-08bb55-7f5b9f8655-k9vlw"
Jul 29 13:27:11.065201 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:11.065125    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-08bb55-7f5b9f8655-k9vlw"
Jul 29 13:27:11.229964 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:11.215775    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-08bb55-7f5b9f8655-k9vlw"]
Jul 29 13:27:11.229964 ip-10-0-128-25 kubenswrapper[2567]: W0729 13:27:11.221725    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cc1d73f_77ea_4915_9095_b4357797c198.slice/crio-181dbef0a6f16fe20f213e895a0e16ea06b98eccb3ae1113397128dc6c4bed54 WatchSource:0}: Error finding container 181dbef0a6f16fe20f213e895a0e16ea06b98eccb3ae1113397128dc6c4bed54: Status 404 returned error can't find the container with id 181dbef0a6f16fe20f213e895a0e16ea06b98eccb3ae1113397128dc6c4bed54
Jul 29 13:27:11.771391 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:11.771361    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-08bb55-7f5b9f8655-k9vlw" event={"ID":"3cc1d73f-77ea-4915-9095-b4357797c198","Type":"ContainerStarted","Data":"181dbef0a6f16fe20f213e895a0e16ea06b98eccb3ae1113397128dc6c4bed54"}
Jul 29 13:27:12.778894 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:12.778856    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-08bb55-7f5b9f8655-k9vlw" event={"ID":"3cc1d73f-77ea-4915-9095-b4357797c198","Type":"ContainerStarted","Data":"b07646b8eec4c78f9683377fd44c96b3c168ffe32061cf6ffb8b9df1f6543ba4"}
Jul 29 13:27:12.779329 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:12.778945    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-08bb55-7f5b9f8655-k9vlw"
Jul 29 13:27:12.796840 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:12.796798    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-08bb55-7f5b9f8655-k9vlw" podStartSLOduration=2.264127388 podStartE2EDuration="2.796784648s" totalImagesPullingTime="532.65726ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 13:27:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 13:27:12.795371001 +0000 UTC m=+3715.489059330" watchObservedRunningTime="2026-07-29 13:27:12.796784648 +0000 UTC m=+3715.490472961"
Jul 29 13:27:18.792056 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:18.792027    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-08bb55-7f5b9f8655-k9vlw"
Jul 29 13:27:35.829371 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:35.829331    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-08bb55-8z8z5"]
Jul 29 13:27:35.833334 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:35.833313    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-08bb55-8z8z5"
Jul 29 13:27:35.836289 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:35.836263    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7gpd2\""
Jul 29 13:27:35.847870 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:35.847843    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-08bb55-8z8z5"]
Jul 29 13:27:35.924120 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:35.924089    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d11b047-a5be-43a0-8f8d-124ffdb1e833-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-08bb55-8z8z5\" (UID: \"9d11b047-a5be-43a0-8f8d-124ffdb1e833\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-08bb55-8z8z5"
Jul 29 13:27:36.025497 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:36.025472    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d11b047-a5be-43a0-8f8d-124ffdb1e833-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-08bb55-8z8z5\" (UID: \"9d11b047-a5be-43a0-8f8d-124ffdb1e833\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-08bb55-8z8z5"
Jul 29 13:27:36.025997 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:36.025979    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d11b047-a5be-43a0-8f8d-124ffdb1e833-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-08bb55-8z8z5\" (UID: \"9d11b047-a5be-43a0-8f8d-124ffdb1e833\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-08bb55-8z8z5"
Jul 29 13:27:36.144479 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:36.144416    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-08bb55-8z8z5"
Jul 29 13:27:36.292150 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:36.292121    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-08bb55-8z8z5"]
Jul 29 13:27:36.292835 ip-10-0-128-25 kubenswrapper[2567]: W0729 13:27:36.292800    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d11b047_a5be_43a0_8f8d_124ffdb1e833.slice/crio-2eca03de240def6becd34b82efefd018648b05aa8ad273011c0185b4297efc71 WatchSource:0}: Error finding container 2eca03de240def6becd34b82efefd018648b05aa8ad273011c0185b4297efc71: Status 404 returned error can't find the container with id 2eca03de240def6becd34b82efefd018648b05aa8ad273011c0185b4297efc71
Jul 29 13:27:36.936162 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:36.936137    2567 generic.go:356] "Generic (PLEG): container finished" podID="9d11b047-a5be-43a0-8f8d-124ffdb1e833" containerID="de5b3b395ec2f31ab6e783d8a317a01829dad517de24ffd706a1a6747d4ab8bf" exitCode=0
Jul 29 13:27:36.936518 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:36.936168    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-08bb55-8z8z5" event={"ID":"9d11b047-a5be-43a0-8f8d-124ffdb1e833","Type":"ContainerDied","Data":"de5b3b395ec2f31ab6e783d8a317a01829dad517de24ffd706a1a6747d4ab8bf"}
Jul 29 13:27:36.936518 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:36.936204    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-08bb55-8z8z5" event={"ID":"9d11b047-a5be-43a0-8f8d-124ffdb1e833","Type":"ContainerStarted","Data":"2eca03de240def6becd34b82efefd018648b05aa8ad273011c0185b4297efc71"}
Jul 29 13:27:38.080094 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:38.080068    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-08bb55-8z8z5"
Jul 29 13:27:38.140665 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:38.140640    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d11b047-a5be-43a0-8f8d-124ffdb1e833-maas-api-service-ca\") pod \"9d11b047-a5be-43a0-8f8d-124ffdb1e833\" (UID: \"9d11b047-a5be-43a0-8f8d-124ffdb1e833\") "
Jul 29 13:27:38.140971 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:38.140951    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d11b047-a5be-43a0-8f8d-124ffdb1e833-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9d11b047-a5be-43a0-8f8d-124ffdb1e833" (UID: "9d11b047-a5be-43a0-8f8d-124ffdb1e833"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 13:27:38.241326 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:38.241273    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d11b047-a5be-43a0-8f8d-124ffdb1e833-maas-api-service-ca\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 13:27:38.958660 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:38.958623    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-08bb55-8z8z5" event={"ID":"9d11b047-a5be-43a0-8f8d-124ffdb1e833","Type":"ContainerDied","Data":"2eca03de240def6becd34b82efefd018648b05aa8ad273011c0185b4297efc71"}
Jul 29 13:27:38.958660 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:38.958647    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-08bb55-8z8z5"
Jul 29 13:27:38.958848 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:38.958701    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2eca03de240def6becd34b82efefd018648b05aa8ad273011c0185b4297efc71"
Jul 29 13:27:40.841626 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:40.841582    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-08bb55-7f5b9f8655-k9vlw"]
Jul 29 13:27:40.842126 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:40.841817    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-08bb55-7f5b9f8655-k9vlw" podUID="3cc1d73f-77ea-4915-9095-b4357797c198" containerName="maas-api" containerID="cri-o://b07646b8eec4c78f9683377fd44c96b3c168ffe32061cf6ffb8b9df1f6543ba4" gracePeriod=30
Jul 29 13:27:40.973941 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:40.973902    2567 generic.go:356] "Generic (PLEG): container finished" podID="3cc1d73f-77ea-4915-9095-b4357797c198" containerID="b07646b8eec4c78f9683377fd44c96b3c168ffe32061cf6ffb8b9df1f6543ba4" exitCode=0
Jul 29 13:27:40.974074 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:40.973971    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-08bb55-7f5b9f8655-k9vlw" event={"ID":"3cc1d73f-77ea-4915-9095-b4357797c198","Type":"ContainerDied","Data":"b07646b8eec4c78f9683377fd44c96b3c168ffe32061cf6ffb8b9df1f6543ba4"}
Jul 29 13:27:41.042671 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:41.042633    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-08bb55-8z8z5"]
Jul 29 13:27:41.045232 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:41.045208    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-08bb55-8z8z5"]
Jul 29 13:27:41.111542 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:41.111518    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-08bb55-7f5b9f8655-k9vlw"
Jul 29 13:27:41.168016 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:41.167988    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k2kdm\" (UniqueName: \"kubernetes.io/projected/3cc1d73f-77ea-4915-9095-b4357797c198-kube-api-access-k2kdm\") pod \"3cc1d73f-77ea-4915-9095-b4357797c198\" (UID: \"3cc1d73f-77ea-4915-9095-b4357797c198\") "
Jul 29 13:27:41.168128 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:41.168067    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cc1d73f-77ea-4915-9095-b4357797c198-openshift-service-ca\") pod \"3cc1d73f-77ea-4915-9095-b4357797c198\" (UID: \"3cc1d73f-77ea-4915-9095-b4357797c198\") "
Jul 29 13:27:41.168178 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:41.168140    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3cc1d73f-77ea-4915-9095-b4357797c198-maas-api-tls\") pod \"3cc1d73f-77ea-4915-9095-b4357797c198\" (UID: \"3cc1d73f-77ea-4915-9095-b4357797c198\") "
Jul 29 13:27:41.168388 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:41.168366    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3cc1d73f-77ea-4915-9095-b4357797c198-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3cc1d73f-77ea-4915-9095-b4357797c198" (UID: "3cc1d73f-77ea-4915-9095-b4357797c198"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 13:27:41.170045 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:41.170004    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3cc1d73f-77ea-4915-9095-b4357797c198-kube-api-access-k2kdm" (OuterVolumeSpecName: "kube-api-access-k2kdm") pod "3cc1d73f-77ea-4915-9095-b4357797c198" (UID: "3cc1d73f-77ea-4915-9095-b4357797c198"). InnerVolumeSpecName "kube-api-access-k2kdm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 13:27:41.170140 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:41.170082    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3cc1d73f-77ea-4915-9095-b4357797c198-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3cc1d73f-77ea-4915-9095-b4357797c198" (UID: "3cc1d73f-77ea-4915-9095-b4357797c198"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 13:27:41.269270 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:41.269247    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cc1d73f-77ea-4915-9095-b4357797c198-openshift-service-ca\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 13:27:41.269270 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:41.269272    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3cc1d73f-77ea-4915-9095-b4357797c198-maas-api-tls\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 13:27:41.269410 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:41.269280    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k2kdm\" (UniqueName: \"kubernetes.io/projected/3cc1d73f-77ea-4915-9095-b4357797c198-kube-api-access-k2kdm\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 13:27:41.921788 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:41.921757    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d11b047-a5be-43a0-8f8d-124ffdb1e833" path="/var/lib/kubelet/pods/9d11b047-a5be-43a0-8f8d-124ffdb1e833/volumes"
Jul 29 13:27:41.981163 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:41.981135    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-08bb55-7f5b9f8655-k9vlw"
Jul 29 13:27:41.981259 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:41.981181    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-08bb55-7f5b9f8655-k9vlw" event={"ID":"3cc1d73f-77ea-4915-9095-b4357797c198","Type":"ContainerDied","Data":"181dbef0a6f16fe20f213e895a0e16ea06b98eccb3ae1113397128dc6c4bed54"}
Jul 29 13:27:41.981259 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:41.981236    2567 scope.go:117] "RemoveContainer" containerID="b07646b8eec4c78f9683377fd44c96b3c168ffe32061cf6ffb8b9df1f6543ba4"
Jul 29 13:27:41.997599 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:41.997579    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-08bb55-7f5b9f8655-k9vlw"]
Jul 29 13:27:42.001249 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:42.001226    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-08bb55-7f5b9f8655-k9vlw"]
Jul 29 13:27:43.922568 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:43.922527    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3cc1d73f-77ea-4915-9095-b4357797c198" path="/var/lib/kubelet/pods/3cc1d73f-77ea-4915-9095-b4357797c198/volumes"
Jul 29 13:27:51.704309 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:51.704269    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-86053a-p7dkf"]
Jul 29 13:27:51.704855 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:51.704815    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3cc1d73f-77ea-4915-9095-b4357797c198" containerName="maas-api"
Jul 29 13:27:51.704855 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:51.704831    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cc1d73f-77ea-4915-9095-b4357797c198" containerName="maas-api"
Jul 29 13:27:51.704855 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:51.704852    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d11b047-a5be-43a0-8f8d-124ffdb1e833" containerName="revoke-keys"
Jul 29 13:27:51.704855 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:51.704858    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d11b047-a5be-43a0-8f8d-124ffdb1e833" containerName="revoke-keys"
Jul 29 13:27:51.705051 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:51.704938    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d11b047-a5be-43a0-8f8d-124ffdb1e833" containerName="revoke-keys"
Jul 29 13:27:51.705051 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:51.704948    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="3cc1d73f-77ea-4915-9095-b4357797c198" containerName="maas-api"
Jul 29 13:27:51.709805 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:51.709778    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-86053a-p7dkf"
Jul 29 13:27:51.712630 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:51.712602    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7gpd2\""
Jul 29 13:27:51.716254 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:51.716226    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-86053a-p7dkf"]
Jul 29 13:27:51.757266 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:51.757237    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f0ef95f-1e5a-4b48-8060-a2d30ecad2db-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-86053a-p7dkf\" (UID: \"1f0ef95f-1e5a-4b48-8060-a2d30ecad2db\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-86053a-p7dkf"
Jul 29 13:27:51.857751 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:51.857725    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f0ef95f-1e5a-4b48-8060-a2d30ecad2db-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-86053a-p7dkf\" (UID: \"1f0ef95f-1e5a-4b48-8060-a2d30ecad2db\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-86053a-p7dkf"
Jul 29 13:27:51.858334 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:51.858310    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f0ef95f-1e5a-4b48-8060-a2d30ecad2db-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-86053a-p7dkf\" (UID: \"1f0ef95f-1e5a-4b48-8060-a2d30ecad2db\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-86053a-p7dkf"
Jul 29 13:27:52.025823 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:52.025803    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-86053a-p7dkf"
Jul 29 13:27:52.165791 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:52.165754    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-86053a-p7dkf"]
Jul 29 13:27:52.166883 ip-10-0-128-25 kubenswrapper[2567]: W0729 13:27:52.166863    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f0ef95f_1e5a_4b48_8060_a2d30ecad2db.slice/crio-a0f44b6123289d089cc41b786a11f9c61b50e73e66ae8e3bf42036465a4ba9b1 WatchSource:0}: Error finding container a0f44b6123289d089cc41b786a11f9c61b50e73e66ae8e3bf42036465a4ba9b1: Status 404 returned error can't find the container with id a0f44b6123289d089cc41b786a11f9c61b50e73e66ae8e3bf42036465a4ba9b1
Jul 29 13:27:53.053793 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:53.053757    2567 generic.go:356] "Generic (PLEG): container finished" podID="1f0ef95f-1e5a-4b48-8060-a2d30ecad2db" containerID="902ea972f6ccce3a9b12c0cc8a29f5fa63de949caef13d4aaec13e043e969865" exitCode=0
Jul 29 13:27:53.054179 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:53.053861    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-86053a-p7dkf" event={"ID":"1f0ef95f-1e5a-4b48-8060-a2d30ecad2db","Type":"ContainerDied","Data":"902ea972f6ccce3a9b12c0cc8a29f5fa63de949caef13d4aaec13e043e969865"}
Jul 29 13:27:53.054179 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:53.053956    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-86053a-p7dkf" event={"ID":"1f0ef95f-1e5a-4b48-8060-a2d30ecad2db","Type":"ContainerStarted","Data":"a0f44b6123289d089cc41b786a11f9c61b50e73e66ae8e3bf42036465a4ba9b1"}
Jul 29 13:27:54.211342 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:54.211322    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-86053a-p7dkf"
Jul 29 13:27:54.279767 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:54.279739    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f0ef95f-1e5a-4b48-8060-a2d30ecad2db-maas-api-service-ca\") pod \"1f0ef95f-1e5a-4b48-8060-a2d30ecad2db\" (UID: \"1f0ef95f-1e5a-4b48-8060-a2d30ecad2db\") "
Jul 29 13:27:54.280060 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:54.280035    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f0ef95f-1e5a-4b48-8060-a2d30ecad2db-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1f0ef95f-1e5a-4b48-8060-a2d30ecad2db" (UID: "1f0ef95f-1e5a-4b48-8060-a2d30ecad2db"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 13:27:54.380901 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:54.380847    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f0ef95f-1e5a-4b48-8060-a2d30ecad2db-maas-api-service-ca\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 13:27:55.070162 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:55.070134    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-86053a-p7dkf"
Jul 29 13:27:55.070315 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:55.070186    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-86053a-p7dkf" event={"ID":"1f0ef95f-1e5a-4b48-8060-a2d30ecad2db","Type":"ContainerDied","Data":"a0f44b6123289d089cc41b786a11f9c61b50e73e66ae8e3bf42036465a4ba9b1"}
Jul 29 13:27:55.070315 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:27:55.070241    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a0f44b6123289d089cc41b786a11f9c61b50e73e66ae8e3bf42036465a4ba9b1"
Jul 29 13:28:07.151413 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:28:07.151370    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-86053a-5cd6f65489-qmx7g"]
Jul 29 13:28:07.151988 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:28:07.151755    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-86053a-5cd6f65489-qmx7g" podUID="78444b1d-6636-4495-b9ad-f35013e35690" containerName="maas-api" containerID="cri-o://2f95fd5b11df10b62f66764a273c2938e370d7b17ec48ac117dc80d78de8abe1" gracePeriod=30
Jul 29 13:28:07.336325 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:28:07.336295    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-86053a-p7dkf"]
Jul 29 13:28:07.339302 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:28:07.339275    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-86053a-p7dkf"]
Jul 29 13:28:07.417026 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:28:07.416973    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-86053a-5cd6f65489-qmx7g"
Jul 29 13:28:07.493601 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:28:07.493577    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/78444b1d-6636-4495-b9ad-f35013e35690-openshift-service-ca\") pod \"78444b1d-6636-4495-b9ad-f35013e35690\" (UID: \"78444b1d-6636-4495-b9ad-f35013e35690\") "
Jul 29 13:28:07.493720 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:28:07.493685    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tcz46\" (UniqueName: \"kubernetes.io/projected/78444b1d-6636-4495-b9ad-f35013e35690-kube-api-access-tcz46\") pod \"78444b1d-6636-4495-b9ad-f35013e35690\" (UID: \"78444b1d-6636-4495-b9ad-f35013e35690\") "
Jul 29 13:28:07.493720 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:28:07.493717    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/78444b1d-6636-4495-b9ad-f35013e35690-maas-api-tls\") pod \"78444b1d-6636-4495-b9ad-f35013e35690\" (UID: \"78444b1d-6636-4495-b9ad-f35013e35690\") "
Jul 29 13:28:07.493898 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:28:07.493877    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78444b1d-6636-4495-b9ad-f35013e35690-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "78444b1d-6636-4495-b9ad-f35013e35690" (UID: "78444b1d-6636-4495-b9ad-f35013e35690"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 13:28:07.494073 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:28:07.494051    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/78444b1d-6636-4495-b9ad-f35013e35690-openshift-service-ca\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 13:28:07.495632 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:28:07.495601    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78444b1d-6636-4495-b9ad-f35013e35690-kube-api-access-tcz46" (OuterVolumeSpecName: "kube-api-access-tcz46") pod "78444b1d-6636-4495-b9ad-f35013e35690" (UID: "78444b1d-6636-4495-b9ad-f35013e35690"). InnerVolumeSpecName "kube-api-access-tcz46". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 13:28:07.495763 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:28:07.495743    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78444b1d-6636-4495-b9ad-f35013e35690-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "78444b1d-6636-4495-b9ad-f35013e35690" (UID: "78444b1d-6636-4495-b9ad-f35013e35690"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 13:28:07.596171 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:28:07.596143    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tcz46\" (UniqueName: \"kubernetes.io/projected/78444b1d-6636-4495-b9ad-f35013e35690-kube-api-access-tcz46\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 13:28:07.596171 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:28:07.596174    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/78444b1d-6636-4495-b9ad-f35013e35690-maas-api-tls\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 13:28:07.922629 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:28:07.922600    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f0ef95f-1e5a-4b48-8060-a2d30ecad2db" path="/var/lib/kubelet/pods/1f0ef95f-1e5a-4b48-8060-a2d30ecad2db/volumes"
Jul 29 13:28:08.154042 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:28:08.154018    2567 generic.go:356] "Generic (PLEG): container finished" podID="78444b1d-6636-4495-b9ad-f35013e35690" containerID="2f95fd5b11df10b62f66764a273c2938e370d7b17ec48ac117dc80d78de8abe1" exitCode=0
Jul 29 13:28:08.154364 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:28:08.154056    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-86053a-5cd6f65489-qmx7g"
Jul 29 13:28:08.154364 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:28:08.154095    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-86053a-5cd6f65489-qmx7g" event={"ID":"78444b1d-6636-4495-b9ad-f35013e35690","Type":"ContainerDied","Data":"2f95fd5b11df10b62f66764a273c2938e370d7b17ec48ac117dc80d78de8abe1"}
Jul 29 13:28:08.154364 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:28:08.154146    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-86053a-5cd6f65489-qmx7g" event={"ID":"78444b1d-6636-4495-b9ad-f35013e35690","Type":"ContainerDied","Data":"06e7ba57c0dd25470abbefef761fdb9999ec46533aff09c9392448284944c4a9"}
Jul 29 13:28:08.154364 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:28:08.154179    2567 scope.go:117] "RemoveContainer" containerID="2f95fd5b11df10b62f66764a273c2938e370d7b17ec48ac117dc80d78de8abe1"
Jul 29 13:28:08.163138 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:28:08.163112    2567 scope.go:117] "RemoveContainer" containerID="2f95fd5b11df10b62f66764a273c2938e370d7b17ec48ac117dc80d78de8abe1"
Jul 29 13:28:08.163378 ip-10-0-128-25 kubenswrapper[2567]: E0729 13:28:08.163359    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f95fd5b11df10b62f66764a273c2938e370d7b17ec48ac117dc80d78de8abe1\": container with ID starting with 2f95fd5b11df10b62f66764a273c2938e370d7b17ec48ac117dc80d78de8abe1 not found: ID does not exist" containerID="2f95fd5b11df10b62f66764a273c2938e370d7b17ec48ac117dc80d78de8abe1"
Jul 29 13:28:08.163443 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:28:08.163387    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f95fd5b11df10b62f66764a273c2938e370d7b17ec48ac117dc80d78de8abe1"} err="failed to get container status \"2f95fd5b11df10b62f66764a273c2938e370d7b17ec48ac117dc80d78de8abe1\": rpc error: code = NotFound desc = could not find container \"2f95fd5b11df10b62f66764a273c2938e370d7b17ec48ac117dc80d78de8abe1\": container with ID starting with 2f95fd5b11df10b62f66764a273c2938e370d7b17ec48ac117dc80d78de8abe1 not found: ID does not exist"
Jul 29 13:28:08.173047 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:28:08.172959    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-86053a-5cd6f65489-qmx7g"]
Jul 29 13:28:08.176158 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:28:08.176140    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-86053a-5cd6f65489-qmx7g"]
Jul 29 13:28:09.927391 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:28:09.927288    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78444b1d-6636-4495-b9ad-f35013e35690" path="/var/lib/kubelet/pods/78444b1d-6636-4495-b9ad-f35013e35690/volumes"
Jul 29 13:28:56.228874 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:28:56.228828    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-bb0da0-n6sfh"]
Jul 29 13:28:56.229374 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:28:56.229293    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f0ef95f-1e5a-4b48-8060-a2d30ecad2db" containerName="revoke-keys"
Jul 29 13:28:56.229374 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:28:56.229306    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f0ef95f-1e5a-4b48-8060-a2d30ecad2db" containerName="revoke-keys"
Jul 29 13:28:56.229374 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:28:56.229316    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78444b1d-6636-4495-b9ad-f35013e35690" containerName="maas-api"
Jul 29 13:28:56.229374 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:28:56.229321    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="78444b1d-6636-4495-b9ad-f35013e35690" containerName="maas-api"
Jul 29 13:28:56.229531 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:28:56.229391    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="78444b1d-6636-4495-b9ad-f35013e35690" containerName="maas-api"
Jul 29 13:28:56.229531 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:28:56.229414    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f0ef95f-1e5a-4b48-8060-a2d30ecad2db" containerName="revoke-keys"
Jul 29 13:28:56.232296 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:28:56.232277    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-bb0da0-n6sfh"
Jul 29 13:28:56.235357 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:28:56.235334    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7gpd2\""
Jul 29 13:28:56.240913 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:28:56.240885    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-bb0da0-n6sfh"]
Jul 29 13:28:56.290557 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:28:56.290524    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d128465-d957-4ff1-8770-fdfeb09e6121-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-bb0da0-n6sfh\" (UID: \"6d128465-d957-4ff1-8770-fdfeb09e6121\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-bb0da0-n6sfh"
Jul 29 13:28:56.391304 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:28:56.391281    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d128465-d957-4ff1-8770-fdfeb09e6121-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-bb0da0-n6sfh\" (UID: \"6d128465-d957-4ff1-8770-fdfeb09e6121\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-bb0da0-n6sfh"
Jul 29 13:28:56.391888 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:28:56.391865    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d128465-d957-4ff1-8770-fdfeb09e6121-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-bb0da0-n6sfh\" (UID: \"6d128465-d957-4ff1-8770-fdfeb09e6121\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-bb0da0-n6sfh"
Jul 29 13:28:56.546020 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:28:56.545997    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-bb0da0-n6sfh"
Jul 29 13:28:56.678140 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:28:56.678114    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-bb0da0-n6sfh"]
Jul 29 13:28:56.679641 ip-10-0-128-25 kubenswrapper[2567]: W0729 13:28:56.679609    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d128465_d957_4ff1_8770_fdfeb09e6121.slice/crio-7b3f0f01d50ee47b00e06ea9a7afcbde677092ac06188397162ec7784cd8d3e9 WatchSource:0}: Error finding container 7b3f0f01d50ee47b00e06ea9a7afcbde677092ac06188397162ec7784cd8d3e9: Status 404 returned error can't find the container with id 7b3f0f01d50ee47b00e06ea9a7afcbde677092ac06188397162ec7784cd8d3e9
Jul 29 13:28:57.464181 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:28:57.464156    2567 generic.go:356] "Generic (PLEG): container finished" podID="6d128465-d957-4ff1-8770-fdfeb09e6121" containerID="6265cbb9c182b0deb5afa865dad47071455f6e0fc4854175a57dce901c80a138" exitCode=0
Jul 29 13:28:57.464483 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:28:57.464237    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-bb0da0-n6sfh" event={"ID":"6d128465-d957-4ff1-8770-fdfeb09e6121","Type":"ContainerDied","Data":"6265cbb9c182b0deb5afa865dad47071455f6e0fc4854175a57dce901c80a138"}
Jul 29 13:28:57.464483 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:28:57.464291    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-bb0da0-n6sfh" event={"ID":"6d128465-d957-4ff1-8770-fdfeb09e6121","Type":"ContainerStarted","Data":"7b3f0f01d50ee47b00e06ea9a7afcbde677092ac06188397162ec7784cd8d3e9"}
Jul 29 13:28:58.608737 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:28:58.608710    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-bb0da0-n6sfh"
Jul 29 13:28:58.709804 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:28:58.709778    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d128465-d957-4ff1-8770-fdfeb09e6121-maas-api-service-ca\") pod \"6d128465-d957-4ff1-8770-fdfeb09e6121\" (UID: \"6d128465-d957-4ff1-8770-fdfeb09e6121\") "
Jul 29 13:28:58.710071 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:28:58.710051    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d128465-d957-4ff1-8770-fdfeb09e6121-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6d128465-d957-4ff1-8770-fdfeb09e6121" (UID: "6d128465-d957-4ff1-8770-fdfeb09e6121"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 13:28:58.811121 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:28:58.811100    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d128465-d957-4ff1-8770-fdfeb09e6121-maas-api-service-ca\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 13:28:59.477133 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:28:59.477106    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-bb0da0-n6sfh" event={"ID":"6d128465-d957-4ff1-8770-fdfeb09e6121","Type":"ContainerDied","Data":"7b3f0f01d50ee47b00e06ea9a7afcbde677092ac06188397162ec7784cd8d3e9"}
Jul 29 13:28:59.477133 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:28:59.477120    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-bb0da0-n6sfh"
Jul 29 13:28:59.477300 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:28:59.477149    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7b3f0f01d50ee47b00e06ea9a7afcbde677092ac06188397162ec7784cd8d3e9"
Jul 29 13:29:06.631925 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:06.631881    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-bb0da0-75bb5b6bb6-qbbhz"]
Jul 29 13:29:06.632472 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:06.632158    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bb0da0-75bb5b6bb6-qbbhz" podUID="c6dbe2f0-4e83-4f60-b80f-90fe7f05b189" containerName="maas-api" containerID="cri-o://3cf890b16d0f81881a9b5a1132555460c395a78f063f963758f6592f5812bd65" gracePeriod=30
Jul 29 13:29:06.838028 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:06.837953    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-bb0da0-n6sfh"]
Jul 29 13:29:06.840727 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:06.840690    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-bb0da0-n6sfh"]
Jul 29 13:29:06.891498 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:06.891444    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bb0da0-75bb5b6bb6-qbbhz"
Jul 29 13:29:06.979650 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:06.979625    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6dbe2f0-4e83-4f60-b80f-90fe7f05b189-openshift-service-ca\") pod \"c6dbe2f0-4e83-4f60-b80f-90fe7f05b189\" (UID: \"c6dbe2f0-4e83-4f60-b80f-90fe7f05b189\") "
Jul 29 13:29:06.979779 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:06.979678    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9ztmd\" (UniqueName: \"kubernetes.io/projected/c6dbe2f0-4e83-4f60-b80f-90fe7f05b189-kube-api-access-9ztmd\") pod \"c6dbe2f0-4e83-4f60-b80f-90fe7f05b189\" (UID: \"c6dbe2f0-4e83-4f60-b80f-90fe7f05b189\") "
Jul 29 13:29:06.979779 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:06.979746    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c6dbe2f0-4e83-4f60-b80f-90fe7f05b189-maas-api-tls\") pod \"c6dbe2f0-4e83-4f60-b80f-90fe7f05b189\" (UID: \"c6dbe2f0-4e83-4f60-b80f-90fe7f05b189\") "
Jul 29 13:29:06.980006 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:06.979980    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6dbe2f0-4e83-4f60-b80f-90fe7f05b189-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c6dbe2f0-4e83-4f60-b80f-90fe7f05b189" (UID: "c6dbe2f0-4e83-4f60-b80f-90fe7f05b189"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 13:29:06.981701 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:06.981672    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6dbe2f0-4e83-4f60-b80f-90fe7f05b189-kube-api-access-9ztmd" (OuterVolumeSpecName: "kube-api-access-9ztmd") pod "c6dbe2f0-4e83-4f60-b80f-90fe7f05b189" (UID: "c6dbe2f0-4e83-4f60-b80f-90fe7f05b189"). InnerVolumeSpecName "kube-api-access-9ztmd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 13:29:06.981807 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:06.981753    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6dbe2f0-4e83-4f60-b80f-90fe7f05b189-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c6dbe2f0-4e83-4f60-b80f-90fe7f05b189" (UID: "c6dbe2f0-4e83-4f60-b80f-90fe7f05b189"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 13:29:07.081123 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:07.081101    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c6dbe2f0-4e83-4f60-b80f-90fe7f05b189-maas-api-tls\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 13:29:07.081123 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:07.081123    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6dbe2f0-4e83-4f60-b80f-90fe7f05b189-openshift-service-ca\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 13:29:07.081273 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:07.081132    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9ztmd\" (UniqueName: \"kubernetes.io/projected/c6dbe2f0-4e83-4f60-b80f-90fe7f05b189-kube-api-access-9ztmd\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 13:29:07.528414 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:07.528377    2567 generic.go:356] "Generic (PLEG): container finished" podID="c6dbe2f0-4e83-4f60-b80f-90fe7f05b189" containerID="3cf890b16d0f81881a9b5a1132555460c395a78f063f963758f6592f5812bd65" exitCode=0
Jul 29 13:29:07.528564 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:07.528442    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bb0da0-75bb5b6bb6-qbbhz"
Jul 29 13:29:07.528564 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:07.528441    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bb0da0-75bb5b6bb6-qbbhz" event={"ID":"c6dbe2f0-4e83-4f60-b80f-90fe7f05b189","Type":"ContainerDied","Data":"3cf890b16d0f81881a9b5a1132555460c395a78f063f963758f6592f5812bd65"}
Jul 29 13:29:07.528564 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:07.528561    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bb0da0-75bb5b6bb6-qbbhz" event={"ID":"c6dbe2f0-4e83-4f60-b80f-90fe7f05b189","Type":"ContainerDied","Data":"4bf256dcc0f3587b76e49aefa1ba592f0446e71a008915eed1fd486f95679c61"}
Jul 29 13:29:07.528765 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:07.528583    2567 scope.go:117] "RemoveContainer" containerID="3cf890b16d0f81881a9b5a1132555460c395a78f063f963758f6592f5812bd65"
Jul 29 13:29:07.538685 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:07.538671    2567 scope.go:117] "RemoveContainer" containerID="3cf890b16d0f81881a9b5a1132555460c395a78f063f963758f6592f5812bd65"
Jul 29 13:29:07.538908 ip-10-0-128-25 kubenswrapper[2567]: E0729 13:29:07.538886    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3cf890b16d0f81881a9b5a1132555460c395a78f063f963758f6592f5812bd65\": container with ID starting with 3cf890b16d0f81881a9b5a1132555460c395a78f063f963758f6592f5812bd65 not found: ID does not exist" containerID="3cf890b16d0f81881a9b5a1132555460c395a78f063f963758f6592f5812bd65"
Jul 29 13:29:07.538969 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:07.538919    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3cf890b16d0f81881a9b5a1132555460c395a78f063f963758f6592f5812bd65"} err="failed to get container status \"3cf890b16d0f81881a9b5a1132555460c395a78f063f963758f6592f5812bd65\": rpc error: code = NotFound desc = could not find container \"3cf890b16d0f81881a9b5a1132555460c395a78f063f963758f6592f5812bd65\": container with ID starting with 3cf890b16d0f81881a9b5a1132555460c395a78f063f963758f6592f5812bd65 not found: ID does not exist"
Jul 29 13:29:07.561777 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:07.561756    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-bb0da0-75bb5b6bb6-qbbhz"]
Jul 29 13:29:07.567769 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:07.567751    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-bb0da0-75bb5b6bb6-qbbhz"]
Jul 29 13:29:07.924171 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:07.924103    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d128465-d957-4ff1-8770-fdfeb09e6121" path="/var/lib/kubelet/pods/6d128465-d957-4ff1-8770-fdfeb09e6121/volumes"
Jul 29 13:29:07.924549 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:07.924530    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6dbe2f0-4e83-4f60-b80f-90fe7f05b189" path="/var/lib/kubelet/pods/c6dbe2f0-4e83-4f60-b80f-90fe7f05b189/volumes"
Jul 29 13:29:13.780962 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:13.780927    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n"]
Jul 29 13:29:13.781397 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:13.781202    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n" podUID="fe280394-648b-4c05-8fd4-30535c2fc262" containerName="istio-proxy" containerID="cri-o://e7292479f3fcb7af0c563de4f4fe1fd51bc70e61d42154f13513293eeda0d92b" gracePeriod=30
Jul 29 13:29:14.157135 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:14.157063    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f0ad55-sbjk4"]
Jul 29 13:29:14.157491 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:14.157478    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6dbe2f0-4e83-4f60-b80f-90fe7f05b189" containerName="maas-api"
Jul 29 13:29:14.157538 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:14.157492    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6dbe2f0-4e83-4f60-b80f-90fe7f05b189" containerName="maas-api"
Jul 29 13:29:14.157538 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:14.157502    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d128465-d957-4ff1-8770-fdfeb09e6121" containerName="revoke-keys"
Jul 29 13:29:14.157538 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:14.157507    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d128465-d957-4ff1-8770-fdfeb09e6121" containerName="revoke-keys"
Jul 29 13:29:14.157640 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:14.157566    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d128465-d957-4ff1-8770-fdfeb09e6121" containerName="revoke-keys"
Jul 29 13:29:14.157640 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:14.157576    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6dbe2f0-4e83-4f60-b80f-90fe7f05b189" containerName="maas-api"
Jul 29 13:29:14.162013 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:14.161983    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f0ad55-sbjk4"
Jul 29 13:29:14.172658 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:14.172626    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7gpd2\""
Jul 29 13:29:14.175938 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:14.175914    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f0ad55-sbjk4"]
Jul 29 13:29:14.242325 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:14.242302    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a7984e5-628c-4875-b1c4-6b08d271d892-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-f0ad55-sbjk4\" (UID: \"8a7984e5-628c-4875-b1c4-6b08d271d892\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f0ad55-sbjk4"
Jul 29 13:29:14.343591 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:14.343569    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a7984e5-628c-4875-b1c4-6b08d271d892-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-f0ad55-sbjk4\" (UID: \"8a7984e5-628c-4875-b1c4-6b08d271d892\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f0ad55-sbjk4"
Jul 29 13:29:14.344089 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:14.344073    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a7984e5-628c-4875-b1c4-6b08d271d892-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-f0ad55-sbjk4\" (UID: \"8a7984e5-628c-4875-b1c4-6b08d271d892\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f0ad55-sbjk4"
Jul 29 13:29:14.472055 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:14.472003    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f0ad55-sbjk4"
Jul 29 13:29:14.630308 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:14.630279    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f0ad55-sbjk4"]
Jul 29 13:29:14.631537 ip-10-0-128-25 kubenswrapper[2567]: W0729 13:29:14.631513    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a7984e5_628c_4875_b1c4_6b08d271d892.slice/crio-c2557ed0dbe650614dd2aeabf209d66288d08ec92ea1ce50d1ad0ab86c9bf255 WatchSource:0}: Error finding container c2557ed0dbe650614dd2aeabf209d66288d08ec92ea1ce50d1ad0ab86c9bf255: Status 404 returned error can't find the container with id c2557ed0dbe650614dd2aeabf209d66288d08ec92ea1ce50d1ad0ab86c9bf255
Jul 29 13:29:15.590038 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:15.590001    2567 generic.go:356] "Generic (PLEG): container finished" podID="8a7984e5-628c-4875-b1c4-6b08d271d892" containerID="8adc938da3e04ce0dd8c488836f38f54a071bdfa64f1ae1260ae6add05d63502" exitCode=0
Jul 29 13:29:15.590420 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:15.590052    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f0ad55-sbjk4" event={"ID":"8a7984e5-628c-4875-b1c4-6b08d271d892","Type":"ContainerDied","Data":"8adc938da3e04ce0dd8c488836f38f54a071bdfa64f1ae1260ae6add05d63502"}
Jul 29 13:29:15.590420 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:15.590122    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f0ad55-sbjk4" event={"ID":"8a7984e5-628c-4875-b1c4-6b08d271d892","Type":"ContainerStarted","Data":"c2557ed0dbe650614dd2aeabf209d66288d08ec92ea1ce50d1ad0ab86c9bf255"}
Jul 29 13:29:16.739178 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:16.739157    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f0ad55-sbjk4"
Jul 29 13:29:16.864998 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:16.864928    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a7984e5-628c-4875-b1c4-6b08d271d892-maas-api-service-ca\") pod \"8a7984e5-628c-4875-b1c4-6b08d271d892\" (UID: \"8a7984e5-628c-4875-b1c4-6b08d271d892\") "
Jul 29 13:29:16.865250 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:16.865227    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a7984e5-628c-4875-b1c4-6b08d271d892-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8a7984e5-628c-4875-b1c4-6b08d271d892" (UID: "8a7984e5-628c-4875-b1c4-6b08d271d892"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 13:29:16.965881 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:16.965852    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a7984e5-628c-4875-b1c4-6b08d271d892-maas-api-service-ca\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 13:29:17.605996 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:17.605967    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f0ad55-sbjk4"
Jul 29 13:29:17.606187 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:17.605986    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f0ad55-sbjk4" event={"ID":"8a7984e5-628c-4875-b1c4-6b08d271d892","Type":"ContainerDied","Data":"c2557ed0dbe650614dd2aeabf209d66288d08ec92ea1ce50d1ad0ab86c9bf255"}
Jul 29 13:29:17.606187 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:17.606049    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c2557ed0dbe650614dd2aeabf209d66288d08ec92ea1ce50d1ad0ab86c9bf255"
Jul 29 13:29:19.041665 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:19.041643    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n"
Jul 29 13:29:19.186188 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:19.186122    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe280394-648b-4c05-8fd4-30535c2fc262-workload-socket\") pod \"fe280394-648b-4c05-8fd4-30535c2fc262\" (UID: \"fe280394-648b-4c05-8fd4-30535c2fc262\") "
Jul 29 13:29:19.186188 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:19.186153    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe280394-648b-4c05-8fd4-30535c2fc262-istio-envoy\") pod \"fe280394-648b-4c05-8fd4-30535c2fc262\" (UID: \"fe280394-648b-4c05-8fd4-30535c2fc262\") "
Jul 29 13:29:19.186188 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:19.186183    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe280394-648b-4c05-8fd4-30535c2fc262-istio-data\") pod \"fe280394-648b-4c05-8fd4-30535c2fc262\" (UID: \"fe280394-648b-4c05-8fd4-30535c2fc262\") "
Jul 29 13:29:19.186467 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:19.186210    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2smpn\" (UniqueName: \"kubernetes.io/projected/fe280394-648b-4c05-8fd4-30535c2fc262-kube-api-access-2smpn\") pod \"fe280394-648b-4c05-8fd4-30535c2fc262\" (UID: \"fe280394-648b-4c05-8fd4-30535c2fc262\") "
Jul 29 13:29:19.186467 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:19.186253    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe280394-648b-4c05-8fd4-30535c2fc262-istio-token\") pod \"fe280394-648b-4c05-8fd4-30535c2fc262\" (UID: \"fe280394-648b-4c05-8fd4-30535c2fc262\") "
Jul 29 13:29:19.186467 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:19.186273    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe280394-648b-4c05-8fd4-30535c2fc262-istio-podinfo\") pod \"fe280394-648b-4c05-8fd4-30535c2fc262\" (UID: \"fe280394-648b-4c05-8fd4-30535c2fc262\") "
Jul 29 13:29:19.186467 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:19.186293    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe280394-648b-4c05-8fd4-30535c2fc262-credential-socket\") pod \"fe280394-648b-4c05-8fd4-30535c2fc262\" (UID: \"fe280394-648b-4c05-8fd4-30535c2fc262\") "
Jul 29 13:29:19.186467 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:19.186322    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe280394-648b-4c05-8fd4-30535c2fc262-workload-certs\") pod \"fe280394-648b-4c05-8fd4-30535c2fc262\" (UID: \"fe280394-648b-4c05-8fd4-30535c2fc262\") "
Jul 29 13:29:19.186467 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:19.186348    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe280394-648b-4c05-8fd4-30535c2fc262-istiod-ca-cert\") pod \"fe280394-648b-4c05-8fd4-30535c2fc262\" (UID: \"fe280394-648b-4c05-8fd4-30535c2fc262\") "
Jul 29 13:29:19.186467 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:19.186452    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe280394-648b-4c05-8fd4-30535c2fc262-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fe280394-648b-4c05-8fd4-30535c2fc262" (UID: "fe280394-648b-4c05-8fd4-30535c2fc262"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 13:29:19.186824 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:19.186572    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe280394-648b-4c05-8fd4-30535c2fc262-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fe280394-648b-4c05-8fd4-30535c2fc262" (UID: "fe280394-648b-4c05-8fd4-30535c2fc262"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 13:29:19.186824 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:19.186640    2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe280394-648b-4c05-8fd4-30535c2fc262-workload-socket\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 13:29:19.186824 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:19.186704    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe280394-648b-4c05-8fd4-30535c2fc262-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fe280394-648b-4c05-8fd4-30535c2fc262" (UID: "fe280394-648b-4c05-8fd4-30535c2fc262"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 13:29:19.186824 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:19.186740    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe280394-648b-4c05-8fd4-30535c2fc262-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fe280394-648b-4c05-8fd4-30535c2fc262" (UID: "fe280394-648b-4c05-8fd4-30535c2fc262"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 13:29:19.187020 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:19.186941    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe280394-648b-4c05-8fd4-30535c2fc262-istio-data" (OuterVolumeSpecName: "istio-data") pod "fe280394-648b-4c05-8fd4-30535c2fc262" (UID: "fe280394-648b-4c05-8fd4-30535c2fc262"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 13:29:19.188238 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:19.188211    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe280394-648b-4c05-8fd4-30535c2fc262-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fe280394-648b-4c05-8fd4-30535c2fc262" (UID: "fe280394-648b-4c05-8fd4-30535c2fc262"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 13:29:19.188361 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:19.188295    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fe280394-648b-4c05-8fd4-30535c2fc262-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fe280394-648b-4c05-8fd4-30535c2fc262" (UID: "fe280394-648b-4c05-8fd4-30535c2fc262"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 29 13:29:19.188796 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:19.188773    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe280394-648b-4c05-8fd4-30535c2fc262-kube-api-access-2smpn" (OuterVolumeSpecName: "kube-api-access-2smpn") pod "fe280394-648b-4c05-8fd4-30535c2fc262" (UID: "fe280394-648b-4c05-8fd4-30535c2fc262"). InnerVolumeSpecName "kube-api-access-2smpn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 13:29:19.188894 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:19.188793    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe280394-648b-4c05-8fd4-30535c2fc262-istio-token" (OuterVolumeSpecName: "istio-token") pod "fe280394-648b-4c05-8fd4-30535c2fc262" (UID: "fe280394-648b-4c05-8fd4-30535c2fc262"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 13:29:19.287558 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:19.287538    2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe280394-648b-4c05-8fd4-30535c2fc262-istio-token\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 13:29:19.287558 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:19.287558    2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe280394-648b-4c05-8fd4-30535c2fc262-istio-podinfo\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 13:29:19.287680 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:19.287567    2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe280394-648b-4c05-8fd4-30535c2fc262-credential-socket\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 13:29:19.287680 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:19.287576    2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe280394-648b-4c05-8fd4-30535c2fc262-workload-certs\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 13:29:19.287680 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:19.287583    2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe280394-648b-4c05-8fd4-30535c2fc262-istiod-ca-cert\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 13:29:19.287680 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:19.287591    2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe280394-648b-4c05-8fd4-30535c2fc262-istio-envoy\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 13:29:19.287680 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:19.287598    2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe280394-648b-4c05-8fd4-30535c2fc262-istio-data\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 13:29:19.287680 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:19.287605    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2smpn\" (UniqueName: \"kubernetes.io/projected/fe280394-648b-4c05-8fd4-30535c2fc262-kube-api-access-2smpn\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 13:29:19.620557 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:19.620532    2567 generic.go:356] "Generic (PLEG): container finished" podID="fe280394-648b-4c05-8fd4-30535c2fc262" containerID="e7292479f3fcb7af0c563de4f4fe1fd51bc70e61d42154f13513293eeda0d92b" exitCode=0
Jul 29 13:29:19.620683 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:19.620587    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n"
Jul 29 13:29:19.620683 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:19.620594    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n" event={"ID":"fe280394-648b-4c05-8fd4-30535c2fc262","Type":"ContainerDied","Data":"e7292479f3fcb7af0c563de4f4fe1fd51bc70e61d42154f13513293eeda0d92b"}
Jul 29 13:29:19.620683 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:19.620656    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n" event={"ID":"fe280394-648b-4c05-8fd4-30535c2fc262","Type":"ContainerDied","Data":"b1c0f8b7e71afe15e7116b7c39eaf592be02fd5bbed50976b7d29c834a4c8be8"}
Jul 29 13:29:19.620868 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:19.620686    2567 scope.go:117] "RemoveContainer" containerID="e7292479f3fcb7af0c563de4f4fe1fd51bc70e61d42154f13513293eeda0d92b"
Jul 29 13:29:19.631213 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:19.631191    2567 scope.go:117] "RemoveContainer" containerID="477d3d6af084aba2e8590db70e57ce37c823779e467eaad9417558952efcbc9b"
Jul 29 13:29:19.640045 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:19.640026    2567 scope.go:117] "RemoveContainer" containerID="e7292479f3fcb7af0c563de4f4fe1fd51bc70e61d42154f13513293eeda0d92b"
Jul 29 13:29:19.640285 ip-10-0-128-25 kubenswrapper[2567]: E0729 13:29:19.640266    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7292479f3fcb7af0c563de4f4fe1fd51bc70e61d42154f13513293eeda0d92b\": container with ID starting with e7292479f3fcb7af0c563de4f4fe1fd51bc70e61d42154f13513293eeda0d92b not found: ID does not exist" containerID="e7292479f3fcb7af0c563de4f4fe1fd51bc70e61d42154f13513293eeda0d92b"
Jul 29 13:29:19.640340 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:19.640296    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7292479f3fcb7af0c563de4f4fe1fd51bc70e61d42154f13513293eeda0d92b"} err="failed to get container status \"e7292479f3fcb7af0c563de4f4fe1fd51bc70e61d42154f13513293eeda0d92b\": rpc error: code = NotFound desc = could not find container \"e7292479f3fcb7af0c563de4f4fe1fd51bc70e61d42154f13513293eeda0d92b\": container with ID starting with e7292479f3fcb7af0c563de4f4fe1fd51bc70e61d42154f13513293eeda0d92b not found: ID does not exist"
Jul 29 13:29:19.640340 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:19.640314    2567 scope.go:117] "RemoveContainer" containerID="477d3d6af084aba2e8590db70e57ce37c823779e467eaad9417558952efcbc9b"
Jul 29 13:29:19.640635 ip-10-0-128-25 kubenswrapper[2567]: E0729 13:29:19.640617    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"477d3d6af084aba2e8590db70e57ce37c823779e467eaad9417558952efcbc9b\": container with ID starting with 477d3d6af084aba2e8590db70e57ce37c823779e467eaad9417558952efcbc9b not found: ID does not exist" containerID="477d3d6af084aba2e8590db70e57ce37c823779e467eaad9417558952efcbc9b"
Jul 29 13:29:19.640675 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:19.640643    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"477d3d6af084aba2e8590db70e57ce37c823779e467eaad9417558952efcbc9b"} err="failed to get container status \"477d3d6af084aba2e8590db70e57ce37c823779e467eaad9417558952efcbc9b\": rpc error: code = NotFound desc = could not find container \"477d3d6af084aba2e8590db70e57ce37c823779e467eaad9417558952efcbc9b\": container with ID starting with 477d3d6af084aba2e8590db70e57ce37c823779e467eaad9417558952efcbc9b not found: ID does not exist"
Jul 29 13:29:19.648252 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:19.648230    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n"]
Jul 29 13:29:19.655443 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:19.655421    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-bb0da0-openshift-default-5b7567777d-qlv5n"]
Jul 29 13:29:19.922464 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:19.922352    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe280394-648b-4c05-8fd4-30535c2fc262" path="/var/lib/kubelet/pods/fe280394-648b-4c05-8fd4-30535c2fc262/volumes"
Jul 29 13:29:24.564428 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:24.564374    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-f0ad55-5c8fcc6b55-gb5dw"]
Jul 29 13:29:24.587305 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:24.587267    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-f0ad55-29755515-jhpb7"]
Jul 29 13:29:24.589968 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:24.589944    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-f0ad55-29755515-jhpb7"]
Jul 29 13:29:24.653735 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:24.653650    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f0ad55-5c8fcc6b55-gb5dw" podUID="a8e9938b-04b9-483f-88de-606e938626bc" containerName="maas-api" containerID="cri-o://977bf2dd6b045be54b55b2f1c59520150baf587a213dd6262b247e5f2a0d2225" gracePeriod=30
Jul 29 13:29:24.768835 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:24.768805    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f0ad55-sbjk4"]
Jul 29 13:29:24.772164 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:24.772134    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f0ad55-sbjk4"]
Jul 29 13:29:24.907299 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:24.907277    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f0ad55-5c8fcc6b55-gb5dw"
Jul 29 13:29:25.034335 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:25.034307    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-27cmw\" (UniqueName: \"kubernetes.io/projected/a8e9938b-04b9-483f-88de-606e938626bc-kube-api-access-27cmw\") pod \"a8e9938b-04b9-483f-88de-606e938626bc\" (UID: \"a8e9938b-04b9-483f-88de-606e938626bc\") "
Jul 29 13:29:25.034494 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:25.034365    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8e9938b-04b9-483f-88de-606e938626bc-openshift-service-ca\") pod \"a8e9938b-04b9-483f-88de-606e938626bc\" (UID: \"a8e9938b-04b9-483f-88de-606e938626bc\") "
Jul 29 13:29:25.034494 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:25.034434    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8e9938b-04b9-483f-88de-606e938626bc-maas-api-tls\") pod \"a8e9938b-04b9-483f-88de-606e938626bc\" (UID: \"a8e9938b-04b9-483f-88de-606e938626bc\") "
Jul 29 13:29:25.034728 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:25.034707    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8e9938b-04b9-483f-88de-606e938626bc-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a8e9938b-04b9-483f-88de-606e938626bc" (UID: "a8e9938b-04b9-483f-88de-606e938626bc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 13:29:25.036350 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:25.036323    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8e9938b-04b9-483f-88de-606e938626bc-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a8e9938b-04b9-483f-88de-606e938626bc" (UID: "a8e9938b-04b9-483f-88de-606e938626bc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 13:29:25.036350 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:25.036337    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8e9938b-04b9-483f-88de-606e938626bc-kube-api-access-27cmw" (OuterVolumeSpecName: "kube-api-access-27cmw") pod "a8e9938b-04b9-483f-88de-606e938626bc" (UID: "a8e9938b-04b9-483f-88de-606e938626bc"). InnerVolumeSpecName "kube-api-access-27cmw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 13:29:25.135999 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:25.135946    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8e9938b-04b9-483f-88de-606e938626bc-openshift-service-ca\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 13:29:25.135999 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:25.135969    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8e9938b-04b9-483f-88de-606e938626bc-maas-api-tls\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 13:29:25.135999 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:25.135978    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-27cmw\" (UniqueName: \"kubernetes.io/projected/a8e9938b-04b9-483f-88de-606e938626bc-kube-api-access-27cmw\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 13:29:25.660904 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:25.660871    2567 generic.go:356] "Generic (PLEG): container finished" podID="a8e9938b-04b9-483f-88de-606e938626bc" containerID="977bf2dd6b045be54b55b2f1c59520150baf587a213dd6262b247e5f2a0d2225" exitCode=0
Jul 29 13:29:25.661300 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:25.660924    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f0ad55-5c8fcc6b55-gb5dw"
Jul 29 13:29:25.661300 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:25.660969    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f0ad55-5c8fcc6b55-gb5dw" event={"ID":"a8e9938b-04b9-483f-88de-606e938626bc","Type":"ContainerDied","Data":"977bf2dd6b045be54b55b2f1c59520150baf587a213dd6262b247e5f2a0d2225"}
Jul 29 13:29:25.661300 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:25.661049    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f0ad55-5c8fcc6b55-gb5dw" event={"ID":"a8e9938b-04b9-483f-88de-606e938626bc","Type":"ContainerDied","Data":"58c5a0a294e187a2319081253224785db2bb56f4373129cbae311cbeda1c73e1"}
Jul 29 13:29:25.661300 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:25.661086    2567 scope.go:117] "RemoveContainer" containerID="977bf2dd6b045be54b55b2f1c59520150baf587a213dd6262b247e5f2a0d2225"
Jul 29 13:29:25.671348 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:25.671321    2567 scope.go:117] "RemoveContainer" containerID="977bf2dd6b045be54b55b2f1c59520150baf587a213dd6262b247e5f2a0d2225"
Jul 29 13:29:25.671646 ip-10-0-128-25 kubenswrapper[2567]: E0729 13:29:25.671625    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"977bf2dd6b045be54b55b2f1c59520150baf587a213dd6262b247e5f2a0d2225\": container with ID starting with 977bf2dd6b045be54b55b2f1c59520150baf587a213dd6262b247e5f2a0d2225 not found: ID does not exist" containerID="977bf2dd6b045be54b55b2f1c59520150baf587a213dd6262b247e5f2a0d2225"
Jul 29 13:29:25.671712 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:25.671653    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"977bf2dd6b045be54b55b2f1c59520150baf587a213dd6262b247e5f2a0d2225"} err="failed to get container status \"977bf2dd6b045be54b55b2f1c59520150baf587a213dd6262b247e5f2a0d2225\": rpc error: code = NotFound desc = could not find container \"977bf2dd6b045be54b55b2f1c59520150baf587a213dd6262b247e5f2a0d2225\": container with ID starting with 977bf2dd6b045be54b55b2f1c59520150baf587a213dd6262b247e5f2a0d2225 not found: ID does not exist"
Jul 29 13:29:25.683980 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:25.683955    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-f0ad55-5c8fcc6b55-gb5dw"]
Jul 29 13:29:25.689037 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:25.689013    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-f0ad55-5c8fcc6b55-gb5dw"]
Jul 29 13:29:25.922009 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:25.921939    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c509298-80f9-432e-b271-e4fdb352f1fc" path="/var/lib/kubelet/pods/7c509298-80f9-432e-b271-e4fdb352f1fc/volumes"
Jul 29 13:29:25.922261 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:25.922247    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a7984e5-628c-4875-b1c4-6b08d271d892" path="/var/lib/kubelet/pods/8a7984e5-628c-4875-b1c4-6b08d271d892/volumes"
Jul 29 13:29:25.922664 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:25.922651    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8e9938b-04b9-483f-88de-606e938626bc" path="/var/lib/kubelet/pods/a8e9938b-04b9-483f-88de-606e938626bc/volumes"
Jul 29 13:29:31.732190 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:31.732157    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-11a904-gfbnq"]
Jul 29 13:29:31.732610 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:31.732577    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe280394-648b-4c05-8fd4-30535c2fc262" containerName="istio-proxy"
Jul 29 13:29:31.732610 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:31.732589    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe280394-648b-4c05-8fd4-30535c2fc262" containerName="istio-proxy"
Jul 29 13:29:31.732610 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:31.732603    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a7984e5-628c-4875-b1c4-6b08d271d892" containerName="revoke-keys"
Jul 29 13:29:31.732610 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:31.732609    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a7984e5-628c-4875-b1c4-6b08d271d892" containerName="revoke-keys"
Jul 29 13:29:31.732782 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:31.732624    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8e9938b-04b9-483f-88de-606e938626bc" containerName="maas-api"
Jul 29 13:29:31.732782 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:31.732630    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8e9938b-04b9-483f-88de-606e938626bc" containerName="maas-api"
Jul 29 13:29:31.732782 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:31.732725    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a7984e5-628c-4875-b1c4-6b08d271d892" containerName="revoke-keys"
Jul 29 13:29:31.732782 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:31.732733    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8e9938b-04b9-483f-88de-606e938626bc" containerName="maas-api"
Jul 29 13:29:31.732782 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:31.732739    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe280394-648b-4c05-8fd4-30535c2fc262" containerName="istio-proxy"
Jul 29 13:29:31.737478 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:31.737460    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-11a904-gfbnq"
Jul 29 13:29:31.739906 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:31.739882    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7gpd2\""
Jul 29 13:29:31.744792 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:31.744769    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-11a904-gfbnq"]
Jul 29 13:29:31.895396 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:31.895363    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f5d235d-e52c-473d-9f22-a4c2dea23d58-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-11a904-gfbnq\" (UID: \"6f5d235d-e52c-473d-9f22-a4c2dea23d58\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-11a904-gfbnq"
Jul 29 13:29:31.996735 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:31.996658    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f5d235d-e52c-473d-9f22-a4c2dea23d58-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-11a904-gfbnq\" (UID: \"6f5d235d-e52c-473d-9f22-a4c2dea23d58\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-11a904-gfbnq"
Jul 29 13:29:31.997295 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:31.997271    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f5d235d-e52c-473d-9f22-a4c2dea23d58-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-11a904-gfbnq\" (UID: \"6f5d235d-e52c-473d-9f22-a4c2dea23d58\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-11a904-gfbnq"
Jul 29 13:29:32.049582 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:32.049560    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-11a904-gfbnq"
Jul 29 13:29:32.179763 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:32.179728    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-11a904-gfbnq"]
Jul 29 13:29:32.181124 ip-10-0-128-25 kubenswrapper[2567]: W0729 13:29:32.181099    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f5d235d_e52c_473d_9f22_a4c2dea23d58.slice/crio-391a5623a7a0befc50ab7ecf5af811615d7fbec1eb280d887344aa29e45aeb0e WatchSource:0}: Error finding container 391a5623a7a0befc50ab7ecf5af811615d7fbec1eb280d887344aa29e45aeb0e: Status 404 returned error can't find the container with id 391a5623a7a0befc50ab7ecf5af811615d7fbec1eb280d887344aa29e45aeb0e
Jul 29 13:29:32.707241 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:32.707170    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-11a904-gfbnq" event={"ID":"6f5d235d-e52c-473d-9f22-a4c2dea23d58","Type":"ContainerStarted","Data":"1e205e4f517c1940bb5b11f065a75538c1ecb01fe621729fbd89328551369ff1"}
Jul 29 13:29:32.707241 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:32.707228    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-11a904-gfbnq" event={"ID":"6f5d235d-e52c-473d-9f22-a4c2dea23d58","Type":"ContainerStarted","Data":"391a5623a7a0befc50ab7ecf5af811615d7fbec1eb280d887344aa29e45aeb0e"}
Jul 29 13:29:32.726310 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:32.726267    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-11a904-gfbnq" podStartSLOduration=1.726252282 podStartE2EDuration="1.726252282s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 13:29:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 13:29:32.723533982 +0000 UTC m=+3855.417222314" watchObservedRunningTime="2026-07-29 13:29:32.726252282 +0000 UTC m=+3855.419940591"
Jul 29 13:29:33.716942 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:33.716909    2567 generic.go:356] "Generic (PLEG): container finished" podID="6f5d235d-e52c-473d-9f22-a4c2dea23d58" containerID="1e205e4f517c1940bb5b11f065a75538c1ecb01fe621729fbd89328551369ff1" exitCode=0
Jul 29 13:29:33.717341 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:33.716997    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-11a904-gfbnq" event={"ID":"6f5d235d-e52c-473d-9f22-a4c2dea23d58","Type":"ContainerDied","Data":"1e205e4f517c1940bb5b11f065a75538c1ecb01fe621729fbd89328551369ff1"}
Jul 29 13:29:34.856982 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:34.856961    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-11a904-gfbnq"
Jul 29 13:29:34.919411 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:34.919379    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f5d235d-e52c-473d-9f22-a4c2dea23d58-maas-api-service-ca\") pod \"6f5d235d-e52c-473d-9f22-a4c2dea23d58\" (UID: \"6f5d235d-e52c-473d-9f22-a4c2dea23d58\") "
Jul 29 13:29:34.919745 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:34.919726    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f5d235d-e52c-473d-9f22-a4c2dea23d58-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6f5d235d-e52c-473d-9f22-a4c2dea23d58" (UID: "6f5d235d-e52c-473d-9f22-a4c2dea23d58"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 13:29:35.020318 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:35.020257    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f5d235d-e52c-473d-9f22-a4c2dea23d58-maas-api-service-ca\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 13:29:35.730942 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:35.730917    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-11a904-gfbnq"
Jul 29 13:29:35.731076 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:35.730956    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-11a904-gfbnq" event={"ID":"6f5d235d-e52c-473d-9f22-a4c2dea23d58","Type":"ContainerDied","Data":"391a5623a7a0befc50ab7ecf5af811615d7fbec1eb280d887344aa29e45aeb0e"}
Jul 29 13:29:35.731076 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:35.731011    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="391a5623a7a0befc50ab7ecf5af811615d7fbec1eb280d887344aa29e45aeb0e"
Jul 29 13:29:42.262054 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:42.261467    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-11a904-7c9668d9d9-msxz6"]
Jul 29 13:29:42.262054 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:42.261751    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-11a904-7c9668d9d9-msxz6" podUID="f2232e07-a88e-4a23-8bf2-f290c0bd7116" containerName="maas-api" containerID="cri-o://c7040e149a75e8385cdc0488e2d30f79195874334ed44ffdf7e9879795331b43" gracePeriod=30
Jul 29 13:29:42.510921 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:42.510882    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-11a904-7c9668d9d9-msxz6"
Jul 29 13:29:42.582067 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:42.582043    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nkhz5\" (UniqueName: \"kubernetes.io/projected/f2232e07-a88e-4a23-8bf2-f290c0bd7116-kube-api-access-nkhz5\") pod \"f2232e07-a88e-4a23-8bf2-f290c0bd7116\" (UID: \"f2232e07-a88e-4a23-8bf2-f290c0bd7116\") "
Jul 29 13:29:42.582191 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:42.582092    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2232e07-a88e-4a23-8bf2-f290c0bd7116-openshift-service-ca\") pod \"f2232e07-a88e-4a23-8bf2-f290c0bd7116\" (UID: \"f2232e07-a88e-4a23-8bf2-f290c0bd7116\") "
Jul 29 13:29:42.582191 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:42.582121    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f2232e07-a88e-4a23-8bf2-f290c0bd7116-maas-api-tls\") pod \"f2232e07-a88e-4a23-8bf2-f290c0bd7116\" (UID: \"f2232e07-a88e-4a23-8bf2-f290c0bd7116\") "
Jul 29 13:29:42.582464 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:42.582434    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2232e07-a88e-4a23-8bf2-f290c0bd7116-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f2232e07-a88e-4a23-8bf2-f290c0bd7116" (UID: "f2232e07-a88e-4a23-8bf2-f290c0bd7116"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 13:29:42.584004 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:42.583981    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2232e07-a88e-4a23-8bf2-f290c0bd7116-kube-api-access-nkhz5" (OuterVolumeSpecName: "kube-api-access-nkhz5") pod "f2232e07-a88e-4a23-8bf2-f290c0bd7116" (UID: "f2232e07-a88e-4a23-8bf2-f290c0bd7116"). InnerVolumeSpecName "kube-api-access-nkhz5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 13:29:42.584078 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:42.584031    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f2232e07-a88e-4a23-8bf2-f290c0bd7116-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f2232e07-a88e-4a23-8bf2-f290c0bd7116" (UID: "f2232e07-a88e-4a23-8bf2-f290c0bd7116"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 13:29:42.657445 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:42.657422    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-11a904-gfbnq"]
Jul 29 13:29:42.667436 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:42.667396    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-11a904-gfbnq"]
Jul 29 13:29:42.683107 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:42.683087    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nkhz5\" (UniqueName: \"kubernetes.io/projected/f2232e07-a88e-4a23-8bf2-f290c0bd7116-kube-api-access-nkhz5\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 13:29:42.683107 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:42.683105    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2232e07-a88e-4a23-8bf2-f290c0bd7116-openshift-service-ca\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 13:29:42.683213 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:42.683116    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f2232e07-a88e-4a23-8bf2-f290c0bd7116-maas-api-tls\") on node \"ip-10-0-128-25.ec2.internal\" DevicePath \"\""
Jul 29 13:29:42.773643 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:42.773619    2567 generic.go:356] "Generic (PLEG): container finished" podID="f2232e07-a88e-4a23-8bf2-f290c0bd7116" containerID="c7040e149a75e8385cdc0488e2d30f79195874334ed44ffdf7e9879795331b43" exitCode=0
Jul 29 13:29:42.773732 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:42.773662    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-11a904-7c9668d9d9-msxz6" event={"ID":"f2232e07-a88e-4a23-8bf2-f290c0bd7116","Type":"ContainerDied","Data":"c7040e149a75e8385cdc0488e2d30f79195874334ed44ffdf7e9879795331b43"}
Jul 29 13:29:42.773732 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:42.773698    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-11a904-7c9668d9d9-msxz6" event={"ID":"f2232e07-a88e-4a23-8bf2-f290c0bd7116","Type":"ContainerDied","Data":"9c45389c81f41c598da2b02348f2692dd9a39cfa26e85b49761d9ba40dd97f40"}
Jul 29 13:29:42.773732 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:42.773719    2567 scope.go:117] "RemoveContainer" containerID="c7040e149a75e8385cdc0488e2d30f79195874334ed44ffdf7e9879795331b43"
Jul 29 13:29:42.773841 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:42.773667    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-11a904-7c9668d9d9-msxz6"
Jul 29 13:29:42.783300 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:42.783284    2567 scope.go:117] "RemoveContainer" containerID="c7040e149a75e8385cdc0488e2d30f79195874334ed44ffdf7e9879795331b43"
Jul 29 13:29:42.783576 ip-10-0-128-25 kubenswrapper[2567]: E0729 13:29:42.783557    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7040e149a75e8385cdc0488e2d30f79195874334ed44ffdf7e9879795331b43\": container with ID starting with c7040e149a75e8385cdc0488e2d30f79195874334ed44ffdf7e9879795331b43 not found: ID does not exist" containerID="c7040e149a75e8385cdc0488e2d30f79195874334ed44ffdf7e9879795331b43"
Jul 29 13:29:42.783645 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:42.783586    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7040e149a75e8385cdc0488e2d30f79195874334ed44ffdf7e9879795331b43"} err="failed to get container status \"c7040e149a75e8385cdc0488e2d30f79195874334ed44ffdf7e9879795331b43\": rpc error: code = NotFound desc = could not find container \"c7040e149a75e8385cdc0488e2d30f79195874334ed44ffdf7e9879795331b43\": container with ID starting with c7040e149a75e8385cdc0488e2d30f79195874334ed44ffdf7e9879795331b43 not found: ID does not exist"
Jul 29 13:29:42.795851 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:42.795831    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-11a904-7c9668d9d9-msxz6"]
Jul 29 13:29:42.800534 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:42.800513    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-11a904-7c9668d9d9-msxz6"]
Jul 29 13:29:43.921752 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:43.921718    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f5d235d-e52c-473d-9f22-a4c2dea23d58" path="/var/lib/kubelet/pods/6f5d235d-e52c-473d-9f22-a4c2dea23d58/volumes"
Jul 29 13:29:43.922135 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:43.922096    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f2232e07-a88e-4a23-8bf2-f290c0bd7116" path="/var/lib/kubelet/pods/f2232e07-a88e-4a23-8bf2-f290c0bd7116/volumes"
Jul 29 13:29:56.497294 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:56.497249    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-zn4kr_0890babf-8771-4220-ad44-be722eb595e3/manager/0.log"
Jul 29 13:29:56.622335 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:56.622296    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-6cn2w_3a610ba1-bc8d-4484-890b-89646572c27b/manager/0.log"
Jul 29 13:29:56.866938 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:56.866919    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-cbbgq_8c660dce-7015-4cee-bb2d-e3a9ec0c2228/server/0.log"
Jul 29 13:29:56.992315 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:56.992291    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-hnbhw_02f86dac-d455-4b0e-af6a-0cdfc0a8b9f2/manager/0.log"
Jul 29 13:29:57.116145 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:57.116124    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-556f8bf875-2lg9f_078c30c5-e329-4de6-85e6-97e5fcef0a7f/copy-manifests/0.log"
Jul 29 13:29:57.123272 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:57.123211    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-556f8bf875-2lg9f_078c30c5-e329-4de6-85e6-97e5fcef0a7f/manager/0.log"
Jul 29 13:29:57.758795 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:57.758769    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-847fd45f6b-njnmj_4e789990-05eb-467f-8a5d-f543fd824811/maas-api/0.log"
Jul 29 13:29:57.873707 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:57.873682    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29755485-gzrqx_c836e992-7c0a-4870-a012-68d4a43a4697/cleanup/0.log"
Jul 29 13:29:58.003355 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:58.003330    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29755500-49hsw_0051e2df-1e85-40a0-9675-c233ac90ff3a/cleanup/0.log"
Jul 29 13:29:58.126910 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:58.126888    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29755515-txk4l_c279df75-8a11-45a6-b915-6ab723f191b0/cleanup/0.log"
Jul 29 13:29:58.255512 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:58.255489    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-ng554_4e94fe7d-6d68-436f-addf-5cb8c2f893c7/postgres/0.log"
Jul 29 13:29:59.123808 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:59.123784    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595psw8_41b3c368-9b73-4152-9cba-3b722aedf467/util/0.log"
Jul 29 13:29:59.130817 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:59.130794    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595psw8_41b3c368-9b73-4152-9cba-3b722aedf467/pull/0.log"
Jul 29 13:29:59.138315 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:59.138296    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595psw8_41b3c368-9b73-4152-9cba-3b722aedf467/extract/0.log"
Jul 29 13:29:59.256086 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:59.256052    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zknzl_e0a1ea4f-b4d2-4e30-8de8-521a4b0b8616/util/0.log"
Jul 29 13:29:59.263037 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:59.263018    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zknzl_e0a1ea4f-b4d2-4e30-8de8-521a4b0b8616/pull/0.log"
Jul 29 13:29:59.270152 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:59.270126    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zknzl_e0a1ea4f-b4d2-4e30-8de8-521a4b0b8616/extract/0.log"
Jul 29 13:29:59.381702 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:59.381645    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73czhfd_0d4a9b8d-ec51-4957-a386-67046db0072f/pull/0.log"
Jul 29 13:29:59.388518 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:59.388500    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73czhfd_0d4a9b8d-ec51-4957-a386-67046db0072f/extract/0.log"
Jul 29 13:29:59.395346 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:59.395325    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73czhfd_0d4a9b8d-ec51-4957-a386-67046db0072f/util/0.log"
Jul 29 13:29:59.522757 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:59.522736    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zd7m6_e9c400e5-5a3f-4c97-a133-af20186cbb59/util/0.log"
Jul 29 13:29:59.529633 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:59.529617    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zd7m6_e9c400e5-5a3f-4c97-a133-af20186cbb59/pull/0.log"
Jul 29 13:29:59.536164 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:29:59.536148    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zd7m6_e9c400e5-5a3f-4c97-a133-af20186cbb59/extract/0.log"
Jul 29 13:30:00.130727 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:00.130698    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29755530-dqhk4"]
Jul 29 13:30:00.131130 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:00.131115    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe280394-648b-4c05-8fd4-30535c2fc262" containerName="istio-proxy"
Jul 29 13:30:00.131130 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:00.131130    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe280394-648b-4c05-8fd4-30535c2fc262" containerName="istio-proxy"
Jul 29 13:30:00.131204 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:00.131160    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2232e07-a88e-4a23-8bf2-f290c0bd7116" containerName="maas-api"
Jul 29 13:30:00.131204 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:00.131165    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2232e07-a88e-4a23-8bf2-f290c0bd7116" containerName="maas-api"
Jul 29 13:30:00.131204 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:00.131172    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f5d235d-e52c-473d-9f22-a4c2dea23d58" containerName="revoke-keys"
Jul 29 13:30:00.131204 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:00.131177    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f5d235d-e52c-473d-9f22-a4c2dea23d58" containerName="revoke-keys"
Jul 29 13:30:00.131325 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:00.131239    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f5d235d-e52c-473d-9f22-a4c2dea23d58" containerName="revoke-keys"
Jul 29 13:30:00.131325 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:00.131247    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2232e07-a88e-4a23-8bf2-f290c0bd7116" containerName="maas-api"
Jul 29 13:30:00.131325 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:00.131253    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe280394-648b-4c05-8fd4-30535c2fc262" containerName="istio-proxy"
Jul 29 13:30:00.135858 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:00.135840    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755530-dqhk4"
Jul 29 13:30:00.139310 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:00.139287    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7gpd2\""
Jul 29 13:30:00.146321 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:00.146302    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755530-dqhk4"
Jul 29 13:30:00.152980 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:00.152956    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29755530-dqhk4"]
Jul 29 13:30:00.303608 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:00.303580    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29755530-dqhk4"]
Jul 29 13:30:00.304706 ip-10-0-128-25 kubenswrapper[2567]: W0729 13:30:00.304676    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8672522_8136_4be2_b504_ca0f51ec8344.slice/crio-60333860bd17ca605d13c35de5492d184da95b0379a575f91701c757edb7e0f9 WatchSource:0}: Error finding container 60333860bd17ca605d13c35de5492d184da95b0379a575f91701c757edb7e0f9: Status 404 returned error can't find the container with id 60333860bd17ca605d13c35de5492d184da95b0379a575f91701c757edb7e0f9
Jul 29 13:30:00.616575 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:00.616552    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-l8jlz_a39801db-46e2-4be7-b99a-083d68ff26fc/manager/0.log"
Jul 29 13:30:00.891300 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:00.891266    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755530-dqhk4" event={"ID":"d8672522-8136-4be2-b504-ca0f51ec8344","Type":"ContainerStarted","Data":"3b4966167ae3a300cbe8fb6d468c2a505524bc8b93197be7cde3122a8ac734cf"}
Jul 29 13:30:00.891446 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:00.891333    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755530-dqhk4" event={"ID":"d8672522-8136-4be2-b504-ca0f51ec8344","Type":"ContainerStarted","Data":"60333860bd17ca605d13c35de5492d184da95b0379a575f91701c757edb7e0f9"}
Jul 29 13:30:00.908839 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:00.908790    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755530-dqhk4" podStartSLOduration=0.90877561 podStartE2EDuration="908.77561ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 13:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 13:30:00.906454125 +0000 UTC m=+3883.600142459" watchObservedRunningTime="2026-07-29 13:30:00.90877561 +0000 UTC m=+3883.602463958"
Jul 29 13:30:01.142032 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:01.141986    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-hwvfh_b7377791-4ee0-4583-a6a9-4339db5c7c40/discovery/0.log"
Jul 29 13:30:01.375095 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:01.375071    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6bb9c5fc49-n286g_52a11e4e-9288-433e-993f-987b09206bf2/kube-auth-proxy/0.log"
Jul 29 13:30:01.758124 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:01.758096    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-k7kgk_c261451c-033e-4484-ae79-c1a2ebb7d81b/payload-processing/0.log"
Jul 29 13:30:01.899083 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:01.899055    2567 generic.go:356] "Generic (PLEG): container finished" podID="d8672522-8136-4be2-b504-ca0f51ec8344" containerID="3b4966167ae3a300cbe8fb6d468c2a505524bc8b93197be7cde3122a8ac734cf" exitCode=0
Jul 29 13:30:01.899228 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:01.899138    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755530-dqhk4" event={"ID":"d8672522-8136-4be2-b504-ca0f51ec8344","Type":"ContainerDied","Data":"3b4966167ae3a300cbe8fb6d468c2a505524bc8b93197be7cde3122a8ac734cf"}
Jul 29 13:30:02.220021 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:02.219994    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-c2qbk_6a26f56b-1c7c-44fb-acc5-9a5a00a93a3f/main/0.log"
Jul 29 13:30:02.439557 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:02.439518    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-vbkqz_a273005c-fe77-4c36-b343-6aa5726ea755/main/0.log"
Jul 29 13:30:03.048961 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:03.048932    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755530-dqhk4"
Jul 29 13:30:03.181451 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:03.181376    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29755485-gzrqx"]
Jul 29 13:30:03.183737 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:03.183719    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29755485-gzrqx"]
Jul 29 13:30:03.915548 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:03.915516    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755530-dqhk4"
Jul 29 13:30:03.924282 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:03.924253    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c836e992-7c0a-4870-a012-68d4a43a4697" path="/var/lib/kubelet/pods/c836e992-7c0a-4870-a012-68d4a43a4697/volumes"
Jul 29 13:30:03.924767 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:03.924684    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755530-dqhk4" event={"ID":"d8672522-8136-4be2-b504-ca0f51ec8344","Type":"ContainerDied","Data":"60333860bd17ca605d13c35de5492d184da95b0379a575f91701c757edb7e0f9"}
Jul 29 13:30:03.924767 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:03.924721    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="60333860bd17ca605d13c35de5492d184da95b0379a575f91701c757edb7e0f9"
Jul 29 13:30:11.398512 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:11.398478    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-w97vv_1635829e-9a40-4521-8430-88adbb07a97a/global-pull-secret-syncer/0.log"
Jul 29 13:30:11.540217 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:11.540187    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-znk2m_7be0cf2c-db65-486e-8545-eabd24ad8428/konnectivity-agent/0.log"
Jul 29 13:30:11.562255 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:11.562228    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-25.ec2.internal_eee88d5337a9ab304c1e57b0fd78f1d9/haproxy/0.log"
Jul 29 13:30:15.444706 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:15.444672    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595psw8_41b3c368-9b73-4152-9cba-3b722aedf467/extract/0.log"
Jul 29 13:30:15.487115 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:15.487085    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595psw8_41b3c368-9b73-4152-9cba-3b722aedf467/util/0.log"
Jul 29 13:30:15.507822 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:15.507796    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595psw8_41b3c368-9b73-4152-9cba-3b722aedf467/pull/0.log"
Jul 29 13:30:15.546452 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:15.546433    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zknzl_e0a1ea4f-b4d2-4e30-8de8-521a4b0b8616/extract/0.log"
Jul 29 13:30:15.566160 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:15.566141    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zknzl_e0a1ea4f-b4d2-4e30-8de8-521a4b0b8616/util/0.log"
Jul 29 13:30:15.585852 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:15.585837    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zknzl_e0a1ea4f-b4d2-4e30-8de8-521a4b0b8616/pull/0.log"
Jul 29 13:30:15.612078 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:15.612060    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73czhfd_0d4a9b8d-ec51-4957-a386-67046db0072f/extract/0.log"
Jul 29 13:30:15.639010 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:15.638992    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73czhfd_0d4a9b8d-ec51-4957-a386-67046db0072f/util/0.log"
Jul 29 13:30:15.666093 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:15.666054    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73czhfd_0d4a9b8d-ec51-4957-a386-67046db0072f/pull/0.log"
Jul 29 13:30:15.696960 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:15.696912    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zd7m6_e9c400e5-5a3f-4c97-a133-af20186cbb59/extract/0.log"
Jul 29 13:30:15.720719 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:15.720700    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zd7m6_e9c400e5-5a3f-4c97-a133-af20186cbb59/util/0.log"
Jul 29 13:30:15.745964 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:15.745947    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zd7m6_e9c400e5-5a3f-4c97-a133-af20186cbb59/pull/0.log"
Jul 29 13:30:16.780612 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:16.780573    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-l8jlz_a39801db-46e2-4be7-b99a-083d68ff26fc/manager/0.log"
Jul 29 13:30:18.294430 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:18.294379    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2118756c-36d8-4f5c-894a-d55d27deb59f/alertmanager/0.log"
Jul 29 13:30:18.294937 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:18.294918    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kj82f_a6d5f3a2-26cc-40a7-9841-2fbfc247661a/ovn-acl-logging/0.log"
Jul 29 13:30:18.306311 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:18.306293    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kj82f_a6d5f3a2-26cc-40a7-9841-2fbfc247661a/ovn-acl-logging/0.log"
Jul 29 13:30:18.317158 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:18.317142    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2118756c-36d8-4f5c-894a-d55d27deb59f/config-reloader/0.log"
Jul 29 13:30:18.341218 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:18.341201    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2118756c-36d8-4f5c-894a-d55d27deb59f/kube-rbac-proxy-web/0.log"
Jul 29 13:30:18.369948 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:18.369928    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2118756c-36d8-4f5c-894a-d55d27deb59f/kube-rbac-proxy/0.log"
Jul 29 13:30:18.396560 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:18.396541    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2118756c-36d8-4f5c-894a-d55d27deb59f/kube-rbac-proxy-metric/0.log"
Jul 29 13:30:18.432212 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:18.432196    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2118756c-36d8-4f5c-894a-d55d27deb59f/prom-label-proxy/0.log"
Jul 29 13:30:18.456679 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:18.456662    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2118756c-36d8-4f5c-894a-d55d27deb59f/init-config-reloader/0.log"
Jul 29 13:30:18.495600 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:18.495585    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-rmcgr_2188f74f-2cb0-4a90-a994-ceb43417c819/cluster-monitoring-operator/0.log"
Jul 29 13:30:18.533081 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:18.533061    2567 scope.go:117] "RemoveContainer" containerID="f00d0147c17a4473f0a352d70b4993d0c8001cb7800f705986c9607bbf6b622a"
Jul 29 13:30:18.541366 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:18.541346    2567 scope.go:117] "RemoveContainer" containerID="6853c8792bd342cd20b25f5a268f2e610599a4e07f6f4e45e8887587491a2ac3"
Jul 29 13:30:18.611492 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:18.611468    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5479989fc7-qqwlm_d9e1e62f-87ee-4c09-b3f5-2393aede7247/metrics-server/0.log"
Jul 29 13:30:18.640101 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:18.640081    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-68b7fc9574-f79zm_75171e3d-f745-40e6-807c-02518fbe54cc/monitoring-plugin/0.log"
Jul 29 13:30:18.831278 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:18.831225    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rzxkb_ab16f311-c741-4ce9-92b2-bd2e46b049f7/node-exporter/0.log"
Jul 29 13:30:18.853307 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:18.853289    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rzxkb_ab16f311-c741-4ce9-92b2-bd2e46b049f7/kube-rbac-proxy/0.log"
Jul 29 13:30:18.876666 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:18.876649    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rzxkb_ab16f311-c741-4ce9-92b2-bd2e46b049f7/init-textfile/0.log"
Jul 29 13:30:19.247540 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:19.247493    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7f8d78f54b-ljpg7_3ba5769c-b445-4466-b7b3-376de6bf5fe2/prometheus-operator-admission-webhook/0.log"
Jul 29 13:30:19.279388 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:19.279373    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7c9474dfd4-fbwrc_f02601fa-bbb1-4088-bc3f-c96dae91017f/telemeter-client/0.log"
Jul 29 13:30:19.302449 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:19.302423    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7c9474dfd4-fbwrc_f02601fa-bbb1-4088-bc3f-c96dae91017f/reload/0.log"
Jul 29 13:30:19.326116 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:19.326098    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7c9474dfd4-fbwrc_f02601fa-bbb1-4088-bc3f-c96dae91017f/kube-rbac-proxy/0.log"
Jul 29 13:30:20.710542 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:20.710511    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-74c9f884b7-d48j6_271b0e50-2034-4105-b7d0-49e85d78c021/networking-console-plugin/0.log"
Jul 29 13:30:21.363918 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:21.363885    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-gv8zd/perf-node-gather-daemonset-x66z9"]
Jul 29 13:30:21.364320 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:21.364305    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8672522-8136-4be2-b504-ca0f51ec8344" containerName="cleanup"
Jul 29 13:30:21.364367 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:21.364321    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8672522-8136-4be2-b504-ca0f51ec8344" containerName="cleanup"
Jul 29 13:30:21.364426 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:21.364395    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8672522-8136-4be2-b504-ca0f51ec8344" containerName="cleanup"
Jul 29 13:30:21.368062 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:21.368037    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gv8zd/perf-node-gather-daemonset-x66z9"
Jul 29 13:30:21.370894 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:21.370868    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gv8zd\"/\"kube-root-ca.crt\""
Jul 29 13:30:21.371044 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:21.370895    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gv8zd\"/\"openshift-service-ca.crt\""
Jul 29 13:30:21.371750 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:21.371728    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-gv8zd\"/\"default-dockercfg-k884m\""
Jul 29 13:30:21.375622 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:21.375597    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gv8zd/perf-node-gather-daemonset-x66z9"]
Jul 29 13:30:21.496541 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:21.496514    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d1af776a-2fe5-42f8-b152-9f3a5e59200b-lib-modules\") pod \"perf-node-gather-daemonset-x66z9\" (UID: \"d1af776a-2fe5-42f8-b152-9f3a5e59200b\") " pod="openshift-must-gather-gv8zd/perf-node-gather-daemonset-x66z9"
Jul 29 13:30:21.496686 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:21.496553    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d1af776a-2fe5-42f8-b152-9f3a5e59200b-podres\") pod \"perf-node-gather-daemonset-x66z9\" (UID: \"d1af776a-2fe5-42f8-b152-9f3a5e59200b\") " pod="openshift-must-gather-gv8zd/perf-node-gather-daemonset-x66z9"
Jul 29 13:30:21.496686 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:21.496641    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d1af776a-2fe5-42f8-b152-9f3a5e59200b-sys\") pod \"perf-node-gather-daemonset-x66z9\" (UID: \"d1af776a-2fe5-42f8-b152-9f3a5e59200b\") " pod="openshift-must-gather-gv8zd/perf-node-gather-daemonset-x66z9"
Jul 29 13:30:21.496761 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:21.496682    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d1af776a-2fe5-42f8-b152-9f3a5e59200b-proc\") pod \"perf-node-gather-daemonset-x66z9\" (UID: \"d1af776a-2fe5-42f8-b152-9f3a5e59200b\") " pod="openshift-must-gather-gv8zd/perf-node-gather-daemonset-x66z9"
Jul 29 13:30:21.496761 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:21.496734    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rdxs\" (UniqueName: \"kubernetes.io/projected/d1af776a-2fe5-42f8-b152-9f3a5e59200b-kube-api-access-5rdxs\") pod \"perf-node-gather-daemonset-x66z9\" (UID: \"d1af776a-2fe5-42f8-b152-9f3a5e59200b\") " pod="openshift-must-gather-gv8zd/perf-node-gather-daemonset-x66z9"
Jul 29 13:30:21.597893 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:21.597863    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d1af776a-2fe5-42f8-b152-9f3a5e59200b-sys\") pod \"perf-node-gather-daemonset-x66z9\" (UID: \"d1af776a-2fe5-42f8-b152-9f3a5e59200b\") " pod="openshift-must-gather-gv8zd/perf-node-gather-daemonset-x66z9"
Jul 29 13:30:21.598009 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:21.597896    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d1af776a-2fe5-42f8-b152-9f3a5e59200b-proc\") pod \"perf-node-gather-daemonset-x66z9\" (UID: \"d1af776a-2fe5-42f8-b152-9f3a5e59200b\") " pod="openshift-must-gather-gv8zd/perf-node-gather-daemonset-x66z9"
Jul 29 13:30:21.598009 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:21.597927    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rdxs\" (UniqueName: \"kubernetes.io/projected/d1af776a-2fe5-42f8-b152-9f3a5e59200b-kube-api-access-5rdxs\") pod \"perf-node-gather-daemonset-x66z9\" (UID: \"d1af776a-2fe5-42f8-b152-9f3a5e59200b\") " pod="openshift-must-gather-gv8zd/perf-node-gather-daemonset-x66z9"
Jul 29 13:30:21.598009 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:21.597998    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d1af776a-2fe5-42f8-b152-9f3a5e59200b-sys\") pod \"perf-node-gather-daemonset-x66z9\" (UID: \"d1af776a-2fe5-42f8-b152-9f3a5e59200b\") " pod="openshift-must-gather-gv8zd/perf-node-gather-daemonset-x66z9"
Jul 29 13:30:21.598117 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:21.598005    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d1af776a-2fe5-42f8-b152-9f3a5e59200b-proc\") pod \"perf-node-gather-daemonset-x66z9\" (UID: \"d1af776a-2fe5-42f8-b152-9f3a5e59200b\") " pod="openshift-must-gather-gv8zd/perf-node-gather-daemonset-x66z9"
Jul 29 13:30:21.598117 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:21.598008    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d1af776a-2fe5-42f8-b152-9f3a5e59200b-lib-modules\") pod \"perf-node-gather-daemonset-x66z9\" (UID: \"d1af776a-2fe5-42f8-b152-9f3a5e59200b\") " pod="openshift-must-gather-gv8zd/perf-node-gather-daemonset-x66z9"
Jul 29 13:30:21.598117 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:21.598071    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d1af776a-2fe5-42f8-b152-9f3a5e59200b-podres\") pod \"perf-node-gather-daemonset-x66z9\" (UID: \"d1af776a-2fe5-42f8-b152-9f3a5e59200b\") " pod="openshift-must-gather-gv8zd/perf-node-gather-daemonset-x66z9"
Jul 29 13:30:21.598117 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:21.598107    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d1af776a-2fe5-42f8-b152-9f3a5e59200b-lib-modules\") pod \"perf-node-gather-daemonset-x66z9\" (UID: \"d1af776a-2fe5-42f8-b152-9f3a5e59200b\") " pod="openshift-must-gather-gv8zd/perf-node-gather-daemonset-x66z9"
Jul 29 13:30:21.598235 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:21.598182    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d1af776a-2fe5-42f8-b152-9f3a5e59200b-podres\") pod \"perf-node-gather-daemonset-x66z9\" (UID: \"d1af776a-2fe5-42f8-b152-9f3a5e59200b\") " pod="openshift-must-gather-gv8zd/perf-node-gather-daemonset-x66z9"
Jul 29 13:30:21.610205 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:21.610173    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rdxs\" (UniqueName: \"kubernetes.io/projected/d1af776a-2fe5-42f8-b152-9f3a5e59200b-kube-api-access-5rdxs\") pod \"perf-node-gather-daemonset-x66z9\" (UID: \"d1af776a-2fe5-42f8-b152-9f3a5e59200b\") " pod="openshift-must-gather-gv8zd/perf-node-gather-daemonset-x66z9"
Jul 29 13:30:21.679823 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:21.679767    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gv8zd/perf-node-gather-daemonset-x66z9"
Jul 29 13:30:21.820758 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:21.820731    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gv8zd/perf-node-gather-daemonset-x66z9"]
Jul 29 13:30:21.990048 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:21.989996    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-76bb87fbc7-6khpt_32b1ff08-d437-4015-85d5-dff245655e8b/console/0.log"
Jul 29 13:30:22.022731 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:22.022713    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-7c97dc4555-lljkn_8ebccfd1-f963-40fb-a3f4-ddcb9b3cf8d6/download-server/0.log"
Jul 29 13:30:22.037357 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:22.037329    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gv8zd/perf-node-gather-daemonset-x66z9" event={"ID":"d1af776a-2fe5-42f8-b152-9f3a5e59200b","Type":"ContainerStarted","Data":"f0b1d73f57d7b18a9667d0c3ece581dd0dbab24309fc76a2d2cafc157ac03fa1"}
Jul 29 13:30:22.037476 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:22.037376    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gv8zd/perf-node-gather-daemonset-x66z9" event={"ID":"d1af776a-2fe5-42f8-b152-9f3a5e59200b","Type":"ContainerStarted","Data":"74a7f7a363e2e3e8c1aab06ddee26934f9430f185956603fe1c52aca67bc1d38"}
Jul 29 13:30:22.037476 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:22.037419    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-gv8zd/perf-node-gather-daemonset-x66z9"
Jul 29 13:30:22.055174 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:22.055136    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-gv8zd/perf-node-gather-daemonset-x66z9" podStartSLOduration=1.055123945 podStartE2EDuration="1.055123945s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 13:30:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 13:30:22.052627915 +0000 UTC m=+3904.746316247" watchObservedRunningTime="2026-07-29 13:30:22.055123945 +0000 UTC m=+3904.748812254"
Jul 29 13:30:23.739416 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:23.739371    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-sdj8z_e9572260-1f9a-4f9d-93db-1cc7f72dcc7a/dns/0.log"
Jul 29 13:30:23.760341 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:23.760318    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-sdj8z_e9572260-1f9a-4f9d-93db-1cc7f72dcc7a/kube-rbac-proxy/0.log"
Jul 29 13:30:23.782905 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:23.782880    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8rddq_4bdec0f6-396e-4392-9e8d-e2bad0289b8f/dns-node-resolver/0.log"
Jul 29 13:30:24.436152 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:24.436122    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-clmgr_caf950ed-dbb9-4b7d-a802-062882ff4899/node-ca/0.log"
Jul 29 13:30:25.518666 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:25.518640    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-hwvfh_b7377791-4ee0-4583-a6a9-4339db5c7c40/discovery/0.log"
Jul 29 13:30:25.564255 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:25.564227    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6bb9c5fc49-n286g_52a11e4e-9288-433e-993f-987b09206bf2/kube-auth-proxy/0.log"
Jul 29 13:30:25.719549 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:25.719528    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-k7kgk_c261451c-033e-4484-ae79-c1a2ebb7d81b/payload-processing/0.log"
Jul 29 13:30:26.573013 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:26.572978    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-w8d7p_5e7bd908-6123-4f1e-a5ef-109f1b3b0069/serve-healthcheck-canary/0.log"
Jul 29 13:30:27.051139 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:27.051112    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2q2mn_5edc73cd-3f85-4c3e-be4a-f311e46492a1/kube-rbac-proxy/0.log"
Jul 29 13:30:27.070844 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:27.070817    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2q2mn_5edc73cd-3f85-4c3e-be4a-f311e46492a1/exporter/0.log"
Jul 29 13:30:27.090650 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:27.090631    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2q2mn_5edc73cd-3f85-4c3e-be4a-f311e46492a1/extractor/0.log"
Jul 29 13:30:28.055213 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:28.055188    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-gv8zd/perf-node-gather-daemonset-x66z9"
Jul 29 13:30:29.299322 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:29.299293    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-zn4kr_0890babf-8771-4220-ad44-be722eb595e3/manager/0.log"
Jul 29 13:30:29.507217 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:29.507191    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-6cn2w_3a610ba1-bc8d-4484-890b-89646572c27b/manager/0.log"
Jul 29 13:30:29.811537 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:29.811511    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-cbbgq_8c660dce-7015-4cee-bb2d-e3a9ec0c2228/server/0.log"
Jul 29 13:30:29.889689 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:29.889665    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-hnbhw_02f86dac-d455-4b0e-af6a-0cdfc0a8b9f2/manager/0.log"
Jul 29 13:30:29.917954 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:29.917933    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-556f8bf875-2lg9f_078c30c5-e329-4de6-85e6-97e5fcef0a7f/manager/0.log"
Jul 29 13:30:29.939991 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:29.939974    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-556f8bf875-2lg9f_078c30c5-e329-4de6-85e6-97e5fcef0a7f/copy-manifests/0.log"
Jul 29 13:30:31.621462 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:31.621439    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-b7fxs_763a3c4a-bd0e-4f64-bbc5-c98f492f36d4/openshift-lws-operator/0.log"
Jul 29 13:30:31.755562 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:31.755513    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-hwvfh_b7377791-4ee0-4583-a6a9-4339db5c7c40/discovery/0.log"
Jul 29 13:30:31.781703 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:31.781680    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6bb9c5fc49-n286g_52a11e4e-9288-433e-993f-987b09206bf2/kube-auth-proxy/0.log"
Jul 29 13:30:31.907270 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:31.907248    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-k7kgk_c261451c-033e-4484-ae79-c1a2ebb7d81b/payload-processing/0.log"
Jul 29 13:30:33.436544 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:33.436519    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-hwvfh_b7377791-4ee0-4583-a6a9-4339db5c7c40/discovery/0.log"
Jul 29 13:30:33.468571 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:33.468542    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6bb9c5fc49-n286g_52a11e4e-9288-433e-993f-987b09206bf2/kube-auth-proxy/0.log"
Jul 29 13:30:33.593846 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:33.593825    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-k7kgk_c261451c-033e-4484-ae79-c1a2ebb7d81b/payload-processing/0.log"
Jul 29 13:30:36.970583 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:36.970552    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-g8xkd_c223418a-dbfe-4991-bc94-feebbfd40bfe/kube-storage-version-migrator-operator/1.log"
Jul 29 13:30:36.971390 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:36.971373    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-g8xkd_c223418a-dbfe-4991-bc94-feebbfd40bfe/kube-storage-version-migrator-operator/0.log"
Jul 29 13:30:38.305295 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:38.305262    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-h2qrx_a09948e8-234f-41ef-866d-10ddb32e33fd/kube-multus-additional-cni-plugins/0.log"
Jul 29 13:30:38.329526 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:38.329496    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-h2qrx_a09948e8-234f-41ef-866d-10ddb32e33fd/egress-router-binary-copy/0.log"
Jul 29 13:30:38.351348 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:38.351319    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-h2qrx_a09948e8-234f-41ef-866d-10ddb32e33fd/cni-plugins/0.log"
Jul 29 13:30:38.374206 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:38.374186    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-h2qrx_a09948e8-234f-41ef-866d-10ddb32e33fd/bond-cni-plugin/0.log"
Jul 29 13:30:38.394933 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:38.394914    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-h2qrx_a09948e8-234f-41ef-866d-10ddb32e33fd/routeoverride-cni/0.log"
Jul 29 13:30:38.417124 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:38.417100    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-h2qrx_a09948e8-234f-41ef-866d-10ddb32e33fd/whereabouts-cni-bincopy/0.log"
Jul 29 13:30:38.437859 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:38.437840    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-h2qrx_a09948e8-234f-41ef-866d-10ddb32e33fd/whereabouts-cni/0.log"
Jul 29 13:30:38.479597 ip-10-0-128-25 kubenswrapper[2567]: I0729 13:30:38.479574    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-ph59r_0f03f162-cb82-442c-8e40-b8877aa1996f/kube-multus/0.log"