Jul 24 23:27:33.869906 ip-10-0-142-133 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 24 23:27:33.869918 ip-10-0-142-133 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 24 23:27:33.869946 ip-10-0-142-133 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 24 23:27:33.870259 ip-10-0-142-133 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 24 23:27:43.990556 ip-10-0-142-133 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 24 23:27:43.990570 ip-10-0-142-133 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 06c42848aab74bb49fb36bc87107e692 --
Jul 24 23:29:19.484836 ip-10-0-142-133 systemd[1]: Starting Kubernetes Kubelet...
Jul 24 23:29:19.936032 ip-10-0-142-133 kubenswrapper[2558]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 24 23:29:19.936032 ip-10-0-142-133 kubenswrapper[2558]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 24 23:29:19.936032 ip-10-0-142-133 kubenswrapper[2558]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 24 23:29:19.936032 ip-10-0-142-133 kubenswrapper[2558]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 24 23:29:19.936032 ip-10-0-142-133 kubenswrapper[2558]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 24 23:29:19.937464 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.937362    2558 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 24 23:29:19.942503 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942485    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 23:29:19.942503 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942501    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 23:29:19.942503 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942505    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 23:29:19.942611 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942509    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 23:29:19.942611 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942512    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 23:29:19.942611 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942514    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 23:29:19.942611 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942517    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 23:29:19.942611 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942520    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 23:29:19.942611 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942523    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 23:29:19.942611 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942525    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 23:29:19.942611 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942539    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 23:29:19.942611 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942542    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 23:29:19.942611 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942545    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 23:29:19.942611 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942547    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 23:29:19.942611 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942550    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 23:29:19.942611 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942553    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 23:29:19.942611 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942557    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 23:29:19.942611 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942560    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 23:29:19.942611 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942563    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 23:29:19.942611 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942569    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 23:29:19.942611 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942572    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 23:29:19.942611 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942574    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 23:29:19.942611 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942577    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 23:29:19.943092 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942580    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 23:29:19.943092 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942582    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 23:29:19.943092 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942585    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 23:29:19.943092 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942587    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 23:29:19.943092 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942590    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 23:29:19.943092 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942593    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 23:29:19.943092 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942595    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 23:29:19.943092 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942598    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 23:29:19.943092 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942600    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 23:29:19.943092 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942603    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 23:29:19.943092 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942605    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 23:29:19.943092 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942609    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 23:29:19.943092 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942612    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 23:29:19.943092 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942615    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 23:29:19.943092 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942618    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 23:29:19.943092 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942621    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 23:29:19.943092 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942624    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 23:29:19.943092 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942627    2558 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 23:29:19.943092 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942631    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 23:29:19.943092 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942634    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 23:29:19.943602 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942637    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 23:29:19.943602 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942640    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 23:29:19.943602 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942642    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 23:29:19.943602 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942645    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 23:29:19.943602 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942648    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 23:29:19.943602 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942650    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 23:29:19.943602 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942653    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 23:29:19.943602 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942655    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 23:29:19.943602 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942658    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 23:29:19.943602 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942661    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 23:29:19.943602 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942665    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 23:29:19.943602 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942669    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 23:29:19.943602 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942673    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 23:29:19.943602 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942675    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 23:29:19.943602 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942678    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 23:29:19.943602 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942681    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 23:29:19.943602 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942684    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 23:29:19.943602 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942686    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 23:29:19.943602 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942689    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 23:29:19.944086 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942691    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 23:29:19.944086 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942694    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 23:29:19.944086 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942697    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 23:29:19.944086 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942699    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 23:29:19.944086 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942702    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 23:29:19.944086 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942705    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 23:29:19.944086 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942708    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 23:29:19.944086 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942711    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 23:29:19.944086 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942713    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 23:29:19.944086 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942717    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 23:29:19.944086 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942720    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 23:29:19.944086 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942722    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 23:29:19.944086 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942725    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 23:29:19.944086 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942728    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 23:29:19.944086 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942732    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 23:29:19.944086 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942734    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 23:29:19.944086 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942736    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 23:29:19.944086 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942739    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 23:29:19.944086 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942741    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 23:29:19.944086 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942744    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 23:29:19.944584 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942747    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 23:29:19.944584 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942749    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 23:29:19.944584 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942752    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 23:29:19.944584 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942755    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 23:29:19.944584 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942759    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 23:29:19.944584 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942763    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 23:29:19.944584 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942767    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 23:29:19.944584 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942770    2558 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 23:29:19.944584 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942772    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 23:29:19.944584 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942775    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 23:29:19.944584 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942779    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 23:29:19.944584 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942782    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 23:29:19.944584 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942785    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 23:29:19.944584 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942788    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 23:29:19.944584 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942791    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 23:29:19.944584 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942795    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 23:29:19.944584 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.942798    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 23:29:19.944584 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944260    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 23:29:19.944584 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944267    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 23:29:19.945051 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944269    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 23:29:19.945051 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944273    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 23:29:19.945051 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944275    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 23:29:19.945051 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944279    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 23:29:19.945051 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944281    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 23:29:19.945051 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944284    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 23:29:19.945051 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944287    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 23:29:19.945051 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944290    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 23:29:19.945051 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944293    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 23:29:19.945051 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944296    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 23:29:19.945051 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944298    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 23:29:19.945051 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944301    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 23:29:19.945051 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944303    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 23:29:19.945051 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944306    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 23:29:19.945051 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944308    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 23:29:19.945051 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944311    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 23:29:19.945051 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944314    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 23:29:19.945051 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944316    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 23:29:19.945051 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944319    2558 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 23:29:19.945051 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944321    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 23:29:19.945624 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944325    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 23:29:19.945624 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944327    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 23:29:19.945624 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944330    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 23:29:19.945624 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944332    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 23:29:19.945624 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944335    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 23:29:19.945624 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944338    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 23:29:19.945624 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944341    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 23:29:19.945624 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944345    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 23:29:19.945624 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944349    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 23:29:19.945624 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944352    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 23:29:19.945624 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944355    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 23:29:19.945624 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944358    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 23:29:19.945624 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944361    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 23:29:19.945624 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944363    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 23:29:19.945624 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944366    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 23:29:19.945624 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944368    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 23:29:19.945624 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944372    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 23:29:19.945624 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944374    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 23:29:19.945624 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944377    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 23:29:19.946092 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944379    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 23:29:19.946092 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944383    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 23:29:19.946092 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944385    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 23:29:19.946092 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944387    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 23:29:19.946092 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944390    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 23:29:19.946092 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944392    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 23:29:19.946092 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944395    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 23:29:19.946092 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944398    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 23:29:19.946092 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944400    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 23:29:19.946092 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944403    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 23:29:19.946092 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944405    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 23:29:19.946092 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944408    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 23:29:19.946092 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944410    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 23:29:19.946092 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944413    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 23:29:19.946092 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944416    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 23:29:19.946092 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944418    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 23:29:19.946092 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944421    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 23:29:19.946092 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944424    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 23:29:19.946092 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944427    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 23:29:19.946092 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944429    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 23:29:19.946600 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944432    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 23:29:19.946600 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944435    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 23:29:19.946600 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944438    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 23:29:19.946600 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944440    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 23:29:19.946600 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944442    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 23:29:19.946600 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944445    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 23:29:19.946600 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944449    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 23:29:19.946600 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944451    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 23:29:19.946600 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944454    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 23:29:19.946600 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944456    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 23:29:19.946600 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944459    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 23:29:19.946600 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944461    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 23:29:19.946600 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944464    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 23:29:19.946600 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944466    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 23:29:19.946600 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944469    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 23:29:19.946600 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944472    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 23:29:19.946600 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944474    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 23:29:19.946600 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944477    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 23:29:19.946600 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944479    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 23:29:19.946600 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944482    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 23:29:19.947103 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944484    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 23:29:19.947103 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944487    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 23:29:19.947103 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944491    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 23:29:19.947103 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944494    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 23:29:19.947103 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944497    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 23:29:19.947103 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944499    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 23:29:19.947103 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944502    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 23:29:19.947103 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944504    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 23:29:19.947103 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944507    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 23:29:19.947103 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944510    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 23:29:19.947103 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944514    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 23:29:19.947103 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944518    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 23:29:19.947103 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944521    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 23:29:19.947103 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944523    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 23:29:19.947103 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944526    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 23:29:19.947103 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944542    2558 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 23:29:19.947103 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944545    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 23:29:19.947103 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944548    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 23:29:19.947103 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944949    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 23:29:19.947591 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944955    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 23:29:19.947591 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944958    2558 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 23:29:19.947591 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944962    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 23:29:19.947591 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944966    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 23:29:19.947591 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944969    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 23:29:19.947591 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944972    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 23:29:19.947591 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944975    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 23:29:19.947591 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944978    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 23:29:19.947591 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944980    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 23:29:19.947591 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944983    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 23:29:19.947591 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944985    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 23:29:19.947591 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944988    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 23:29:19.947591 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944991    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 23:29:19.947591 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944994    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 23:29:19.947591 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944996    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 23:29:19.947591 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.944999    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 23:29:19.947591 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945002    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 23:29:19.947591 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945004    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 23:29:19.947591 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945007    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 23:29:19.947591 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945009    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 23:29:19.948100 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945012    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 23:29:19.948100 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945014    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 23:29:19.948100 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945017    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 23:29:19.948100 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945021    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 23:29:19.948100 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945025    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 23:29:19.948100 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945028    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 23:29:19.948100 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945031    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 23:29:19.948100 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945033    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 23:29:19.948100 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945037    2558 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 23:29:19.948100 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945040    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 23:29:19.948100 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945043    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 23:29:19.948100 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945046    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 23:29:19.948100 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945048    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 23:29:19.948100 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945051    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 23:29:19.948100 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945054    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 23:29:19.948100 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945057    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 23:29:19.948100 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945060    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 23:29:19.948100 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945062    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 23:29:19.948100 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945066    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 23:29:19.948579 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945068    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 23:29:19.948579 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945071    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 23:29:19.948579 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945074    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 23:29:19.948579 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945077    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 23:29:19.948579 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945079    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 23:29:19.948579 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945081    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 23:29:19.948579 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945084    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 23:29:19.948579 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945086    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 23:29:19.948579 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945089    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 23:29:19.948579 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945091    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 23:29:19.948579 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945094    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 23:29:19.948579 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945097    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 23:29:19.948579 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945099    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 23:29:19.948579 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945104    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 23:29:19.948579 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945107    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 23:29:19.948579 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945110    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 23:29:19.948579 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945113    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 23:29:19.948579 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945116    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 23:29:19.948579 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945119    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 23:29:19.949036 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945122    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 23:29:19.949036 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945125    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 23:29:19.949036 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945128    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 23:29:19.949036 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945130    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 23:29:19.949036 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945133    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 23:29:19.949036 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945136    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 23:29:19.949036 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945138    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 23:29:19.949036 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945141    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 23:29:19.949036 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945144    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 23:29:19.949036 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945147    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 23:29:19.949036 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945150    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 23:29:19.949036 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945152    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 23:29:19.949036 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945155    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 23:29:19.949036 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945157    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 23:29:19.949036 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945160    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 23:29:19.949036 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945163    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 23:29:19.949036 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945166    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 23:29:19.949036 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945168    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 23:29:19.949036 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945171    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 23:29:19.949557 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945173    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 23:29:19.949557 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945175    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 23:29:19.949557 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945178    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 23:29:19.949557 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945180    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 23:29:19.949557 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945183    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 23:29:19.949557 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945185    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 23:29:19.949557 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945188    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 23:29:19.949557 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945191    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 23:29:19.949557 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945193    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 23:29:19.949557 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945196    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 23:29:19.949557 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945198    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 23:29:19.949557 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945201    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 23:29:19.949557 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945203    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 23:29:19.949557 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945206    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 23:29:19.949557 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945209    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 23:29:19.949557 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945211    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 23:29:19.949557 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945214    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 23:29:19.949557 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945216    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 23:29:19.949557 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945218    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 23:29:19.949557 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945221    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 23:29:19.950102 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945223    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 23:29:19.950102 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945297    2558 flags.go:64] FLAG: --address="0.0.0.0"
Jul 24 23:29:19.950102 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945308    2558 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 24 23:29:19.950102 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945315    2558 flags.go:64] FLAG: --anonymous-auth="true"
Jul 24 23:29:19.950102 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945320    2558 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 24 23:29:19.950102 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945325    2558 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 24 23:29:19.950102 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945329    2558 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 24 23:29:19.950102 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945333    2558 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 24 23:29:19.950102 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945338    2558 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 24 23:29:19.950102 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945342    2558 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 24 23:29:19.950102 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945346    2558 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 24 23:29:19.950102 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945349    2558 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 24 23:29:19.950102 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945352    2558 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 24 23:29:19.950102 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945355    2558 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 24 23:29:19.950102 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945361    2558 flags.go:64] FLAG: --cgroup-root=""
Jul 24 23:29:19.950102 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945364    2558 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 24 23:29:19.950102 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945368    2558 flags.go:64] FLAG: --client-ca-file=""
Jul 24 23:29:19.950102 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945371    2558 flags.go:64] FLAG: --cloud-provider="external"
Jul 24 23:29:19.950102 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945374    2558 flags.go:64] FLAG: --cluster-dns="[]"
Jul 24 23:29:19.950102 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945378    2558 flags.go:64] FLAG: --cluster-domain=""
Jul 24 23:29:19.950102 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945382    2558 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 24 23:29:19.950102 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945385    2558 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 24 23:29:19.950102 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945388    2558 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 24 23:29:19.950670 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945392    2558 flags.go:64] FLAG: --container-log-max-files="5"
Jul 24 23:29:19.950670 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945403    2558 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 24 23:29:19.950670 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945411    2558 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 24 23:29:19.950670 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945414    2558 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 24 23:29:19.950670 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945418    2558 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 24 23:29:19.950670 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945421    2558 flags.go:64] FLAG: --contention-profiling="false"
Jul 24 23:29:19.950670 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945424    2558 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 24 23:29:19.950670 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945427    2558 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 24 23:29:19.950670 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945430    2558 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 24 23:29:19.950670 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945433    2558 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 24 23:29:19.950670 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945437    2558 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 24 23:29:19.950670 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945440    2558 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 24 23:29:19.950670 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945443    2558 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 24 23:29:19.950670 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945447    2558 flags.go:64] FLAG: --enable-load-reader="false"
Jul 24 23:29:19.950670 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945449    2558 flags.go:64] FLAG: --enable-server="true"
Jul 24 23:29:19.950670 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945452    2558 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 24 23:29:19.950670 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945458    2558 flags.go:64] FLAG: --event-burst="100"
Jul 24 23:29:19.950670 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945464    2558 flags.go:64] FLAG: --event-qps="50"
Jul 24 23:29:19.950670 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945467    2558 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 24 23:29:19.950670 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945471    2558 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 24 23:29:19.950670 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945474    2558 flags.go:64] FLAG: --eviction-hard=""
Jul 24 23:29:19.950670 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945479    2558 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 24 23:29:19.950670 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945481    2558 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 24 23:29:19.950670 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945487    2558 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 24 23:29:19.950670 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945490    2558 flags.go:64] FLAG: --eviction-soft=""
Jul 24 23:29:19.951342 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945493    2558 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 24 23:29:19.951342 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945495    2558 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 24 23:29:19.951342 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945498    2558 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 24 23:29:19.951342 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945502    2558 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 24 23:29:19.951342 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945504    2558 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 24 23:29:19.951342 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945507    2558 flags.go:64] FLAG: --fail-swap-on="true"
Jul 24 23:29:19.951342 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945510    2558 flags.go:64] FLAG: --feature-gates=""
Jul 24 23:29:19.951342 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945514    2558 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 24 23:29:19.951342 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945517    2558 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 24 23:29:19.951342 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945520    2558 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 24 23:29:19.951342 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945523    2558 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 24 23:29:19.951342 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945541    2558 flags.go:64] FLAG: --healthz-port="10248"
Jul 24 23:29:19.951342 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945545    2558 flags.go:64] FLAG: --help="false"
Jul 24 23:29:19.951342 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945548    2558 flags.go:64] FLAG: --hostname-override="ip-10-0-142-133.ec2.internal"
Jul 24 23:29:19.951342 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945552    2558 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 24 23:29:19.951342 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945561    2558 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 24 23:29:19.951342 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945564    2558 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 24 23:29:19.951342 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945568    2558 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 24 23:29:19.951342 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945571    2558 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 24 23:29:19.951342 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945574    2558 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 24 23:29:19.951342 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945577    2558 flags.go:64] FLAG: --image-service-endpoint=""
Jul 24 23:29:19.951342 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945580    2558 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 24 23:29:19.951342 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945583    2558 flags.go:64] FLAG: --kube-api-burst="100"
Jul 24 23:29:19.951952 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945586    2558 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 24 23:29:19.951952 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945591    2558 flags.go:64] FLAG: --kube-api-qps="50"
Jul 24 23:29:19.951952 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945594    2558 flags.go:64] FLAG: --kube-reserved=""
Jul 24 23:29:19.951952 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945597    2558 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 24 23:29:19.951952 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945601    2558 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 24 23:29:19.951952 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945604    2558 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 24 23:29:19.951952 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945607    2558 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 24 23:29:19.951952 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945611    2558 flags.go:64] FLAG: --lock-file=""
Jul 24 23:29:19.951952 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945614    2558 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 24 23:29:19.951952 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945617    2558 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 24 23:29:19.951952 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945620    2558 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 24 23:29:19.951952 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945625    2558 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 24 23:29:19.951952 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945628    2558 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 24 23:29:19.951952 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945632    2558 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 24 23:29:19.951952 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945635    2558 flags.go:64] FLAG: --logging-format="text"
Jul 24 23:29:19.951952 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945637    2558 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 24 23:29:19.951952 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945641    2558 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 24 23:29:19.951952 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945644    2558 flags.go:64] FLAG: --manifest-url=""
Jul 24 23:29:19.951952 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945646    2558 flags.go:64] FLAG: --manifest-url-header=""
Jul 24 23:29:19.951952 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945651    2558 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 24 23:29:19.951952 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945654    2558 flags.go:64] FLAG: --max-open-files="1000000"
Jul 24 23:29:19.951952 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945658    2558 flags.go:64] FLAG: --max-pods="110"
Jul 24 23:29:19.951952 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945661    2558 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 24 23:29:19.951952 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945664    2558 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 24 23:29:19.951952 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945667    2558 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 24 23:29:19.952608 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945670    2558 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 24 23:29:19.952608 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945673    2558 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 24 23:29:19.952608 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945676    2558 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 24 23:29:19.952608 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945679    2558 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 24 23:29:19.952608 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945687    2558 flags.go:64] FLAG: --node-status-max-images="50"
Jul 24 23:29:19.952608 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945690    2558 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 24 23:29:19.952608 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945693    2558 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 24 23:29:19.952608 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945696    2558 flags.go:64] FLAG: --pod-cidr=""
Jul 24 23:29:19.952608 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945700    2558 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 24 23:29:19.952608 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945706    2558 flags.go:64] FLAG: --pod-manifest-path=""
Jul 24 23:29:19.952608 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945708    2558 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 24 23:29:19.952608 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945712    2558 flags.go:64] FLAG: --pods-per-core="0"
Jul 24 23:29:19.952608 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945715    2558 flags.go:64] FLAG: --port="10250"
Jul 24 23:29:19.952608 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945718    2558 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 24 23:29:19.952608 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945722    2558 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-008241b2b660d070c"
Jul 24 23:29:19.952608 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945725    2558 flags.go:64] FLAG: --qos-reserved=""
Jul 24 23:29:19.952608 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945728    2558 flags.go:64] FLAG: --read-only-port="10255"
Jul 24 23:29:19.952608 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945731    2558 flags.go:64] FLAG: --register-node="true"
Jul 24 23:29:19.952608 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945734    2558 flags.go:64] FLAG: --register-with-taints=""
Jul 24 23:29:19.952608 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945739    2558 flags.go:64] FLAG: --registry-burst="10"
Jul 24 23:29:19.952608 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945742    2558 flags.go:64] FLAG: --registry-qps="5"
Jul 24 23:29:19.952608 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945744    2558 flags.go:64] FLAG: --reserved-cpus=""
Jul 24 23:29:19.952608 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945747    2558 flags.go:64] FLAG: --reserved-memory=""
Jul 24 23:29:19.952608 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945751    2558 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 24 23:29:19.952608 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945754    2558 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 24 23:29:19.953224 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945758    2558 flags.go:64] FLAG: --rotate-certificates="false"
Jul 24 23:29:19.953224 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945760    2558 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 24 23:29:19.953224 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945763    2558 flags.go:64] FLAG: --runonce="false"
Jul 24 23:29:19.953224 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945766    2558 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 24 23:29:19.953224 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945770    2558 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 24 23:29:19.953224 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945773    2558 flags.go:64] FLAG: --seccomp-default="false"
Jul 24 23:29:19.953224 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945776    2558 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 24 23:29:19.953224 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945778    2558 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 24 23:29:19.953224 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945781    2558 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 24 23:29:19.953224 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945784    2558 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 24 23:29:19.953224 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945787    2558 flags.go:64] FLAG: --storage-driver-password="root"
Jul 24 23:29:19.953224 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945790    2558 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 24 23:29:19.953224 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945793    2558 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 24 23:29:19.953224 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945796    2558 flags.go:64] FLAG: --storage-driver-user="root"
Jul 24 23:29:19.953224 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945799    2558 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 24 23:29:19.953224 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945802    2558 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 24 23:29:19.953224 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945806    2558 flags.go:64] FLAG: --system-cgroups=""
Jul 24 23:29:19.953224 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945808    2558 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 24 23:29:19.953224 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945814    2558 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 24 23:29:19.953224 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945817    2558 flags.go:64] FLAG: --tls-cert-file=""
Jul 24 23:29:19.953224 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945820    2558 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 24 23:29:19.953224 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945825    2558 flags.go:64] FLAG: --tls-min-version=""
Jul 24 23:29:19.953224 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945828    2558 flags.go:64] FLAG: --tls-private-key-file=""
Jul 24 23:29:19.953224 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945831    2558 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 24 23:29:19.953224 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945834    2558 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 24 23:29:19.953940 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945837    2558 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 24 23:29:19.953940 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945840    2558 flags.go:64] FLAG: --v="2"
Jul 24 23:29:19.953940 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945843    2558 flags.go:64] FLAG: --version="false"
Jul 24 23:29:19.953940 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945847    2558 flags.go:64] FLAG: --vmodule=""
Jul 24 23:29:19.953940 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945851    2558 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 24 23:29:19.953940 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.945854    2558 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 24 23:29:19.953940 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945991    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 23:29:19.953940 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945996    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 23:29:19.953940 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.945999    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 23:29:19.953940 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946003    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 23:29:19.953940 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946007    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 23:29:19.953940 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946009    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 23:29:19.953940 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946012    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 23:29:19.953940 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946014    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 23:29:19.953940 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946017    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 23:29:19.953940 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946020    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 23:29:19.953940 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946023    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 23:29:19.953940 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946025    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 23:29:19.953940 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946028    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 23:29:19.953940 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946031    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 23:29:19.953940 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946033    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 23:29:19.954461 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946036    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 23:29:19.954461 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946038    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 23:29:19.954461 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946042    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 23:29:19.954461 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946045    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 23:29:19.954461 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946048    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 23:29:19.954461 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946050    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 23:29:19.954461 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946053    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 23:29:19.954461 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946058    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 23:29:19.954461 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946061    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 23:29:19.954461 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946064    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 23:29:19.954461 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946066    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 23:29:19.954461 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946069    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 23:29:19.954461 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946072    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 23:29:19.954461 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946074    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 23:29:19.954461 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946077    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 23:29:19.954461 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946080    2558 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 23:29:19.954461 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946082    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 23:29:19.954461 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946084    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 23:29:19.954461 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946087    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 23:29:19.954461 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946090    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 23:29:19.954996 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946093    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 23:29:19.954996 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946096    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 23:29:19.954996 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946098    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 23:29:19.954996 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946101    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 23:29:19.954996 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946104    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 23:29:19.954996 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946107    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 23:29:19.954996 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946109    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 23:29:19.954996 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946112    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 23:29:19.954996 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946114    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 23:29:19.954996 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946117    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 23:29:19.954996 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946119    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 23:29:19.954996 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946122    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 23:29:19.954996 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946125    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 23:29:19.954996 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946127    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 23:29:19.954996 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946130    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 23:29:19.954996 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946133    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 23:29:19.954996 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946135    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 23:29:19.954996 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946138    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 23:29:19.954996 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946141    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 23:29:19.954996 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946144    2558 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 23:29:19.955490 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946147    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 23:29:19.955490 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946150    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 23:29:19.955490 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946153    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 23:29:19.955490 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946155    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 23:29:19.955490 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946158    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 23:29:19.955490 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946160    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 23:29:19.955490 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946164    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 23:29:19.955490 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946168    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 23:29:19.955490 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946171    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 23:29:19.955490 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946174    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 23:29:19.955490 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946176    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 23:29:19.955490 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946179    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 23:29:19.955490 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946182    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 23:29:19.955490 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946184    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 23:29:19.955490 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946188    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 23:29:19.955490 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946191    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 23:29:19.955490 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946194    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 23:29:19.955490 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946197    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 23:29:19.955945 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946200    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 23:29:19.955945 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946203    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 23:29:19.955945 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946206    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 23:29:19.955945 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946208    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 23:29:19.955945 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946211    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 23:29:19.955945 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946213    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 23:29:19.955945 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946216    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 23:29:19.955945 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946218    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 23:29:19.955945 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946221    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 23:29:19.955945 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946223    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 23:29:19.955945 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946226    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 23:29:19.955945 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946229    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 23:29:19.955945 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946231    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 23:29:19.955945 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946235    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 23:29:19.955945 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946238    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 23:29:19.955945 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946241    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 23:29:19.955945 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946243    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 23:29:19.955945 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946246    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 23:29:19.955945 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946248    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 23:29:19.955945 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946251    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 23:29:19.956450 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946254    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 23:29:19.956450 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946256    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 23:29:19.956450 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946259    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 23:29:19.956450 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946261    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 23:29:19.956450 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946264    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 23:29:19.956450 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.946266    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 23:29:19.956450 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.946912    2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 24 23:29:19.956450 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.953974    2558 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 24 23:29:19.956450 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.953989    2558 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 24 23:29:19.956450 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.954009    2558 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 24 23:29:19.956450 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.954015    2558 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 24 23:29:19.956450 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954079    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 23:29:19.956450 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954084    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 23:29:19.956450 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954087    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 23:29:19.956450 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954090    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 23:29:19.956828 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954093    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 23:29:19.956828 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954096    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 23:29:19.956828 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954099    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 23:29:19.956828 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954102    2558 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 23:29:19.956828 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954104    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 23:29:19.956828 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954107    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 23:29:19.956828 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954110    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 23:29:19.956828 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954113    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 23:29:19.956828 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954115    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 23:29:19.956828 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954118    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 23:29:19.956828 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954121    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 23:29:19.956828 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954124    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 23:29:19.956828 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954126    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 23:29:19.956828 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954129    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 23:29:19.956828 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954132    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 23:29:19.956828 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954134    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 23:29:19.956828 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954137    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 23:29:19.956828 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954140    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 23:29:19.956828 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954142    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 23:29:19.956828 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954145    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 23:29:19.957415 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954147    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 23:29:19.957415 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954150    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 23:29:19.957415 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954153    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 23:29:19.957415 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954155    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 23:29:19.957415 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954158    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 23:29:19.957415 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954162    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 23:29:19.957415 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954164    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 23:29:19.957415 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954167    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 23:29:19.957415 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954170    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 23:29:19.957415 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954174    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 23:29:19.957415 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954177    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 23:29:19.957415 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954180    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 23:29:19.957415 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954183    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 23:29:19.957415 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954186    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 23:29:19.957415 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954189    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 23:29:19.957415 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954191    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 23:29:19.957415 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954194    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 23:29:19.957415 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954196    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 23:29:19.957415 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954199    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 23:29:19.957415 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954201    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 23:29:19.957942 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954204    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 23:29:19.957942 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954206    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 23:29:19.957942 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954209    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 23:29:19.957942 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954212    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 23:29:19.957942 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954215    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 23:29:19.957942 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954217    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 23:29:19.957942 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954220    2558 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 23:29:19.957942 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954222    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 23:29:19.957942 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954225    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 23:29:19.957942 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954227    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 23:29:19.957942 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954230    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 23:29:19.957942 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954232    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 23:29:19.957942 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954234    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 23:29:19.957942 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954237    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 23:29:19.957942 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954239    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 23:29:19.957942 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954242    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 23:29:19.957942 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954244    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 23:29:19.957942 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954247    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 23:29:19.957942 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954250    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 23:29:19.957942 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954253    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 23:29:19.958481 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954256    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 23:29:19.958481 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954260    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 23:29:19.958481 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954263    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 23:29:19.958481 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954265    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 23:29:19.958481 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954268    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 23:29:19.958481 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954271    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 23:29:19.958481 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954273    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 23:29:19.958481 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954275    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 23:29:19.958481 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954278    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 23:29:19.958481 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954281    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 23:29:19.958481 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954283    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 23:29:19.958481 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954286    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 23:29:19.958481 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954288    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 23:29:19.958481 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954291    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 23:29:19.958481 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954293    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 23:29:19.958481 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954296    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 23:29:19.958481 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954299    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 23:29:19.958481 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954301    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 23:29:19.958481 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954304    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 23:29:19.958973 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954306    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 23:29:19.958973 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954310    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 23:29:19.958973 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954314    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 23:29:19.958973 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954316    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 23:29:19.958973 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954320    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 23:29:19.958973 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954323    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 23:29:19.958973 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954325    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 23:29:19.958973 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954328    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 23:29:19.958973 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954331    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 23:29:19.958973 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954333    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 23:29:19.958973 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954336    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 23:29:19.958973 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954339    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 23:29:19.958973 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954342    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 23:29:19.958973 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954345    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 23:29:19.958973 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954347    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 23:29:19.958973 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954350    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 23:29:19.959354 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.954354    2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 24 23:29:19.959354 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954482    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 23:29:19.959354 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954488    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 23:29:19.959354 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954492    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 23:29:19.959354 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954494    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 23:29:19.959354 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954498    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 23:29:19.959354 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954501    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 23:29:19.959354 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954503    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 23:29:19.959354 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954506    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 23:29:19.959354 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954509    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 23:29:19.959354 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954512    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 23:29:19.959354 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954515    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 23:29:19.959354 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954517    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 23:29:19.959354 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954520    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 23:29:19.959354 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954522    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 23:29:19.959821 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954525    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 23:29:19.959821 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954544    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 23:29:19.959821 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954550    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 23:29:19.959821 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954553    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 23:29:19.959821 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954556    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 23:29:19.959821 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954559    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 23:29:19.959821 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954562    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 23:29:19.959821 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954565    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 23:29:19.959821 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954568    2558 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 23:29:19.959821 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954571    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 23:29:19.959821 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954573    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 23:29:19.959821 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954576    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 23:29:19.959821 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954579    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 23:29:19.959821 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954581    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 23:29:19.959821 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954585    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 23:29:19.959821 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954588    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 23:29:19.959821 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954591    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 23:29:19.959821 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954593    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 23:29:19.959821 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954595    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 23:29:19.959821 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954598    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 23:29:19.960311 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954601    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 23:29:19.960311 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954603    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 23:29:19.960311 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954606    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 23:29:19.960311 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954610    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 23:29:19.960311 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954614    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 23:29:19.960311 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954616    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 23:29:19.960311 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954619    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 23:29:19.960311 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954622    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 23:29:19.960311 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954625    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 23:29:19.960311 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954628    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 23:29:19.960311 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954630    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 23:29:19.960311 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954633    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 23:29:19.960311 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954636    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 23:29:19.960311 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954638    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 23:29:19.960311 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954640    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 23:29:19.960311 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954643    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 23:29:19.960311 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954645    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 23:29:19.960311 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954648    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 23:29:19.960311 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954650    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 23:29:19.960311 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954653    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 23:29:19.960834 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954655    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 23:29:19.960834 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954658    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 23:29:19.960834 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954660    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 23:29:19.960834 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954663    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 23:29:19.960834 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954665    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 23:29:19.960834 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954668    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 23:29:19.960834 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954670    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 23:29:19.960834 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954673    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 23:29:19.960834 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954676    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 23:29:19.960834 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954678    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 23:29:19.960834 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954682    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 23:29:19.960834 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954685    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 23:29:19.960834 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954688    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 23:29:19.960834 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954690    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 23:29:19.960834 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954693    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 23:29:19.960834 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954695    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 23:29:19.960834 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954698    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 23:29:19.960834 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954700    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 23:29:19.960834 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954703    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 23:29:19.961504 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954705    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 23:29:19.961504 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954707    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 23:29:19.961504 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954710    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 23:29:19.961504 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954715    2558 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 23:29:19.961504 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954717    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 23:29:19.961504 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954720    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 23:29:19.961504 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954722    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 23:29:19.961504 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954725    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 23:29:19.961504 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954728    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 23:29:19.961504 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954730    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 23:29:19.961504 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954732    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 23:29:19.961504 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954735    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 23:29:19.961504 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954737    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 23:29:19.961504 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954740    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 23:29:19.961504 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954743    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 23:29:19.961504 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954745    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 23:29:19.961504 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954748    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 23:29:19.961504 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954750    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 23:29:19.961504 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954753    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 23:29:19.961504 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954755    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 23:29:19.962012 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954758    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 23:29:19.962012 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954761    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 23:29:19.962012 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954764    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 23:29:19.962012 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954767    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 23:29:19.962012 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954769    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 23:29:19.962012 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:19.954772    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 23:29:19.962012 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.954776    2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 24 23:29:19.962012 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.954900    2558 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 24 23:29:19.962012 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.957454    2558 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 24 23:29:19.962012 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.958943    2558 server.go:1019] "Starting client certificate rotation"
Jul 24 23:29:19.962012 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.959037    2558 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 24 23:29:19.962012 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.959068    2558 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 24 23:29:19.963586 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.963547    2558 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 24 23:29:19.964868 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.964852    2558 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 24 23:29:19.974416 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.974398    2558 log.go:25] "Validated CRI v1 runtime API"
Jul 24 23:29:19.977389 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.977374    2558 log.go:25] "Validated CRI v1 image API"
Jul 24 23:29:19.978338 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.978318    2558 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 24 23:29:19.982827 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.982805    2558 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 99350a1e-e99a-4677-bd58-5a3d031893b0:/dev/nvme0n1p3 e2842133-23dc-4058-b25d-501f40d8b61d:/dev/nvme0n1p4]
Jul 24 23:29:19.982921 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.982824    2558 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 24 23:29:19.989023 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.988911    2558 manager.go:216] Machine: {Timestamp:2026-07-24 23:29:19.986496906 +0000 UTC m=+0.372927625 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3104611 MemoryCapacity:33164484608 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2ccf6274c4aeb45f4ab5181bc60d24 SystemUUID:ec2ccf62-74c4-aeb4-5f4a-b5181bc60d24 BootID:06c42848-aab7-4bb4-9fb3-6bc87107e692 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:9d:2e:dd:b0:19 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:9d:2e:dd:b0:19 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:8a:00:be:d7:e7:ef Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164484608 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 24 23:29:19.989023 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.989020    2558 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 24 23:29:19.989152 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.989140    2558 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 24 23:29:19.992740 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.992708    2558 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 24 23:29:19.992878 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.992741    2558 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-133.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 24 23:29:19.993273 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.993257    2558 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 24 23:29:19.993555 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.993524    2558 topology_manager.go:138] "Creating topology manager with none policy"
Jul 24 23:29:19.993555 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.993554    2558 container_manager_linux.go:309] "Creating device plugin manager"
Jul 24 23:29:19.993677 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.993562    2558 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 24 23:29:19.994248 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.994237    2558 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 24 23:29:19.995253 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.995244    2558 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 24 23:29:19.996991 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.996980    2558 state_mem.go:36] "Initialized new in-memory state store"
Jul 24 23:29:19.997116 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.997107    2558 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 24 23:29:19.997151 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.997142    2558 kubelet.go:486] "Attempting to sync node with API server"
Jul 24 23:29:19.997184 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.997158    2558 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 24 23:29:19.997184 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.997170    2558 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 24 23:29:19.997184 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.997179    2558 kubelet.go:398] "Adding apiserver pod source"
Jul 24 23:29:19.997279 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.997187    2558 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 24 23:29:19.997625 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.997610    2558 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 24 23:29:19.997675 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:19.997642    2558 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 24 23:29:20.001366 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.001315    2558 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 24 23:29:20.003745 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.003728    2558 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 24 23:29:20.003814 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.003761    2558 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 24 23:29:20.005942 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.005929    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 24 23:29:20.005988 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.005953    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 24 23:29:20.005988 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.005964    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 24 23:29:20.005988 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.005971    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 24 23:29:20.005988 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.005979    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 24 23:29:20.005988 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.005985    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 24 23:29:20.006148 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.005993    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 24 23:29:20.006148 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.005999    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 24 23:29:20.006148 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.006006    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 24 23:29:20.006148 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.006012    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 24 23:29:20.006148 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.006020    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 24 23:29:20.006148 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.006025    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 24 23:29:20.006831 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.006821    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 24 23:29:20.006831 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.006830    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 24 23:29:20.007664 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:20.007643    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-133.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 24 23:29:20.007726 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:20.007672    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 24 23:29:20.010095 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.010082    2558 server.go:1268] "Started kubelet"
Jul 24 23:29:20.010448 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.010430    2558 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 24 23:29:20.010744 ip-10-0-142-133 systemd[1]: Started Kubernetes Kubelet.
Jul 24 23:29:20.012093 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.012080    2558 server.go:310] "Adding debug handlers to kubelet server"
Jul 24 23:29:20.012851 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.012814    2558 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 24 23:29:20.012896 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.012867    2558 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 24 23:29:20.013901 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.013866    2558 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-mwtxg"
Jul 24 23:29:20.015106 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.015010    2558 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 24 23:29:20.019483 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.019460    2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-133.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 23:29:20.020316 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:20.019231    2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-133.ec2.internal.18c55d9de24a967b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-133.ec2.internal,UID:ip-10-0-142-133.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-133.ec2.internal,},FirstTimestamp:2026-07-24 23:29:20.010057339 +0000 UTC m=+0.396488058,LastTimestamp:2026-07-24 23:29:20.010057339 +0000 UTC m=+0.396488058,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-133.ec2.internal,}"
Jul 24 23:29:20.020613 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.020567    2558 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-mwtxg"
Jul 24 23:29:20.021176 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:20.021156    2558 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 24 23:29:20.023264 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.023246    2558 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 24 23:29:20.023264 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.023254    2558 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 24 23:29:20.023514 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:20.023488    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-133.ec2.internal\" not found"
Jul 24 23:29:20.024243 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.024026    2558 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 24 23:29:20.024243 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.024072    2558 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 24 23:29:20.024243 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.024031    2558 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 24 23:29:20.025124 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.025102    2558 reconstruct.go:97] "Volume reconstruction finished"
Jul 24 23:29:20.025252 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.025132    2558 reconciler.go:29] "Reconciler: start to sync state"
Jul 24 23:29:20.025545 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.025514    2558 factory.go:55] Registering systemd factory
Jul 24 23:29:20.025643 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.025549    2558 factory.go:223] Registration of the systemd container factory successfully
Jul 24 23:29:20.025915 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.025896    2558 factory.go:153] Registering CRI-O factory
Jul 24 23:29:20.025915 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:20.025904    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 24 23:29:20.025915 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.025918    2558 factory.go:223] Registration of the crio container factory successfully
Jul 24 23:29:20.025915 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:20.025890    2558 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-133.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 24 23:29:20.026111 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.025965    2558 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 24 23:29:20.026111 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.025989    2558 factory.go:103] Registering Raw factory
Jul 24 23:29:20.026111 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.026004    2558 manager.go:1196] Started watching for new ooms in manager
Jul 24 23:29:20.026501 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.026487    2558 manager.go:349] Starting recovery of all containers
Jul 24 23:29:20.040267 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.040166    2558 manager.go:354] Recovery completed
Jul 24 23:29:20.044542 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.044518    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 23:29:20.048378 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.048362    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-133.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 23:29:20.048457 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.048389    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-133.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 23:29:20.048457 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.048400    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-133.ec2.internal" event="NodeHasSufficientPID"
Jul 24 23:29:20.048956 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.048941    2558 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 24 23:29:20.048956 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.048955    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 23:29:20.049078 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.048955    2558 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 24 23:29:20.049078 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.049018    2558 state_mem.go:36] "Initialized new in-memory state store"
Jul 24 23:29:20.049685 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.049671    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-133.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 23:29:20.049744 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.049694    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-133.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 23:29:20.049744 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.049704    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-133.ec2.internal" event="NodeHasSufficientPID"
Jul 24 23:29:20.050889 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.050877    2558 policy_none.go:49] "None policy: Start"
Jul 24 23:29:20.050923 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.050893    2558 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 24 23:29:20.050923 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.050904    2558 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 24 23:29:20.123794 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:20.123763    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-133.ec2.internal\" not found"
Jul 24 23:29:20.186762 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.186038    2558 policy_none.go:47] "Start"
Jul 24 23:29:20.196745 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.196719    2558 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 24 23:29:20.197995 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.197975    2558 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 24 23:29:20.198102 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.198023    2558 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 24 23:29:20.198102 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.198050    2558 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 24 23:29:20.198102 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:20.198086    2558 kubelet.go:2472] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Jul 24 23:29:20.200332 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.200300    2558 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 23:29:20.218077 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.218057    2558 manager.go:337] "Starting Device Plugin manager"
Jul 24 23:29:20.218154 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:20.218089    2558 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 24 23:29:20.218154 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.218099    2558 server.go:87] "Starting device plugin registration server"
Jul 24 23:29:20.218340 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.218328    2558 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 24 23:29:20.218406 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.218341    2558 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 24 23:29:20.218458 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.218436    2558 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 24 23:29:20.218521 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.218509    2558 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 24 23:29:20.218598 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.218521    2558 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 24 23:29:20.218978 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:20.218962    2558 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 24 23:29:20.219055 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:20.219006    2558 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-133.ec2.internal\" not found"
Jul 24 23:29:20.230902 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:20.230884    2558 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-142-133.ec2.internal\" not found" node="ip-10-0-142-133.ec2.internal"
Jul 24 23:29:20.299050 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.299022    2558 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-133.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-133.ec2.internal"]
Jul 24 23:29:20.299190 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.299150    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 23:29:20.300140 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.300122    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-133.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 23:29:20.300223 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.300157    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-133.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 23:29:20.300223 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.300172    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-133.ec2.internal" event="NodeHasSufficientPID"
Jul 24 23:29:20.301439 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.301427    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 23:29:20.301544 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.301520    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-133.ec2.internal"
Jul 24 23:29:20.301602 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.301562    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 23:29:20.302188 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.302172    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-133.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 23:29:20.302258 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.302206    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-133.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 23:29:20.302258 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.302218    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-133.ec2.internal" event="NodeHasSufficientPID"
Jul 24 23:29:20.302326 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.302174    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-133.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 23:29:20.302326 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.302283    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-133.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 23:29:20.302326 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.302295    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-133.ec2.internal" event="NodeHasSufficientPID"
Jul 24 23:29:20.303309 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.303293    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-133.ec2.internal"
Jul 24 23:29:20.303389 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.303317    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 23:29:20.304183 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.304169    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-133.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 23:29:20.304266 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.304194    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-133.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 23:29:20.304266 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.304207    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-133.ec2.internal" event="NodeHasSufficientPID"
Jul 24 23:29:20.318849 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.318831    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 23:29:20.320432 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.320417    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-133.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 23:29:20.320505 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.320443    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-133.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 23:29:20.320505 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.320453    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-133.ec2.internal" event="NodeHasSufficientPID"
Jul 24 23:29:20.320505 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.320474    2558 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-133.ec2.internal"
Jul 24 23:29:20.326382 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.326363    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b8b5e88d55d068e81c55bbf522837bfe-config\") pod \"kube-apiserver-proxy-ip-10-0-142-133.ec2.internal\" (UID: \"b8b5e88d55d068e81c55bbf522837bfe\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-133.ec2.internal"
Jul 24 23:29:20.326470 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.326395    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4e4ac0d94c86ae9a285d0e7940c49e40-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-133.ec2.internal\" (UID: \"4e4ac0d94c86ae9a285d0e7940c49e40\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-133.ec2.internal"
Jul 24 23:29:20.326470 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.326427    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4e4ac0d94c86ae9a285d0e7940c49e40-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-133.ec2.internal\" (UID: \"4e4ac0d94c86ae9a285d0e7940c49e40\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-133.ec2.internal"
Jul 24 23:29:20.328069 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.328053    2558 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-133.ec2.internal"
Jul 24 23:29:20.328127 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:20.328074    2558 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-133.ec2.internal\": node \"ip-10-0-142-133.ec2.internal\" not found"
Jul 24 23:29:20.328413 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:20.328399    2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-133.ec2.internal\" not found" node="ip-10-0-142-133.ec2.internal"
Jul 24 23:29:20.333077 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:20.333063    2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-133.ec2.internal\" not found" node="ip-10-0-142-133.ec2.internal"
Jul 24 23:29:20.340372 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:20.340355    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-133.ec2.internal\" not found"
Jul 24 23:29:20.427523 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.427489    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b8b5e88d55d068e81c55bbf522837bfe-config\") pod \"kube-apiserver-proxy-ip-10-0-142-133.ec2.internal\" (UID: \"b8b5e88d55d068e81c55bbf522837bfe\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-133.ec2.internal"
Jul 24 23:29:20.427523 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.427523    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4e4ac0d94c86ae9a285d0e7940c49e40-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-133.ec2.internal\" (UID: \"4e4ac0d94c86ae9a285d0e7940c49e40\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-133.ec2.internal"
Jul 24 23:29:20.427725 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.427561    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4e4ac0d94c86ae9a285d0e7940c49e40-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-133.ec2.internal\" (UID: \"4e4ac0d94c86ae9a285d0e7940c49e40\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-133.ec2.internal"
Jul 24 23:29:20.427725 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.427593    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b8b5e88d55d068e81c55bbf522837bfe-config\") pod \"kube-apiserver-proxy-ip-10-0-142-133.ec2.internal\" (UID: \"b8b5e88d55d068e81c55bbf522837bfe\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-133.ec2.internal"
Jul 24 23:29:20.427725 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.427609    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4e4ac0d94c86ae9a285d0e7940c49e40-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-133.ec2.internal\" (UID: \"4e4ac0d94c86ae9a285d0e7940c49e40\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-133.ec2.internal"
Jul 24 23:29:20.427725 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.427610    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4e4ac0d94c86ae9a285d0e7940c49e40-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-133.ec2.internal\" (UID: \"4e4ac0d94c86ae9a285d0e7940c49e40\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-133.ec2.internal"
Jul 24 23:29:20.440634 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:20.440577    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-133.ec2.internal\" not found"
Jul 24 23:29:20.541460 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:20.541410    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-133.ec2.internal\" not found"
Jul 24 23:29:20.631285 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.631244    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-133.ec2.internal"
Jul 24 23:29:20.636342 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.636324    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-133.ec2.internal"
Jul 24 23:29:20.641703 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:20.641684    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-133.ec2.internal\" not found"
Jul 24 23:29:20.742180 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:20.742081    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-133.ec2.internal\" not found"
Jul 24 23:29:20.842526 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:20.842487    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-133.ec2.internal\" not found"
Jul 24 23:29:20.943572 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:20.943510    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-133.ec2.internal\" not found"
Jul 24 23:29:20.958806 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.958780    2558 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 24 23:29:20.958978 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:20.958951    2558 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 24 23:29:21.027777 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:21.027677    2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-23 23:24:20 +0000 UTC" deadline="2028-03-02 03:14:30.721035993 +0000 UTC"
Jul 24 23:29:21.027777 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:21.027719    2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14067h45m9.69332065s"
Jul 24 23:29:21.044436 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:21.044406    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-133.ec2.internal\" not found"
Jul 24 23:29:21.145478 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:21.145448    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-133.ec2.internal\" not found"
Jul 24 23:29:21.232693 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:21.232660    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8b5e88d55d068e81c55bbf522837bfe.slice/crio-a573a5980e6a33ccc41d595650ffe5beabc32f0d71e6561f63bd8f4e90809e8b WatchSource:0}: Error finding container a573a5980e6a33ccc41d595650ffe5beabc32f0d71e6561f63bd8f4e90809e8b: Status 404 returned error can't find the container with id a573a5980e6a33ccc41d595650ffe5beabc32f0d71e6561f63bd8f4e90809e8b
Jul 24 23:29:21.233117 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:21.233090    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e4ac0d94c86ae9a285d0e7940c49e40.slice/crio-961fca4919e3291c44ed9de67351d67aa2e5dcceb988a5c2cd4687cab30361aa WatchSource:0}: Error finding container 961fca4919e3291c44ed9de67351d67aa2e5dcceb988a5c2cd4687cab30361aa: Status 404 returned error can't find the container with id 961fca4919e3291c44ed9de67351d67aa2e5dcceb988a5c2cd4687cab30361aa
Jul 24 23:29:21.238919 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:21.238900    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 23:29:21.246073 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:21.246050    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-133.ec2.internal\" not found"
Jul 24 23:29:21.346918 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:21.346804    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-133.ec2.internal\" not found"
Jul 24 23:29:21.355464 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:21.355438    2558 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 23:29:21.397157 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:21.397127    2558 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 23:29:21.448061 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:21.448030    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-133.ec2.internal\" not found"
Jul 24 23:29:21.548593 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:21.548548    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-133.ec2.internal\" not found"
Jul 24 23:29:21.567054 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:21.567025    2558 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 23:29:21.624084 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:21.623986    2558 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-133.ec2.internal"
Jul 24 23:29:21.634970 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:21.634942    2558 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 24 23:29:21.636116 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:21.636100    2558 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-133.ec2.internal"
Jul 24 23:29:21.646146 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:21.646123    2558 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 24 23:29:21.998586 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:21.998486    2558 apiserver.go:52] "Watching apiserver"
Jul 24 23:29:22.005519 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.005485    2558 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 24 23:29:22.006000 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.005974    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8llw9","openshift-dns/node-resolver-7f7qw","openshift-image-registry/node-ca-x8chk","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-133.ec2.internal","openshift-multus/multus-additional-cni-plugins-pck26","openshift-multus/multus-rf4fp","openshift-network-diagnostics/network-check-target-gltb5","kube-system/global-pull-secret-syncer-rplqt","kube-system/konnectivity-agent-hjb7w","openshift-cluster-node-tuning-operator/tuned-mhq9b","openshift-multus/network-metrics-daemon-54fxx","openshift-network-operator/iptables-alerter-kbtxb","openshift-ovn-kubernetes/ovnkube-node-lxdc5","kube-system/kube-apiserver-proxy-ip-10-0-142-133.ec2.internal"]
Jul 24 23:29:22.007636 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.007612    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mhq9b"
Jul 24 23:29:22.009930 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.009228    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5"
Jul 24 23:29:22.010593 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.010266    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 24 23:29:22.010593 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.010301    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-96jlv\""
Jul 24 23:29:22.010593 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.010539    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 24 23:29:22.010881 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.010871    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-x8chk"
Jul 24 23:29:22.012831 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.011688    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-x89zs\""
Jul 24 23:29:22.012831 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.011890    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 24 23:29:22.012831 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.012290    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 24 23:29:22.012831 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.012505    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 24 23:29:22.012831 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.012619    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 24 23:29:22.012831 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.012768    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 24 23:29:22.013141 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.013127    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rplqt"
Jul 24 23:29:22.013221 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:22.013193    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rplqt" podUID="68864bf3-c771-4894-9f14-52ee9e8052f9"
Jul 24 23:29:22.013284 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.013231    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 24 23:29:22.013407 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.013385    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-qdqgr\""
Jul 24 23:29:22.013602 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.013470    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 24 23:29:22.013602 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.013569    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 24 23:29:22.014080 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.013986    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 24 23:29:22.015089 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.014308    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-pck26"
Jul 24 23:29:22.015822 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.015802    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rf4fp"
Jul 24 23:29:22.016620 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.016531    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-c6sfz\""
Jul 24 23:29:22.017087 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.016863    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 24 23:29:22.017087 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.016886    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 24 23:29:22.017087 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.016950    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 24 23:29:22.017087 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.017054    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 24 23:29:22.017370 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.017235    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 24 23:29:22.017782 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.017504    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-hjb7w"
Jul 24 23:29:22.017782 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.017770    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 24 23:29:22.019082 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.019059    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-6cc4z\""
Jul 24 23:29:22.019403 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.019384    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gltb5"
Jul 24 23:29:22.019487 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:22.019458    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gltb5" podUID="7b7b9a28-3166-4af3-b27f-1ab5795c5f92"
Jul 24 23:29:22.019966 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.019822    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 24 23:29:22.020090 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.020032    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 24 23:29:22.020260 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.020222    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-xxzm7\""
Jul 24 23:29:22.020604 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.020582    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7f7qw"
Jul 24 23:29:22.022219 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.022195    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8llw9"
Jul 24 23:29:22.022504 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.022485    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 24 23:29:22.024090 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.024065    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 24 23:29:22.024478 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.024457    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-54fxx"
Jul 24 23:29:22.026347 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.024822    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-79lvf\""
Jul 24 23:29:22.026347 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:22.024913    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-54fxx" podUID="dcf99aef-00e1-4d16-94ca-12050dc95547"
Jul 24 23:29:22.026347 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.025233    2558 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 24 23:29:22.026347 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.025503    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 24 23:29:22.026347 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.025663    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 24 23:29:22.027585 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.026709    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-7nt64\""
Jul 24 23:29:22.028097 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.027762    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 24 23:29:22.031384 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.031363    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kbtxb"
Jul 24 23:29:22.035513 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.035483    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 24 23:29:22.035674 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.035512    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 24 23:29:22.035862 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.035844    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 24 23:29:22.036246 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.036213    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-49kj6\""
Jul 24 23:29:22.037909 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.037853    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5e81d551-3cb1-4707-998f-713e083fefe8-system-cni-dir\") pod \"multus-rf4fp\" (UID: \"5e81d551-3cb1-4707-998f-713e083fefe8\") " pod="openshift-multus/multus-rf4fp"
Jul 24 23:29:22.039982 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.039245    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5e81d551-3cb1-4707-998f-713e083fefe8-cni-binary-copy\") pod \"multus-rf4fp\" (UID: \"5e81d551-3cb1-4707-998f-713e083fefe8\") " pod="openshift-multus/multus-rf4fp"
Jul 24 23:29:22.039982 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.039283    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9dc9bac0-7bd8-4756-9a96-25594b1e3bd2-etc-sysctl-d\") pod \"tuned-mhq9b\" (UID: \"9dc9bac0-7bd8-4756-9a96-25594b1e3bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-mhq9b"
Jul 24 23:29:22.039982 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.039306    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/77f6315f-d3d1-47ea-a83d-2b036ac39189-host\") pod \"node-ca-x8chk\" (UID: \"77f6315f-d3d1-47ea-a83d-2b036ac39189\") " pod="openshift-image-registry/node-ca-x8chk"
Jul 24 23:29:22.039982 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.039332    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/725f7657-c52d-46d2-bf7d-bc6d045cb903-ovn-node-metrics-cert\") pod \"ovnkube-node-lxdc5\" (UID: \"725f7657-c52d-46d2-bf7d-bc6d045cb903\") " pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5"
Jul 24 23:29:22.039982 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.039359    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5e81d551-3cb1-4707-998f-713e083fefe8-host-var-lib-cni-multus\") pod \"multus-rf4fp\" (UID: \"5e81d551-3cb1-4707-998f-713e083fefe8\") " pod="openshift-multus/multus-rf4fp"
Jul 24 23:29:22.039982 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.039382    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5e81d551-3cb1-4707-998f-713e083fefe8-hostroot\") pod \"multus-rf4fp\" (UID: \"5e81d551-3cb1-4707-998f-713e083fefe8\") " pod="openshift-multus/multus-rf4fp"
Jul 24 23:29:22.039982 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.039404    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/725f7657-c52d-46d2-bf7d-bc6d045cb903-host-cni-bin\") pod \"ovnkube-node-lxdc5\" (UID: \"725f7657-c52d-46d2-bf7d-bc6d045cb903\") " pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5"
Jul 24 23:29:22.039982 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.039429    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/725f7657-c52d-46d2-bf7d-bc6d045cb903-host-slash\") pod \"ovnkube-node-lxdc5\" (UID: \"725f7657-c52d-46d2-bf7d-bc6d045cb903\") " pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5"
Jul 24 23:29:22.039982 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.039452    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3504a0bb-6580-4187-8f2d-7f71494675e4-tmp-dir\") pod \"node-resolver-7f7qw\" (UID: \"3504a0bb-6580-4187-8f2d-7f71494675e4\") " pod="openshift-dns/node-resolver-7f7qw"
Jul 24 23:29:22.039982 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.039479    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79c2s\" (UniqueName: \"kubernetes.io/projected/3504a0bb-6580-4187-8f2d-7f71494675e4-kube-api-access-79c2s\") pod \"node-resolver-7f7qw\" (UID: \"3504a0bb-6580-4187-8f2d-7f71494675e4\") " pod="openshift-dns/node-resolver-7f7qw"
Jul 24 23:29:22.039982 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.039504    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/334638da-1923-4fd5-88b4-8ca936adf513-etc-selinux\") pod \"aws-ebs-csi-driver-node-8llw9\" (UID: \"334638da-1923-4fd5-88b4-8ca936adf513\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8llw9"
Jul 24 23:29:22.039982 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.039530    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1d4690c2-2cef-4ede-8db1-fe6d22add6e2-agent-certs\") pod \"konnectivity-agent-hjb7w\" (UID: \"1d4690c2-2cef-4ede-8db1-fe6d22add6e2\") " pod="kube-system/konnectivity-agent-hjb7w"
Jul 24 23:29:22.039982 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.039555    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/725f7657-c52d-46d2-bf7d-bc6d045cb903-ovnkube-config\") pod \"ovnkube-node-lxdc5\" (UID: \"725f7657-c52d-46d2-bf7d-bc6d045cb903\") " pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5"
Jul 24 23:29:22.039982 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.039581    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnq9w\" (UniqueName: \"kubernetes.io/projected/03bb2978-3f29-4167-8b41-c257a2b3878d-kube-api-access-fnq9w\") pod \"multus-additional-cni-plugins-pck26\" (UID: \"03bb2978-3f29-4167-8b41-c257a2b3878d\") " pod="openshift-multus/multus-additional-cni-plugins-pck26"
Jul 24 23:29:22.039982 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.039625    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5e81d551-3cb1-4707-998f-713e083fefe8-multus-cni-dir\") pod \"multus-rf4fp\" (UID: \"5e81d551-3cb1-4707-998f-713e083fefe8\") " pod="openshift-multus/multus-rf4fp"
Jul 24 23:29:22.039982 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.039650    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5e81d551-3cb1-4707-998f-713e083fefe8-os-release\") pod \"multus-rf4fp\" (UID: \"5e81d551-3cb1-4707-998f-713e083fefe8\") " pod="openshift-multus/multus-rf4fp"
Jul 24 23:29:22.039982 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.039676    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5e81d551-3cb1-4707-998f-713e083fefe8-multus-daemon-config\") pod \"multus-rf4fp\" (UID: \"5e81d551-3cb1-4707-998f-713e083fefe8\") " pod="openshift-multus/multus-rf4fp"
Jul 24 23:29:22.040914 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.039702    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpcqp\" (UniqueName: \"kubernetes.io/projected/7b7b9a28-3166-4af3-b27f-1ab5795c5f92-kube-api-access-bpcqp\") pod \"network-check-target-gltb5\" (UID: \"7b7b9a28-3166-4af3-b27f-1ab5795c5f92\") " pod="openshift-network-diagnostics/network-check-target-gltb5"
Jul 24 23:29:22.040914 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.039726    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9dc9bac0-7bd8-4756-9a96-25594b1e3bd2-lib-modules\") pod \"tuned-mhq9b\" (UID: \"9dc9bac0-7bd8-4756-9a96-25594b1e3bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-mhq9b"
Jul 24 23:29:22.040914 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.039758    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/725f7657-c52d-46d2-bf7d-bc6d045cb903-host-run-netns\") pod \"ovnkube-node-lxdc5\" (UID: \"725f7657-c52d-46d2-bf7d-bc6d045cb903\") " pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5"
Jul 24 23:29:22.040914 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.039781    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkh5g\" (UniqueName: \"kubernetes.io/projected/5e81d551-3cb1-4707-998f-713e083fefe8-kube-api-access-xkh5g\") pod \"multus-rf4fp\" (UID: \"5e81d551-3cb1-4707-998f-713e083fefe8\") " pod="openshift-multus/multus-rf4fp"
Jul 24 23:29:22.040914 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.039804    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/334638da-1923-4fd5-88b4-8ca936adf513-device-dir\") pod \"aws-ebs-csi-driver-node-8llw9\" (UID: \"334638da-1923-4fd5-88b4-8ca936adf513\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8llw9"
Jul 24 23:29:22.040914 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.039828    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l42dl\" (UniqueName: \"kubernetes.io/projected/dcf99aef-00e1-4d16-94ca-12050dc95547-kube-api-access-l42dl\") pod \"network-metrics-daemon-54fxx\" (UID: \"dcf99aef-00e1-4d16-94ca-12050dc95547\") " pod="openshift-multus/network-metrics-daemon-54fxx"
Jul 24 23:29:22.040914 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.039850    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6mtw\" (UniqueName: \"kubernetes.io/projected/9dc9bac0-7bd8-4756-9a96-25594b1e3bd2-kube-api-access-c6mtw\") pod \"tuned-mhq9b\" (UID: \"9dc9bac0-7bd8-4756-9a96-25594b1e3bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-mhq9b"
Jul 24 23:29:22.040914 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.039874    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqg9m\" (UniqueName: \"kubernetes.io/projected/6ad632db-cb9d-460c-885f-0c4c1f495a1a-kube-api-access-xqg9m\") pod \"iptables-alerter-kbtxb\" (UID: \"6ad632db-cb9d-460c-885f-0c4c1f495a1a\") " pod="openshift-network-operator/iptables-alerter-kbtxb"
Jul 24 23:29:22.040914 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.039899    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/725f7657-c52d-46d2-bf7d-bc6d045cb903-node-log\") pod \"ovnkube-node-lxdc5\" (UID: \"725f7657-c52d-46d2-bf7d-bc6d045cb903\") " pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5"
Jul 24 23:29:22.040914 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.039948    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/77f6315f-d3d1-47ea-a83d-2b036ac39189-serviceca\") pod \"node-ca-x8chk\" (UID: \"77f6315f-d3d1-47ea-a83d-2b036ac39189\") " pod="openshift-image-registry/node-ca-x8chk"
Jul 24 23:29:22.040914 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.039982    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n88sg\" (UniqueName: \"kubernetes.io/projected/77f6315f-d3d1-47ea-a83d-2b036ac39189-kube-api-access-n88sg\") pod \"node-ca-x8chk\" (UID: \"77f6315f-d3d1-47ea-a83d-2b036ac39189\") " pod="openshift-image-registry/node-ca-x8chk"
Jul 24 23:29:22.040914 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.040019    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/68864bf3-c771-4894-9f14-52ee9e8052f9-original-pull-secret\") pod \"global-pull-secret-syncer-rplqt\" (UID: \"68864bf3-c771-4894-9f14-52ee9e8052f9\") " pod="kube-system/global-pull-secret-syncer-rplqt"
Jul 24 23:29:22.040914 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.040044    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/725f7657-c52d-46d2-bf7d-bc6d045cb903-etc-openvswitch\") pod \"ovnkube-node-lxdc5\" (UID: \"725f7657-c52d-46d2-bf7d-bc6d045cb903\") " pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5"
Jul 24 23:29:22.040914 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.040117    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/334638da-1923-4fd5-88b4-8ca936adf513-socket-dir\") pod \"aws-ebs-csi-driver-node-8llw9\" (UID: \"334638da-1923-4fd5-88b4-8ca936adf513\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8llw9"
Jul 24 23:29:22.040914 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.040146    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhfmb\" (UniqueName: \"kubernetes.io/projected/334638da-1923-4fd5-88b4-8ca936adf513-kube-api-access-fhfmb\") pod \"aws-ebs-csi-driver-node-8llw9\" (UID: \"334638da-1923-4fd5-88b4-8ca936adf513\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8llw9"
Jul 24 23:29:22.040914 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.040172    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dcf99aef-00e1-4d16-94ca-12050dc95547-metrics-certs\") pod \"network-metrics-daemon-54fxx\" (UID: \"dcf99aef-00e1-4d16-94ca-12050dc95547\") " pod="openshift-multus/network-metrics-daemon-54fxx"
Jul 24 23:29:22.041725 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.040195    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/68864bf3-c771-4894-9f14-52ee9e8052f9-dbus\") pod \"global-pull-secret-syncer-rplqt\" (UID: \"68864bf3-c771-4894-9f14-52ee9e8052f9\") " pod="kube-system/global-pull-secret-syncer-rplqt"
Jul 24 23:29:22.041725 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.040233    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/334638da-1923-4fd5-88b4-8ca936adf513-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8llw9\" (UID: \"334638da-1923-4fd5-88b4-8ca936adf513\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8llw9"
Jul 24 23:29:22.041725 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.040257    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/334638da-1923-4fd5-88b4-8ca936adf513-registration-dir\") pod \"aws-ebs-csi-driver-node-8llw9\" (UID: \"334638da-1923-4fd5-88b4-8ca936adf513\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8llw9"
Jul 24 23:29:22.041725 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.040280    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9dc9bac0-7bd8-4756-9a96-25594b1e3bd2-etc-sysconfig\") pod \"tuned-mhq9b\" (UID: \"9dc9bac0-7bd8-4756-9a96-25594b1e3bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-mhq9b"
Jul 24 23:29:22.041725 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.040303    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/725f7657-c52d-46d2-bf7d-bc6d045cb903-systemd-units\") pod \"ovnkube-node-lxdc5\" (UID: \"725f7657-c52d-46d2-bf7d-bc6d045cb903\") " pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5"
Jul 24 23:29:22.041725 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.040327    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/725f7657-c52d-46d2-bf7d-bc6d045cb903-host-run-ovn-kubernetes\") pod \"ovnkube-node-lxdc5\" (UID: \"725f7657-c52d-46d2-bf7d-bc6d045cb903\") " pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5"
Jul 24 23:29:22.041725 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.040352    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/03bb2978-3f29-4167-8b41-c257a2b3878d-system-cni-dir\") pod \"multus-additional-cni-plugins-pck26\" (UID: \"03bb2978-3f29-4167-8b41-c257a2b3878d\") " pod="openshift-multus/multus-additional-cni-plugins-pck26"
Jul 24 23:29:22.041725 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.040378    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5e81d551-3cb1-4707-998f-713e083fefe8-host-run-k8s-cni-cncf-io\") pod \"multus-rf4fp\" (UID: \"5e81d551-3cb1-4707-998f-713e083fefe8\") " pod="openshift-multus/multus-rf4fp"
Jul 24 23:29:22.041725 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.040420    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5e81d551-3cb1-4707-998f-713e083fefe8-host-var-lib-kubelet\") pod \"multus-rf4fp\" (UID: \"5e81d551-3cb1-4707-998f-713e083fefe8\") " pod="openshift-multus/multus-rf4fp"
Jul 24 23:29:22.041725 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.040447    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9dc9bac0-7bd8-4756-9a96-25594b1e3bd2-etc-sysctl-conf\") pod \"tuned-mhq9b\" (UID: \"9dc9bac0-7bd8-4756-9a96-25594b1e3bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-mhq9b"
Jul 24 23:29:22.041725 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.040471    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9dc9bac0-7bd8-4756-9a96-25594b1e3bd2-run\") pod \"tuned-mhq9b\" (UID: \"9dc9bac0-7bd8-4756-9a96-25594b1e3bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-mhq9b"
Jul 24 23:29:22.041725 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.040501    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9dc9bac0-7bd8-4756-9a96-25594b1e3bd2-var-lib-kubelet\") pod \"tuned-mhq9b\" (UID: \"9dc9bac0-7bd8-4756-9a96-25594b1e3bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-mhq9b"
Jul 24 23:29:22.041725 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.040524    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6ad632db-cb9d-460c-885f-0c4c1f495a1a-host-slash\") pod \"iptables-alerter-kbtxb\" (UID: \"6ad632db-cb9d-460c-885f-0c4c1f495a1a\") " pod="openshift-network-operator/iptables-alerter-kbtxb"
Jul 24 23:29:22.041725 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.040550    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/03bb2978-3f29-4167-8b41-c257a2b3878d-cni-binary-copy\") pod \"multus-additional-cni-plugins-pck26\" (UID: \"03bb2978-3f29-4167-8b41-c257a2b3878d\") " pod="openshift-multus/multus-additional-cni-plugins-pck26"
Jul 24 23:29:22.041725 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.040574    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5e81d551-3cb1-4707-998f-713e083fefe8-multus-conf-dir\") pod \"multus-rf4fp\" (UID: \"5e81d551-3cb1-4707-998f-713e083fefe8\") " pod="openshift-multus/multus-rf4fp"
Jul 24 23:29:22.041725 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.040615    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/334638da-1923-4fd5-88b4-8ca936adf513-sys-fs\") pod \"aws-ebs-csi-driver-node-8llw9\" (UID: \"334638da-1923-4fd5-88b4-8ca936adf513\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8llw9"
Jul 24 23:29:22.042653 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.040640    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9dc9bac0-7bd8-4756-9a96-25594b1e3bd2-host\") pod \"tuned-mhq9b\" (UID: \"9dc9bac0-7bd8-4756-9a96-25594b1e3bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-mhq9b"
Jul 24 23:29:22.042653 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.040663    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/725f7657-c52d-46d2-bf7d-bc6d045cb903-run-systemd\") pod \"ovnkube-node-lxdc5\" (UID: \"725f7657-c52d-46d2-bf7d-bc6d045cb903\") " pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5"
Jul 24 23:29:22.042653 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.040689    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/03bb2978-3f29-4167-8b41-c257a2b3878d-os-release\") pod \"multus-additional-cni-plugins-pck26\" (UID: \"03bb2978-3f29-4167-8b41-c257a2b3878d\") " pod="openshift-multus/multus-additional-cni-plugins-pck26"
Jul 24 23:29:22.042653 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.040714    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/03bb2978-3f29-4167-8b41-c257a2b3878d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pck26\" (UID: \"03bb2978-3f29-4167-8b41-c257a2b3878d\") " pod="openshift-multus/multus-additional-cni-plugins-pck26"
Jul 24 23:29:22.042653 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.040741    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5e81d551-3cb1-4707-998f-713e083fefe8-multus-socket-dir-parent\") pod \"multus-rf4fp\" (UID: \"5e81d551-3cb1-4707-998f-713e083fefe8\") " pod="openshift-multus/multus-rf4fp"
Jul 24 23:29:22.042653 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.040764    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/725f7657-c52d-46d2-bf7d-bc6d045cb903-run-openvswitch\") pod \"ovnkube-node-lxdc5\" (UID: \"725f7657-c52d-46d2-bf7d-bc6d045cb903\") " pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5"
Jul 24 23:29:22.042653 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.040790    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/725f7657-c52d-46d2-bf7d-bc6d045cb903-log-socket\") pod \"ovnkube-node-lxdc5\" (UID: \"725f7657-c52d-46d2-bf7d-bc6d045cb903\") " pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5"
Jul 24 23:29:22.042653 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.040899    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/725f7657-c52d-46d2-bf7d-bc6d045cb903-env-overrides\") pod \"ovnkube-node-lxdc5\" (UID: \"725f7657-c52d-46d2-bf7d-bc6d045cb903\") " pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5"
Jul 24 23:29:22.042653 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.041082    2558 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 24 23:29:22.043116 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.042681    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1d4690c2-2cef-4ede-8db1-fe6d22add6e2-konnectivity-ca\") pod \"konnectivity-agent-hjb7w\" (UID: \"1d4690c2-2cef-4ede-8db1-fe6d22add6e2\") " pod="kube-system/konnectivity-agent-hjb7w"
Jul 24 23:29:22.043116 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.042820    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/725f7657-c52d-46d2-bf7d-bc6d045cb903-run-ovn\") pod \"ovnkube-node-lxdc5\" (UID: \"725f7657-c52d-46d2-bf7d-bc6d045cb903\") " pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5"
Jul 24 23:29:22.043116 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.042847    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgds7\" (UniqueName: \"kubernetes.io/projected/725f7657-c52d-46d2-bf7d-bc6d045cb903-kube-api-access-dgds7\") pod \"ovnkube-node-lxdc5\" (UID: \"725f7657-c52d-46d2-bf7d-bc6d045cb903\") " pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5"
Jul 24 23:29:22.043116 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.042880    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/03bb2978-3f29-4167-8b41-c257a2b3878d-cnibin\") pod \"multus-additional-cni-plugins-pck26\" (UID: \"03bb2978-3f29-4167-8b41-c257a2b3878d\") " pod="openshift-multus/multus-additional-cni-plugins-pck26"
Jul 24 23:29:22.043116 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.042904    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5e81d551-3cb1-4707-998f-713e083fefe8-host-run-netns\") pod \"multus-rf4fp\" (UID: \"5e81d551-3cb1-4707-998f-713e083fefe8\") " pod="openshift-multus/multus-rf4fp"
Jul 24 23:29:22.043116 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.042925    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9dc9bac0-7bd8-4756-9a96-25594b1e3bd2-etc-kubernetes\") pod \"tuned-mhq9b\" (UID: \"9dc9bac0-7bd8-4756-9a96-25594b1e3bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-mhq9b"
Jul 24 23:29:22.043565 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.042950    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6ad632db-cb9d-460c-885f-0c4c1f495a1a-iptables-alerter-script\") pod \"iptables-alerter-kbtxb\" (UID: \"6ad632db-cb9d-460c-885f-0c4c1f495a1a\") " pod="openshift-network-operator/iptables-alerter-kbtxb"
Jul 24 23:29:22.043644 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.043625    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/725f7657-c52d-46d2-bf7d-bc6d045cb903-host-cni-netd\") pod \"ovnkube-node-lxdc5\" (UID: \"725f7657-c52d-46d2-bf7d-bc6d045cb903\") " pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5"
Jul 24 23:29:22.043689 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.043656    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5e81d551-3cb1-4707-998f-713e083fefe8-cnibin\") pod \"multus-rf4fp\" (UID: \"5e81d551-3cb1-4707-998f-713e083fefe8\") " pod="openshift-multus/multus-rf4fp"
Jul 24 23:29:22.043689 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.043682    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5e81d551-3cb1-4707-998f-713e083fefe8-etc-kubernetes\") pod \"multus-rf4fp\" (UID: \"5e81d551-3cb1-4707-998f-713e083fefe8\") " pod="openshift-multus/multus-rf4fp"
Jul 24 23:29:22.043796 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.043709    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9dc9bac0-7bd8-4756-9a96-25594b1e3bd2-etc-modprobe-d\") pod \"tuned-mhq9b\" (UID: \"9dc9bac0-7bd8-4756-9a96-25594b1e3bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-mhq9b"
Jul 24 23:29:22.043796 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.043731    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9dc9bac0-7bd8-4756-9a96-25594b1e3bd2-sys\") pod \"tuned-mhq9b\" (UID: \"9dc9bac0-7bd8-4756-9a96-25594b1e3bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-mhq9b"
Jul 24 23:29:22.043902 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.043758    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/725f7657-c52d-46d2-bf7d-bc6d045cb903-var-lib-openvswitch\") pod \"ovnkube-node-lxdc5\" (UID: \"725f7657-c52d-46d2-bf7d-bc6d045cb903\") " pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5"
Jul 24 23:29:22.043958 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.043905    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5e81d551-3cb1-4707-998f-713e083fefe8-host-run-multus-certs\") pod \"multus-rf4fp\" (UID: \"5e81d551-3cb1-4707-998f-713e083fefe8\") " pod="openshift-multus/multus-rf4fp"
Jul 24 23:29:22.044014 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.043993    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3504a0bb-6580-4187-8f2d-7f71494675e4-hosts-file\") pod \"node-resolver-7f7qw\" (UID: \"3504a0bb-6580-4187-8f2d-7f71494675e4\") " pod="openshift-dns/node-resolver-7f7qw"
Jul 24 23:29:22.044063 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.044053    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/03bb2978-3f29-4167-8b41-c257a2b3878d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pck26\" (UID: \"03bb2978-3f29-4167-8b41-c257a2b3878d\") " pod="openshift-multus/multus-additional-cni-plugins-pck26"
Jul 24 23:29:22.044120 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.044080    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5e81d551-3cb1-4707-998f-713e083fefe8-host-var-lib-cni-bin\") pod \"multus-rf4fp\" (UID: \"5e81d551-3cb1-4707-998f-713e083fefe8\") " pod="openshift-multus/multus-rf4fp"
Jul 24 23:29:22.044175 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.044125    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9dc9bac0-7bd8-4756-9a96-25594b1e3bd2-etc-systemd\") pod \"tuned-mhq9b\" (UID: \"9dc9bac0-7bd8-4756-9a96-25594b1e3bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-mhq9b"
Jul 24 23:29:22.044175 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.044149    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9dc9bac0-7bd8-4756-9a96-25594b1e3bd2-etc-tuned\") pod \"tuned-mhq9b\" (UID: \"9dc9bac0-7bd8-4756-9a96-25594b1e3bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-mhq9b"
Jul 24 23:29:22.044329 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.044310    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/725f7657-c52d-46d2-bf7d-bc6d045cb903-host-kubelet\") pod \"ovnkube-node-lxdc5\" (UID: \"725f7657-c52d-46d2-bf7d-bc6d045cb903\") " pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5"
Jul 24 23:29:22.044445 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.044420    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/725f7657-c52d-46d2-bf7d-bc6d045cb903-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lxdc5\" (UID: \"725f7657-c52d-46d2-bf7d-bc6d045cb903\") " pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5"
Jul 24 23:29:22.044491 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.044458    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9dc9bac0-7bd8-4756-9a96-25594b1e3bd2-tmp\") pod \"tuned-mhq9b\" (UID: \"9dc9bac0-7bd8-4756-9a96-25594b1e3bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-mhq9b"
Jul 24 23:29:22.044801 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.044776    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/68864bf3-c771-4894-9f14-52ee9e8052f9-kubelet-config\") pod \"global-pull-secret-syncer-rplqt\" (UID: \"68864bf3-c771-4894-9f14-52ee9e8052f9\") " pod="kube-system/global-pull-secret-syncer-rplqt"
Jul 24 23:29:22.044872 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.044821    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/725f7657-c52d-46d2-bf7d-bc6d045cb903-ovnkube-script-lib\") pod \"ovnkube-node-lxdc5\" (UID: \"725f7657-c52d-46d2-bf7d-bc6d045cb903\") " pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5"
Jul 24 23:29:22.044872 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.044849    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/03bb2978-3f29-4167-8b41-c257a2b3878d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pck26\" (UID: \"03bb2978-3f29-4167-8b41-c257a2b3878d\") " pod="openshift-multus/multus-additional-cni-plugins-pck26"
Jul 24 23:29:22.064669 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.064642    2558 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-jkdcj"
Jul 24 23:29:22.072696 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.072670    2558 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-jkdcj"
Jul 24 23:29:22.125132 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.125105    2558 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 24 23:29:22.145422 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.145379    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/334638da-1923-4fd5-88b4-8ca936adf513-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8llw9\" (UID: \"334638da-1923-4fd5-88b4-8ca936adf513\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8llw9"
Jul 24 23:29:22.145625 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.145424    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/334638da-1923-4fd5-88b4-8ca936adf513-registration-dir\") pod \"aws-ebs-csi-driver-node-8llw9\" (UID: \"334638da-1923-4fd5-88b4-8ca936adf513\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8llw9"
Jul 24 23:29:22.145625 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.145462    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9dc9bac0-7bd8-4756-9a96-25594b1e3bd2-etc-sysconfig\") pod \"tuned-mhq9b\" (UID: \"9dc9bac0-7bd8-4756-9a96-25594b1e3bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-mhq9b"
Jul 24 23:29:22.145625 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.145489    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/725f7657-c52d-46d2-bf7d-bc6d045cb903-systemd-units\") pod \"ovnkube-node-lxdc5\" (UID: \"725f7657-c52d-46d2-bf7d-bc6d045cb903\") " pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5"
Jul 24 23:29:22.145625 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.145504    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/334638da-1923-4fd5-88b4-8ca936adf513-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8llw9\" (UID: \"334638da-1923-4fd5-88b4-8ca936adf513\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8llw9"
Jul 24 23:29:22.145625 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.145512    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/725f7657-c52d-46d2-bf7d-bc6d045cb903-host-run-ovn-kubernetes\") pod \"ovnkube-node-lxdc5\" (UID: \"725f7657-c52d-46d2-bf7d-bc6d045cb903\") " pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5"
Jul 24 23:29:22.145625 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.145548    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/334638da-1923-4fd5-88b4-8ca936adf513-registration-dir\") pod \"aws-ebs-csi-driver-node-8llw9\" (UID: \"334638da-1923-4fd5-88b4-8ca936adf513\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8llw9"
Jul 24 23:29:22.145625 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.145570    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/03bb2978-3f29-4167-8b41-c257a2b3878d-system-cni-dir\") pod \"multus-additional-cni-plugins-pck26\" (UID: \"03bb2978-3f29-4167-8b41-c257a2b3878d\") " pod="openshift-multus/multus-additional-cni-plugins-pck26"
Jul 24 23:29:22.145625 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.145584    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/725f7657-c52d-46d2-bf7d-bc6d045cb903-systemd-units\") pod \"ovnkube-node-lxdc5\" (UID: \"725f7657-c52d-46d2-bf7d-bc6d045cb903\") " pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5"
Jul 24 23:29:22.145625 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.145600    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5e81d551-3cb1-4707-998f-713e083fefe8-host-run-k8s-cni-cncf-io\") pod \"multus-rf4fp\" (UID: \"5e81d551-3cb1-4707-998f-713e083fefe8\") " pod="openshift-multus/multus-rf4fp"
Jul 24 23:29:22.145625 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.145556    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9dc9bac0-7bd8-4756-9a96-25594b1e3bd2-etc-sysconfig\") pod \"tuned-mhq9b\" (UID: \"9dc9bac0-7bd8-4756-9a96-25594b1e3bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-mhq9b"
Jul 24 23:29:22.145625 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.145572    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/725f7657-c52d-46d2-bf7d-bc6d045cb903-host-run-ovn-kubernetes\") pod \"ovnkube-node-lxdc5\" (UID: \"725f7657-c52d-46d2-bf7d-bc6d045cb903\") " pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5"
Jul 24 23:29:22.145625 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.145628    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5e81d551-3cb1-4707-998f-713e083fefe8-host-var-lib-kubelet\") pod \"multus-rf4fp\" (UID: \"5e81d551-3cb1-4707-998f-713e083fefe8\") " pod="openshift-multus/multus-rf4fp"
Jul 24 23:29:22.146247 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.145661    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/03bb2978-3f29-4167-8b41-c257a2b3878d-system-cni-dir\") pod \"multus-additional-cni-plugins-pck26\" (UID: \"03bb2978-3f29-4167-8b41-c257a2b3878d\") " pod="openshift-multus/multus-additional-cni-plugins-pck26"
Jul 24 23:29:22.146247 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.145681    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5e81d551-3cb1-4707-998f-713e083fefe8-host-var-lib-kubelet\") pod \"multus-rf4fp\" (UID: \"5e81d551-3cb1-4707-998f-713e083fefe8\") " pod="openshift-multus/multus-rf4fp"
Jul 24 23:29:22.146247 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.145684    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5e81d551-3cb1-4707-998f-713e083fefe8-host-run-k8s-cni-cncf-io\") pod \"multus-rf4fp\" (UID: \"5e81d551-3cb1-4707-998f-713e083fefe8\") " pod="openshift-multus/multus-rf4fp"
Jul 24 23:29:22.146247 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.145651    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9dc9bac0-7bd8-4756-9a96-25594b1e3bd2-etc-sysctl-conf\") pod \"tuned-mhq9b\" (UID: \"9dc9bac0-7bd8-4756-9a96-25594b1e3bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-mhq9b"
Jul 24 23:29:22.146247 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.145742    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9dc9bac0-7bd8-4756-9a96-25594b1e3bd2-run\") pod \"tuned-mhq9b\" (UID: \"9dc9bac0-7bd8-4756-9a96-25594b1e3bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-mhq9b"
Jul 24 23:29:22.146247 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.145784    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9dc9bac0-7bd8-4756-9a96-25594b1e3bd2-var-lib-kubelet\") pod \"tuned-mhq9b\" (UID: \"9dc9bac0-7bd8-4756-9a96-25594b1e3bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-mhq9b"
Jul 24 23:29:22.146247 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.145805    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6ad632db-cb9d-460c-885f-0c4c1f495a1a-host-slash\") pod \"iptables-alerter-kbtxb\" (UID: \"6ad632db-cb9d-460c-885f-0c4c1f495a1a\") " pod="openshift-network-operator/iptables-alerter-kbtxb"
Jul 24 23:29:22.146247 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.145828    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/03bb2978-3f29-4167-8b41-c257a2b3878d-cni-binary-copy\") pod \"multus-additional-cni-plugins-pck26\" (UID: \"03bb2978-3f29-4167-8b41-c257a2b3878d\") " pod="openshift-multus/multus-additional-cni-plugins-pck26"
Jul 24 23:29:22.146247 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.145850    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5e81d551-3cb1-4707-998f-713e083fefe8-multus-conf-dir\") pod \"multus-rf4fp\" (UID: \"5e81d551-3cb1-4707-998f-713e083fefe8\") " pod="openshift-multus/multus-rf4fp"
Jul 24 23:29:22.146247 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.145874    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/334638da-1923-4fd5-88b4-8ca936adf513-sys-fs\") pod \"aws-ebs-csi-driver-node-8llw9\" (UID: \"334638da-1923-4fd5-88b4-8ca936adf513\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8llw9"
Jul 24 23:29:22.146247 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.145876    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9dc9bac0-7bd8-4756-9a96-25594b1e3bd2-var-lib-kubelet\") pod \"tuned-mhq9b\" (UID: \"9dc9bac0-7bd8-4756-9a96-25594b1e3bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-mhq9b"
Jul 24 23:29:22.146247 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.145885    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6ad632db-cb9d-460c-885f-0c4c1f495a1a-host-slash\") pod \"iptables-alerter-kbtxb\" (UID: \"6ad632db-cb9d-460c-885f-0c4c1f495a1a\") " pod="openshift-network-operator/iptables-alerter-kbtxb"
Jul 24 23:29:22.146247 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.145896    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9dc9bac0-7bd8-4756-9a96-25594b1e3bd2-etc-sysctl-conf\") pod \"tuned-mhq9b\" (UID: \"9dc9bac0-7bd8-4756-9a96-25594b1e3bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-mhq9b"
Jul 24 23:29:22.146247 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.145898    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9dc9bac0-7bd8-4756-9a96-25594b1e3bd2-host\") pod \"tuned-mhq9b\" (UID: \"9dc9bac0-7bd8-4756-9a96-25594b1e3bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-mhq9b"
Jul 24 23:29:22.146927 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.146745    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/334638da-1923-4fd5-88b4-8ca936adf513-sys-fs\") pod \"aws-ebs-csi-driver-node-8llw9\" (UID: \"334638da-1923-4fd5-88b4-8ca936adf513\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8llw9"
Jul 24 23:29:22.146927 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.146911    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5e81d551-3cb1-4707-998f-713e083fefe8-multus-conf-dir\") pod \"multus-rf4fp\" (UID: \"5e81d551-3cb1-4707-998f-713e083fefe8\") " pod="openshift-multus/multus-rf4fp"
Jul 24 23:29:22.147068 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.145942    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/725f7657-c52d-46d2-bf7d-bc6d045cb903-run-systemd\") pod \"ovnkube-node-lxdc5\" (UID: \"725f7657-c52d-46d2-bf7d-bc6d045cb903\") " pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5"
Jul 24 23:29:22.147068 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.146971    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/03bb2978-3f29-4167-8b41-c257a2b3878d-os-release\") pod \"multus-additional-cni-plugins-pck26\" (UID: \"03bb2978-3f29-4167-8b41-c257a2b3878d\") " pod="openshift-multus/multus-additional-cni-plugins-pck26"
Jul 24 23:29:22.147068 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.147020    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/03bb2978-3f29-4167-8b41-c257a2b3878d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pck26\" (UID: \"03bb2978-3f29-4167-8b41-c257a2b3878d\") " pod="openshift-multus/multus-additional-cni-plugins-pck26"
Jul 24 23:29:22.147230 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.147131    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5e81d551-3cb1-4707-998f-713e083fefe8-multus-socket-dir-parent\") pod \"multus-rf4fp\" (UID: \"5e81d551-3cb1-4707-998f-713e083fefe8\") " pod="openshift-multus/multus-rf4fp"
Jul 24 23:29:22.147230 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.147210    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/03bb2978-3f29-4167-8b41-c257a2b3878d-os-release\") pod \"multus-additional-cni-plugins-pck26\" (UID: \"03bb2978-3f29-4167-8b41-c257a2b3878d\") " pod="openshift-multus/multus-additional-cni-plugins-pck26"
Jul 24 23:29:22.148277 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.148244    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/03bb2978-3f29-4167-8b41-c257a2b3878d-cni-binary-copy\") pod \"multus-additional-cni-plugins-pck26\" (UID: \"03bb2978-3f29-4167-8b41-c257a2b3878d\") " pod="openshift-multus/multus-additional-cni-plugins-pck26"
Jul 24 23:29:22.148398 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.148316    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9dc9bac0-7bd8-4756-9a96-25594b1e3bd2-host\") pod \"tuned-mhq9b\" (UID: \"9dc9bac0-7bd8-4756-9a96-25594b1e3bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-mhq9b"
Jul 24 23:29:22.148398 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.148371    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9dc9bac0-7bd8-4756-9a96-25594b1e3bd2-run\") pod \"tuned-mhq9b\" (UID: \"9dc9bac0-7bd8-4756-9a96-25594b1e3bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-mhq9b"
Jul 24 23:29:22.148524 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.147054    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5e81d551-3cb1-4707-998f-713e083fefe8-multus-socket-dir-parent\") pod \"multus-rf4fp\" (UID: \"5e81d551-3cb1-4707-998f-713e083fefe8\") " pod="openshift-multus/multus-rf4fp"
Jul 24 23:29:22.148593 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.148553    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/725f7657-c52d-46d2-bf7d-bc6d045cb903-run-openvswitch\") pod \"ovnkube-node-lxdc5\" (UID: \"725f7657-c52d-46d2-bf7d-bc6d045cb903\") " pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5"
Jul 24 23:29:22.148593 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.148556    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/725f7657-c52d-46d2-bf7d-bc6d045cb903-run-systemd\") pod \"ovnkube-node-lxdc5\" (UID: \"725f7657-c52d-46d2-bf7d-bc6d045cb903\") " pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5"
Jul 24 23:29:22.148593 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.148588    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/725f7657-c52d-46d2-bf7d-bc6d045cb903-log-socket\") pod \"ovnkube-node-lxdc5\" (UID: \"725f7657-c52d-46d2-bf7d-bc6d045cb903\") " pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5"
Jul 24 23:29:22.148765 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.148621    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/725f7657-c52d-46d2-bf7d-bc6d045cb903-run-openvswitch\") pod \"ovnkube-node-lxdc5\" (UID: \"725f7657-c52d-46d2-bf7d-bc6d045cb903\") " pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5"
Jul 24 23:29:22.148765 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.148623    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/725f7657-c52d-46d2-bf7d-bc6d045cb903-env-overrides\") pod \"ovnkube-node-lxdc5\" (UID: \"725f7657-c52d-46d2-bf7d-bc6d045cb903\") " pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5"
Jul 24 23:29:22.149024 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.149003    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/03bb2978-3f29-4167-8b41-c257a2b3878d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pck26\" (UID: \"03bb2978-3f29-4167-8b41-c257a2b3878d\") " pod="openshift-multus/multus-additional-cni-plugins-pck26"
Jul 24 23:29:22.149111 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.149065    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1d4690c2-2cef-4ede-8db1-fe6d22add6e2-konnectivity-ca\") pod \"konnectivity-agent-hjb7w\" (UID: \"1d4690c2-2cef-4ede-8db1-fe6d22add6e2\") " pod="kube-system/konnectivity-agent-hjb7w"
Jul 24 23:29:22.149111 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.149103    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/725f7657-c52d-46d2-bf7d-bc6d045cb903-run-ovn\") pod \"ovnkube-node-lxdc5\" (UID: \"725f7657-c52d-46d2-bf7d-bc6d045cb903\") " pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5"
Jul 24 23:29:22.149219 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.149140    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgds7\" (UniqueName: \"kubernetes.io/projected/725f7657-c52d-46d2-bf7d-bc6d045cb903-kube-api-access-dgds7\") pod \"ovnkube-node-lxdc5\" (UID: \"725f7657-c52d-46d2-bf7d-bc6d045cb903\") " pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5"
Jul 24 23:29:22.149219 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.149171    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/03bb2978-3f29-4167-8b41-c257a2b3878d-cnibin\") pod \"multus-additional-cni-plugins-pck26\" (UID: \"03bb2978-3f29-4167-8b41-c257a2b3878d\") " pod="openshift-multus/multus-additional-cni-plugins-pck26"
Jul 24 23:29:22.149219 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.149197    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5e81d551-3cb1-4707-998f-713e083fefe8-host-run-netns\") pod \"multus-rf4fp\" (UID: \"5e81d551-3cb1-4707-998f-713e083fefe8\") " pod="openshift-multus/multus-rf4fp"
Jul 24 23:29:22.149376 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.149216    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/725f7657-c52d-46d2-bf7d-bc6d045cb903-env-overrides\") pod \"ovnkube-node-lxdc5\" (UID: \"725f7657-c52d-46d2-bf7d-bc6d045cb903\") " pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5"
Jul 24 23:29:22.149376 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.149228    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9dc9bac0-7bd8-4756-9a96-25594b1e3bd2-etc-kubernetes\") pod \"tuned-mhq9b\" (UID: \"9dc9bac0-7bd8-4756-9a96-25594b1e3bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-mhq9b"
Jul 24 23:29:22.149376 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.149261    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6ad632db-cb9d-460c-885f-0c4c1f495a1a-iptables-alerter-script\") pod \"iptables-alerter-kbtxb\" (UID: \"6ad632db-cb9d-460c-885f-0c4c1f495a1a\") " pod="openshift-network-operator/iptables-alerter-kbtxb"
Jul 24 23:29:22.149376 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.149292    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/725f7657-c52d-46d2-bf7d-bc6d045cb903-host-cni-netd\") pod \"ovnkube-node-lxdc5\" (UID: \"725f7657-c52d-46d2-bf7d-bc6d045cb903\") " pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5"
Jul 24 23:29:22.149376 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.149321    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5e81d551-3cb1-4707-998f-713e083fefe8-cnibin\") pod \"multus-rf4fp\" (UID: \"5e81d551-3cb1-4707-998f-713e083fefe8\") " pod="openshift-multus/multus-rf4fp"
Jul 24 23:29:22.149376 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.149353    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5e81d551-3cb1-4707-998f-713e083fefe8-etc-kubernetes\") pod \"multus-rf4fp\" (UID: \"5e81d551-3cb1-4707-998f-713e083fefe8\") " pod="openshift-multus/multus-rf4fp"
Jul 24 23:29:22.149696 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.149385    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9dc9bac0-7bd8-4756-9a96-25594b1e3bd2-etc-modprobe-d\") pod \"tuned-mhq9b\" (UID: \"9dc9bac0-7bd8-4756-9a96-25594b1e3bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-mhq9b"
Jul 24 23:29:22.149696 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.149414    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9dc9bac0-7bd8-4756-9a96-25594b1e3bd2-sys\") pod \"tuned-mhq9b\" (UID: \"9dc9bac0-7bd8-4756-9a96-25594b1e3bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-mhq9b"
Jul 24 23:29:22.149696 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.149448    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/725f7657-c52d-46d2-bf7d-bc6d045cb903-var-lib-openvswitch\") pod \"ovnkube-node-lxdc5\" (UID: \"725f7657-c52d-46d2-bf7d-bc6d045cb903\") " pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5"
Jul 24 23:29:22.149696 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.149475    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5e81d551-3cb1-4707-998f-713e083fefe8-host-run-multus-certs\") pod \"multus-rf4fp\" (UID: \"5e81d551-3cb1-4707-998f-713e083fefe8\") " pod="openshift-multus/multus-rf4fp"
Jul 24 23:29:22.149696 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.149507    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3504a0bb-6580-4187-8f2d-7f71494675e4-hosts-file\") pod \"node-resolver-7f7qw\" (UID: \"3504a0bb-6580-4187-8f2d-7f71494675e4\") " pod="openshift-dns/node-resolver-7f7qw"
Jul 24 23:29:22.149696 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.149538    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/03bb2978-3f29-4167-8b41-c257a2b3878d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pck26\" (UID: \"03bb2978-3f29-4167-8b41-c257a2b3878d\") " pod="openshift-multus/multus-additional-cni-plugins-pck26"
Jul 24 23:29:22.149696 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.149570    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5e81d551-3cb1-4707-998f-713e083fefe8-host-var-lib-cni-bin\") pod \"multus-rf4fp\" (UID: \"5e81d551-3cb1-4707-998f-713e083fefe8\") " pod="openshift-multus/multus-rf4fp"
Jul 24 23:29:22.149696 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.149622    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9dc9bac0-7bd8-4756-9a96-25594b1e3bd2-etc-systemd\") pod \"tuned-mhq9b\" (UID: \"9dc9bac0-7bd8-4756-9a96-25594b1e3bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-mhq9b"
Jul 24 23:29:22.149696 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.149648    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9dc9bac0-7bd8-4756-9a96-25594b1e3bd2-etc-tuned\") pod \"tuned-mhq9b\" (UID: \"9dc9bac0-7bd8-4756-9a96-25594b1e3bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-mhq9b"
Jul 24 23:29:22.149696 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.149652    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/725f7657-c52d-46d2-bf7d-bc6d045cb903-log-socket\") pod \"ovnkube-node-lxdc5\" (UID: \"725f7657-c52d-46d2-bf7d-bc6d045cb903\") " pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5"
Jul 24 23:29:22.149696 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.149683    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/725f7657-c52d-46d2-bf7d-bc6d045cb903-host-kubelet\") pod \"ovnkube-node-lxdc5\" (UID: \"725f7657-c52d-46d2-bf7d-bc6d045cb903\") " pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5"
Jul 24 23:29:22.150279 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.149719    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/725f7657-c52d-46d2-bf7d-bc6d045cb903-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lxdc5\" (UID: \"725f7657-c52d-46d2-bf7d-bc6d045cb903\") " pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5"
Jul 24 23:29:22.150279 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.149751    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9dc9bac0-7bd8-4756-9a96-25594b1e3bd2-tmp\") pod \"tuned-mhq9b\" (UID: \"9dc9bac0-7bd8-4756-9a96-25594b1e3bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-mhq9b"
Jul 24 23:29:22.150279 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.149778    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/68864bf3-c771-4894-9f14-52ee9e8052f9-kubelet-config\") pod \"global-pull-secret-syncer-rplqt\" (UID: \"68864bf3-c771-4894-9f14-52ee9e8052f9\") " pod="kube-system/global-pull-secret-syncer-rplqt"
Jul 24 23:29:22.150279 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.149809    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/725f7657-c52d-46d2-bf7d-bc6d045cb903-ovnkube-script-lib\") pod \"ovnkube-node-lxdc5\" (UID: \"725f7657-c52d-46d2-bf7d-bc6d045cb903\") " pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5"
Jul 24 23:29:22.150279 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.149842    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/03bb2978-3f29-4167-8b41-c257a2b3878d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pck26\" (UID: \"03bb2978-3f29-4167-8b41-c257a2b3878d\") " pod="openshift-multus/multus-additional-cni-plugins-pck26"
Jul 24 23:29:22.150279 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.149874    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5e81d551-3cb1-4707-998f-713e083fefe8-system-cni-dir\") pod \"multus-rf4fp\" (UID: \"5e81d551-3cb1-4707-998f-713e083fefe8\") " pod="openshift-multus/multus-rf4fp"
Jul 24 23:29:22.150279 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.149906    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5e81d551-3cb1-4707-998f-713e083fefe8-cni-binary-copy\") pod \"multus-rf4fp\" (UID: \"5e81d551-3cb1-4707-998f-713e083fefe8\") " pod="openshift-multus/multus-rf4fp"
Jul 24 23:29:22.150279 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.149934    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9dc9bac0-7bd8-4756-9a96-25594b1e3bd2-etc-sysctl-d\") pod \"tuned-mhq9b\" (UID: \"9dc9bac0-7bd8-4756-9a96-25594b1e3bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-mhq9b"
Jul 24 23:29:22.150279 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.149964    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/77f6315f-d3d1-47ea-a83d-2b036ac39189-host\") pod \"node-ca-x8chk\" (UID: \"77f6315f-d3d1-47ea-a83d-2b036ac39189\") " pod="openshift-image-registry/node-ca-x8chk"
Jul 24 23:29:22.150279 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.149997    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/725f7657-c52d-46d2-bf7d-bc6d045cb903-ovn-node-metrics-cert\") pod \"ovnkube-node-lxdc5\" (UID: \"725f7657-c52d-46d2-bf7d-bc6d045cb903\") " pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5"
Jul 24 23:29:22.150279 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.150031    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5e81d551-3cb1-4707-998f-713e083fefe8-host-var-lib-cni-multus\") pod \"multus-rf4fp\" (UID: \"5e81d551-3cb1-4707-998f-713e083fefe8\") " pod="openshift-multus/multus-rf4fp"
Jul 24 23:29:22.150279 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.150061    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5e81d551-3cb1-4707-998f-713e083fefe8-hostroot\") pod \"multus-rf4fp\" (UID: \"5e81d551-3cb1-4707-998f-713e083fefe8\") " pod="openshift-multus/multus-rf4fp"
Jul 24 23:29:22.150279 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.150088    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/725f7657-c52d-46d2-bf7d-bc6d045cb903-host-cni-bin\") pod \"ovnkube-node-lxdc5\" (UID: \"725f7657-c52d-46d2-bf7d-bc6d045cb903\") " pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5"
Jul 24 23:29:22.150279 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.150122    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/725f7657-c52d-46d2-bf7d-bc6d045cb903-host-slash\") pod \"ovnkube-node-lxdc5\" (UID: \"725f7657-c52d-46d2-bf7d-bc6d045cb903\") " pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5"
Jul 24 23:29:22.150279 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.150159    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3504a0bb-6580-4187-8f2d-7f71494675e4-tmp-dir\") pod \"node-resolver-7f7qw\" (UID: \"3504a0bb-6580-4187-8f2d-7f71494675e4\") " pod="openshift-dns/node-resolver-7f7qw"
Jul 24 23:29:22.150279 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.150192    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79c2s\" (UniqueName: \"kubernetes.io/projected/3504a0bb-6580-4187-8f2d-7f71494675e4-kube-api-access-79c2s\") pod \"node-resolver-7f7qw\" (UID: \"3504a0bb-6580-4187-8f2d-7f71494675e4\") " pod="openshift-dns/node-resolver-7f7qw"
Jul 24 23:29:22.150279 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.150222    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/334638da-1923-4fd5-88b4-8ca936adf513-etc-selinux\") pod \"aws-ebs-csi-driver-node-8llw9\" (UID: \"334638da-1923-4fd5-88b4-8ca936adf513\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8llw9"
Jul 24 23:29:22.151027 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.150253    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1d4690c2-2cef-4ede-8db1-fe6d22add6e2-agent-certs\") pod \"konnectivity-agent-hjb7w\" (UID: \"1d4690c2-2cef-4ede-8db1-fe6d22add6e2\") " pod="kube-system/konnectivity-agent-hjb7w"
Jul 24 23:29:22.151027 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.150271    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1d4690c2-2cef-4ede-8db1-fe6d22add6e2-konnectivity-ca\") pod \"konnectivity-agent-hjb7w\" (UID: \"1d4690c2-2cef-4ede-8db1-fe6d22add6e2\") " pod="kube-system/konnectivity-agent-hjb7w"
Jul 24 23:29:22.151027 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.150281    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/725f7657-c52d-46d2-bf7d-bc6d045cb903-ovnkube-config\") pod \"ovnkube-node-lxdc5\" (UID: \"725f7657-c52d-46d2-bf7d-bc6d045cb903\") " pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5"
Jul 24 23:29:22.151027 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.150314    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnq9w\" (UniqueName: \"kubernetes.io/projected/03bb2978-3f29-4167-8b41-c257a2b3878d-kube-api-access-fnq9w\") pod \"multus-additional-cni-plugins-pck26\" (UID: \"03bb2978-3f29-4167-8b41-c257a2b3878d\") " pod="openshift-multus/multus-additional-cni-plugins-pck26"
Jul 24 23:29:22.151027 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.150338    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/725f7657-c52d-46d2-bf7d-bc6d045cb903-run-ovn\") pod \"ovnkube-node-lxdc5\" (UID: \"725f7657-c52d-46d2-bf7d-bc6d045cb903\") " pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5"
Jul 24 23:29:22.151027 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.150349    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5e81d551-3cb1-4707-998f-713e083fefe8-multus-cni-dir\") pod \"multus-rf4fp\" (UID: \"5e81d551-3cb1-4707-998f-713e083fefe8\") " pod="openshift-multus/multus-rf4fp"
Jul 24 23:29:22.151027 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.150516    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5e81d551-3cb1-4707-998f-713e083fefe8-multus-cni-dir\") pod \"multus-rf4fp\" (UID: \"5e81d551-3cb1-4707-998f-713e083fefe8\") " pod="openshift-multus/multus-rf4fp"
Jul 24 23:29:22.151027 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.150585    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/68864bf3-c771-4894-9f14-52ee9e8052f9-kubelet-config\") pod \"global-pull-secret-syncer-rplqt\" (UID: \"68864bf3-c771-4894-9f14-52ee9e8052f9\") " pod="kube-system/global-pull-secret-syncer-rplqt"
Jul 24 23:29:22.151027 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.150803    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9dc9bac0-7bd8-4756-9a96-25594b1e3bd2-etc-modprobe-d\") pod \"tuned-mhq9b\" (UID: \"9dc9bac0-7bd8-4756-9a96-25594b1e3bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-mhq9b"
Jul 24 23:29:22.151027 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.150862    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5e81d551-3cb1-4707-998f-713e083fefe8-host-var-lib-cni-bin\") pod \"multus-rf4fp\" (UID: \"5e81d551-3cb1-4707-998f-713e083fefe8\") " pod="openshift-multus/multus-rf4fp"
Jul 24 23:29:22.151027 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.150912    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/725f7657-c52d-46d2-bf7d-bc6d045cb903-var-lib-openvswitch\") pod \"ovnkube-node-lxdc5\" (UID: \"725f7657-c52d-46d2-bf7d-bc6d045cb903\") " pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5"
Jul 24 23:29:22.151027 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.150967    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5e81d551-3cb1-4707-998f-713e083fefe8-host-run-multus-certs\") pod \"multus-rf4fp\" (UID: \"5e81d551-3cb1-4707-998f-713e083fefe8\") " pod="openshift-multus/multus-rf4fp"
Jul 24 23:29:22.151027 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.151025    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3504a0bb-6580-4187-8f2d-7f71494675e4-hosts-file\") pod \"node-resolver-7f7qw\" (UID: \"3504a0bb-6580-4187-8f2d-7f71494675e4\") " pod="openshift-dns/node-resolver-7f7qw"
Jul 24 23:29:22.151708 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.151120    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/03bb2978-3f29-4167-8b41-c257a2b3878d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pck26\" (UID: \"03bb2978-3f29-4167-8b41-c257a2b3878d\") " pod="openshift-multus/multus-additional-cni-plugins-pck26"
Jul 24 23:29:22.151708 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.151175    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/03bb2978-3f29-4167-8b41-c257a2b3878d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pck26\" (UID: \"03bb2978-3f29-4167-8b41-c257a2b3878d\") " pod="openshift-multus/multus-additional-cni-plugins-pck26"
Jul 24 23:29:22.151708 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.151235    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/725f7657-c52d-46d2-bf7d-bc6d045cb903-host-cni-bin\") pod \"ovnkube-node-lxdc5\" (UID: \"725f7657-c52d-46d2-bf7d-bc6d045cb903\") " pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5"
Jul 24 23:29:22.151708 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.151398    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5e81d551-3cb1-4707-998f-713e083fefe8-system-cni-dir\") pod \"multus-rf4fp\" (UID: \"5e81d551-3cb1-4707-998f-713e083fefe8\") " pod="openshift-multus/multus-rf4fp"
Jul 24 23:29:22.151708 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.151595    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6ad632db-cb9d-460c-885f-0c4c1f495a1a-iptables-alerter-script\") pod \"iptables-alerter-kbtxb\" (UID: \"6ad632db-cb9d-460c-885f-0c4c1f495a1a\") " pod="openshift-network-operator/iptables-alerter-kbtxb"
Jul 24 23:29:22.151708 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.151683    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/03bb2978-3f29-4167-8b41-c257a2b3878d-cnibin\") pod \"multus-additional-cni-plugins-pck26\" (UID: \"03bb2978-3f29-4167-8b41-c257a2b3878d\") " pod="openshift-multus/multus-additional-cni-plugins-pck26"
Jul 24 23:29:22.152059 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.152038    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5e81d551-3cb1-4707-998f-713e083fefe8-cni-binary-copy\") pod \"multus-rf4fp\" (UID: \"5e81d551-3cb1-4707-998f-713e083fefe8\") " pod="openshift-multus/multus-rf4fp"
Jul 24 23:29:22.152216 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.152186    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/725f7657-c52d-46d2-bf7d-bc6d045cb903-ovnkube-script-lib\") pod \"ovnkube-node-lxdc5\" (UID: \"725f7657-c52d-46d2-bf7d-bc6d045cb903\") " pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5"
Jul 24 23:29:22.152216 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.152207    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9dc9bac0-7bd8-4756-9a96-25594b1e3bd2-etc-sysctl-d\") pod \"tuned-mhq9b\" (UID: \"9dc9bac0-7bd8-4756-9a96-25594b1e3bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-mhq9b"
Jul 24 23:29:22.152337 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.152260    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5e81d551-3cb1-4707-998f-713e083fefe8-host-run-netns\") pod \"multus-rf4fp\" (UID: \"5e81d551-3cb1-4707-998f-713e083fefe8\") " pod="openshift-multus/multus-rf4fp"
Jul 24 23:29:22.152337 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.152309    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/77f6315f-d3d1-47ea-a83d-2b036ac39189-host\") pod \"node-ca-x8chk\" (UID: \"77f6315f-d3d1-47ea-a83d-2b036ac39189\") " pod="openshift-image-registry/node-ca-x8chk"
Jul 24 23:29:22.152442 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.152394    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5e81d551-3cb1-4707-998f-713e083fefe8-cnibin\") pod \"multus-rf4fp\" (UID: \"5e81d551-3cb1-4707-998f-713e083fefe8\") " pod="openshift-multus/multus-rf4fp"
Jul 24 23:29:22.152498 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.152441    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5e81d551-3cb1-4707-998f-713e083fefe8-etc-kubernetes\") pod \"multus-rf4fp\" (UID: \"5e81d551-3cb1-4707-998f-713e083fefe8\") " pod="openshift-multus/multus-rf4fp"
Jul 24 23:29:22.152498 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.152488    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/725f7657-c52d-46d2-bf7d-bc6d045cb903-host-slash\") pod \"ovnkube-node-lxdc5\" (UID: \"725f7657-c52d-46d2-bf7d-bc6d045cb903\") " pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5"
Jul 24 23:29:22.152623 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.152579    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5e81d551-3cb1-4707-998f-713e083fefe8-host-var-lib-cni-multus\") pod \"multus-rf4fp\" (UID: \"5e81d551-3cb1-4707-998f-713e083fefe8\") " pod="openshift-multus/multus-rf4fp"
Jul 24 23:29:22.152688 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.152657    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5e81d551-3cb1-4707-998f-713e083fefe8-hostroot\") pod \"multus-rf4fp\" (UID: \"5e81d551-3cb1-4707-998f-713e083fefe8\") " pod="openshift-multus/multus-rf4fp"
Jul 24 23:29:22.152791 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.152771    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3504a0bb-6580-4187-8f2d-7f71494675e4-tmp-dir\") pod \"node-resolver-7f7qw\" (UID: \"3504a0bb-6580-4187-8f2d-7f71494675e4\") " pod="openshift-dns/node-resolver-7f7qw"
Jul 24 23:29:22.152875 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.152856    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9dc9bac0-7bd8-4756-9a96-25594b1e3bd2-etc-systemd\") pod \"tuned-mhq9b\" (UID: \"9dc9bac0-7bd8-4756-9a96-25594b1e3bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-mhq9b"
Jul 24 23:29:22.152946 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.152931    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/725f7657-c52d-46d2-bf7d-bc6d045cb903-host-kubelet\") pod \"ovnkube-node-lxdc5\" (UID: \"725f7657-c52d-46d2-bf7d-bc6d045cb903\") " pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5"
Jul 24 23:29:22.153178 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.153156    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9dc9bac0-7bd8-4756-9a96-25594b1e3bd2-sys\") pod \"tuned-mhq9b\" (UID: \"9dc9bac0-7bd8-4756-9a96-25594b1e3bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-mhq9b"
Jul 24 23:29:22.153259 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.153224    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9dc9bac0-7bd8-4756-9a96-25594b1e3bd2-etc-kubernetes\") pod \"tuned-mhq9b\" (UID: \"9dc9bac0-7bd8-4756-9a96-25594b1e3bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-mhq9b"
Jul 24 23:29:22.153318 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.153275    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/725f7657-c52d-46d2-bf7d-bc6d045cb903-host-cni-netd\") pod \"ovnkube-node-lxdc5\" (UID: \"725f7657-c52d-46d2-bf7d-bc6d045cb903\") " pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5"
Jul 24 23:29:22.153374 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.153329    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/725f7657-c52d-46d2-bf7d-bc6d045cb903-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lxdc5\" (UID: \"725f7657-c52d-46d2-bf7d-bc6d045cb903\") " pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5"
Jul 24 23:29:22.153374 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.153366    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/334638da-1923-4fd5-88b4-8ca936adf513-etc-selinux\") pod \"aws-ebs-csi-driver-node-8llw9\" (UID: \"334638da-1923-4fd5-88b4-8ca936adf513\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8llw9"
Jul 24 23:29:22.153741 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.153719    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5e81d551-3cb1-4707-998f-713e083fefe8-os-release\") pod \"multus-rf4fp\" (UID: \"5e81d551-3cb1-4707-998f-713e083fefe8\") " pod="openshift-multus/multus-rf4fp"
Jul 24 23:29:22.153830 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.153768    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5e81d551-3cb1-4707-998f-713e083fefe8-multus-daemon-config\") pod \"multus-rf4fp\" (UID: \"5e81d551-3cb1-4707-998f-713e083fefe8\") " pod="openshift-multus/multus-rf4fp"
Jul 24 23:29:22.153830 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.153804    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpcqp\" (UniqueName: \"kubernetes.io/projected/7b7b9a28-3166-4af3-b27f-1ab5795c5f92-kube-api-access-bpcqp\") pod \"network-check-target-gltb5\" (UID: \"7b7b9a28-3166-4af3-b27f-1ab5795c5f92\") " pod="openshift-network-diagnostics/network-check-target-gltb5"
Jul 24 23:29:22.153830 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.153772    2558 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 24 23:29:22.155713 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.153830    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9dc9bac0-7bd8-4756-9a96-25594b1e3bd2-lib-modules\") pod \"tuned-mhq9b\" (UID: \"9dc9bac0-7bd8-4756-9a96-25594b1e3bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-mhq9b"
Jul 24 23:29:22.155713 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.153861    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/725f7657-c52d-46d2-bf7d-bc6d045cb903-host-run-netns\") pod \"ovnkube-node-lxdc5\" (UID: \"725f7657-c52d-46d2-bf7d-bc6d045cb903\") " pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5"
Jul 24 23:29:22.155713 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.153891    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkh5g\" (UniqueName: \"kubernetes.io/projected/5e81d551-3cb1-4707-998f-713e083fefe8-kube-api-access-xkh5g\") pod \"multus-rf4fp\" (UID: \"5e81d551-3cb1-4707-998f-713e083fefe8\") " pod="openshift-multus/multus-rf4fp"
Jul 24 23:29:22.155713 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.154166    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/334638da-1923-4fd5-88b4-8ca936adf513-device-dir\") pod \"aws-ebs-csi-driver-node-8llw9\" (UID: \"334638da-1923-4fd5-88b4-8ca936adf513\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8llw9"
Jul 24 23:29:22.155713 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.154215    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l42dl\" (UniqueName: \"kubernetes.io/projected/dcf99aef-00e1-4d16-94ca-12050dc95547-kube-api-access-l42dl\") pod \"network-metrics-daemon-54fxx\" (UID: \"dcf99aef-00e1-4d16-94ca-12050dc95547\") " pod="openshift-multus/network-metrics-daemon-54fxx"
Jul 24 23:29:22.155713 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.154256    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6mtw\" (UniqueName: \"kubernetes.io/projected/9dc9bac0-7bd8-4756-9a96-25594b1e3bd2-kube-api-access-c6mtw\") pod \"tuned-mhq9b\" (UID: \"9dc9bac0-7bd8-4756-9a96-25594b1e3bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-mhq9b"
Jul 24 23:29:22.155713 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.154293    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqg9m\" (UniqueName: \"kubernetes.io/projected/6ad632db-cb9d-460c-885f-0c4c1f495a1a-kube-api-access-xqg9m\") pod \"iptables-alerter-kbtxb\" (UID: \"6ad632db-cb9d-460c-885f-0c4c1f495a1a\") " pod="openshift-network-operator/iptables-alerter-kbtxb"
Jul 24 23:29:22.155713 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.154325    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/725f7657-c52d-46d2-bf7d-bc6d045cb903-node-log\") pod \"ovnkube-node-lxdc5\" (UID: \"725f7657-c52d-46d2-bf7d-bc6d045cb903\") " pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5"
Jul 24 23:29:22.155713 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.154363    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/77f6315f-d3d1-47ea-a83d-2b036ac39189-serviceca\") pod \"node-ca-x8chk\" (UID: \"77f6315f-d3d1-47ea-a83d-2b036ac39189\") " pod="openshift-image-registry/node-ca-x8chk"
Jul 24 23:29:22.155713 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.154403    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n88sg\" (UniqueName: \"kubernetes.io/projected/77f6315f-d3d1-47ea-a83d-2b036ac39189-kube-api-access-n88sg\") pod \"node-ca-x8chk\" (UID: \"77f6315f-d3d1-47ea-a83d-2b036ac39189\") " pod="openshift-image-registry/node-ca-x8chk"
Jul 24 23:29:22.155713 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.154442    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/68864bf3-c771-4894-9f14-52ee9e8052f9-original-pull-secret\") pod \"global-pull-secret-syncer-rplqt\" (UID: \"68864bf3-c771-4894-9f14-52ee9e8052f9\") " pod="kube-system/global-pull-secret-syncer-rplqt"
Jul 24 23:29:22.155713 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.154450    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5e81d551-3cb1-4707-998f-713e083fefe8-multus-daemon-config\") pod \"multus-rf4fp\" (UID: \"5e81d551-3cb1-4707-998f-713e083fefe8\") " pod="openshift-multus/multus-rf4fp"
Jul 24 23:29:22.155713 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.154484    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/725f7657-c52d-46d2-bf7d-bc6d045cb903-etc-openvswitch\") pod \"ovnkube-node-lxdc5\" (UID: \"725f7657-c52d-46d2-bf7d-bc6d045cb903\") " pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5"
Jul 24 23:29:22.155713 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.154522    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/334638da-1923-4fd5-88b4-8ca936adf513-socket-dir\") pod \"aws-ebs-csi-driver-node-8llw9\" (UID: \"334638da-1923-4fd5-88b4-8ca936adf513\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8llw9"
Jul 24 23:29:22.155713 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.154952    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/334638da-1923-4fd5-88b4-8ca936adf513-device-dir\") pod \"aws-ebs-csi-driver-node-8llw9\" (UID: \"334638da-1923-4fd5-88b4-8ca936adf513\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8llw9"
Jul 24 23:29:22.155713 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.154440    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5e81d551-3cb1-4707-998f-713e083fefe8-os-release\") pod \"multus-rf4fp\" (UID: \"5e81d551-3cb1-4707-998f-713e083fefe8\") " pod="openshift-multus/multus-rf4fp"
Jul 24 23:29:22.155713 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.155135    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhfmb\" (UniqueName: \"kubernetes.io/projected/334638da-1923-4fd5-88b4-8ca936adf513-kube-api-access-fhfmb\") pod \"aws-ebs-csi-driver-node-8llw9\" (UID: \"334638da-1923-4fd5-88b4-8ca936adf513\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8llw9"
Jul 24 23:29:22.156697 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.155383    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dcf99aef-00e1-4d16-94ca-12050dc95547-metrics-certs\") pod \"network-metrics-daemon-54fxx\" (UID: \"dcf99aef-00e1-4d16-94ca-12050dc95547\") " pod="openshift-multus/network-metrics-daemon-54fxx"
Jul 24 23:29:22.156697 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.155511    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/68864bf3-c771-4894-9f14-52ee9e8052f9-dbus\") pod \"global-pull-secret-syncer-rplqt\" (UID: \"68864bf3-c771-4894-9f14-52ee9e8052f9\") " pod="kube-system/global-pull-secret-syncer-rplqt"
Jul 24 23:29:22.156697 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.155538    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/725f7657-c52d-46d2-bf7d-bc6d045cb903-ovnkube-config\") pod \"ovnkube-node-lxdc5\" (UID: \"725f7657-c52d-46d2-bf7d-bc6d045cb903\") " pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5"
Jul 24 23:29:22.156697 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.155745    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/68864bf3-c771-4894-9f14-52ee9e8052f9-dbus\") pod \"global-pull-secret-syncer-rplqt\" (UID: \"68864bf3-c771-4894-9f14-52ee9e8052f9\") " pod="kube-system/global-pull-secret-syncer-rplqt"
Jul 24 23:29:22.156697 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.155881    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/725f7657-c52d-46d2-bf7d-bc6d045cb903-host-run-netns\") pod \"ovnkube-node-lxdc5\" (UID: \"725f7657-c52d-46d2-bf7d-bc6d045cb903\") " pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5"
Jul 24 23:29:22.156948 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.156724    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/725f7657-c52d-46d2-bf7d-bc6d045cb903-etc-openvswitch\") pod \"ovnkube-node-lxdc5\" (UID: \"725f7657-c52d-46d2-bf7d-bc6d045cb903\") " pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5"
Jul 24 23:29:22.156948 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.156725    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/77f6315f-d3d1-47ea-a83d-2b036ac39189-serviceca\") pod \"node-ca-x8chk\" (UID: \"77f6315f-d3d1-47ea-a83d-2b036ac39189\") " pod="openshift-image-registry/node-ca-x8chk"
Jul 24 23:29:22.156948 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.156839    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/334638da-1923-4fd5-88b4-8ca936adf513-socket-dir\") pod \"aws-ebs-csi-driver-node-8llw9\" (UID: \"334638da-1923-4fd5-88b4-8ca936adf513\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8llw9"
Jul 24 23:29:22.156948 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:22.156842    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 23:29:22.157155 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:22.156956    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 23:29:22.157155 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.156955    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9dc9bac0-7bd8-4756-9a96-25594b1e3bd2-lib-modules\") pod \"tuned-mhq9b\" (UID: \"9dc9bac0-7bd8-4756-9a96-25594b1e3bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-mhq9b"
Jul 24 23:29:22.157155 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:22.157043    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68864bf3-c771-4894-9f14-52ee9e8052f9-original-pull-secret podName:68864bf3-c771-4894-9f14-52ee9e8052f9 nodeName:}" failed. No retries permitted until 2026-07-24 23:29:22.65699949 +0000 UTC m=+3.043430224 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/68864bf3-c771-4894-9f14-52ee9e8052f9-original-pull-secret") pod "global-pull-secret-syncer-rplqt" (UID: "68864bf3-c771-4894-9f14-52ee9e8052f9") : object "kube-system"/"original-pull-secret" not registered
Jul 24 23:29:22.157155 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.157037    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/725f7657-c52d-46d2-bf7d-bc6d045cb903-node-log\") pod \"ovnkube-node-lxdc5\" (UID: \"725f7657-c52d-46d2-bf7d-bc6d045cb903\") " pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5"
Jul 24 23:29:22.157155 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:22.157086    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcf99aef-00e1-4d16-94ca-12050dc95547-metrics-certs podName:dcf99aef-00e1-4d16-94ca-12050dc95547 nodeName:}" failed. No retries permitted until 2026-07-24 23:29:22.657055763 +0000 UTC m=+3.043486553 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dcf99aef-00e1-4d16-94ca-12050dc95547-metrics-certs") pod "network-metrics-daemon-54fxx" (UID: "dcf99aef-00e1-4d16-94ca-12050dc95547") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 23:29:22.160223 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.159095    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9dc9bac0-7bd8-4756-9a96-25594b1e3bd2-tmp\") pod \"tuned-mhq9b\" (UID: \"9dc9bac0-7bd8-4756-9a96-25594b1e3bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-mhq9b"
Jul 24 23:29:22.160223 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.159165    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9dc9bac0-7bd8-4756-9a96-25594b1e3bd2-etc-tuned\") pod \"tuned-mhq9b\" (UID: \"9dc9bac0-7bd8-4756-9a96-25594b1e3bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-mhq9b"
Jul 24 23:29:22.160223 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.159749    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/725f7657-c52d-46d2-bf7d-bc6d045cb903-ovn-node-metrics-cert\") pod \"ovnkube-node-lxdc5\" (UID: \"725f7657-c52d-46d2-bf7d-bc6d045cb903\") " pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5"
Jul 24 23:29:22.160891 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.160867    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgds7\" (UniqueName: \"kubernetes.io/projected/725f7657-c52d-46d2-bf7d-bc6d045cb903-kube-api-access-dgds7\") pod \"ovnkube-node-lxdc5\" (UID: \"725f7657-c52d-46d2-bf7d-bc6d045cb903\") " pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5"
Jul 24 23:29:22.161192 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.161068    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1d4690c2-2cef-4ede-8db1-fe6d22add6e2-agent-certs\") pod \"konnectivity-agent-hjb7w\" (UID: \"1d4690c2-2cef-4ede-8db1-fe6d22add6e2\") " pod="kube-system/konnectivity-agent-hjb7w"
Jul 24 23:29:22.161719 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:22.161561    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 23:29:22.161719 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:22.161584    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 23:29:22.161719 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:22.161600    2558 projected.go:196] Error preparing data for projected volume kube-api-access-bpcqp for pod openshift-network-diagnostics/network-check-target-gltb5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 23:29:22.161719 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:22.161704    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7b7b9a28-3166-4af3-b27f-1ab5795c5f92-kube-api-access-bpcqp podName:7b7b9a28-3166-4af3-b27f-1ab5795c5f92 nodeName:}" failed. No retries permitted until 2026-07-24 23:29:22.661686729 +0000 UTC m=+3.048117451 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-bpcqp" (UniqueName: "kubernetes.io/projected/7b7b9a28-3166-4af3-b27f-1ab5795c5f92-kube-api-access-bpcqp") pod "network-check-target-gltb5" (UID: "7b7b9a28-3166-4af3-b27f-1ab5795c5f92") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 23:29:22.162187 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.162014    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-79c2s\" (UniqueName: \"kubernetes.io/projected/3504a0bb-6580-4187-8f2d-7f71494675e4-kube-api-access-79c2s\") pod \"node-resolver-7f7qw\" (UID: \"3504a0bb-6580-4187-8f2d-7f71494675e4\") " pod="openshift-dns/node-resolver-7f7qw"
Jul 24 23:29:22.162331 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.162263    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnq9w\" (UniqueName: \"kubernetes.io/projected/03bb2978-3f29-4167-8b41-c257a2b3878d-kube-api-access-fnq9w\") pod \"multus-additional-cni-plugins-pck26\" (UID: \"03bb2978-3f29-4167-8b41-c257a2b3878d\") " pod="openshift-multus/multus-additional-cni-plugins-pck26"
Jul 24 23:29:22.165223 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.165012    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqg9m\" (UniqueName: \"kubernetes.io/projected/6ad632db-cb9d-460c-885f-0c4c1f495a1a-kube-api-access-xqg9m\") pod \"iptables-alerter-kbtxb\" (UID: \"6ad632db-cb9d-460c-885f-0c4c1f495a1a\") " pod="openshift-network-operator/iptables-alerter-kbtxb"
Jul 24 23:29:22.165766 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.165617    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n88sg\" (UniqueName: \"kubernetes.io/projected/77f6315f-d3d1-47ea-a83d-2b036ac39189-kube-api-access-n88sg\") pod \"node-ca-x8chk\" (UID: \"77f6315f-d3d1-47ea-a83d-2b036ac39189\") " pod="openshift-image-registry/node-ca-x8chk"
Jul 24 23:29:22.165766 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.165715    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkh5g\" (UniqueName: \"kubernetes.io/projected/5e81d551-3cb1-4707-998f-713e083fefe8-kube-api-access-xkh5g\") pod \"multus-rf4fp\" (UID: \"5e81d551-3cb1-4707-998f-713e083fefe8\") " pod="openshift-multus/multus-rf4fp"
Jul 24 23:29:22.166832 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.166737    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6mtw\" (UniqueName: \"kubernetes.io/projected/9dc9bac0-7bd8-4756-9a96-25594b1e3bd2-kube-api-access-c6mtw\") pod \"tuned-mhq9b\" (UID: \"9dc9bac0-7bd8-4756-9a96-25594b1e3bd2\") " pod="openshift-cluster-node-tuning-operator/tuned-mhq9b"
Jul 24 23:29:22.167124 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.167105    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l42dl\" (UniqueName: \"kubernetes.io/projected/dcf99aef-00e1-4d16-94ca-12050dc95547-kube-api-access-l42dl\") pod \"network-metrics-daemon-54fxx\" (UID: \"dcf99aef-00e1-4d16-94ca-12050dc95547\") " pod="openshift-multus/network-metrics-daemon-54fxx"
Jul 24 23:29:22.169807 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.169787    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhfmb\" (UniqueName: \"kubernetes.io/projected/334638da-1923-4fd5-88b4-8ca936adf513-kube-api-access-fhfmb\") pod \"aws-ebs-csi-driver-node-8llw9\" (UID: \"334638da-1923-4fd5-88b4-8ca936adf513\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8llw9"
Jul 24 23:29:22.206219 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.206155    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-133.ec2.internal" event={"ID":"4e4ac0d94c86ae9a285d0e7940c49e40","Type":"ContainerStarted","Data":"961fca4919e3291c44ed9de67351d67aa2e5dcceb988a5c2cd4687cab30361aa"}
Jul 24 23:29:22.207387 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.207344    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-133.ec2.internal" event={"ID":"b8b5e88d55d068e81c55bbf522837bfe","Type":"ContainerStarted","Data":"a573a5980e6a33ccc41d595650ffe5beabc32f0d71e6561f63bd8f4e90809e8b"}
Jul 24 23:29:22.213572 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.213531    2558 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 23:29:22.322838 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.322743    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mhq9b"
Jul 24 23:29:22.336556 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.336509    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5"
Jul 24 23:29:22.347693 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.347668    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-x8chk"
Jul 24 23:29:22.354671 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.354648    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-pck26"
Jul 24 23:29:22.362700 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.362680    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rf4fp"
Jul 24 23:29:22.371381 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.371360    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-hjb7w"
Jul 24 23:29:22.380273 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.380251    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7f7qw"
Jul 24 23:29:22.389129 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.389104    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8llw9"
Jul 24 23:29:22.396036 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.396017    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kbtxb"
Jul 24 23:29:22.660408 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.660316    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/68864bf3-c771-4894-9f14-52ee9e8052f9-original-pull-secret\") pod \"global-pull-secret-syncer-rplqt\" (UID: \"68864bf3-c771-4894-9f14-52ee9e8052f9\") " pod="kube-system/global-pull-secret-syncer-rplqt"
Jul 24 23:29:22.660408 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.660364    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dcf99aef-00e1-4d16-94ca-12050dc95547-metrics-certs\") pod \"network-metrics-daemon-54fxx\" (UID: \"dcf99aef-00e1-4d16-94ca-12050dc95547\") " pod="openshift-multus/network-metrics-daemon-54fxx"
Jul 24 23:29:22.660665 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:22.660455    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 23:29:22.660665 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:22.660496    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 23:29:22.660665 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:22.660565    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68864bf3-c771-4894-9f14-52ee9e8052f9-original-pull-secret podName:68864bf3-c771-4894-9f14-52ee9e8052f9 nodeName:}" failed. No retries permitted until 2026-07-24 23:29:23.660522702 +0000 UTC m=+4.046953412 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/68864bf3-c771-4894-9f14-52ee9e8052f9-original-pull-secret") pod "global-pull-secret-syncer-rplqt" (UID: "68864bf3-c771-4894-9f14-52ee9e8052f9") : object "kube-system"/"original-pull-secret" not registered
Jul 24 23:29:22.660665 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:22.660591    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcf99aef-00e1-4d16-94ca-12050dc95547-metrics-certs podName:dcf99aef-00e1-4d16-94ca-12050dc95547 nodeName:}" failed. No retries permitted until 2026-07-24 23:29:23.660579094 +0000 UTC m=+4.047009803 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dcf99aef-00e1-4d16-94ca-12050dc95547-metrics-certs") pod "network-metrics-daemon-54fxx" (UID: "dcf99aef-00e1-4d16-94ca-12050dc95547") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 23:29:22.761091 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:22.761052    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpcqp\" (UniqueName: \"kubernetes.io/projected/7b7b9a28-3166-4af3-b27f-1ab5795c5f92-kube-api-access-bpcqp\") pod \"network-check-target-gltb5\" (UID: \"7b7b9a28-3166-4af3-b27f-1ab5795c5f92\") " pod="openshift-network-diagnostics/network-check-target-gltb5"
Jul 24 23:29:22.761272 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:22.761243    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 23:29:22.761272 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:22.761265    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 23:29:22.761351 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:22.761275    2558 projected.go:196] Error preparing data for projected volume kube-api-access-bpcqp for pod openshift-network-diagnostics/network-check-target-gltb5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 23:29:22.761351 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:22.761332    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7b7b9a28-3166-4af3-b27f-1ab5795c5f92-kube-api-access-bpcqp podName:7b7b9a28-3166-4af3-b27f-1ab5795c5f92 nodeName:}" failed. No retries permitted until 2026-07-24 23:29:23.761317683 +0000 UTC m=+4.147748390 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-bpcqp" (UniqueName: "kubernetes.io/projected/7b7b9a28-3166-4af3-b27f-1ab5795c5f92-kube-api-access-bpcqp") pod "network-check-target-gltb5" (UID: "7b7b9a28-3166-4af3-b27f-1ab5795c5f92") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 23:29:23.074686 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:23.074588    2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 23:24:22 +0000 UTC" deadline="2028-04-22 21:49:24.33398203 +0000 UTC"
Jul 24 23:29:23.074686 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:23.074635    2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15310h20m1.259353275s"
Jul 24 23:29:23.219850 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:23.219822    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d4690c2_2cef_4ede_8db1_fe6d22add6e2.slice/crio-705c8f266825a6d2b3aae6a5dced4b17254b825948a174ea65b37f1249b37af0 WatchSource:0}: Error finding container 705c8f266825a6d2b3aae6a5dced4b17254b825948a174ea65b37f1249b37af0: Status 404 returned error can't find the container with id 705c8f266825a6d2b3aae6a5dced4b17254b825948a174ea65b37f1249b37af0
Jul 24 23:29:23.222449 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:23.222419    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod725f7657_c52d_46d2_bf7d_bc6d045cb903.slice/crio-5c4111c6e4315907b02c973afbaa39bce138cd76a80b434758c09055a5276adb WatchSource:0}: Error finding container 5c4111c6e4315907b02c973afbaa39bce138cd76a80b434758c09055a5276adb: Status 404 returned error can't find the container with id 5c4111c6e4315907b02c973afbaa39bce138cd76a80b434758c09055a5276adb
Jul 24 23:29:23.226604 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:23.226166    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9dc9bac0_7bd8_4756_9a96_25594b1e3bd2.slice/crio-8eac1c796e2663dfa23e062864459205bc658b4441c2ca247c991115bbccd05a WatchSource:0}: Error finding container 8eac1c796e2663dfa23e062864459205bc658b4441c2ca247c991115bbccd05a: Status 404 returned error can't find the container with id 8eac1c796e2663dfa23e062864459205bc658b4441c2ca247c991115bbccd05a
Jul 24 23:29:23.228789 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:23.228697    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3504a0bb_6580_4187_8f2d_7f71494675e4.slice/crio-7cad27ef8dd5e40d1bcf3af111fff4e11e0281d6da0b21482e7a391481365247 WatchSource:0}: Error finding container 7cad27ef8dd5e40d1bcf3af111fff4e11e0281d6da0b21482e7a391481365247: Status 404 returned error can't find the container with id 7cad27ef8dd5e40d1bcf3af111fff4e11e0281d6da0b21482e7a391481365247
Jul 24 23:29:23.230856 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:23.230818    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ad632db_cb9d_460c_885f_0c4c1f495a1a.slice/crio-4ea354a7346ac3f18da154699ff056ae3dc762c91fd0050e8bcc9729abc41f4f WatchSource:0}: Error finding container 4ea354a7346ac3f18da154699ff056ae3dc762c91fd0050e8bcc9729abc41f4f: Status 404 returned error can't find the container with id 4ea354a7346ac3f18da154699ff056ae3dc762c91fd0050e8bcc9729abc41f4f
Jul 24 23:29:23.231649 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:23.231628    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e81d551_3cb1_4707_998f_713e083fefe8.slice/crio-311f5c4764c08bcf1407603cf464d3bd5e4d021c5a448843331542dfa94d0c2b WatchSource:0}: Error finding container 311f5c4764c08bcf1407603cf464d3bd5e4d021c5a448843331542dfa94d0c2b: Status 404 returned error can't find the container with id 311f5c4764c08bcf1407603cf464d3bd5e4d021c5a448843331542dfa94d0c2b
Jul 24 23:29:23.233383 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:23.233330    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod334638da_1923_4fd5_88b4_8ca936adf513.slice/crio-e54ff574c89886e30ff9bac2173e438af68091c6b90d296f3c34fa552ddd6f2a WatchSource:0}: Error finding container e54ff574c89886e30ff9bac2173e438af68091c6b90d296f3c34fa552ddd6f2a: Status 404 returned error can't find the container with id e54ff574c89886e30ff9bac2173e438af68091c6b90d296f3c34fa552ddd6f2a
Jul 24 23:29:23.233759 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:23.233738    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03bb2978_3f29_4167_8b41_c257a2b3878d.slice/crio-56ce010cb0a930ccfdd2e7d1246d900543a47ac685caabc2f330e22e6e7c4d6c WatchSource:0}: Error finding container 56ce010cb0a930ccfdd2e7d1246d900543a47ac685caabc2f330e22e6e7c4d6c: Status 404 returned error can't find the container with id 56ce010cb0a930ccfdd2e7d1246d900543a47ac685caabc2f330e22e6e7c4d6c
Jul 24 23:29:23.236530 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:29:23.236447    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77f6315f_d3d1_47ea_a83d_2b036ac39189.slice/crio-556acec21894a0049d3eacc311d0c61c6ff0eb93ffaefbda5fca9d84e0cc5784 WatchSource:0}: Error finding container 556acec21894a0049d3eacc311d0c61c6ff0eb93ffaefbda5fca9d84e0cc5784: Status 404 returned error can't find the container with id 556acec21894a0049d3eacc311d0c61c6ff0eb93ffaefbda5fca9d84e0cc5784
Jul 24 23:29:23.668417 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:23.668091    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/68864bf3-c771-4894-9f14-52ee9e8052f9-original-pull-secret\") pod \"global-pull-secret-syncer-rplqt\" (UID: \"68864bf3-c771-4894-9f14-52ee9e8052f9\") " pod="kube-system/global-pull-secret-syncer-rplqt"
Jul 24 23:29:23.668417 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:23.668364    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dcf99aef-00e1-4d16-94ca-12050dc95547-metrics-certs\") pod \"network-metrics-daemon-54fxx\" (UID: \"dcf99aef-00e1-4d16-94ca-12050dc95547\") " pod="openshift-multus/network-metrics-daemon-54fxx"
Jul 24 23:29:23.668662 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:23.668418    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 23:29:23.668662 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:23.668554    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68864bf3-c771-4894-9f14-52ee9e8052f9-original-pull-secret podName:68864bf3-c771-4894-9f14-52ee9e8052f9 nodeName:}" failed. No retries permitted until 2026-07-24 23:29:25.668527313 +0000 UTC m=+6.054958025 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/68864bf3-c771-4894-9f14-52ee9e8052f9-original-pull-secret") pod "global-pull-secret-syncer-rplqt" (UID: "68864bf3-c771-4894-9f14-52ee9e8052f9") : object "kube-system"/"original-pull-secret" not registered
Jul 24 23:29:23.668662 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:23.668579    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 23:29:23.668824 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:23.668670    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcf99aef-00e1-4d16-94ca-12050dc95547-metrics-certs podName:dcf99aef-00e1-4d16-94ca-12050dc95547 nodeName:}" failed. No retries permitted until 2026-07-24 23:29:25.668651891 +0000 UTC m=+6.055082604 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dcf99aef-00e1-4d16-94ca-12050dc95547-metrics-certs") pod "network-metrics-daemon-54fxx" (UID: "dcf99aef-00e1-4d16-94ca-12050dc95547") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 23:29:23.769395 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:23.769222    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpcqp\" (UniqueName: \"kubernetes.io/projected/7b7b9a28-3166-4af3-b27f-1ab5795c5f92-kube-api-access-bpcqp\") pod \"network-check-target-gltb5\" (UID: \"7b7b9a28-3166-4af3-b27f-1ab5795c5f92\") " pod="openshift-network-diagnostics/network-check-target-gltb5"
Jul 24 23:29:23.769603 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:23.769407    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 23:29:23.769603 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:23.769429    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 23:29:23.769603 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:23.769439    2558 projected.go:196] Error preparing data for projected volume kube-api-access-bpcqp for pod openshift-network-diagnostics/network-check-target-gltb5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 23:29:23.769603 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:23.769504    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7b7b9a28-3166-4af3-b27f-1ab5795c5f92-kube-api-access-bpcqp podName:7b7b9a28-3166-4af3-b27f-1ab5795c5f92 nodeName:}" failed. No retries permitted until 2026-07-24 23:29:25.769482133 +0000 UTC m=+6.155912856 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-bpcqp" (UniqueName: "kubernetes.io/projected/7b7b9a28-3166-4af3-b27f-1ab5795c5f92-kube-api-access-bpcqp") pod "network-check-target-gltb5" (UID: "7b7b9a28-3166-4af3-b27f-1ab5795c5f92") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 23:29:24.075105 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:24.074945    2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 23:24:22 +0000 UTC" deadline="2028-01-04 12:53:41.770803174 +0000 UTC"
Jul 24 23:29:24.075105 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:24.075012    2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12685h24m17.695795394s"
Jul 24 23:29:24.200516 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:24.199744    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rplqt"
Jul 24 23:29:24.200516 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:24.199883    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rplqt" podUID="68864bf3-c771-4894-9f14-52ee9e8052f9"
Jul 24 23:29:24.200516 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:24.200311    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gltb5"
Jul 24 23:29:24.200516 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:24.200409    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gltb5" podUID="7b7b9a28-3166-4af3-b27f-1ab5795c5f92"
Jul 24 23:29:24.201210 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:24.201047    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-54fxx"
Jul 24 23:29:24.201210 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:24.201163    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-54fxx" podUID="dcf99aef-00e1-4d16-94ca-12050dc95547"
Jul 24 23:29:24.228341 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:24.228302    2558 generic.go:356] "Generic (PLEG): container finished" podID="4e4ac0d94c86ae9a285d0e7940c49e40" containerID="f25c5a8cfe91ca6c8846b2f9c764099f6629d0308d576cbeec959cbc74547c75" exitCode=0
Jul 24 23:29:24.228496 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:24.228409    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-133.ec2.internal" event={"ID":"4e4ac0d94c86ae9a285d0e7940c49e40","Type":"ContainerDied","Data":"f25c5a8cfe91ca6c8846b2f9c764099f6629d0308d576cbeec959cbc74547c75"}
Jul 24 23:29:24.232850 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:24.232748    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8llw9" event={"ID":"334638da-1923-4fd5-88b4-8ca936adf513","Type":"ContainerStarted","Data":"e54ff574c89886e30ff9bac2173e438af68091c6b90d296f3c34fa552ddd6f2a"}
Jul 24 23:29:24.236711 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:24.236622    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pck26" event={"ID":"03bb2978-3f29-4167-8b41-c257a2b3878d","Type":"ContainerStarted","Data":"56ce010cb0a930ccfdd2e7d1246d900543a47ac685caabc2f330e22e6e7c4d6c"}
Jul 24 23:29:24.241320 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:24.241292    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rf4fp" event={"ID":"5e81d551-3cb1-4707-998f-713e083fefe8","Type":"ContainerStarted","Data":"311f5c4764c08bcf1407603cf464d3bd5e4d021c5a448843331542dfa94d0c2b"}
Jul 24 23:29:24.245473 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:24.245445    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mhq9b" event={"ID":"9dc9bac0-7bd8-4756-9a96-25594b1e3bd2","Type":"ContainerStarted","Data":"8eac1c796e2663dfa23e062864459205bc658b4441c2ca247c991115bbccd05a"}
Jul 24 23:29:24.251717 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:24.251689    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5" event={"ID":"725f7657-c52d-46d2-bf7d-bc6d045cb903","Type":"ContainerStarted","Data":"5c4111c6e4315907b02c973afbaa39bce138cd76a80b434758c09055a5276adb"}
Jul 24 23:29:24.262496 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:24.262467    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-133.ec2.internal" event={"ID":"b8b5e88d55d068e81c55bbf522837bfe","Type":"ContainerStarted","Data":"0588a9b2a90b39f363271c9309e18b9edfa5299c352ccaf3dc60e6a846ad322e"}
Jul 24 23:29:24.267625 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:24.267592    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-x8chk" event={"ID":"77f6315f-d3d1-47ea-a83d-2b036ac39189","Type":"ContainerStarted","Data":"556acec21894a0049d3eacc311d0c61c6ff0eb93ffaefbda5fca9d84e0cc5784"}
Jul 24 23:29:24.270828 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:24.270793    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kbtxb" event={"ID":"6ad632db-cb9d-460c-885f-0c4c1f495a1a","Type":"ContainerStarted","Data":"4ea354a7346ac3f18da154699ff056ae3dc762c91fd0050e8bcc9729abc41f4f"}
Jul 24 23:29:24.272790 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:24.272766    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7f7qw" event={"ID":"3504a0bb-6580-4187-8f2d-7f71494675e4","Type":"ContainerStarted","Data":"7cad27ef8dd5e40d1bcf3af111fff4e11e0281d6da0b21482e7a391481365247"}
Jul 24 23:29:24.277772 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:24.277729    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-hjb7w" event={"ID":"1d4690c2-2cef-4ede-8db1-fe6d22add6e2","Type":"ContainerStarted","Data":"705c8f266825a6d2b3aae6a5dced4b17254b825948a174ea65b37f1249b37af0"}
Jul 24 23:29:25.300719 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:25.299919    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-133.ec2.internal" event={"ID":"4e4ac0d94c86ae9a285d0e7940c49e40","Type":"ContainerStarted","Data":"71b3d20ebc14e5e8bec965d460628d2fb6f57a2b45afd7c8aa9e27990b6467c0"}
Jul 24 23:29:25.316399 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:25.314895    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-133.ec2.internal" podStartSLOduration=3.314872949 podStartE2EDuration="3.314872949s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 23:29:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:29:25.314748424 +0000 UTC m=+5.701179156" watchObservedRunningTime="2026-07-24 23:29:25.314872949 +0000 UTC m=+5.701303661"
Jul 24 23:29:25.316399 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:25.315840    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-133.ec2.internal" podStartSLOduration=3.315825968 podStartE2EDuration="3.315825968s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 23:29:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:29:24.276835725 +0000 UTC m=+4.663266457" watchObservedRunningTime="2026-07-24 23:29:25.315825968 +0000 UTC m=+5.702256701"
Jul 24 23:29:25.690227 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:25.690131    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/68864bf3-c771-4894-9f14-52ee9e8052f9-original-pull-secret\") pod \"global-pull-secret-syncer-rplqt\" (UID: \"68864bf3-c771-4894-9f14-52ee9e8052f9\") " pod="kube-system/global-pull-secret-syncer-rplqt"
Jul 24 23:29:25.690227 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:25.690215    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dcf99aef-00e1-4d16-94ca-12050dc95547-metrics-certs\") pod \"network-metrics-daemon-54fxx\" (UID: \"dcf99aef-00e1-4d16-94ca-12050dc95547\") " pod="openshift-multus/network-metrics-daemon-54fxx"
Jul 24 23:29:25.690499 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:25.690356    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 23:29:25.690499 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:25.690423    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcf99aef-00e1-4d16-94ca-12050dc95547-metrics-certs podName:dcf99aef-00e1-4d16-94ca-12050dc95547 nodeName:}" failed. No retries permitted until 2026-07-24 23:29:29.690403726 +0000 UTC m=+10.076834439 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dcf99aef-00e1-4d16-94ca-12050dc95547-metrics-certs") pod "network-metrics-daemon-54fxx" (UID: "dcf99aef-00e1-4d16-94ca-12050dc95547") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 23:29:25.690737 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:25.690718    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 23:29:25.690836 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:25.690773    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68864bf3-c771-4894-9f14-52ee9e8052f9-original-pull-secret podName:68864bf3-c771-4894-9f14-52ee9e8052f9 nodeName:}" failed. No retries permitted until 2026-07-24 23:29:29.690758229 +0000 UTC m=+10.077188939 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/68864bf3-c771-4894-9f14-52ee9e8052f9-original-pull-secret") pod "global-pull-secret-syncer-rplqt" (UID: "68864bf3-c771-4894-9f14-52ee9e8052f9") : object "kube-system"/"original-pull-secret" not registered
Jul 24 23:29:25.790741 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:25.790697    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpcqp\" (UniqueName: \"kubernetes.io/projected/7b7b9a28-3166-4af3-b27f-1ab5795c5f92-kube-api-access-bpcqp\") pod \"network-check-target-gltb5\" (UID: \"7b7b9a28-3166-4af3-b27f-1ab5795c5f92\") " pod="openshift-network-diagnostics/network-check-target-gltb5"
Jul 24 23:29:25.790957 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:25.790938    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 23:29:25.791049 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:25.790967    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 23:29:25.791049 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:25.790981    2558 projected.go:196] Error preparing data for projected volume kube-api-access-bpcqp for pod openshift-network-diagnostics/network-check-target-gltb5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 23:29:25.791162 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:25.791058    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7b7b9a28-3166-4af3-b27f-1ab5795c5f92-kube-api-access-bpcqp podName:7b7b9a28-3166-4af3-b27f-1ab5795c5f92 nodeName:}" failed. No retries permitted until 2026-07-24 23:29:29.791028091 +0000 UTC m=+10.177458816 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-bpcqp" (UniqueName: "kubernetes.io/projected/7b7b9a28-3166-4af3-b27f-1ab5795c5f92-kube-api-access-bpcqp") pod "network-check-target-gltb5" (UID: "7b7b9a28-3166-4af3-b27f-1ab5795c5f92") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 23:29:26.202486 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:26.201470    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gltb5"
Jul 24 23:29:26.202486 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:26.201690    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gltb5" podUID="7b7b9a28-3166-4af3-b27f-1ab5795c5f92"
Jul 24 23:29:26.202486 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:26.202166    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-54fxx"
Jul 24 23:29:26.202486 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:26.202287    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-54fxx" podUID="dcf99aef-00e1-4d16-94ca-12050dc95547"
Jul 24 23:29:26.202486 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:26.202349    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rplqt"
Jul 24 23:29:26.202486 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:26.202427    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rplqt" podUID="68864bf3-c771-4894-9f14-52ee9e8052f9"
Jul 24 23:29:28.198879 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:28.198845    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gltb5"
Jul 24 23:29:28.198879 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:28.198882    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rplqt"
Jul 24 23:29:28.199443 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:28.198850    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-54fxx"
Jul 24 23:29:28.199443 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:28.198984    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gltb5" podUID="7b7b9a28-3166-4af3-b27f-1ab5795c5f92"
Jul 24 23:29:28.199443 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:28.199102    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-54fxx" podUID="dcf99aef-00e1-4d16-94ca-12050dc95547"
Jul 24 23:29:28.199443 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:28.199203    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rplqt" podUID="68864bf3-c771-4894-9f14-52ee9e8052f9"
Jul 24 23:29:29.727408 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:29.727365    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/68864bf3-c771-4894-9f14-52ee9e8052f9-original-pull-secret\") pod \"global-pull-secret-syncer-rplqt\" (UID: \"68864bf3-c771-4894-9f14-52ee9e8052f9\") " pod="kube-system/global-pull-secret-syncer-rplqt"
Jul 24 23:29:29.727970 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:29.727426    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dcf99aef-00e1-4d16-94ca-12050dc95547-metrics-certs\") pod \"network-metrics-daemon-54fxx\" (UID: \"dcf99aef-00e1-4d16-94ca-12050dc95547\") " pod="openshift-multus/network-metrics-daemon-54fxx"
Jul 24 23:29:29.727970 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:29.727549    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 23:29:29.727970 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:29.727600    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 23:29:29.727970 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:29.727632    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68864bf3-c771-4894-9f14-52ee9e8052f9-original-pull-secret podName:68864bf3-c771-4894-9f14-52ee9e8052f9 nodeName:}" failed. No retries permitted until 2026-07-24 23:29:37.727610083 +0000 UTC m=+18.114040812 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/68864bf3-c771-4894-9f14-52ee9e8052f9-original-pull-secret") pod "global-pull-secret-syncer-rplqt" (UID: "68864bf3-c771-4894-9f14-52ee9e8052f9") : object "kube-system"/"original-pull-secret" not registered
Jul 24 23:29:29.727970 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:29.727657    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcf99aef-00e1-4d16-94ca-12050dc95547-metrics-certs podName:dcf99aef-00e1-4d16-94ca-12050dc95547 nodeName:}" failed. No retries permitted until 2026-07-24 23:29:37.727645818 +0000 UTC m=+18.114076532 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dcf99aef-00e1-4d16-94ca-12050dc95547-metrics-certs") pod "network-metrics-daemon-54fxx" (UID: "dcf99aef-00e1-4d16-94ca-12050dc95547") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 23:29:29.828262 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:29.828213    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpcqp\" (UniqueName: \"kubernetes.io/projected/7b7b9a28-3166-4af3-b27f-1ab5795c5f92-kube-api-access-bpcqp\") pod \"network-check-target-gltb5\" (UID: \"7b7b9a28-3166-4af3-b27f-1ab5795c5f92\") " pod="openshift-network-diagnostics/network-check-target-gltb5"
Jul 24 23:29:29.828442 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:29.828398    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 23:29:29.828442 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:29.828422    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 23:29:29.828442 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:29.828433    2558 projected.go:196] Error preparing data for projected volume kube-api-access-bpcqp for pod openshift-network-diagnostics/network-check-target-gltb5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 23:29:29.828618 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:29.828480    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7b7b9a28-3166-4af3-b27f-1ab5795c5f92-kube-api-access-bpcqp podName:7b7b9a28-3166-4af3-b27f-1ab5795c5f92 nodeName:}" failed. No retries permitted until 2026-07-24 23:29:37.828466523 +0000 UTC m=+18.214897235 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-bpcqp" (UniqueName: "kubernetes.io/projected/7b7b9a28-3166-4af3-b27f-1ab5795c5f92-kube-api-access-bpcqp") pod "network-check-target-gltb5" (UID: "7b7b9a28-3166-4af3-b27f-1ab5795c5f92") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 23:29:30.200213 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:30.200051    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-54fxx"
Jul 24 23:29:30.200213 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:30.200157    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rplqt"
Jul 24 23:29:30.200213 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:30.200199    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gltb5"
Jul 24 23:29:30.200484 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:30.200231    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-54fxx" podUID="dcf99aef-00e1-4d16-94ca-12050dc95547"
Jul 24 23:29:30.200484 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:30.200278    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gltb5" podUID="7b7b9a28-3166-4af3-b27f-1ab5795c5f92"
Jul 24 23:29:30.200484 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:30.200359    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rplqt" podUID="68864bf3-c771-4894-9f14-52ee9e8052f9"
Jul 24 23:29:32.198694 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:32.198660    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rplqt"
Jul 24 23:29:32.199158 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:32.198658    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-54fxx"
Jul 24 23:29:32.199158 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:32.198794    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rplqt" podUID="68864bf3-c771-4894-9f14-52ee9e8052f9"
Jul 24 23:29:32.199158 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:32.198883    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-54fxx" podUID="dcf99aef-00e1-4d16-94ca-12050dc95547"
Jul 24 23:29:32.199158 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:32.198658    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gltb5"
Jul 24 23:29:32.199158 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:32.199001    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gltb5" podUID="7b7b9a28-3166-4af3-b27f-1ab5795c5f92"
Jul 24 23:29:34.198831 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:34.198794    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gltb5"
Jul 24 23:29:34.198831 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:34.198815    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-54fxx"
Jul 24 23:29:34.199302 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:34.198815    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rplqt"
Jul 24 23:29:34.199302 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:34.198889    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gltb5" podUID="7b7b9a28-3166-4af3-b27f-1ab5795c5f92"
Jul 24 23:29:34.199302 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:34.198992    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-54fxx" podUID="dcf99aef-00e1-4d16-94ca-12050dc95547"
Jul 24 23:29:34.199302 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:34.199055    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rplqt" podUID="68864bf3-c771-4894-9f14-52ee9e8052f9"
Jul 24 23:29:36.199008 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:36.198959    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-54fxx"
Jul 24 23:29:36.199008 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:36.198992    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rplqt"
Jul 24 23:29:36.199524 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:36.198960    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gltb5"
Jul 24 23:29:36.199524 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:36.199104    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-54fxx" podUID="dcf99aef-00e1-4d16-94ca-12050dc95547"
Jul 24 23:29:36.199524 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:36.199218    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gltb5" podUID="7b7b9a28-3166-4af3-b27f-1ab5795c5f92"
Jul 24 23:29:36.199524 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:36.199301    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rplqt" podUID="68864bf3-c771-4894-9f14-52ee9e8052f9"
Jul 24 23:29:37.793190 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:37.793147    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/68864bf3-c771-4894-9f14-52ee9e8052f9-original-pull-secret\") pod \"global-pull-secret-syncer-rplqt\" (UID: \"68864bf3-c771-4894-9f14-52ee9e8052f9\") " pod="kube-system/global-pull-secret-syncer-rplqt"
Jul 24 23:29:37.793705 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:37.793202    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dcf99aef-00e1-4d16-94ca-12050dc95547-metrics-certs\") pod \"network-metrics-daemon-54fxx\" (UID: \"dcf99aef-00e1-4d16-94ca-12050dc95547\") " pod="openshift-multus/network-metrics-daemon-54fxx"
Jul 24 23:29:37.793705 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:37.793301    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 23:29:37.793705 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:37.793370    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68864bf3-c771-4894-9f14-52ee9e8052f9-original-pull-secret podName:68864bf3-c771-4894-9f14-52ee9e8052f9 nodeName:}" failed. No retries permitted until 2026-07-24 23:29:53.793353086 +0000 UTC m=+34.179783792 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/68864bf3-c771-4894-9f14-52ee9e8052f9-original-pull-secret") pod "global-pull-secret-syncer-rplqt" (UID: "68864bf3-c771-4894-9f14-52ee9e8052f9") : object "kube-system"/"original-pull-secret" not registered
Jul 24 23:29:37.793705 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:37.793310    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 23:29:37.793705 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:37.793458    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcf99aef-00e1-4d16-94ca-12050dc95547-metrics-certs podName:dcf99aef-00e1-4d16-94ca-12050dc95547 nodeName:}" failed. No retries permitted until 2026-07-24 23:29:53.793438723 +0000 UTC m=+34.179869449 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dcf99aef-00e1-4d16-94ca-12050dc95547-metrics-certs") pod "network-metrics-daemon-54fxx" (UID: "dcf99aef-00e1-4d16-94ca-12050dc95547") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 23:29:37.893993 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:37.893955    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpcqp\" (UniqueName: \"kubernetes.io/projected/7b7b9a28-3166-4af3-b27f-1ab5795c5f92-kube-api-access-bpcqp\") pod \"network-check-target-gltb5\" (UID: \"7b7b9a28-3166-4af3-b27f-1ab5795c5f92\") " pod="openshift-network-diagnostics/network-check-target-gltb5"
Jul 24 23:29:37.894145 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:37.894114    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 23:29:37.894145 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:37.894134    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 23:29:37.894145 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:37.894143    2558 projected.go:196] Error preparing data for projected volume kube-api-access-bpcqp for pod openshift-network-diagnostics/network-check-target-gltb5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 23:29:37.894237 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:37.894198    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7b7b9a28-3166-4af3-b27f-1ab5795c5f92-kube-api-access-bpcqp podName:7b7b9a28-3166-4af3-b27f-1ab5795c5f92 nodeName:}" failed. No retries permitted until 2026-07-24 23:29:53.894182746 +0000 UTC m=+34.280613474 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-bpcqp" (UniqueName: "kubernetes.io/projected/7b7b9a28-3166-4af3-b27f-1ab5795c5f92-kube-api-access-bpcqp") pod "network-check-target-gltb5" (UID: "7b7b9a28-3166-4af3-b27f-1ab5795c5f92") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 23:29:38.199209 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:38.199105    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rplqt"
Jul 24 23:29:38.199377 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:38.199105    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gltb5"
Jul 24 23:29:38.199377 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:38.199243    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rplqt" podUID="68864bf3-c771-4894-9f14-52ee9e8052f9"
Jul 24 23:29:38.199377 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:38.199345    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gltb5" podUID="7b7b9a28-3166-4af3-b27f-1ab5795c5f92"
Jul 24 23:29:38.199377 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:38.199105    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-54fxx"
Jul 24 23:29:38.199562 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:38.199450    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-54fxx" podUID="dcf99aef-00e1-4d16-94ca-12050dc95547"
Jul 24 23:29:40.200066 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:40.200030    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-54fxx"
Jul 24 23:29:40.200625 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:40.200070    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rplqt"
Jul 24 23:29:40.200625 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:40.200163    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-54fxx" podUID="dcf99aef-00e1-4d16-94ca-12050dc95547"
Jul 24 23:29:40.200625 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:40.200239    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rplqt" podUID="68864bf3-c771-4894-9f14-52ee9e8052f9"
Jul 24 23:29:40.200625 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:40.200263    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gltb5"
Jul 24 23:29:40.200625 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:40.200367    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gltb5" podUID="7b7b9a28-3166-4af3-b27f-1ab5795c5f92"
Jul 24 23:29:42.199771 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:42.198949    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-54fxx"
Jul 24 23:29:42.199771 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:42.199161    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-54fxx" podUID="dcf99aef-00e1-4d16-94ca-12050dc95547"
Jul 24 23:29:42.199771 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:42.199498    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rplqt"
Jul 24 23:29:42.199771 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:42.199687    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rplqt" podUID="68864bf3-c771-4894-9f14-52ee9e8052f9"
Jul 24 23:29:42.200734 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:42.199850    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gltb5"
Jul 24 23:29:42.200734 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:42.199942    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gltb5" podUID="7b7b9a28-3166-4af3-b27f-1ab5795c5f92"
Jul 24 23:29:42.343596 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:42.343369    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-x8chk" event={"ID":"77f6315f-d3d1-47ea-a83d-2b036ac39189","Type":"ContainerStarted","Data":"3c0e49c65cff09a16d6a2aa5b5c18bc81a2aadd0e61ed913492380e7fdcd22f2"}
Jul 24 23:29:42.344865 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:42.344837    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7f7qw" event={"ID":"3504a0bb-6580-4187-8f2d-7f71494675e4","Type":"ContainerStarted","Data":"3804d0141391c718d5f402c0ec7a5fe9af45c6133f0d31722f68e8bb49383631"}
Jul 24 23:29:42.346373 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:42.346346    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-hjb7w" event={"ID":"1d4690c2-2cef-4ede-8db1-fe6d22add6e2","Type":"ContainerStarted","Data":"f4a764817a6e9a6046c55d7e9b13a884dd790827a75bac5b233c0e9198757590"}
Jul 24 23:29:42.347846 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:42.347825    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8llw9" event={"ID":"334638da-1923-4fd5-88b4-8ca936adf513","Type":"ContainerStarted","Data":"16a54d8a2101c1652fdf28eb985c3356374072bec90a738c8f4cb4f6d79058f0"}
Jul 24 23:29:42.349157 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:42.349137    2558 generic.go:356] "Generic (PLEG): container finished" podID="03bb2978-3f29-4167-8b41-c257a2b3878d" containerID="d1cc7097738ec96db32db416fa8834420cb1b94eda3b9a3d58950d8d200a8e52" exitCode=0
Jul 24 23:29:42.349241 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:42.349220    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pck26" event={"ID":"03bb2978-3f29-4167-8b41-c257a2b3878d","Type":"ContainerDied","Data":"d1cc7097738ec96db32db416fa8834420cb1b94eda3b9a3d58950d8d200a8e52"}
Jul 24 23:29:42.350650 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:42.350632    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rf4fp" event={"ID":"5e81d551-3cb1-4707-998f-713e083fefe8","Type":"ContainerStarted","Data":"b236008d0d62ccbfbcb3a066945601e691de1f737e3793e873c5ed96c871419a"}
Jul 24 23:29:42.352665 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:42.352132    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mhq9b" event={"ID":"9dc9bac0-7bd8-4756-9a96-25594b1e3bd2","Type":"ContainerStarted","Data":"571988507cc7aa7fff164f36aa07b73d015fb1648971f9a2d1bd3d91cace515e"}
Jul 24 23:29:42.353984 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:42.353962    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5" event={"ID":"725f7657-c52d-46d2-bf7d-bc6d045cb903","Type":"ContainerStarted","Data":"c16634484cfe1d841474392276149dcdd7fdf1ad6d1c3f1d68ab74e537659b5d"}
Jul 24 23:29:42.354132 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:42.354037    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5" event={"ID":"725f7657-c52d-46d2-bf7d-bc6d045cb903","Type":"ContainerStarted","Data":"2487830eb559ab3a7e359b89428e82f611adce170ee20f9ba2b2f77b4a40f910"}
Jul 24 23:29:42.357862 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:42.357825    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-x8chk" podStartSLOduration=2.648184363 podStartE2EDuration="21.357813904s" totalImagesPullingTime="18.709629541s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 23:29:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:29:42.357618716 +0000 UTC m=+22.744049454" watchObservedRunningTime="2026-07-24 23:29:42.357813904 +0000 UTC m=+22.744244637"
Jul 24 23:29:42.372148 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:42.372109    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-mhq9b" podStartSLOduration=2.6504818390000002 podStartE2EDuration="21.372095969s" totalImagesPullingTime="18.72161413s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 23:29:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:29:42.372016858 +0000 UTC m=+22.758447588" watchObservedRunningTime="2026-07-24 23:29:42.372095969 +0000 UTC m=+22.758526696"
Jul 24 23:29:42.387615 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:42.387576    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-hjb7w" podStartSLOduration=6.742928671 podStartE2EDuration="21.387562497s" totalImagesPullingTime="14.644633826s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:29:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:29:42.387306843 +0000 UTC m=+22.773737571" watchObservedRunningTime="2026-07-24 23:29:42.387562497 +0000 UTC m=+22.773993224"
Jul 24 23:29:42.400338 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:42.400299    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-7f7qw" podStartSLOduration=2.682766547 podStartE2EDuration="21.400286038s" totalImagesPullingTime="18.717519491s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 23:29:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:29:42.399922121 +0000 UTC m=+22.786352848" watchObservedRunningTime="2026-07-24 23:29:42.400286038 +0000 UTC m=+22.786716766"
Jul 24 23:29:42.435740 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:42.435695    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-rf4fp" podStartSLOduration=2.662485433 podStartE2EDuration="21.435679785s" totalImagesPullingTime="18.773194352s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 23:29:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:29:42.435419034 +0000 UTC m=+22.821849763" watchObservedRunningTime="2026-07-24 23:29:42.435679785 +0000 UTC m=+22.822110513"
Jul 24 23:29:42.576522 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:42.576495    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:42.658327 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:42.657015    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:42.756045 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:42.756014    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:42.929287 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:42.929067    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:43.029202 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:43.029116    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:43.361925 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:43.361884    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5" event={"ID":"725f7657-c52d-46d2-bf7d-bc6d045cb903","Type":"ContainerStarted","Data":"adf95479c3759d872182083017aee6f3d69695e3d692e8e803eb1ff8b3594f5e"}
Jul 24 23:29:43.362343 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:43.362020    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5" event={"ID":"725f7657-c52d-46d2-bf7d-bc6d045cb903","Type":"ContainerStarted","Data":"8b576acb8afe11c4f9e158bd709e518b5abf83ce257ac59507e1f81892ed132f"}
Jul 24 23:29:43.362343 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:43.362068    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5" event={"ID":"725f7657-c52d-46d2-bf7d-bc6d045cb903","Type":"ContainerStarted","Data":"8b1af39e91d00b4252e34e3eed572896a9253ea8ce7a222e92e5650b70efc458"}
Jul 24 23:29:43.362343 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:43.362116    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5" event={"ID":"725f7657-c52d-46d2-bf7d-bc6d045cb903","Type":"ContainerStarted","Data":"5803f2075fe88388a94be4a4c3aa2568416c1617e4d61845bfd73200c5a22388"}
Jul 24 23:29:43.506075 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:43.506033    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:43.548469 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:43.548444    2558 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 24 23:29:43.898480 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:43.898444    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:43.953276 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:43.953236    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:44.198872 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:44.198776    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gltb5"
Jul 24 23:29:44.198872 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:44.198818    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rplqt"
Jul 24 23:29:44.199197 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:44.198776    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-54fxx"
Jul 24 23:29:44.199197 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:44.198916    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gltb5" podUID="7b7b9a28-3166-4af3-b27f-1ab5795c5f92"
Jul 24 23:29:44.199197 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:44.198993    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-54fxx" podUID="dcf99aef-00e1-4d16-94ca-12050dc95547"
Jul 24 23:29:44.199197 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:44.199071    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rplqt" podUID="68864bf3-c771-4894-9f14-52ee9e8052f9"
Jul 24 23:29:44.210281 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:44.210249    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-hjb7w"
Jul 24 23:29:44.211654 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:44.211635    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-hjb7w"
Jul 24 23:29:44.231940 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:44.231826    2558 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-24T23:29:43.548461232Z","UUID":"3b4f11d3-cabb-4384-9776-08977ff01775","Handler":null,"Name":"","Endpoint":""}
Jul 24 23:29:44.233635 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:44.233610    2558 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 24 23:29:44.233788 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:44.233654    2558 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 24 23:29:44.366191 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:44.366147    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kbtxb" event={"ID":"6ad632db-cb9d-460c-885f-0c4c1f495a1a","Type":"ContainerStarted","Data":"eb0271c53f5be46494d014f9cddca6439fad28d07314f9d0306977d1cdefb68e"}
Jul 24 23:29:44.368697 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:44.368665    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8llw9" event={"ID":"334638da-1923-4fd5-88b4-8ca936adf513","Type":"ContainerStarted","Data":"2c0bfc72a842ca99da977614db3ceaa1842f238bfbd9e558e16c9c48c54f7b6f"}
Jul 24 23:29:44.381910 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:44.381842    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-kbtxb" podStartSLOduration=4.664994585 podStartE2EDuration="23.381819998s" totalImagesPullingTime="18.716825413s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 23:29:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:29:44.381286171 +0000 UTC m=+24.767716901" watchObservedRunningTime="2026-07-24 23:29:44.381819998 +0000 UTC m=+24.768250729"
Jul 24 23:29:44.705065 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:44.705029    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:45.096544 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:45.096453    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:45.168725 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:45.168690    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:45.288324 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:45.288287    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:45.356084 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:45.355988    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:45.372236 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:45.372200    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8llw9" event={"ID":"334638da-1923-4fd5-88b4-8ca936adf513","Type":"ContainerStarted","Data":"bec2113a88f55d7a93656a5b98f228eef49c7c8db08a6bc6a129c61e806f80a8"}
Jul 24 23:29:45.375376 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:45.375353    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5" event={"ID":"725f7657-c52d-46d2-bf7d-bc6d045cb903","Type":"ContainerStarted","Data":"1d2d97f39d6cf3528a8ba7be7ad6b9bd057163e5810af2ffe917b7076481c821"}
Jul 24 23:29:45.375496 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:45.375404    2558 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 24 23:29:45.390073 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:45.390033    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8llw9" podStartSLOduration=3.408722407 podStartE2EDuration="24.390020074s" totalImagesPullingTime="20.981297667s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 23:29:21 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:29:45.389133826 +0000 UTC m=+25.775564554" watchObservedRunningTime="2026-07-24 23:29:45.390020074 +0000 UTC m=+25.776450802"
Jul 24 23:29:45.824904 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:45.824869    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-hjb7w"
Jul 24 23:29:45.825608 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:45.825585    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-hjb7w"
Jul 24 23:29:45.885881 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:45.885852    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:45.937827 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:45.937791    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:46.153316 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:46.153223    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:46.199373 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:46.199340    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gltb5"
Jul 24 23:29:46.199571 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:46.199345    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-54fxx"
Jul 24 23:29:46.199571 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:46.199469    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gltb5" podUID="7b7b9a28-3166-4af3-b27f-1ab5795c5f92"
Jul 24 23:29:46.199571 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:46.199492    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rplqt"
Jul 24 23:29:46.199740 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:46.199629    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-54fxx" podUID="dcf99aef-00e1-4d16-94ca-12050dc95547"
Jul 24 23:29:46.199740 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:46.199719    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rplqt" podUID="68864bf3-c771-4894-9f14-52ee9e8052f9"
Jul 24 23:29:46.208213 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:46.208183    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:46.318443 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:46.318410    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:46.412117 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:46.412014    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:46.512398 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:46.512358    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:46.520331 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:46.520300    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7f7qw_3504a0bb-6580-4187-8f2d-7f71494675e4/dns-node-resolver/0.log"
Jul 24 23:29:46.864047 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:46.863832    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:46.964651 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:46.964616    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:47.171596 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:47.171567    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:47.276653 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:47.276622    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:47.382672 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:47.382586    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pck26" event={"ID":"03bb2978-3f29-4167-8b41-c257a2b3878d","Type":"ContainerStarted","Data":"b5957b20baa8510e277b7f1f5d42e530b66f0d6a6db2305339b3d333ef8ca769"}
Jul 24 23:29:47.386358 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:47.386324    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5" event={"ID":"725f7657-c52d-46d2-bf7d-bc6d045cb903","Type":"ContainerStarted","Data":"cddfe78237dad70b7afe8d5ac51c6305cefbe6547489d8852d266011eba4b48e"}
Jul 24 23:29:47.430698 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:47.430650    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5" podStartSLOduration=7.643786457 podStartE2EDuration="26.430632325s" totalImagesPullingTime="18.786845868s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 23:29:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:29:47.430235846 +0000 UTC m=+27.816666584" watchObservedRunningTime="2026-07-24 23:29:47.430632325 +0000 UTC m=+27.817063036"
Jul 24 23:29:47.444229 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:47.444199    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:47.504723 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:47.504688    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:47.521089 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:47.521062    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-x8chk_77f6315f-d3d1-47ea-a83d-2b036ac39189/node-ca/0.log"
Jul 24 23:29:47.604641 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:47.604407    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:48.198477 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:48.198435    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gltb5"
Jul 24 23:29:48.198697 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:48.198435    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rplqt"
Jul 24 23:29:48.198697 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:48.198570    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gltb5" podUID="7b7b9a28-3166-4af3-b27f-1ab5795c5f92"
Jul 24 23:29:48.198697 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:48.198579    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-54fxx"
Jul 24 23:29:48.198697 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:48.198659    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rplqt" podUID="68864bf3-c771-4894-9f14-52ee9e8052f9"
Jul 24 23:29:48.198915 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:48.198742    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-54fxx" podUID="dcf99aef-00e1-4d16-94ca-12050dc95547"
Jul 24 23:29:48.390117 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:48.390087    2558 generic.go:356] "Generic (PLEG): container finished" podID="03bb2978-3f29-4167-8b41-c257a2b3878d" containerID="b5957b20baa8510e277b7f1f5d42e530b66f0d6a6db2305339b3d333ef8ca769" exitCode=0
Jul 24 23:29:48.390292 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:48.390173    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pck26" event={"ID":"03bb2978-3f29-4167-8b41-c257a2b3878d","Type":"ContainerDied","Data":"b5957b20baa8510e277b7f1f5d42e530b66f0d6a6db2305339b3d333ef8ca769"}
Jul 24 23:29:48.390292 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:48.390221    2558 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 24 23:29:48.390659 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:48.390639    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5"
Jul 24 23:29:48.390728 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:48.390718    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5"
Jul 24 23:29:48.405870 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:48.405847    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5"
Jul 24 23:29:48.405980 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:48.405940    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5"
Jul 24 23:29:48.445428 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:48.445396    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:48.509442 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:48.509410    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:49.151654 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:49.151611    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5"
Jul 24 23:29:49.153685 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:49.153657    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-rplqt"]
Jul 24 23:29:49.153835 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:49.153766    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rplqt"
Jul 24 23:29:49.153908 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:49.153880    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rplqt" podUID="68864bf3-c771-4894-9f14-52ee9e8052f9"
Jul 24 23:29:49.157223 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:49.157192    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-54fxx"]
Jul 24 23:29:49.157345 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:49.157306    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-54fxx"
Jul 24 23:29:49.157552 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:49.157434    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-54fxx" podUID="dcf99aef-00e1-4d16-94ca-12050dc95547"
Jul 24 23:29:49.158002 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:49.157979    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-gltb5"]
Jul 24 23:29:49.158092 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:49.158053    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gltb5"
Jul 24 23:29:49.158179 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:49.158154    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gltb5" podUID="7b7b9a28-3166-4af3-b27f-1ab5795c5f92"
Jul 24 23:29:49.523557 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:49.523512    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:50.185786 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:50.185755    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:50.201543 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:50.201511    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-54fxx"
Jul 24 23:29:50.201672 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:50.201628    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-54fxx" podUID="dcf99aef-00e1-4d16-94ca-12050dc95547"
Jul 24 23:29:50.282071 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:50.282035    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:50.380864 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:50.380824    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:50.396453 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:50.396418    2558 generic.go:356] "Generic (PLEG): container finished" podID="03bb2978-3f29-4167-8b41-c257a2b3878d" containerID="6d9e68f5b6f1cfd00b911cee32815a587ee84acf7470967f490f76bf0ae07423" exitCode=0
Jul 24 23:29:50.396646 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:50.396500    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pck26" event={"ID":"03bb2978-3f29-4167-8b41-c257a2b3878d","Type":"ContainerDied","Data":"6d9e68f5b6f1cfd00b911cee32815a587ee84acf7470967f490f76bf0ae07423"}
Jul 24 23:29:50.434969 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:50.434935    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:50.488709 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:50.488680    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:50.540921 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:50.540882    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:50.877436 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:50.877347    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:51.198431 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:51.198360    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gltb5"
Jul 24 23:29:51.198431 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:51.198382    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rplqt"
Jul 24 23:29:51.198631 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:51.198464    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gltb5" podUID="7b7b9a28-3166-4af3-b27f-1ab5795c5f92"
Jul 24 23:29:51.198631 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:51.198597    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rplqt" podUID="68864bf3-c771-4894-9f14-52ee9e8052f9"
Jul 24 23:29:51.402019 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:51.401849    2558 generic.go:356] "Generic (PLEG): container finished" podID="03bb2978-3f29-4167-8b41-c257a2b3878d" containerID="6959475af28ce22f6b3f311f6d8c330721cb8c9f74ae5faa33d4a19dc0285750" exitCode=0
Jul 24 23:29:51.402120 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:51.401925    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pck26" event={"ID":"03bb2978-3f29-4167-8b41-c257a2b3878d","Type":"ContainerDied","Data":"6959475af28ce22f6b3f311f6d8c330721cb8c9f74ae5faa33d4a19dc0285750"}
Jul 24 23:29:51.694610 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:51.694577    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:51.887862 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:51.887776    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:52.120052 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:52.119993    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:52.198995 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:52.198961    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-54fxx"
Jul 24 23:29:52.199153 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:52.199097    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-54fxx" podUID="dcf99aef-00e1-4d16-94ca-12050dc95547"
Jul 24 23:29:52.278800 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:52.278769    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:52.477242 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:52.477205    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:52.687940 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:52.687904    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:53.075622 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:53.075574    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:53.198882 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:53.198851    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gltb5"
Jul 24 23:29:53.199044 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:53.198851    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rplqt"
Jul 24 23:29:53.199044 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:53.198973    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gltb5" podUID="7b7b9a28-3166-4af3-b27f-1ab5795c5f92"
Jul 24 23:29:53.199146 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:53.199055    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rplqt" podUID="68864bf3-c771-4894-9f14-52ee9e8052f9"
Jul 24 23:29:53.815018 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:53.814975    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/68864bf3-c771-4894-9f14-52ee9e8052f9-original-pull-secret\") pod \"global-pull-secret-syncer-rplqt\" (UID: \"68864bf3-c771-4894-9f14-52ee9e8052f9\") " pod="kube-system/global-pull-secret-syncer-rplqt"
Jul 24 23:29:53.815018 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:53.815023    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dcf99aef-00e1-4d16-94ca-12050dc95547-metrics-certs\") pod \"network-metrics-daemon-54fxx\" (UID: \"dcf99aef-00e1-4d16-94ca-12050dc95547\") " pod="openshift-multus/network-metrics-daemon-54fxx"
Jul 24 23:29:53.815271 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:53.815144    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 23:29:53.815271 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:53.815202    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcf99aef-00e1-4d16-94ca-12050dc95547-metrics-certs podName:dcf99aef-00e1-4d16-94ca-12050dc95547 nodeName:}" failed. No retries permitted until 2026-07-24 23:30:25.815184988 +0000 UTC m=+66.201615698 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dcf99aef-00e1-4d16-94ca-12050dc95547-metrics-certs") pod "network-metrics-daemon-54fxx" (UID: "dcf99aef-00e1-4d16-94ca-12050dc95547") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 23:29:53.815271 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:53.815145    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 23:29:53.815404 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:53.815289    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68864bf3-c771-4894-9f14-52ee9e8052f9-original-pull-secret podName:68864bf3-c771-4894-9f14-52ee9e8052f9 nodeName:}" failed. No retries permitted until 2026-07-24 23:30:25.815270705 +0000 UTC m=+66.201701424 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/68864bf3-c771-4894-9f14-52ee9e8052f9-original-pull-secret") pod "global-pull-secret-syncer-rplqt" (UID: "68864bf3-c771-4894-9f14-52ee9e8052f9") : object "kube-system"/"original-pull-secret" not registered
Jul 24 23:29:53.875610 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:53.875573    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:53.915648 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:53.915606    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpcqp\" (UniqueName: \"kubernetes.io/projected/7b7b9a28-3166-4af3-b27f-1ab5795c5f92-kube-api-access-bpcqp\") pod \"network-check-target-gltb5\" (UID: \"7b7b9a28-3166-4af3-b27f-1ab5795c5f92\") " pod="openshift-network-diagnostics/network-check-target-gltb5"
Jul 24 23:29:53.915894 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:53.915758    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 23:29:53.915894 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:53.915848    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 23:29:53.915894 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:53.915861    2558 projected.go:196] Error preparing data for projected volume kube-api-access-bpcqp for pod openshift-network-diagnostics/network-check-target-gltb5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 23:29:53.916063 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:53.915929    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7b7b9a28-3166-4af3-b27f-1ab5795c5f92-kube-api-access-bpcqp podName:7b7b9a28-3166-4af3-b27f-1ab5795c5f92 nodeName:}" failed. No retries permitted until 2026-07-24 23:30:25.915910982 +0000 UTC m=+66.302341708 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-bpcqp" (UniqueName: "kubernetes.io/projected/7b7b9a28-3166-4af3-b27f-1ab5795c5f92-kube-api-access-bpcqp") pod "network-check-target-gltb5" (UID: "7b7b9a28-3166-4af3-b27f-1ab5795c5f92") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 23:29:53.926916 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:53.926881    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:54.077680 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:54.077521    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:54.203065 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:54.203032    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-54fxx"
Jul 24 23:29:54.203245 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:54.203169    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-54fxx" podUID="dcf99aef-00e1-4d16-94ca-12050dc95547"
Jul 24 23:29:54.272302 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:54.272259    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:54.480211 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:54.480174    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:54.531857 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:54.531822    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:55.198665 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:55.198630    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rplqt"
Jul 24 23:29:55.199128 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:55.198673    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gltb5"
Jul 24 23:29:55.199128 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:55.198764    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rplqt" podUID="68864bf3-c771-4894-9f14-52ee9e8052f9"
Jul 24 23:29:55.199128 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:55.198918    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gltb5" podUID="7b7b9a28-3166-4af3-b27f-1ab5795c5f92"
Jul 24 23:29:55.418389 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:55.418347    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:55.501641 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:55.501601    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:55.877933 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:55.877838    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:56.079583 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:56.079545    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:56.176627 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:56.176519    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:56.201434 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:56.201414    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-54fxx"
Jul 24 23:29:56.201830 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:56.201512    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-54fxx" podUID="dcf99aef-00e1-4d16-94ca-12050dc95547"
Jul 24 23:29:56.258894 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:56.258866    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:56.314349 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:56.314316    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:56.418397 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:56.418336    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:56.420931 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:56.420906    2558 generic.go:356] "Generic (PLEG): container finished" podID="03bb2978-3f29-4167-8b41-c257a2b3878d" containerID="a08bf459f2c950a85f14e6d47402b15bc004843e02340c1df29752bc8a882c79" exitCode=0
Jul 24 23:29:56.421046 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:56.420944    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pck26" event={"ID":"03bb2978-3f29-4167-8b41-c257a2b3878d","Type":"ContainerDied","Data":"a08bf459f2c950a85f14e6d47402b15bc004843e02340c1df29752bc8a882c79"}
Jul 24 23:29:56.475505 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:56.475471    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:56.531852 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:56.531821    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:56.797580 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:56.797540    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:56.897978 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:56.897937    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:57.199172 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:57.199079    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rplqt"
Jul 24 23:29:57.199346 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:57.199181    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rplqt" podUID="68864bf3-c771-4894-9f14-52ee9e8052f9"
Jul 24 23:29:57.199346 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:57.199243    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gltb5"
Jul 24 23:29:57.199462 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:57.199348    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gltb5" podUID="7b7b9a28-3166-4af3-b27f-1ab5795c5f92"
Jul 24 23:29:57.217442 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:57.217410    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:57.318833 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:57.318800    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:57.421484 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:57.421448    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:57.425736 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:57.425698    2558 generic.go:356] "Generic (PLEG): container finished" podID="03bb2978-3f29-4167-8b41-c257a2b3878d" containerID="2030c1f8970bc4d993e7aea81554965eac7c96a00c4d0d8d0284ed32de33038c" exitCode=0
Jul 24 23:29:57.425876 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:57.425764    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pck26" event={"ID":"03bb2978-3f29-4167-8b41-c257a2b3878d","Type":"ContainerDied","Data":"2030c1f8970bc4d993e7aea81554965eac7c96a00c4d0d8d0284ed32de33038c"}
Jul 24 23:29:57.472895 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:57.472866    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:57.528752 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:57.528716    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:57.807294 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:57.807264    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:57.907932 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:57.907896    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:58.008612 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:58.008576    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:58.109130 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:58.109040    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:58.199348 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:58.199317    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-54fxx"
Jul 24 23:29:58.199464 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:58.199434    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-54fxx" podUID="dcf99aef-00e1-4d16-94ca-12050dc95547"
Jul 24 23:29:58.209280 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:58.209249    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:58.309981 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:58.309950    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:58.431850 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:58.431770    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pck26" event={"ID":"03bb2978-3f29-4167-8b41-c257a2b3878d","Type":"ContainerStarted","Data":"aa87105a80b1bde06fc0b94fa10085aa06bbbb5d02a5db2fff93fa7c461b756c"}
Jul 24 23:29:58.455544 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:58.455481    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-pck26" podStartSLOduration=7.749548636 podStartE2EDuration="37.455467151s" totalImagesPullingTime="29.705918515s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 23:29:21 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:29:58.45481634 +0000 UTC m=+38.841247068" watchObservedRunningTime="2026-07-24 23:29:58.455467151 +0000 UTC m=+38.841897899"
Jul 24 23:29:58.674219 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:58.674182    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:59.198500 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:59.198471    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rplqt"
Jul 24 23:29:59.198684 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:59.198506    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gltb5"
Jul 24 23:29:59.198684 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:59.198610    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rplqt" podUID="68864bf3-c771-4894-9f14-52ee9e8052f9"
Jul 24 23:29:59.198764 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:29:59.198726    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gltb5" podUID="7b7b9a28-3166-4af3-b27f-1ab5795c5f92"
Jul 24 23:29:59.219929 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:59.219893    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:59.320360 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:59.320324    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:59.376519 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:59.376479    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:59.673781 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:59.673744    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:59.875509 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:59.875472    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:29:59.975845 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:29:59.975815    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:00.077087 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:00.077057    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:00.199000 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:00.198970    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-54fxx"
Jul 24 23:30:00.199166 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:30:00.199056    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-54fxx" podUID="dcf99aef-00e1-4d16-94ca-12050dc95547"
Jul 24 23:30:00.275165 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:00.275064    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:00.678980 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:00.678890    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:01.082784 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:01.082749    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:01.198437 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:01.198395    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rplqt"
Jul 24 23:30:01.198437 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:01.198422    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gltb5"
Jul 24 23:30:01.198677 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:30:01.198501    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rplqt" podUID="68864bf3-c771-4894-9f14-52ee9e8052f9"
Jul 24 23:30:01.198677 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:30:01.198616    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gltb5" podUID="7b7b9a28-3166-4af3-b27f-1ab5795c5f92"
Jul 24 23:30:01.290708 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:01.290670    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:02.166904 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:02.166868    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:02.199074 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:02.199042    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-54fxx"
Jul 24 23:30:02.199228 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:30:02.199164    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-54fxx" podUID="dcf99aef-00e1-4d16-94ca-12050dc95547"
Jul 24 23:30:02.261494 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:02.261459    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:03.198947 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:03.198914    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rplqt"
Jul 24 23:30:03.199327 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:03.198913    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gltb5"
Jul 24 23:30:03.199327 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:30:03.199022    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rplqt" podUID="68864bf3-c771-4894-9f14-52ee9e8052f9"
Jul 24 23:30:03.199327 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:30:03.199083    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gltb5" podUID="7b7b9a28-3166-4af3-b27f-1ab5795c5f92"
Jul 24 23:30:04.198807 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:04.198772    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-54fxx"
Jul 24 23:30:04.198997 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:30:04.198910    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-54fxx" podUID="dcf99aef-00e1-4d16-94ca-12050dc95547"
Jul 24 23:30:04.828790 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:04.828756    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:04.882778 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:04.882741    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:05.198319 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:05.198245    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gltb5"
Jul 24 23:30:05.198574 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:05.198245    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rplqt"
Jul 24 23:30:05.198574 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:30:05.198366    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gltb5" podUID="7b7b9a28-3166-4af3-b27f-1ab5795c5f92"
Jul 24 23:30:05.198574 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:30:05.198423    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rplqt" podUID="68864bf3-c771-4894-9f14-52ee9e8052f9"
Jul 24 23:30:06.198933 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:06.198904    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-54fxx"
Jul 24 23:30:06.199326 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:30:06.199023    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-54fxx" podUID="dcf99aef-00e1-4d16-94ca-12050dc95547"
Jul 24 23:30:07.199012 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:07.198976    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rplqt"
Jul 24 23:30:07.199433 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:07.198976    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gltb5"
Jul 24 23:30:07.199433 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:30:07.199087    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rplqt" podUID="68864bf3-c771-4894-9f14-52ee9e8052f9"
Jul 24 23:30:07.199433 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:30:07.199165    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gltb5" podUID="7b7b9a28-3166-4af3-b27f-1ab5795c5f92"
Jul 24 23:30:08.198515 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:08.198481    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-54fxx"
Jul 24 23:30:08.198693 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:30:08.198612    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-54fxx" podUID="dcf99aef-00e1-4d16-94ca-12050dc95547"
Jul 24 23:30:08.330655 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:08.330607    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:08.422238 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:08.422204    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:08.518594 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:08.518560    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:08.618395 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:08.618356    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:08.926370 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:08.926344    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-133.ec2.internal" event="NodeReady"
Jul 24 23:30:08.926561 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:08.926456    2558 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 24 23:30:08.962020 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:08.961989    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-544bc854f6-wqshw"]
Jul 24 23:30:08.979938 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:08.979904    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-lrp6m"]
Jul 24 23:30:08.980114 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:08.980039    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-544bc854f6-wqshw"
Jul 24 23:30:08.982392 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:08.982252    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 24 23:30:08.982392 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:08.982323    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-bgcmt\""
Jul 24 23:30:08.982392 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:08.982355    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 24 23:30:08.982722 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:08.982702    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 24 23:30:08.993895 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:08.993868    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-n9sjh"]
Jul 24 23:30:08.994043 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:08.993981    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-lrp6m"
Jul 24 23:30:08.994136 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:08.994118    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 24 23:30:08.996579 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:08.996561    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 24 23:30:08.996697 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:08.996596    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 24 23:30:08.996697 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:08.996603    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-cqz6m\""
Jul 24 23:30:08.996875 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:08.996724    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 24 23:30:09.010262 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.010230    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-544bc854f6-wqshw"]
Jul 24 23:30:09.010262 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.010271    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-lrp6m"]
Jul 24 23:30:09.010484 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.010288    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-n9sjh"]
Jul 24 23:30:09.010484 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.010358    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-n9sjh"
Jul 24 23:30:09.012801 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.012778    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-gkx27\""
Jul 24 23:30:09.013047 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.013031    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 24 23:30:09.013102 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.013045    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 24 23:30:09.013364 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.013349    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 24 23:30:09.013466 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.013408    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 24 23:30:09.021282 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.021264    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7d4cc02f-cfdb-4f1e-aef4-5477808977aa-bound-sa-token\") pod \"image-registry-544bc854f6-wqshw\" (UID: \"7d4cc02f-cfdb-4f1e-aef4-5477808977aa\") " pod="openshift-image-registry/image-registry-544bc854f6-wqshw"
Jul 24 23:30:09.021381 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.021291    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7d4cc02f-cfdb-4f1e-aef4-5477808977aa-registry-certificates\") pod \"image-registry-544bc854f6-wqshw\" (UID: \"7d4cc02f-cfdb-4f1e-aef4-5477808977aa\") " pod="openshift-image-registry/image-registry-544bc854f6-wqshw"
Jul 24 23:30:09.021381 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.021314    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7d4cc02f-cfdb-4f1e-aef4-5477808977aa-image-registry-private-configuration\") pod \"image-registry-544bc854f6-wqshw\" (UID: \"7d4cc02f-cfdb-4f1e-aef4-5477808977aa\") " pod="openshift-image-registry/image-registry-544bc854f6-wqshw"
Jul 24 23:30:09.021381 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.021329    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a1fb734a-d36c-4a41-a7f3-281f7cbacb5e-cert\") pod \"ingress-canary-lrp6m\" (UID: \"a1fb734a-d36c-4a41-a7f3-281f7cbacb5e\") " pod="openshift-ingress-canary/ingress-canary-lrp6m"
Jul 24 23:30:09.021381 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.021358    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7d4cc02f-cfdb-4f1e-aef4-5477808977aa-trusted-ca\") pod \"image-registry-544bc854f6-wqshw\" (UID: \"7d4cc02f-cfdb-4f1e-aef4-5477808977aa\") " pod="openshift-image-registry/image-registry-544bc854f6-wqshw"
Jul 24 23:30:09.021511 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.021396    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7d4cc02f-cfdb-4f1e-aef4-5477808977aa-installation-pull-secrets\") pod \"image-registry-544bc854f6-wqshw\" (UID: \"7d4cc02f-cfdb-4f1e-aef4-5477808977aa\") " pod="openshift-image-registry/image-registry-544bc854f6-wqshw"
Jul 24 23:30:09.021511 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.021468    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7d4cc02f-cfdb-4f1e-aef4-5477808977aa-registry-tls\") pod \"image-registry-544bc854f6-wqshw\" (UID: \"7d4cc02f-cfdb-4f1e-aef4-5477808977aa\") " pod="openshift-image-registry/image-registry-544bc854f6-wqshw"
Jul 24 23:30:09.021511 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.021494    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5x5c7\" (UniqueName: \"kubernetes.io/projected/a1fb734a-d36c-4a41-a7f3-281f7cbacb5e-kube-api-access-5x5c7\") pod \"ingress-canary-lrp6m\" (UID: \"a1fb734a-d36c-4a41-a7f3-281f7cbacb5e\") " pod="openshift-ingress-canary/ingress-canary-lrp6m"
Jul 24 23:30:09.021631 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.021514    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7d4cc02f-cfdb-4f1e-aef4-5477808977aa-ca-trust-extracted\") pod \"image-registry-544bc854f6-wqshw\" (UID: \"7d4cc02f-cfdb-4f1e-aef4-5477808977aa\") " pod="openshift-image-registry/image-registry-544bc854f6-wqshw"
Jul 24 23:30:09.021631 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.021577    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kq2c9\" (UniqueName: \"kubernetes.io/projected/7d4cc02f-cfdb-4f1e-aef4-5477808977aa-kube-api-access-kq2c9\") pod \"image-registry-544bc854f6-wqshw\" (UID: \"7d4cc02f-cfdb-4f1e-aef4-5477808977aa\") " pod="openshift-image-registry/image-registry-544bc854f6-wqshw"
Jul 24 23:30:09.073571 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.073515    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-jcc9s"]
Jul 24 23:30:09.092080 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.092049    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jcc9s"]
Jul 24 23:30:09.092226 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.092186    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jcc9s"
Jul 24 23:30:09.094724 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.094699    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2kgw4\""
Jul 24 23:30:09.094724 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.094699    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 24 23:30:09.094895 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.094877    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 24 23:30:09.122656 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.122617    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7d4cc02f-cfdb-4f1e-aef4-5477808977aa-registry-tls\") pod \"image-registry-544bc854f6-wqshw\" (UID: \"7d4cc02f-cfdb-4f1e-aef4-5477808977aa\") " pod="openshift-image-registry/image-registry-544bc854f6-wqshw"
Jul 24 23:30:09.122656 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.122659    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5x5c7\" (UniqueName: \"kubernetes.io/projected/a1fb734a-d36c-4a41-a7f3-281f7cbacb5e-kube-api-access-5x5c7\") pod \"ingress-canary-lrp6m\" (UID: \"a1fb734a-d36c-4a41-a7f3-281f7cbacb5e\") " pod="openshift-ingress-canary/ingress-canary-lrp6m"
Jul 24 23:30:09.122912 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.122687    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3b9e67d-4964-4049-975a-464a9fe43f23-tmp-dir\") pod \"dns-default-jcc9s\" (UID: \"d3b9e67d-4964-4049-975a-464a9fe43f23\") " pod="openshift-dns/dns-default-jcc9s"
Jul 24 23:30:09.122912 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.122743    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8qft\" (UniqueName: \"kubernetes.io/projected/b5d56d9a-1eab-409c-9c69-5939576e4836-kube-api-access-j8qft\") pod \"insights-runtime-extractor-n9sjh\" (UID: \"b5d56d9a-1eab-409c-9c69-5939576e4836\") " pod="openshift-insights/insights-runtime-extractor-n9sjh"
Jul 24 23:30:09.122912 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.122878    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7d4cc02f-cfdb-4f1e-aef4-5477808977aa-ca-trust-extracted\") pod \"image-registry-544bc854f6-wqshw\" (UID: \"7d4cc02f-cfdb-4f1e-aef4-5477808977aa\") " pod="openshift-image-registry/image-registry-544bc854f6-wqshw"
Jul 24 23:30:09.123047 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.122928    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kq2c9\" (UniqueName: \"kubernetes.io/projected/7d4cc02f-cfdb-4f1e-aef4-5477808977aa-kube-api-access-kq2c9\") pod \"image-registry-544bc854f6-wqshw\" (UID: \"7d4cc02f-cfdb-4f1e-aef4-5477808977aa\") " pod="openshift-image-registry/image-registry-544bc854f6-wqshw"
Jul 24 23:30:09.123047 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.122948    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b5d56d9a-1eab-409c-9c69-5939576e4836-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n9sjh\" (UID: \"b5d56d9a-1eab-409c-9c69-5939576e4836\") " pod="openshift-insights/insights-runtime-extractor-n9sjh"
Jul 24 23:30:09.123047 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.122981    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d3b9e67d-4964-4049-975a-464a9fe43f23-config-volume\") pod \"dns-default-jcc9s\" (UID: \"d3b9e67d-4964-4049-975a-464a9fe43f23\") " pod="openshift-dns/dns-default-jcc9s"
Jul 24 23:30:09.123047 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.123011    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b5d56d9a-1eab-409c-9c69-5939576e4836-data-volume\") pod \"insights-runtime-extractor-n9sjh\" (UID: \"b5d56d9a-1eab-409c-9c69-5939576e4836\") " pod="openshift-insights/insights-runtime-extractor-n9sjh"
Jul 24 23:30:09.123047 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.123037    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7d4cc02f-cfdb-4f1e-aef4-5477808977aa-bound-sa-token\") pod \"image-registry-544bc854f6-wqshw\" (UID: \"7d4cc02f-cfdb-4f1e-aef4-5477808977aa\") " pod="openshift-image-registry/image-registry-544bc854f6-wqshw"
Jul 24 23:30:09.123240 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.123055    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7d4cc02f-cfdb-4f1e-aef4-5477808977aa-registry-certificates\") pod \"image-registry-544bc854f6-wqshw\" (UID: \"7d4cc02f-cfdb-4f1e-aef4-5477808977aa\") " pod="openshift-image-registry/image-registry-544bc854f6-wqshw"
Jul 24 23:30:09.123240 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.123072    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b5d56d9a-1eab-409c-9c69-5939576e4836-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n9sjh\" (UID: \"b5d56d9a-1eab-409c-9c69-5939576e4836\") " pod="openshift-insights/insights-runtime-extractor-n9sjh"
Jul 24 23:30:09.123240 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.123095    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwvjc\" (UniqueName: \"kubernetes.io/projected/d3b9e67d-4964-4049-975a-464a9fe43f23-kube-api-access-wwvjc\") pod \"dns-default-jcc9s\" (UID: \"d3b9e67d-4964-4049-975a-464a9fe43f23\") " pod="openshift-dns/dns-default-jcc9s"
Jul 24 23:30:09.123240 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.123151    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7d4cc02f-cfdb-4f1e-aef4-5477808977aa-image-registry-private-configuration\") pod \"image-registry-544bc854f6-wqshw\" (UID: \"7d4cc02f-cfdb-4f1e-aef4-5477808977aa\") " pod="openshift-image-registry/image-registry-544bc854f6-wqshw"
Jul 24 23:30:09.123240 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.123174    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a1fb734a-d36c-4a41-a7f3-281f7cbacb5e-cert\") pod \"ingress-canary-lrp6m\" (UID: \"a1fb734a-d36c-4a41-a7f3-281f7cbacb5e\") " pod="openshift-ingress-canary/ingress-canary-lrp6m"
Jul 24 23:30:09.123240 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.123198    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d3b9e67d-4964-4049-975a-464a9fe43f23-metrics-tls\") pod \"dns-default-jcc9s\" (UID: \"d3b9e67d-4964-4049-975a-464a9fe43f23\") " pod="openshift-dns/dns-default-jcc9s"
Jul 24 23:30:09.123240 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.123232    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7d4cc02f-cfdb-4f1e-aef4-5477808977aa-trusted-ca\") pod \"image-registry-544bc854f6-wqshw\" (UID: \"7d4cc02f-cfdb-4f1e-aef4-5477808977aa\") " pod="openshift-image-registry/image-registry-544bc854f6-wqshw"
Jul 24 23:30:09.123557 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.123269    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7d4cc02f-cfdb-4f1e-aef4-5477808977aa-installation-pull-secrets\") pod \"image-registry-544bc854f6-wqshw\" (UID: \"7d4cc02f-cfdb-4f1e-aef4-5477808977aa\") " pod="openshift-image-registry/image-registry-544bc854f6-wqshw"
Jul 24 23:30:09.123557 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.123296    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b5d56d9a-1eab-409c-9c69-5939576e4836-crio-socket\") pod \"insights-runtime-extractor-n9sjh\" (UID: \"b5d56d9a-1eab-409c-9c69-5939576e4836\") " pod="openshift-insights/insights-runtime-extractor-n9sjh"
Jul 24 23:30:09.123557 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.123272    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7d4cc02f-cfdb-4f1e-aef4-5477808977aa-ca-trust-extracted\") pod \"image-registry-544bc854f6-wqshw\" (UID: \"7d4cc02f-cfdb-4f1e-aef4-5477808977aa\") " pod="openshift-image-registry/image-registry-544bc854f6-wqshw"
Jul 24 23:30:09.124097 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.124069    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7d4cc02f-cfdb-4f1e-aef4-5477808977aa-registry-certificates\") pod \"image-registry-544bc854f6-wqshw\" (UID: \"7d4cc02f-cfdb-4f1e-aef4-5477808977aa\") " pod="openshift-image-registry/image-registry-544bc854f6-wqshw"
Jul 24 23:30:09.124256 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.124169    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7d4cc02f-cfdb-4f1e-aef4-5477808977aa-trusted-ca\") pod \"image-registry-544bc854f6-wqshw\" (UID: \"7d4cc02f-cfdb-4f1e-aef4-5477808977aa\") " pod="openshift-image-registry/image-registry-544bc854f6-wqshw"
Jul 24 23:30:09.127167 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.127139    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7d4cc02f-cfdb-4f1e-aef4-5477808977aa-installation-pull-secrets\") pod \"image-registry-544bc854f6-wqshw\" (UID: \"7d4cc02f-cfdb-4f1e-aef4-5477808977aa\") " pod="openshift-image-registry/image-registry-544bc854f6-wqshw"
Jul 24 23:30:09.127245 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.127184    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7d4cc02f-cfdb-4f1e-aef4-5477808977aa-image-registry-private-configuration\") pod \"image-registry-544bc854f6-wqshw\" (UID: \"7d4cc02f-cfdb-4f1e-aef4-5477808977aa\") " pod="openshift-image-registry/image-registry-544bc854f6-wqshw"
Jul 24 23:30:09.127245 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.127194    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a1fb734a-d36c-4a41-a7f3-281f7cbacb5e-cert\") pod \"ingress-canary-lrp6m\" (UID: \"a1fb734a-d36c-4a41-a7f3-281f7cbacb5e\") " pod="openshift-ingress-canary/ingress-canary-lrp6m"
Jul 24 23:30:09.127245 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.127217    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7d4cc02f-cfdb-4f1e-aef4-5477808977aa-registry-tls\") pod \"image-registry-544bc854f6-wqshw\" (UID: \"7d4cc02f-cfdb-4f1e-aef4-5477808977aa\") " pod="openshift-image-registry/image-registry-544bc854f6-wqshw"
Jul 24 23:30:09.130130 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.130104    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5x5c7\" (UniqueName: \"kubernetes.io/projected/a1fb734a-d36c-4a41-a7f3-281f7cbacb5e-kube-api-access-5x5c7\") pod \"ingress-canary-lrp6m\" (UID: \"a1fb734a-d36c-4a41-a7f3-281f7cbacb5e\") " pod="openshift-ingress-canary/ingress-canary-lrp6m"
Jul 24 23:30:09.130397 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.130377    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7d4cc02f-cfdb-4f1e-aef4-5477808977aa-bound-sa-token\") pod \"image-registry-544bc854f6-wqshw\" (UID: \"7d4cc02f-cfdb-4f1e-aef4-5477808977aa\") " pod="openshift-image-registry/image-registry-544bc854f6-wqshw"
Jul 24 23:30:09.130443 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.130384    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kq2c9\" (UniqueName: \"kubernetes.io/projected/7d4cc02f-cfdb-4f1e-aef4-5477808977aa-kube-api-access-kq2c9\") pod \"image-registry-544bc854f6-wqshw\" (UID: \"7d4cc02f-cfdb-4f1e-aef4-5477808977aa\") " pod="openshift-image-registry/image-registry-544bc854f6-wqshw"
Jul 24 23:30:09.169479 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.169441    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:09.199002 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.198919    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rplqt"
Jul 24 23:30:09.199002 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.198951    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gltb5"
Jul 24 23:30:09.201245 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.201222    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 24 23:30:09.201368 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.201325    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 24 23:30:09.201368 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.201332    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-nzdt8\""
Jul 24 23:30:09.201368 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.201345    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 24 23:30:09.223871 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.223847    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d3b9e67d-4964-4049-975a-464a9fe43f23-metrics-tls\") pod \"dns-default-jcc9s\" (UID: \"d3b9e67d-4964-4049-975a-464a9fe43f23\") " pod="openshift-dns/dns-default-jcc9s"
Jul 24 23:30:09.223993 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.223882    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b5d56d9a-1eab-409c-9c69-5939576e4836-crio-socket\") pod \"insights-runtime-extractor-n9sjh\" (UID: \"b5d56d9a-1eab-409c-9c69-5939576e4836\") " pod="openshift-insights/insights-runtime-extractor-n9sjh"
Jul 24 23:30:09.223993 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.223906    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3b9e67d-4964-4049-975a-464a9fe43f23-tmp-dir\") pod \"dns-default-jcc9s\" (UID: \"d3b9e67d-4964-4049-975a-464a9fe43f23\") " pod="openshift-dns/dns-default-jcc9s"
Jul 24 23:30:09.223993 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.223922    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8qft\" (UniqueName: \"kubernetes.io/projected/b5d56d9a-1eab-409c-9c69-5939576e4836-kube-api-access-j8qft\") pod \"insights-runtime-extractor-n9sjh\" (UID: \"b5d56d9a-1eab-409c-9c69-5939576e4836\") " pod="openshift-insights/insights-runtime-extractor-n9sjh"
Jul 24 23:30:09.224136 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.224059    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b5d56d9a-1eab-409c-9c69-5939576e4836-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n9sjh\" (UID: \"b5d56d9a-1eab-409c-9c69-5939576e4836\") " pod="openshift-insights/insights-runtime-extractor-n9sjh"
Jul 24 23:30:09.224136 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.224084    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d3b9e67d-4964-4049-975a-464a9fe43f23-config-volume\") pod \"dns-default-jcc9s\" (UID: \"d3b9e67d-4964-4049-975a-464a9fe43f23\") " pod="openshift-dns/dns-default-jcc9s"
Jul 24 23:30:09.224136 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.224096    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b5d56d9a-1eab-409c-9c69-5939576e4836-crio-socket\") pod \"insights-runtime-extractor-n9sjh\" (UID: \"b5d56d9a-1eab-409c-9c69-5939576e4836\") " pod="openshift-insights/insights-runtime-extractor-n9sjh"
Jul 24 23:30:09.224136 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.224111    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b5d56d9a-1eab-409c-9c69-5939576e4836-data-volume\") pod \"insights-runtime-extractor-n9sjh\" (UID: \"b5d56d9a-1eab-409c-9c69-5939576e4836\") " pod="openshift-insights/insights-runtime-extractor-n9sjh"
Jul 24 23:30:09.224324 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.224172    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b5d56d9a-1eab-409c-9c69-5939576e4836-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n9sjh\" (UID: \"b5d56d9a-1eab-409c-9c69-5939576e4836\") " pod="openshift-insights/insights-runtime-extractor-n9sjh"
Jul 24 23:30:09.224324 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.224204    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwvjc\" (UniqueName: \"kubernetes.io/projected/d3b9e67d-4964-4049-975a-464a9fe43f23-kube-api-access-wwvjc\") pod \"dns-default-jcc9s\" (UID: \"d3b9e67d-4964-4049-975a-464a9fe43f23\") " pod="openshift-dns/dns-default-jcc9s"
Jul 24 23:30:09.224324 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.224230    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3b9e67d-4964-4049-975a-464a9fe43f23-tmp-dir\") pod \"dns-default-jcc9s\" (UID: \"d3b9e67d-4964-4049-975a-464a9fe43f23\") " pod="openshift-dns/dns-default-jcc9s"
Jul 24 23:30:09.224462 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.224427    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b5d56d9a-1eab-409c-9c69-5939576e4836-data-volume\") pod \"insights-runtime-extractor-n9sjh\" (UID: \"b5d56d9a-1eab-409c-9c69-5939576e4836\") " pod="openshift-insights/insights-runtime-extractor-n9sjh"
Jul 24 23:30:09.224634 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.224611    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d3b9e67d-4964-4049-975a-464a9fe43f23-config-volume\") pod \"dns-default-jcc9s\" (UID: \"d3b9e67d-4964-4049-975a-464a9fe43f23\") " pod="openshift-dns/dns-default-jcc9s"
Jul 24 23:30:09.224716 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.224691    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b5d56d9a-1eab-409c-9c69-5939576e4836-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n9sjh\" (UID: \"b5d56d9a-1eab-409c-9c69-5939576e4836\") " pod="openshift-insights/insights-runtime-extractor-n9sjh"
Jul 24 23:30:09.226101 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.226080    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d3b9e67d-4964-4049-975a-464a9fe43f23-metrics-tls\") pod \"dns-default-jcc9s\" (UID: \"d3b9e67d-4964-4049-975a-464a9fe43f23\") " pod="openshift-dns/dns-default-jcc9s"
Jul 24 23:30:09.226419 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.226398    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b5d56d9a-1eab-409c-9c69-5939576e4836-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n9sjh\" (UID: \"b5d56d9a-1eab-409c-9c69-5939576e4836\") " pod="openshift-insights/insights-runtime-extractor-n9sjh"
Jul 24 23:30:09.229993 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.229929    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:09.231753 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.231734    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwvjc\" (UniqueName: \"kubernetes.io/projected/d3b9e67d-4964-4049-975a-464a9fe43f23-kube-api-access-wwvjc\") pod \"dns-default-jcc9s\" (UID: \"d3b9e67d-4964-4049-975a-464a9fe43f23\") " pod="openshift-dns/dns-default-jcc9s"
Jul 24 23:30:09.232355 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.232339    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8qft\" (UniqueName: \"kubernetes.io/projected/b5d56d9a-1eab-409c-9c69-5939576e4836-kube-api-access-j8qft\") pod \"insights-runtime-extractor-n9sjh\" (UID: \"b5d56d9a-1eab-409c-9c69-5939576e4836\") " pod="openshift-insights/insights-runtime-extractor-n9sjh"
Jul 24 23:30:09.291203 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.291162    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-544bc854f6-wqshw"
Jul 24 23:30:09.302299 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.302272    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-lrp6m"
Jul 24 23:30:09.318623 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.318598    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-n9sjh"
Jul 24 23:30:09.400612 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.400581    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jcc9s"
Jul 24 23:30:09.513189 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.513016    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-lrp6m"]
Jul 24 23:30:09.587474 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.587440    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jcc9s"]
Jul 24 23:30:09.590107 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:30:09.590070    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3b9e67d_4964_4049_975a_464a9fe43f23.slice/crio-c0b518c5a122fce96aff0589cb1136ca7e5fbc425db14d13f4ce2bc88a458321 WatchSource:0}: Error finding container c0b518c5a122fce96aff0589cb1136ca7e5fbc425db14d13f4ce2bc88a458321: Status 404 returned error can't find the container with id c0b518c5a122fce96aff0589cb1136ca7e5fbc425db14d13f4ce2bc88a458321
Jul 24 23:30:09.731439 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.731408    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-n9sjh"]
Jul 24 23:30:09.734353 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:09.734327    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-544bc854f6-wqshw"]
Jul 24 23:30:09.735668 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:30:09.735639    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5d56d9a_1eab_409c_9c69_5939576e4836.slice/crio-36d0c2056fd14402cbc420acb62298101d0e10119772b3efb108f64875cad116 WatchSource:0}: Error finding container 36d0c2056fd14402cbc420acb62298101d0e10119772b3efb108f64875cad116: Status 404 returned error can't find the container with id 36d0c2056fd14402cbc420acb62298101d0e10119772b3efb108f64875cad116
Jul 24 23:30:09.739954 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:30:09.739931    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d4cc02f_cfdb_4f1e_aef4_5477808977aa.slice/crio-071a79cba5ebede379dc684d5519518a2e496a18bf8d522b4d1c9fe128524cb3 WatchSource:0}: Error finding container 071a79cba5ebede379dc684d5519518a2e496a18bf8d522b4d1c9fe128524cb3: Status 404 returned error can't find the container with id 071a79cba5ebede379dc684d5519518a2e496a18bf8d522b4d1c9fe128524cb3
Jul 24 23:30:10.202338 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:10.202255    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-54fxx"
Jul 24 23:30:10.205401 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:10.205370    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-t27k5\""
Jul 24 23:30:10.205716 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:10.205690    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 24 23:30:10.469642 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:10.469551    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-lrp6m" event={"ID":"a1fb734a-d36c-4a41-a7f3-281f7cbacb5e","Type":"ContainerStarted","Data":"f7b024c7cd298181c183117515faeb74830ddba3d5bfbe6a55351e774be795a0"}
Jul 24 23:30:10.475975 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:10.475484    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-544bc854f6-wqshw" event={"ID":"7d4cc02f-cfdb-4f1e-aef4-5477808977aa","Type":"ContainerStarted","Data":"78d9256dc601bb454fd26977d518f12e0f7822bb6ae3319b1e5110d3812bbc70"}
Jul 24 23:30:10.475975 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:10.475559    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-544bc854f6-wqshw" event={"ID":"7d4cc02f-cfdb-4f1e-aef4-5477808977aa","Type":"ContainerStarted","Data":"071a79cba5ebede379dc684d5519518a2e496a18bf8d522b4d1c9fe128524cb3"}
Jul 24 23:30:10.475975 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:10.475772    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-544bc854f6-wqshw"
Jul 24 23:30:10.484004 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:10.483908    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-n9sjh" event={"ID":"b5d56d9a-1eab-409c-9c69-5939576e4836","Type":"ContainerStarted","Data":"36d0c2056fd14402cbc420acb62298101d0e10119772b3efb108f64875cad116"}
Jul 24 23:30:10.486501 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:10.486260    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jcc9s" event={"ID":"d3b9e67d-4964-4049-975a-464a9fe43f23","Type":"ContainerStarted","Data":"c0b518c5a122fce96aff0589cb1136ca7e5fbc425db14d13f4ce2bc88a458321"}
Jul 24 23:30:10.496660 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:10.495701    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-544bc854f6-wqshw" podStartSLOduration=2.4956845100000002 podStartE2EDuration="2.49568451s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:30:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:30:10.494165634 +0000 UTC m=+50.880596362" watchObservedRunningTime="2026-07-24 23:30:10.49568451 +0000 UTC m=+50.882115240"
Jul 24 23:30:13.453962 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:13.453870    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:13.967922 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:13.967838    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:14.070567 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:14.070516    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:15.040468 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:15.040413    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:16.462470 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:16.462433    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:16.509041 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:16.508947    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-lrp6m" event={"ID":"a1fb734a-d36c-4a41-a7f3-281f7cbacb5e","Type":"ContainerStarted","Data":"6a059ecd498528f96d7f4dc3777d9f7e6834f6af188ab86b1ddc007c9dfdc509"}
Jul 24 23:30:16.510922 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:16.510857    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-n9sjh" event={"ID":"b5d56d9a-1eab-409c-9c69-5939576e4836","Type":"ContainerStarted","Data":"ccb598c77d0c3b5aa35e93e435af4c569afdcc1f878165f21b6c2dc312b0bfee"}
Jul 24 23:30:16.513558 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:16.513514    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jcc9s" event={"ID":"d3b9e67d-4964-4049-975a-464a9fe43f23","Type":"ContainerStarted","Data":"d2850c875b1c1d0554e6d6313dab4468c6d63066b3c4015390713583d065d618"}
Jul 24 23:30:16.513693 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:16.513581    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jcc9s" event={"ID":"d3b9e67d-4964-4049-975a-464a9fe43f23","Type":"ContainerStarted","Data":"7bee128779ddebe629adc608c0fc034aa1abc8aa838a3bfe0e6ce3d8543aca50"}
Jul 24 23:30:16.513768 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:16.513699    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-jcc9s"
Jul 24 23:30:16.519646 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:16.519572    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:16.523956 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:16.523907    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-lrp6m" podStartSLOduration=2.56743547 podStartE2EDuration="8.523889823s" totalImagesPullingTime="5.956454353s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:30:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:30:16.523246873 +0000 UTC m=+56.909677602" watchObservedRunningTime="2026-07-24 23:30:16.523889823 +0000 UTC m=+56.910320553"
Jul 24 23:30:16.538689 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:16.538490    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-jcc9s" podStartSLOduration=1.6554373679999999 podStartE2EDuration="7.538470712s" totalImagesPullingTime="5.883033344s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:30:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:30:16.538460516 +0000 UTC m=+56.924891248" watchObservedRunningTime="2026-07-24 23:30:16.538470712 +0000 UTC m=+56.924901420"
Jul 24 23:30:16.843173 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:16.843129    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:16.946514 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:16.946472    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:17.046653 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:17.046616    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:17.120664 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:17.120463    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:17.122992 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:17.122957    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-zvt4z"]
Jul 24 23:30:17.126104 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:17.126085    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-zvt4z"
Jul 24 23:30:17.128470 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:17.128448    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 24 23:30:17.128594 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:17.128448    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-kqdt9\""
Jul 24 23:30:17.134175 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:17.134150    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-zvt4z"]
Jul 24 23:30:17.178206 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:17.178161    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:17.187987 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:17.187942    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/8cba79ff-e0ae-4e55-aca5-13f58f7b7541-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-zvt4z\" (UID: \"8cba79ff-e0ae-4e55-aca5-13f58f7b7541\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-zvt4z"
Jul 24 23:30:17.233365 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:17.233325    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:17.288701 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:17.288655    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/8cba79ff-e0ae-4e55-aca5-13f58f7b7541-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-zvt4z\" (UID: \"8cba79ff-e0ae-4e55-aca5-13f58f7b7541\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-zvt4z"
Jul 24 23:30:17.288888 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:30:17.288840    2558 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Jul 24 23:30:17.288947 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:30:17.288917    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8cba79ff-e0ae-4e55-aca5-13f58f7b7541-tls-certificates podName:8cba79ff-e0ae-4e55-aca5-13f58f7b7541 nodeName:}" failed. No retries permitted until 2026-07-24 23:30:17.788894954 +0000 UTC m=+58.175325674 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/8cba79ff-e0ae-4e55-aca5-13f58f7b7541-tls-certificates") pod "prometheus-operator-admission-webhook-7f8d78f54b-zvt4z" (UID: "8cba79ff-e0ae-4e55-aca5-13f58f7b7541") : secret "prometheus-operator-admission-webhook-tls" not found
Jul 24 23:30:17.596747 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:17.596712    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:17.692415 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:17.692374    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:17.791888 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:17.791846    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/8cba79ff-e0ae-4e55-aca5-13f58f7b7541-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-zvt4z\" (UID: \"8cba79ff-e0ae-4e55-aca5-13f58f7b7541\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-zvt4z"
Jul 24 23:30:17.792060 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:30:17.791986    2558 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Jul 24 23:30:17.792119 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:30:17.792066    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8cba79ff-e0ae-4e55-aca5-13f58f7b7541-tls-certificates podName:8cba79ff-e0ae-4e55-aca5-13f58f7b7541 nodeName:}" failed. No retries permitted until 2026-07-24 23:30:18.792045733 +0000 UTC m=+59.178476442 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/8cba79ff-e0ae-4e55-aca5-13f58f7b7541-tls-certificates") pod "prometheus-operator-admission-webhook-7f8d78f54b-zvt4z" (UID: "8cba79ff-e0ae-4e55-aca5-13f58f7b7541") : secret "prometheus-operator-admission-webhook-tls" not found
Jul 24 23:30:18.249364 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:18.249328    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:18.799636 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:18.799591    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/8cba79ff-e0ae-4e55-aca5-13f58f7b7541-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-zvt4z\" (UID: \"8cba79ff-e0ae-4e55-aca5-13f58f7b7541\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-zvt4z"
Jul 24 23:30:18.802444 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:18.802407    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/8cba79ff-e0ae-4e55-aca5-13f58f7b7541-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-zvt4z\" (UID: \"8cba79ff-e0ae-4e55-aca5-13f58f7b7541\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-zvt4z"
Jul 24 23:30:18.935857 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:18.935819    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-zvt4z"
Jul 24 23:30:19.074210 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:19.074143    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-zvt4z"]
Jul 24 23:30:19.076770 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:30:19.076746    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cba79ff_e0ae_4e55_aca5_13f58f7b7541.slice/crio-9db844396807f2e53b9744101ccc79acb654a9c4375bb1214b8494f7e7effaea WatchSource:0}: Error finding container 9db844396807f2e53b9744101ccc79acb654a9c4375bb1214b8494f7e7effaea: Status 404 returned error can't find the container with id 9db844396807f2e53b9744101ccc79acb654a9c4375bb1214b8494f7e7effaea
Jul 24 23:30:19.460800 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:19.460700    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:19.526224 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:19.526174    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-n9sjh" event={"ID":"b5d56d9a-1eab-409c-9c69-5939576e4836","Type":"ContainerStarted","Data":"dd2e753d7e33d06594da3c727782b41523b10891aee74fa6606e6cd02f33c183"}
Jul 24 23:30:19.526999 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:19.526958    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:19.527660 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:19.527633    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-zvt4z" event={"ID":"8cba79ff-e0ae-4e55-aca5-13f58f7b7541","Type":"ContainerStarted","Data":"9db844396807f2e53b9744101ccc79acb654a9c4375bb1214b8494f7e7effaea"}
Jul 24 23:30:20.412894 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:20.412861    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lxdc5"
Jul 24 23:30:20.581620 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:20.581581    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:20.682679 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:20.682589    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:21.295103 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:21.295064    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:21.348341 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:21.348294    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:21.537113 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:21.537073    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-zvt4z" event={"ID":"8cba79ff-e0ae-4e55-aca5-13f58f7b7541","Type":"ContainerStarted","Data":"861a55b0ecd34bbc3d93d575e3459d058289e2974a2452ccc171c0e8a322740d"}
Jul 24 23:30:21.537578 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:21.537327    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-zvt4z"
Jul 24 23:30:21.543570 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:21.543526    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-zvt4z"
Jul 24 23:30:21.553720 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:21.553624    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-zvt4z" podStartSLOduration=3.092096868 podStartE2EDuration="4.553603419s" totalImagesPullingTime="1.461506551s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:30:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:30:21.552550998 +0000 UTC m=+61.938981727" watchObservedRunningTime="2026-07-24 23:30:21.553603419 +0000 UTC m=+61.940034147"
Jul 24 23:30:22.157321 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:22.157283    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:22.196740 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:22.196702    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-z9hzm"]
Jul 24 23:30:22.214393 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:22.214313    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-z9hzm"
Jul 24 23:30:22.216741 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:22.216712    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 24 23:30:22.217073 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:22.217050    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 24 23:30:22.217195 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:22.217098    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 24 23:30:22.218039 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:22.218022    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 24 23:30:22.218721 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:22.218316    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-jmm7b\""
Jul 24 23:30:22.218721 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:22.218393    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 24 23:30:22.219334 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:22.219312    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-z9hzm"]
Jul 24 23:30:22.269148 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:22.269105    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:22.328262 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:22.328229    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7b3e8ae2-f6bb-4296-b607-9d144e32bb26-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-z9hzm\" (UID: \"7b3e8ae2-f6bb-4296-b607-9d144e32bb26\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-z9hzm"
Jul 24 23:30:22.328459 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:22.328269    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7b3e8ae2-f6bb-4296-b607-9d144e32bb26-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-z9hzm\" (UID: \"7b3e8ae2-f6bb-4296-b607-9d144e32bb26\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-z9hzm"
Jul 24 23:30:22.328459 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:22.328416    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7b3e8ae2-f6bb-4296-b607-9d144e32bb26-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-z9hzm\" (UID: \"7b3e8ae2-f6bb-4296-b607-9d144e32bb26\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-z9hzm"
Jul 24 23:30:22.328604 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:22.328462    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxg9h\" (UniqueName: \"kubernetes.io/projected/7b3e8ae2-f6bb-4296-b607-9d144e32bb26-kube-api-access-lxg9h\") pod \"prometheus-operator-84d69f794d-z9hzm\" (UID: \"7b3e8ae2-f6bb-4296-b607-9d144e32bb26\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-z9hzm"
Jul 24 23:30:22.429882 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:22.429800    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7b3e8ae2-f6bb-4296-b607-9d144e32bb26-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-z9hzm\" (UID: \"7b3e8ae2-f6bb-4296-b607-9d144e32bb26\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-z9hzm"
Jul 24 23:30:22.429882 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:22.429839    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7b3e8ae2-f6bb-4296-b607-9d144e32bb26-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-z9hzm\" (UID: \"7b3e8ae2-f6bb-4296-b607-9d144e32bb26\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-z9hzm"
Jul 24 23:30:22.430114 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:22.429914    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7b3e8ae2-f6bb-4296-b607-9d144e32bb26-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-z9hzm\" (UID: \"7b3e8ae2-f6bb-4296-b607-9d144e32bb26\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-z9hzm"
Jul 24 23:30:22.430114 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:22.429953    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxg9h\" (UniqueName: \"kubernetes.io/projected/7b3e8ae2-f6bb-4296-b607-9d144e32bb26-kube-api-access-lxg9h\") pod \"prometheus-operator-84d69f794d-z9hzm\" (UID: \"7b3e8ae2-f6bb-4296-b607-9d144e32bb26\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-z9hzm"
Jul 24 23:30:22.430114 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:30:22.429980    2558 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 24 23:30:22.430114 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:30:22.430059    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b3e8ae2-f6bb-4296-b607-9d144e32bb26-prometheus-operator-tls podName:7b3e8ae2-f6bb-4296-b607-9d144e32bb26 nodeName:}" failed. No retries permitted until 2026-07-24 23:30:22.930038001 +0000 UTC m=+63.316468712 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/7b3e8ae2-f6bb-4296-b607-9d144e32bb26-prometheus-operator-tls") pod "prometheus-operator-84d69f794d-z9hzm" (UID: "7b3e8ae2-f6bb-4296-b607-9d144e32bb26") : secret "prometheus-operator-tls" not found
Jul 24 23:30:22.430667 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:22.430649    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7b3e8ae2-f6bb-4296-b607-9d144e32bb26-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-z9hzm\" (UID: \"7b3e8ae2-f6bb-4296-b607-9d144e32bb26\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-z9hzm"
Jul 24 23:30:22.432921 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:22.432900    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7b3e8ae2-f6bb-4296-b607-9d144e32bb26-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-z9hzm\" (UID: \"7b3e8ae2-f6bb-4296-b607-9d144e32bb26\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-z9hzm"
Jul 24 23:30:22.438318 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:22.438289    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxg9h\" (UniqueName: \"kubernetes.io/projected/7b3e8ae2-f6bb-4296-b607-9d144e32bb26-kube-api-access-lxg9h\") pod \"prometheus-operator-84d69f794d-z9hzm\" (UID: \"7b3e8ae2-f6bb-4296-b607-9d144e32bb26\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-z9hzm"
Jul 24 23:30:22.563865 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:22.563835    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:22.932870 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:22.932834    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7b3e8ae2-f6bb-4296-b607-9d144e32bb26-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-z9hzm\" (UID: \"7b3e8ae2-f6bb-4296-b607-9d144e32bb26\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-z9hzm"
Jul 24 23:30:22.935162 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:22.935141    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7b3e8ae2-f6bb-4296-b607-9d144e32bb26-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-z9hzm\" (UID: \"7b3e8ae2-f6bb-4296-b607-9d144e32bb26\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-z9hzm"
Jul 24 23:30:23.125272 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:23.125238    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-z9hzm"
Jul 24 23:30:23.243925 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:23.243897    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-z9hzm"]
Jul 24 23:30:23.247426 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:30:23.247401    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b3e8ae2_f6bb_4296_b607_9d144e32bb26.slice/crio-8343ad59972d89d01d53affdc2c244cad29884def9620043e011202138727af8 WatchSource:0}: Error finding container 8343ad59972d89d01d53affdc2c244cad29884def9620043e011202138727af8: Status 404 returned error can't find the container with id 8343ad59972d89d01d53affdc2c244cad29884def9620043e011202138727af8
Jul 24 23:30:23.546125 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:23.546090    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-n9sjh" event={"ID":"b5d56d9a-1eab-409c-9c69-5939576e4836","Type":"ContainerStarted","Data":"3fb1f3525bbb2d04e31cc0b35040e74edcd6c2643ea2d6dcaa3ad9734970c6dc"}
Jul 24 23:30:23.547265 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:23.547242    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-z9hzm" event={"ID":"7b3e8ae2-f6bb-4296-b607-9d144e32bb26","Type":"ContainerStarted","Data":"8343ad59972d89d01d53affdc2c244cad29884def9620043e011202138727af8"}
Jul 24 23:30:23.563759 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:23.563715    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-n9sjh" podStartSLOduration=2.899766306 podStartE2EDuration="15.563699417s" totalImagesPullingTime="12.663933111s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 23:30:08 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:30:23.562325831 +0000 UTC m=+63.948756562" watchObservedRunningTime="2026-07-24 23:30:23.563699417 +0000 UTC m=+63.950130144"
Jul 24 23:30:24.433742 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:24.433699    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:25.131705 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:25.131591    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:25.510553 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:25.510504    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:25.555364 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:25.555333    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-z9hzm" event={"ID":"7b3e8ae2-f6bb-4296-b607-9d144e32bb26","Type":"ContainerStarted","Data":"be3c2a44bbbd5daf4698d2d803e932c8ae588788db501593dba7a164c7754e80"}
Jul 24 23:30:25.555518 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:25.555422    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-z9hzm" event={"ID":"7b3e8ae2-f6bb-4296-b607-9d144e32bb26","Type":"ContainerStarted","Data":"c1abb2a726598da389d43d0e4d15a8b6de9f8080f43c1db433f037cefa248df4"}
Jul 24 23:30:25.562030 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:25.562004    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:25.571486 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:25.571440    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-84d69f794d-z9hzm" podStartSLOduration=1.956283673 podStartE2EDuration="3.571426466s" totalImagesPullingTime="1.615142793s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:30:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:30:25.570407384 +0000 UTC m=+65.956838113" watchObservedRunningTime="2026-07-24 23:30:25.571426466 +0000 UTC m=+65.957857195"
Jul 24 23:30:25.859168 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:25.859080    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/68864bf3-c771-4894-9f14-52ee9e8052f9-original-pull-secret\") pod \"global-pull-secret-syncer-rplqt\" (UID: \"68864bf3-c771-4894-9f14-52ee9e8052f9\") " pod="kube-system/global-pull-secret-syncer-rplqt"
Jul 24 23:30:25.859168 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:25.859118    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dcf99aef-00e1-4d16-94ca-12050dc95547-metrics-certs\") pod \"network-metrics-daemon-54fxx\" (UID: \"dcf99aef-00e1-4d16-94ca-12050dc95547\") " pod="openshift-multus/network-metrics-daemon-54fxx"
Jul 24 23:30:25.862343 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:25.862321    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 24 23:30:25.862687 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:25.862669    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 24 23:30:25.872049 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:25.872019    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/68864bf3-c771-4894-9f14-52ee9e8052f9-original-pull-secret\") pod \"global-pull-secret-syncer-rplqt\" (UID: \"68864bf3-c771-4894-9f14-52ee9e8052f9\") " pod="kube-system/global-pull-secret-syncer-rplqt"
Jul 24 23:30:25.872134 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:25.872085    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dcf99aef-00e1-4d16-94ca-12050dc95547-metrics-certs\") pod \"network-metrics-daemon-54fxx\" (UID: \"dcf99aef-00e1-4d16-94ca-12050dc95547\") " pod="openshift-multus/network-metrics-daemon-54fxx"
Jul 24 23:30:25.959470 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:25.959431    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpcqp\" (UniqueName: \"kubernetes.io/projected/7b7b9a28-3166-4af3-b27f-1ab5795c5f92-kube-api-access-bpcqp\") pod \"network-check-target-gltb5\" (UID: \"7b7b9a28-3166-4af3-b27f-1ab5795c5f92\") " pod="openshift-network-diagnostics/network-check-target-gltb5"
Jul 24 23:30:25.962472 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:25.962454    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 24 23:30:25.972811 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:25.972791    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 24 23:30:25.983135 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:25.983115    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpcqp\" (UniqueName: \"kubernetes.io/projected/7b7b9a28-3166-4af3-b27f-1ab5795c5f92-kube-api-access-bpcqp\") pod \"network-check-target-gltb5\" (UID: \"7b7b9a28-3166-4af3-b27f-1ab5795c5f92\") " pod="openshift-network-diagnostics/network-check-target-gltb5"
Jul 24 23:30:26.009119 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:26.009092    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rplqt"
Jul 24 23:30:26.015676 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:26.015655    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-nzdt8\""
Jul 24 23:30:26.023154 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:26.023127    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gltb5"
Jul 24 23:30:26.121251 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:26.121190    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-t27k5\""
Jul 24 23:30:26.128371 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:26.128344    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-54fxx"
Jul 24 23:30:26.137276 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:26.137252    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-rplqt"]
Jul 24 23:30:26.140464 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:30:26.140431    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68864bf3_c771_4894_9f14_52ee9e8052f9.slice/crio-b94a45ef7ef5ba77fc3f7089c1b610776c9446381e48a9f50d7407db884f605a WatchSource:0}: Error finding container b94a45ef7ef5ba77fc3f7089c1b610776c9446381e48a9f50d7407db884f605a: Status 404 returned error can't find the container with id b94a45ef7ef5ba77fc3f7089c1b610776c9446381e48a9f50d7407db884f605a
Jul 24 23:30:26.156021 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:26.155994    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-gltb5"]
Jul 24 23:30:26.157598 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:30:26.157572    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b7b9a28_3166_4af3_b27f_1ab5795c5f92.slice/crio-62a3ed9cef23bf2cffa7afcf13482d5435c83ec267adf8d436c85b29d75b9907 WatchSource:0}: Error finding container 62a3ed9cef23bf2cffa7afcf13482d5435c83ec267adf8d436c85b29d75b9907: Status 404 returned error can't find the container with id 62a3ed9cef23bf2cffa7afcf13482d5435c83ec267adf8d436c85b29d75b9907
Jul 24 23:30:26.249161 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:26.249125    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-54fxx"]
Jul 24 23:30:26.253028 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:30:26.253001    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcf99aef_00e1_4d16_94ca_12050dc95547.slice/crio-1517188cc7b8e5df1b7557cec3411f34cb2c52e52ddc9c16032d4117b17afcce WatchSource:0}: Error finding container 1517188cc7b8e5df1b7557cec3411f34cb2c52e52ddc9c16032d4117b17afcce: Status 404 returned error can't find the container with id 1517188cc7b8e5df1b7557cec3411f34cb2c52e52ddc9c16032d4117b17afcce
Jul 24 23:30:26.517885 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:26.517858    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-jcc9s"
Jul 24 23:30:26.561635 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:26.561603    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-gltb5" event={"ID":"7b7b9a28-3166-4af3-b27f-1ab5795c5f92","Type":"ContainerStarted","Data":"62a3ed9cef23bf2cffa7afcf13482d5435c83ec267adf8d436c85b29d75b9907"}
Jul 24 23:30:26.563157 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:26.563128    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-54fxx" event={"ID":"dcf99aef-00e1-4d16-94ca-12050dc95547","Type":"ContainerStarted","Data":"1517188cc7b8e5df1b7557cec3411f34cb2c52e52ddc9c16032d4117b17afcce"}
Jul 24 23:30:26.564624 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:26.564595    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-rplqt" event={"ID":"68864bf3-c771-4894-9f14-52ee9e8052f9","Type":"ContainerStarted","Data":"b94a45ef7ef5ba77fc3f7089c1b610776c9446381e48a9f50d7407db884f605a"}
Jul 24 23:30:27.308082 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:27.308023    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:27.481829 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:27.481787    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:27.564976 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:27.564814    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:27.571114 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:27.571081    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-54fxx" event={"ID":"dcf99aef-00e1-4d16-94ca-12050dc95547","Type":"ContainerStarted","Data":"0eb3fe3a48f1ef8bc4fece898e59dde76dc73f1aef0443a74a9f0b443f02f6ca"}
Jul 24 23:30:27.593488 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:27.593457    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-9jpxt"]
Jul 24 23:30:27.622468 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:27.622434    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-rhrlf"]
Jul 24 23:30:27.622918 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:27.622891    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9jpxt"
Jul 24 23:30:27.627158 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:27.627133    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 24 23:30:27.627248 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:27.627134    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 24 23:30:27.628085 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:27.628061    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-vh87k\""
Jul 24 23:30:27.628327 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:27.628305    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 24 23:30:27.636410 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:27.636387    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-9jpxt"]
Jul 24 23:30:27.636519 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:27.636512    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rhrlf"
Jul 24 23:30:27.637097 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:27.636835    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:27.639981 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:27.639958    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 24 23:30:27.640079 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:27.640041    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 24 23:30:27.640142 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:27.639958    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 24 23:30:27.640219 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:27.640203    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-n4stc\""
Jul 24 23:30:27.676336 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:27.676306    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/18c4ebca-b677-451a-bb9a-a2acda06fd79-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-9jpxt\" (UID: \"18c4ebca-b677-451a-bb9a-a2acda06fd79\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9jpxt"
Jul 24 23:30:27.676676 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:27.676592    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/18c4ebca-b677-451a-bb9a-a2acda06fd79-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-9jpxt\" (UID: \"18c4ebca-b677-451a-bb9a-a2acda06fd79\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9jpxt"
Jul 24 23:30:27.676676 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:27.676642    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/18c4ebca-b677-451a-bb9a-a2acda06fd79-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-9jpxt\" (UID: \"18c4ebca-b677-451a-bb9a-a2acda06fd79\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9jpxt"
Jul 24 23:30:27.676834 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:27.676724    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/18c4ebca-b677-451a-bb9a-a2acda06fd79-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-9jpxt\" (UID: \"18c4ebca-b677-451a-bb9a-a2acda06fd79\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9jpxt"
Jul 24 23:30:27.676834 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:27.676753    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlmdg\" (UniqueName: \"kubernetes.io/projected/18c4ebca-b677-451a-bb9a-a2acda06fd79-kube-api-access-tlmdg\") pod \"kube-state-metrics-56d9f944b4-9jpxt\" (UID: \"18c4ebca-b677-451a-bb9a-a2acda06fd79\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9jpxt"
Jul 24 23:30:27.676834 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:27.676775    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/18c4ebca-b677-451a-bb9a-a2acda06fd79-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-9jpxt\" (UID: \"18c4ebca-b677-451a-bb9a-a2acda06fd79\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9jpxt"
Jul 24 23:30:27.730142 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:27.730096    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:27.778118 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:27.778079    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/18c4ebca-b677-451a-bb9a-a2acda06fd79-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-9jpxt\" (UID: \"18c4ebca-b677-451a-bb9a-a2acda06fd79\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9jpxt"
Jul 24 23:30:27.778308 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:27.778140    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/18c4ebca-b677-451a-bb9a-a2acda06fd79-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-9jpxt\" (UID: \"18c4ebca-b677-451a-bb9a-a2acda06fd79\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9jpxt"
Jul 24 23:30:27.778308 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:27.778182    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmmj9\" (UniqueName: \"kubernetes.io/projected/39b86669-df55-45b0-ab05-f4a4df0926b1-kube-api-access-lmmj9\") pod \"node-exporter-rhrlf\" (UID: \"39b86669-df55-45b0-ab05-f4a4df0926b1\") " pod="openshift-monitoring/node-exporter-rhrlf"
Jul 24 23:30:27.778308 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:27.778218    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/18c4ebca-b677-451a-bb9a-a2acda06fd79-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-9jpxt\" (UID: \"18c4ebca-b677-451a-bb9a-a2acda06fd79\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9jpxt"
Jul 24 23:30:27.778308 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:30:27.778248    2558 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 24 23:30:27.778499 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:27.778246    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlmdg\" (UniqueName: \"kubernetes.io/projected/18c4ebca-b677-451a-bb9a-a2acda06fd79-kube-api-access-tlmdg\") pod \"kube-state-metrics-56d9f944b4-9jpxt\" (UID: \"18c4ebca-b677-451a-bb9a-a2acda06fd79\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9jpxt"
Jul 24 23:30:27.778499 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:30:27.778335    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/18c4ebca-b677-451a-bb9a-a2acda06fd79-kube-state-metrics-tls podName:18c4ebca-b677-451a-bb9a-a2acda06fd79 nodeName:}" failed. No retries permitted until 2026-07-24 23:30:28.278312214 +0000 UTC m=+68.664742924 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/18c4ebca-b677-451a-bb9a-a2acda06fd79-kube-state-metrics-tls") pod "kube-state-metrics-56d9f944b4-9jpxt" (UID: "18c4ebca-b677-451a-bb9a-a2acda06fd79") : secret "kube-state-metrics-tls" not found
Jul 24 23:30:27.778499 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:27.778374    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/39b86669-df55-45b0-ab05-f4a4df0926b1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rhrlf\" (UID: \"39b86669-df55-45b0-ab05-f4a4df0926b1\") " pod="openshift-monitoring/node-exporter-rhrlf"
Jul 24 23:30:27.778499 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:27.778403    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/39b86669-df55-45b0-ab05-f4a4df0926b1-metrics-client-ca\") pod \"node-exporter-rhrlf\" (UID: \"39b86669-df55-45b0-ab05-f4a4df0926b1\") " pod="openshift-monitoring/node-exporter-rhrlf"
Jul 24 23:30:27.778499 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:27.778429    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/39b86669-df55-45b0-ab05-f4a4df0926b1-node-exporter-accelerators-collector-config\") pod \"node-exporter-rhrlf\" (UID: \"39b86669-df55-45b0-ab05-f4a4df0926b1\") " pod="openshift-monitoring/node-exporter-rhrlf"
Jul 24 23:30:27.778499 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:27.778458    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/18c4ebca-b677-451a-bb9a-a2acda06fd79-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-9jpxt\" (UID: \"18c4ebca-b677-451a-bb9a-a2acda06fd79\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9jpxt"
Jul 24 23:30:27.778499 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:27.778492    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/39b86669-df55-45b0-ab05-f4a4df0926b1-root\") pod \"node-exporter-rhrlf\" (UID: \"39b86669-df55-45b0-ab05-f4a4df0926b1\") " pod="openshift-monitoring/node-exporter-rhrlf"
Jul 24 23:30:27.778868 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:27.778521    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/39b86669-df55-45b0-ab05-f4a4df0926b1-node-exporter-textfile\") pod \"node-exporter-rhrlf\" (UID: \"39b86669-df55-45b0-ab05-f4a4df0926b1\") " pod="openshift-monitoring/node-exporter-rhrlf"
Jul 24 23:30:27.778868 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:27.778664    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/39b86669-df55-45b0-ab05-f4a4df0926b1-sys\") pod \"node-exporter-rhrlf\" (UID: \"39b86669-df55-45b0-ab05-f4a4df0926b1\") " pod="openshift-monitoring/node-exporter-rhrlf"
Jul 24 23:30:27.778868 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:27.778752    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/39b86669-df55-45b0-ab05-f4a4df0926b1-node-exporter-tls\") pod \"node-exporter-rhrlf\" (UID: \"39b86669-df55-45b0-ab05-f4a4df0926b1\") " pod="openshift-monitoring/node-exporter-rhrlf"
Jul 24 23:30:27.778868 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:27.778804    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/18c4ebca-b677-451a-bb9a-a2acda06fd79-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-9jpxt\" (UID: \"18c4ebca-b677-451a-bb9a-a2acda06fd79\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9jpxt"
Jul 24 23:30:27.778868 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:27.778834    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/39b86669-df55-45b0-ab05-f4a4df0926b1-node-exporter-wtmp\") pod \"node-exporter-rhrlf\" (UID: \"39b86669-df55-45b0-ab05-f4a4df0926b1\") " pod="openshift-monitoring/node-exporter-rhrlf"
Jul 24 23:30:27.778868 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:27.778843    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/18c4ebca-b677-451a-bb9a-a2acda06fd79-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-9jpxt\" (UID: \"18c4ebca-b677-451a-bb9a-a2acda06fd79\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9jpxt"
Jul 24 23:30:27.779203 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:27.779174    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/18c4ebca-b677-451a-bb9a-a2acda06fd79-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-9jpxt\" (UID: \"18c4ebca-b677-451a-bb9a-a2acda06fd79\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9jpxt"
Jul 24 23:30:27.779490 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:27.779471    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/18c4ebca-b677-451a-bb9a-a2acda06fd79-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-9jpxt\" (UID: \"18c4ebca-b677-451a-bb9a-a2acda06fd79\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9jpxt"
Jul 24 23:30:27.782117 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:27.782092    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/18c4ebca-b677-451a-bb9a-a2acda06fd79-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-9jpxt\" (UID: \"18c4ebca-b677-451a-bb9a-a2acda06fd79\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9jpxt"
Jul 24 23:30:27.786435 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:27.786414    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlmdg\" (UniqueName: \"kubernetes.io/projected/18c4ebca-b677-451a-bb9a-a2acda06fd79-kube-api-access-tlmdg\") pod \"kube-state-metrics-56d9f944b4-9jpxt\" (UID: \"18c4ebca-b677-451a-bb9a-a2acda06fd79\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9jpxt"
Jul 24 23:30:27.831600 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:27.831490    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:27.879789 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:27.879744    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmmj9\" (UniqueName: \"kubernetes.io/projected/39b86669-df55-45b0-ab05-f4a4df0926b1-kube-api-access-lmmj9\") pod \"node-exporter-rhrlf\" (UID: \"39b86669-df55-45b0-ab05-f4a4df0926b1\") " pod="openshift-monitoring/node-exporter-rhrlf"
Jul 24 23:30:27.879950 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:27.879799    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/39b86669-df55-45b0-ab05-f4a4df0926b1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rhrlf\" (UID: \"39b86669-df55-45b0-ab05-f4a4df0926b1\") " pod="openshift-monitoring/node-exporter-rhrlf"
Jul 24 23:30:27.879950 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:27.879826    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/39b86669-df55-45b0-ab05-f4a4df0926b1-metrics-client-ca\") pod \"node-exporter-rhrlf\" (UID: \"39b86669-df55-45b0-ab05-f4a4df0926b1\") " pod="openshift-monitoring/node-exporter-rhrlf"
Jul 24 23:30:27.879950 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:27.879847    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/39b86669-df55-45b0-ab05-f4a4df0926b1-node-exporter-accelerators-collector-config\") pod \"node-exporter-rhrlf\" (UID: \"39b86669-df55-45b0-ab05-f4a4df0926b1\") " pod="openshift-monitoring/node-exporter-rhrlf"
Jul 24 23:30:27.879950 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:27.879872    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/39b86669-df55-45b0-ab05-f4a4df0926b1-root\") pod \"node-exporter-rhrlf\" (UID: \"39b86669-df55-45b0-ab05-f4a4df0926b1\") " pod="openshift-monitoring/node-exporter-rhrlf"
Jul 24 23:30:27.879950 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:27.879892    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/39b86669-df55-45b0-ab05-f4a4df0926b1-node-exporter-textfile\") pod \"node-exporter-rhrlf\" (UID: \"39b86669-df55-45b0-ab05-f4a4df0926b1\") " pod="openshift-monitoring/node-exporter-rhrlf"
Jul 24 23:30:27.879950 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:27.879910    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/39b86669-df55-45b0-ab05-f4a4df0926b1-sys\") pod \"node-exporter-rhrlf\" (UID: \"39b86669-df55-45b0-ab05-f4a4df0926b1\") " pod="openshift-monitoring/node-exporter-rhrlf"
Jul 24 23:30:27.879950 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:27.879927    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/39b86669-df55-45b0-ab05-f4a4df0926b1-node-exporter-tls\") pod \"node-exporter-rhrlf\" (UID: \"39b86669-df55-45b0-ab05-f4a4df0926b1\") " pod="openshift-monitoring/node-exporter-rhrlf"
Jul 24 23:30:27.879950 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:27.879948    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/39b86669-df55-45b0-ab05-f4a4df0926b1-node-exporter-wtmp\") pod \"node-exporter-rhrlf\" (UID: \"39b86669-df55-45b0-ab05-f4a4df0926b1\") " pod="openshift-monitoring/node-exporter-rhrlf"
Jul 24 23:30:27.880284 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:27.880105    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/39b86669-df55-45b0-ab05-f4a4df0926b1-node-exporter-wtmp\") pod \"node-exporter-rhrlf\" (UID: \"39b86669-df55-45b0-ab05-f4a4df0926b1\") " pod="openshift-monitoring/node-exporter-rhrlf"
Jul 24 23:30:27.880284 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:27.880156    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/39b86669-df55-45b0-ab05-f4a4df0926b1-root\") pod \"node-exporter-rhrlf\" (UID: \"39b86669-df55-45b0-ab05-f4a4df0926b1\") " pod="openshift-monitoring/node-exporter-rhrlf"
Jul 24 23:30:27.880398 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:27.880370    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/39b86669-df55-45b0-ab05-f4a4df0926b1-sys\") pod \"node-exporter-rhrlf\" (UID: \"39b86669-df55-45b0-ab05-f4a4df0926b1\") " pod="openshift-monitoring/node-exporter-rhrlf"
Jul 24 23:30:27.880464 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:27.880436    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/39b86669-df55-45b0-ab05-f4a4df0926b1-node-exporter-textfile\") pod \"node-exporter-rhrlf\" (UID: \"39b86669-df55-45b0-ab05-f4a4df0926b1\") " pod="openshift-monitoring/node-exporter-rhrlf"
Jul 24 23:30:27.880835 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:27.880805    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/39b86669-df55-45b0-ab05-f4a4df0926b1-node-exporter-accelerators-collector-config\") pod \"node-exporter-rhrlf\" (UID: \"39b86669-df55-45b0-ab05-f4a4df0926b1\") " pod="openshift-monitoring/node-exporter-rhrlf"
Jul 24 23:30:27.880975 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:30:27.880523    2558 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 24 23:30:27.880975 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:27.880886    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/39b86669-df55-45b0-ab05-f4a4df0926b1-metrics-client-ca\") pod \"node-exporter-rhrlf\" (UID: \"39b86669-df55-45b0-ab05-f4a4df0926b1\") " pod="openshift-monitoring/node-exporter-rhrlf"
Jul 24 23:30:27.880975 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:30:27.880915    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39b86669-df55-45b0-ab05-f4a4df0926b1-node-exporter-tls podName:39b86669-df55-45b0-ab05-f4a4df0926b1 nodeName:}" failed. No retries permitted until 2026-07-24 23:30:28.380891765 +0000 UTC m=+68.767322476 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/39b86669-df55-45b0-ab05-f4a4df0926b1-node-exporter-tls") pod "node-exporter-rhrlf" (UID: "39b86669-df55-45b0-ab05-f4a4df0926b1") : secret "node-exporter-tls" not found
Jul 24 23:30:27.883161 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:27.883136    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/39b86669-df55-45b0-ab05-f4a4df0926b1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rhrlf\" (UID: \"39b86669-df55-45b0-ab05-f4a4df0926b1\") " pod="openshift-monitoring/node-exporter-rhrlf"
Jul 24 23:30:27.888990 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:27.888944    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmmj9\" (UniqueName: \"kubernetes.io/projected/39b86669-df55-45b0-ab05-f4a4df0926b1-kube-api-access-lmmj9\") pod \"node-exporter-rhrlf\" (UID: \"39b86669-df55-45b0-ab05-f4a4df0926b1\") " pod="openshift-monitoring/node-exporter-rhrlf"
Jul 24 23:30:28.070304 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:28.070233    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:28.178286 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:28.178192    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:28.283799 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:28.283763    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/18c4ebca-b677-451a-bb9a-a2acda06fd79-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-9jpxt\" (UID: \"18c4ebca-b677-451a-bb9a-a2acda06fd79\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9jpxt"
Jul 24 23:30:28.283991 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:30:28.283931    2558 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 24 23:30:28.284063 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:30:28.284025    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/18c4ebca-b677-451a-bb9a-a2acda06fd79-kube-state-metrics-tls podName:18c4ebca-b677-451a-bb9a-a2acda06fd79 nodeName:}" failed. No retries permitted until 2026-07-24 23:30:29.284004294 +0000 UTC m=+69.670435019 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/18c4ebca-b677-451a-bb9a-a2acda06fd79-kube-state-metrics-tls") pod "kube-state-metrics-56d9f944b4-9jpxt" (UID: "18c4ebca-b677-451a-bb9a-a2acda06fd79") : secret "kube-state-metrics-tls" not found
Jul 24 23:30:28.340325 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:28.340279    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:28.384228 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:28.384186    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/39b86669-df55-45b0-ab05-f4a4df0926b1-node-exporter-tls\") pod \"node-exporter-rhrlf\" (UID: \"39b86669-df55-45b0-ab05-f4a4df0926b1\") " pod="openshift-monitoring/node-exporter-rhrlf"
Jul 24 23:30:28.386848 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:28.386817    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/39b86669-df55-45b0-ab05-f4a4df0926b1-node-exporter-tls\") pod \"node-exporter-rhrlf\" (UID: \"39b86669-df55-45b0-ab05-f4a4df0926b1\") " pod="openshift-monitoring/node-exporter-rhrlf"
Jul 24 23:30:28.449089 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:28.448994    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:28.550308 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:28.550279    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rhrlf"
Jul 24 23:30:28.551867 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:28.551828    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:28.578110 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:28.578070    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-54fxx" event={"ID":"dcf99aef-00e1-4d16-94ca-12050dc95547","Type":"ContainerStarted","Data":"322bc747e76a20cc6c8b30709adf6fd6a38aa6091fc819d46c7c12132600b0b2"}
Jul 24 23:30:28.595824 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:28.595763    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-54fxx" podStartSLOduration=66.529964372 podStartE2EDuration="1m7.595744041s" totalImagesPullingTime="1.065779669s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:29:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:30:28.595446589 +0000 UTC m=+68.981877333" watchObservedRunningTime="2026-07-24 23:30:28.595744041 +0000 UTC m=+68.982174761"
Jul 24 23:30:28.654721 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:28.654681    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 24 23:30:28.657414 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:28.657373    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:28.671887 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:28.671779    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 24 23:30:28.676003 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:28.675782    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 24 23:30:28.676003 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:28.675822    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 24 23:30:28.676003 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:28.675880    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 24 23:30:28.676003 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:28.675896    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 24 23:30:28.676003 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:28.675949    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 24 23:30:28.676415 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:28.676020    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 24 23:30:28.676415 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:28.676026    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 24 23:30:28.676415 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:28.676089    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-5llm4\""
Jul 24 23:30:28.676415 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:28.676194    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 24 23:30:28.676415 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:28.676235    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 24 23:30:28.676678 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:28.676435    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 24 23:30:28.765458 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:28.765412    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:28.788601 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:28.788568    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1dcf2959-0da6-410b-80cd-3408b640aa7d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1dcf2959-0da6-410b-80cd-3408b640aa7d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 23:30:28.788601 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:28.788607    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1dcf2959-0da6-410b-80cd-3408b640aa7d-config-out\") pod \"alertmanager-main-0\" (UID: \"1dcf2959-0da6-410b-80cd-3408b640aa7d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 23:30:28.788844 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:28.788630    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1dcf2959-0da6-410b-80cd-3408b640aa7d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1dcf2959-0da6-410b-80cd-3408b640aa7d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 23:30:28.788844 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:28.788694    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1dcf2959-0da6-410b-80cd-3408b640aa7d-config-volume\") pod \"alertmanager-main-0\" (UID: \"1dcf2959-0da6-410b-80cd-3408b640aa7d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 23:30:28.788844 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:28.788735    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1dcf2959-0da6-410b-80cd-3408b640aa7d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1dcf2959-0da6-410b-80cd-3408b640aa7d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 23:30:28.788844 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:28.788778    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1dcf2959-0da6-410b-80cd-3408b640aa7d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1dcf2959-0da6-410b-80cd-3408b640aa7d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 23:30:28.789032 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:28.788861    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1dcf2959-0da6-410b-80cd-3408b640aa7d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1dcf2959-0da6-410b-80cd-3408b640aa7d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 23:30:28.789032 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:28.788897    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1dcf2959-0da6-410b-80cd-3408b640aa7d-web-config\") pod \"alertmanager-main-0\" (UID: \"1dcf2959-0da6-410b-80cd-3408b640aa7d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 23:30:28.789032 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:28.788927    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1dcf2959-0da6-410b-80cd-3408b640aa7d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1dcf2959-0da6-410b-80cd-3408b640aa7d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 23:30:28.789032 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:28.788959    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1dcf2959-0da6-410b-80cd-3408b640aa7d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1dcf2959-0da6-410b-80cd-3408b640aa7d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 23:30:28.789032 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:28.789014    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1dcf2959-0da6-410b-80cd-3408b640aa7d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1dcf2959-0da6-410b-80cd-3408b640aa7d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 23:30:28.789271 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:28.789058    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pg8bx\" (UniqueName: \"kubernetes.io/projected/1dcf2959-0da6-410b-80cd-3408b640aa7d-kube-api-access-pg8bx\") pod \"alertmanager-main-0\" (UID: \"1dcf2959-0da6-410b-80cd-3408b640aa7d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 23:30:28.789271 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:28.789111    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1dcf2959-0da6-410b-80cd-3408b640aa7d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1dcf2959-0da6-410b-80cd-3408b640aa7d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 23:30:28.865843 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:28.865748    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:28.890251 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:28.890213    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1dcf2959-0da6-410b-80cd-3408b640aa7d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1dcf2959-0da6-410b-80cd-3408b640aa7d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 23:30:28.890251 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:28.890257    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1dcf2959-0da6-410b-80cd-3408b640aa7d-config-volume\") pod \"alertmanager-main-0\" (UID: \"1dcf2959-0da6-410b-80cd-3408b640aa7d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 23:30:28.890476 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:28.890277    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1dcf2959-0da6-410b-80cd-3408b640aa7d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1dcf2959-0da6-410b-80cd-3408b640aa7d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 23:30:28.890476 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:28.890303    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1dcf2959-0da6-410b-80cd-3408b640aa7d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1dcf2959-0da6-410b-80cd-3408b640aa7d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 23:30:28.890476 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:28.890332    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1dcf2959-0da6-410b-80cd-3408b640aa7d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1dcf2959-0da6-410b-80cd-3408b640aa7d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 23:30:28.890476 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:28.890362    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1dcf2959-0da6-410b-80cd-3408b640aa7d-web-config\") pod \"alertmanager-main-0\" (UID: \"1dcf2959-0da6-410b-80cd-3408b640aa7d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 23:30:28.890476 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:28.890388    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1dcf2959-0da6-410b-80cd-3408b640aa7d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1dcf2959-0da6-410b-80cd-3408b640aa7d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 23:30:28.890476 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:28.890420    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1dcf2959-0da6-410b-80cd-3408b640aa7d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1dcf2959-0da6-410b-80cd-3408b640aa7d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 23:30:28.890800 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:28.890558    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1dcf2959-0da6-410b-80cd-3408b640aa7d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1dcf2959-0da6-410b-80cd-3408b640aa7d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 23:30:28.890800 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:28.890589    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pg8bx\" (UniqueName: \"kubernetes.io/projected/1dcf2959-0da6-410b-80cd-3408b640aa7d-kube-api-access-pg8bx\") pod \"alertmanager-main-0\" (UID: \"1dcf2959-0da6-410b-80cd-3408b640aa7d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 23:30:28.890800 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:28.890618    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1dcf2959-0da6-410b-80cd-3408b640aa7d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1dcf2959-0da6-410b-80cd-3408b640aa7d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 23:30:28.890800 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:28.890668    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1dcf2959-0da6-410b-80cd-3408b640aa7d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1dcf2959-0da6-410b-80cd-3408b640aa7d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 23:30:28.890800 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:28.890694    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1dcf2959-0da6-410b-80cd-3408b640aa7d-config-out\") pod \"alertmanager-main-0\" (UID: \"1dcf2959-0da6-410b-80cd-3408b640aa7d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 23:30:28.892281 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:30:28.891192    2558 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Jul 24 23:30:28.892281 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:30:28.891262    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1dcf2959-0da6-410b-80cd-3408b640aa7d-secret-alertmanager-main-tls podName:1dcf2959-0da6-410b-80cd-3408b640aa7d nodeName:}" failed. No retries permitted until 2026-07-24 23:30:29.391242417 +0000 UTC m=+69.777673137 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/1dcf2959-0da6-410b-80cd-3408b640aa7d-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "1dcf2959-0da6-410b-80cd-3408b640aa7d") : secret "alertmanager-main-tls" not found
Jul 24 23:30:28.892281 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:28.891376    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1dcf2959-0da6-410b-80cd-3408b640aa7d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1dcf2959-0da6-410b-80cd-3408b640aa7d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 23:30:28.892281 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:28.891876    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1dcf2959-0da6-410b-80cd-3408b640aa7d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1dcf2959-0da6-410b-80cd-3408b640aa7d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 23:30:28.892281 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:28.892170    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1dcf2959-0da6-410b-80cd-3408b640aa7d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1dcf2959-0da6-410b-80cd-3408b640aa7d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 23:30:28.893326 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:28.893301    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1dcf2959-0da6-410b-80cd-3408b640aa7d-config-out\") pod \"alertmanager-main-0\" (UID: \"1dcf2959-0da6-410b-80cd-3408b640aa7d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 23:30:28.893421 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:28.893359    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1dcf2959-0da6-410b-80cd-3408b640aa7d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1dcf2959-0da6-410b-80cd-3408b640aa7d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 23:30:28.893724 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:28.893699    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1dcf2959-0da6-410b-80cd-3408b640aa7d-config-volume\") pod \"alertmanager-main-0\" (UID: \"1dcf2959-0da6-410b-80cd-3408b640aa7d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 23:30:28.895267 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:28.895244    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1dcf2959-0da6-410b-80cd-3408b640aa7d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1dcf2959-0da6-410b-80cd-3408b640aa7d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 23:30:28.895349 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:28.895296    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1dcf2959-0da6-410b-80cd-3408b640aa7d-web-config\") pod \"alertmanager-main-0\" (UID: \"1dcf2959-0da6-410b-80cd-3408b640aa7d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 23:30:28.895487 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:28.895465    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1dcf2959-0da6-410b-80cd-3408b640aa7d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1dcf2959-0da6-410b-80cd-3408b640aa7d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 23:30:28.895609 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:28.895587    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1dcf2959-0da6-410b-80cd-3408b640aa7d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1dcf2959-0da6-410b-80cd-3408b640aa7d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 23:30:28.895702 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:28.895677    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1dcf2959-0da6-410b-80cd-3408b640aa7d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1dcf2959-0da6-410b-80cd-3408b640aa7d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 23:30:28.901275 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:28.901247    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pg8bx\" (UniqueName: \"kubernetes.io/projected/1dcf2959-0da6-410b-80cd-3408b640aa7d-kube-api-access-pg8bx\") pod \"alertmanager-main-0\" (UID: \"1dcf2959-0da6-410b-80cd-3408b640aa7d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 23:30:29.231938 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:29.231893    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:29.294793 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:29.294749    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/18c4ebca-b677-451a-bb9a-a2acda06fd79-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-9jpxt\" (UID: \"18c4ebca-b677-451a-bb9a-a2acda06fd79\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9jpxt"
Jul 24 23:30:29.297081 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:29.296927    2558 patch_prober.go:28] interesting pod/image-registry-544bc854f6-wqshw container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 24 23:30:29.297081 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:29.297010    2558 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-544bc854f6-wqshw" podUID="7d4cc02f-cfdb-4f1e-aef4-5477808977aa" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 24 23:30:29.297666 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:29.297640    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/18c4ebca-b677-451a-bb9a-a2acda06fd79-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-9jpxt\" (UID: \"18c4ebca-b677-451a-bb9a-a2acda06fd79\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9jpxt"
Jul 24 23:30:29.396075 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:29.396037    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1dcf2959-0da6-410b-80cd-3408b640aa7d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1dcf2959-0da6-410b-80cd-3408b640aa7d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 23:30:29.398827 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:29.398802    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1dcf2959-0da6-410b-80cd-3408b640aa7d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1dcf2959-0da6-410b-80cd-3408b640aa7d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 23:30:29.441945 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:29.441909    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9jpxt"
Jul 24 23:30:29.486118 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:29.486013    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:29.561232 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:29.561199    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:29.586246 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:29.586218    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 24 23:30:30.497138 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:30.497097    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:30.544736 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:30.544689    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:30.870241 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:30.870148    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:31.067267 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:31.067238    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-9jpxt"]
Jul 24 23:30:31.070059 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:30:31.070031    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18c4ebca_b677_451a_bb9a_a2acda06fd79.slice/crio-1e5860c1eef600e1f5051243b4efcb2e7c425ca9f376894ac7fa07d901c89007 WatchSource:0}: Error finding container 1e5860c1eef600e1f5051243b4efcb2e7c425ca9f376894ac7fa07d901c89007: Status 404 returned error can't find the container with id 1e5860c1eef600e1f5051243b4efcb2e7c425ca9f376894ac7fa07d901c89007
Jul 24 23:30:31.095956 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:31.095738    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 24 23:30:31.098655 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:30:31.098626    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1dcf2959_0da6_410b_80cd_3408b640aa7d.slice/crio-da0a662af5b715cc505eab6004df64fe6e234ada853029800eb36ddd159b6d7b WatchSource:0}: Error finding container da0a662af5b715cc505eab6004df64fe6e234ada853029800eb36ddd159b6d7b: Status 404 returned error can't find the container with id da0a662af5b715cc505eab6004df64fe6e234ada853029800eb36ddd159b6d7b
Jul 24 23:30:31.137028 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:31.136912    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:31.494917 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:31.494892    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-544bc854f6-wqshw"
Jul 24 23:30:31.592900 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:31.592866    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1dcf2959-0da6-410b-80cd-3408b640aa7d","Type":"ContainerStarted","Data":"da0a662af5b715cc505eab6004df64fe6e234ada853029800eb36ddd159b6d7b"}
Jul 24 23:30:31.593950 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:31.593929    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rhrlf" event={"ID":"39b86669-df55-45b0-ab05-f4a4df0926b1","Type":"ContainerStarted","Data":"d10d4b08b614e885eaed2c95bb8c569fd4e9cf9ae02b6d55af5ef802b40c6916"}
Jul 24 23:30:31.595363 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:31.595340    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-rplqt" event={"ID":"68864bf3-c771-4894-9f14-52ee9e8052f9","Type":"ContainerStarted","Data":"f64468b5c82940068161d083fffd417321db35922d343f15aa625aa9f9ce628b"}
Jul 24 23:30:31.596898 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:31.596871    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-gltb5" event={"ID":"7b7b9a28-3166-4af3-b27f-1ab5795c5f92","Type":"ContainerStarted","Data":"959b9443c951158568745c57a0c4cc2af2a07ab73b336095808147a8e76bca54"}
Jul 24 23:30:31.596997 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:31.596922    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-gltb5"
Jul 24 23:30:31.597867 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:31.597849    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9jpxt" event={"ID":"18c4ebca-b677-451a-bb9a-a2acda06fd79","Type":"ContainerStarted","Data":"1e5860c1eef600e1f5051243b4efcb2e7c425ca9f376894ac7fa07d901c89007"}
Jul 24 23:30:31.611169 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:31.611127    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-rplqt" podStartSLOduration=65.814231878 podStartE2EDuration="1m10.61111474s" totalImagesPullingTime="4.796882862s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 23:29:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:30:31.610292674 +0000 UTC m=+71.996723414" watchObservedRunningTime="2026-07-24 23:30:31.61111474 +0000 UTC m=+71.997545445"
Jul 24 23:30:31.623772 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:31.623725    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-gltb5" podStartSLOduration=65.858252215 podStartE2EDuration="1m10.62371s" totalImagesPullingTime="4.765457785s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:29:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:30:31.622966461 +0000 UTC m=+72.009397192" watchObservedRunningTime="2026-07-24 23:30:31.62371 +0000 UTC m=+72.010140773"
Jul 24 23:30:31.630853 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:31.630811    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:31.681739 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:31.680978    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:31.852824 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:31.852687    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:31.954449 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:31.954411    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:32.174311 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:32.174219    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:32.227397 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:32.227358    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:32.352056 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:32.352012    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:32.421119 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:32.421063    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:32.916699 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:32.916614    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:33.607601 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:33.607567    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9jpxt" event={"ID":"18c4ebca-b677-451a-bb9a-a2acda06fd79","Type":"ContainerStarted","Data":"2ceac00e2faf19c45ff39ed1f02d1b9734d43bdaa11e8f7a146675e3da4f9a3a"}
Jul 24 23:30:33.608061 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:33.607627    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9jpxt" event={"ID":"18c4ebca-b677-451a-bb9a-a2acda06fd79","Type":"ContainerStarted","Data":"54e6d582f96edbff86b490a8c09fa3250ed8ae625c945b171dff123d02b317e7"}
Jul 24 23:30:33.608061 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:33.607659    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9jpxt" event={"ID":"18c4ebca-b677-451a-bb9a-a2acda06fd79","Type":"ContainerStarted","Data":"97d701b1fe45488e29cadf23a4fd91f87c563967612f9eb31646feac83b78b02"}
Jul 24 23:30:33.609342 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:33.609311    2558 generic.go:356] "Generic (PLEG): container finished" podID="1dcf2959-0da6-410b-80cd-3408b640aa7d" containerID="60c09327e30eb6468720195a73f51e3a891817f3b14041f9b70a408b845cf480" exitCode=0
Jul 24 23:30:33.609476 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:33.609401    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1dcf2959-0da6-410b-80cd-3408b640aa7d","Type":"ContainerDied","Data":"60c09327e30eb6468720195a73f51e3a891817f3b14041f9b70a408b845cf480"}
Jul 24 23:30:33.611277 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:33.611179    2558 generic.go:356] "Generic (PLEG): container finished" podID="39b86669-df55-45b0-ab05-f4a4df0926b1" containerID="0a0d30cd3102a31040c4c87b14e93d795abec264ee78515ea12615ff420cf413" exitCode=0
Jul 24 23:30:33.611277 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:33.611227    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rhrlf" event={"ID":"39b86669-df55-45b0-ab05-f4a4df0926b1","Type":"ContainerDied","Data":"0a0d30cd3102a31040c4c87b14e93d795abec264ee78515ea12615ff420cf413"}
Jul 24 23:30:33.629851 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:33.629799    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9jpxt" podStartSLOduration=4.918837652 podStartE2EDuration="6.629778159s" totalImagesPullingTime="1.710940507s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:30:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:30:33.628161774 +0000 UTC m=+74.014592503" watchObservedRunningTime="2026-07-24 23:30:33.629778159 +0000 UTC m=+74.016208867"
Jul 24 23:30:33.788982 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:33.788946    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:33.856266 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:33.856229    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:34.100204 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:34.100161    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:34.183757 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:34.183714    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:34.619826 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:34.619783    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rhrlf" event={"ID":"39b86669-df55-45b0-ab05-f4a4df0926b1","Type":"ContainerStarted","Data":"42cca155bb6bb5fc8a221de675961d89c6ec088fc4bc746c8781c15947ca5e46"}
Jul 24 23:30:34.620266 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:34.619853    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rhrlf" event={"ID":"39b86669-df55-45b0-ab05-f4a4df0926b1","Type":"ContainerStarted","Data":"f7d45c6aabbce82c68695a3470e83306ac6ffb7859af9afb87e6f449e0d35e69"}
Jul 24 23:30:34.645141 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:34.640125    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-rhrlf" podStartSLOduration=5.706820401 podStartE2EDuration="7.640104544s" totalImagesPullingTime="1.85149917s" totalInitContainerRuntime="81.784973ms" isStatefulPod=true podCreationTimestamp="2026-07-24 23:30:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:30:34.638392442 +0000 UTC m=+75.024823166" watchObservedRunningTime="2026-07-24 23:30:34.640104544 +0000 UTC m=+75.026535273"
Jul 24 23:30:35.627022 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:35.626987    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1dcf2959-0da6-410b-80cd-3408b640aa7d","Type":"ContainerStarted","Data":"b164be19fb384784d0981732f5d9c58a7aafeb1fd314ca84f91ce5d3382c1015"}
Jul 24 23:30:35.627403 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:35.627087    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1dcf2959-0da6-410b-80cd-3408b640aa7d","Type":"ContainerStarted","Data":"9d44912b7436fc8b4c8d74049b213f19d2a680bcc8b1cb617288678652c1541c"}
Jul 24 23:30:35.627403 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:35.627115    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1dcf2959-0da6-410b-80cd-3408b640aa7d","Type":"ContainerStarted","Data":"5ceac84696b09dfb6b19c2d0095f8cde93c2a99feba266400bcca4bba01d9853"}
Jul 24 23:30:35.627403 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:35.627145    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1dcf2959-0da6-410b-80cd-3408b640aa7d","Type":"ContainerStarted","Data":"951688c1f70647cb5577da29833661708801e0903108bd94c58d67ec22c71e79"}
Jul 24 23:30:35.627403 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:35.627169    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1dcf2959-0da6-410b-80cd-3408b640aa7d","Type":"ContainerStarted","Data":"9291dc4db6d84ac78708196fa3ed2564880c7588a05f2160ff35b5c88ba16e19"}
Jul 24 23:30:35.641442 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:35.641410    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:35.696091 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:35.696057    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:36.634851 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:36.634808    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1dcf2959-0da6-410b-80cd-3408b640aa7d","Type":"ContainerStarted","Data":"2edf37abba6f17e6e6df98e6d29f9404956a96da2dd213a7d3427f0212d8a53f"}
Jul 24 23:30:36.664361 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:36.664293    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.862994808 podStartE2EDuration="8.66427217s" totalImagesPullingTime="3.781674455s" totalInitContainerRuntime="19.602907ms" isStatefulPod=false podCreationTimestamp="2026-07-24 23:30:28 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:30:36.662610667 +0000 UTC m=+77.049041396" watchObservedRunningTime="2026-07-24 23:30:36.66427217 +0000 UTC m=+77.050702899"
Jul 24 23:30:36.934459 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:36.934369    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:36.990753 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:36.990720    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:38.394869 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:38.394828    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:38.444907 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:38.444848    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:39.193051 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:39.193015    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:39.244697 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:39.244662    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:40.148452 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:40.148415    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:40.247712 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:40.247683    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:40.657201 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:40.657149    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:40.703819 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:40.703772    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:41.257064 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:41.257024    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:44.647587 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:44.647448    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:47.178669 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:47.178631    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:47.299886 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:47.299854    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:49.170688 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:49.170641    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:49.270340 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:49.270307    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:51.859523 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:51.859481    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:51.906962 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:51.906926    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:51.963414 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:51.963375    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:30:52.439987 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:30:52.439949    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:31:02.174739 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:31:02.174672    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:31:02.275359 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:31:02.275324    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:31:02.604696 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:31:02.604664    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-gltb5"
Jul 24 23:31:04.060431 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:31:04.060383    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:31:04.160502 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:31:04.160466    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:31:43.644347 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:31:43.644307    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:31:43.696519 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:31:43.696481    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:31:55.648138 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:31:55.648101    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:31:55.701438 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:31:55.701404    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:31:56.221912 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:31:56.221876    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:32:02.179281 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:32:02.179225    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:32:02.277265 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:32:02.277231    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:32:07.176215 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:32:07.176174    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:32:07.276367 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:32:07.276323    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:32:07.324061 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:32:07.324028    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:32:21.593123 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:32:21.593037    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:32:21.643341 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:32:21.643300    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:32:38.749586 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:32:38.749545    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:32:44.527483 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:32:44.527447    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:32:44.624434 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:32:44.624399    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:32:45.085471 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:32:45.085441    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qtbs4"]
Jul 24 23:32:45.088555 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:32:45.088520    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qtbs4"
Jul 24 23:32:45.091062 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:32:45.091034    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 24 23:32:45.092209 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:32:45.092186    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-wtdzv\""
Jul 24 23:32:45.092331 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:32:45.092219    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 24 23:32:45.096972 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:32:45.096947    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qtbs4"]
Jul 24 23:32:45.192161 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:32:45.192124    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/067a1e78-106b-4bb5-8fd3-063b670292b4-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qtbs4\" (UID: \"067a1e78-106b-4bb5-8fd3-063b670292b4\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qtbs4"
Jul 24 23:32:45.192161 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:32:45.192164    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p78kp\" (UniqueName: \"kubernetes.io/projected/067a1e78-106b-4bb5-8fd3-063b670292b4-kube-api-access-p78kp\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qtbs4\" (UID: \"067a1e78-106b-4bb5-8fd3-063b670292b4\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qtbs4"
Jul 24 23:32:45.192378 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:32:45.192202    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/067a1e78-106b-4bb5-8fd3-063b670292b4-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qtbs4\" (UID: \"067a1e78-106b-4bb5-8fd3-063b670292b4\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qtbs4"
Jul 24 23:32:45.293171 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:32:45.293123    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/067a1e78-106b-4bb5-8fd3-063b670292b4-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qtbs4\" (UID: \"067a1e78-106b-4bb5-8fd3-063b670292b4\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qtbs4"
Jul 24 23:32:45.293171 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:32:45.293177    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p78kp\" (UniqueName: \"kubernetes.io/projected/067a1e78-106b-4bb5-8fd3-063b670292b4-kube-api-access-p78kp\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qtbs4\" (UID: \"067a1e78-106b-4bb5-8fd3-063b670292b4\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qtbs4"
Jul 24 23:32:45.293373 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:32:45.293216    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/067a1e78-106b-4bb5-8fd3-063b670292b4-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qtbs4\" (UID: \"067a1e78-106b-4bb5-8fd3-063b670292b4\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qtbs4"
Jul 24 23:32:45.293509 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:32:45.293488    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/067a1e78-106b-4bb5-8fd3-063b670292b4-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qtbs4\" (UID: \"067a1e78-106b-4bb5-8fd3-063b670292b4\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qtbs4"
Jul 24 23:32:45.293569 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:32:45.293509    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/067a1e78-106b-4bb5-8fd3-063b670292b4-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qtbs4\" (UID: \"067a1e78-106b-4bb5-8fd3-063b670292b4\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qtbs4"
Jul 24 23:32:45.302425 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:32:45.302395    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p78kp\" (UniqueName: \"kubernetes.io/projected/067a1e78-106b-4bb5-8fd3-063b670292b4-kube-api-access-p78kp\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qtbs4\" (UID: \"067a1e78-106b-4bb5-8fd3-063b670292b4\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qtbs4"
Jul 24 23:32:45.397770 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:32:45.397689    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qtbs4"
Jul 24 23:32:45.523054 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:32:45.523030    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qtbs4"]
Jul 24 23:32:45.525368 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:32:45.525343    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod067a1e78_106b_4bb5_8fd3_063b670292b4.slice/crio-23f85a3c29a6dc1792373d8b537791d29f43310cb2266a715210672d36168254 WatchSource:0}: Error finding container 23f85a3c29a6dc1792373d8b537791d29f43310cb2266a715210672d36168254: Status 404 returned error can't find the container with id 23f85a3c29a6dc1792373d8b537791d29f43310cb2266a715210672d36168254
Jul 24 23:32:46.081741 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:32:46.081705    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qtbs4" event={"ID":"067a1e78-106b-4bb5-8fd3-063b670292b4","Type":"ContainerStarted","Data":"23f85a3c29a6dc1792373d8b537791d29f43310cb2266a715210672d36168254"}
Jul 24 23:32:53.109519 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:32:53.109479    2558 generic.go:356] "Generic (PLEG): container finished" podID="067a1e78-106b-4bb5-8fd3-063b670292b4" containerID="74e6c09cbca733ced30140e27ff8ed74d501fab18107cda99667416a1ae9bd4e" exitCode=0
Jul 24 23:32:53.110128 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:32:53.109581    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qtbs4" event={"ID":"067a1e78-106b-4bb5-8fd3-063b670292b4","Type":"ContainerDied","Data":"74e6c09cbca733ced30140e27ff8ed74d501fab18107cda99667416a1ae9bd4e"}
Jul 24 23:32:56.122134 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:32:56.122102    2558 generic.go:356] "Generic (PLEG): container finished" podID="067a1e78-106b-4bb5-8fd3-063b670292b4" containerID="0bfabf3d6f2464802114775078ddda9724da542bb5d3c7ae8f5a62ef57112eab" exitCode=0
Jul 24 23:32:56.122551 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:32:56.122191    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qtbs4" event={"ID":"067a1e78-106b-4bb5-8fd3-063b670292b4","Type":"ContainerDied","Data":"0bfabf3d6f2464802114775078ddda9724da542bb5d3c7ae8f5a62ef57112eab"}
Jul 24 23:33:04.155648 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:04.155615    2558 generic.go:356] "Generic (PLEG): container finished" podID="067a1e78-106b-4bb5-8fd3-063b670292b4" containerID="17e8f0f81bbdfe7164dc9834fd8f88d689e85f6d7c7b69705b6f939828375c32" exitCode=0
Jul 24 23:33:04.156039 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:04.155685    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qtbs4" event={"ID":"067a1e78-106b-4bb5-8fd3-063b670292b4","Type":"ContainerDied","Data":"17e8f0f81bbdfe7164dc9834fd8f88d689e85f6d7c7b69705b6f939828375c32"}
Jul 24 23:33:05.284825 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:05.284798    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qtbs4"
Jul 24 23:33:05.349705 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:05.349654    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p78kp\" (UniqueName: \"kubernetes.io/projected/067a1e78-106b-4bb5-8fd3-063b670292b4-kube-api-access-p78kp\") pod \"067a1e78-106b-4bb5-8fd3-063b670292b4\" (UID: \"067a1e78-106b-4bb5-8fd3-063b670292b4\") "
Jul 24 23:33:05.349705 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:05.349712    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/067a1e78-106b-4bb5-8fd3-063b670292b4-bundle\") pod \"067a1e78-106b-4bb5-8fd3-063b670292b4\" (UID: \"067a1e78-106b-4bb5-8fd3-063b670292b4\") "
Jul 24 23:33:05.349980 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:05.349744    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/067a1e78-106b-4bb5-8fd3-063b670292b4-util\") pod \"067a1e78-106b-4bb5-8fd3-063b670292b4\" (UID: \"067a1e78-106b-4bb5-8fd3-063b670292b4\") "
Jul 24 23:33:05.350343 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:05.350318    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/067a1e78-106b-4bb5-8fd3-063b670292b4-bundle" (OuterVolumeSpecName: "bundle") pod "067a1e78-106b-4bb5-8fd3-063b670292b4" (UID: "067a1e78-106b-4bb5-8fd3-063b670292b4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:33:05.351906 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:05.351885    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/067a1e78-106b-4bb5-8fd3-063b670292b4-kube-api-access-p78kp" (OuterVolumeSpecName: "kube-api-access-p78kp") pod "067a1e78-106b-4bb5-8fd3-063b670292b4" (UID: "067a1e78-106b-4bb5-8fd3-063b670292b4"). InnerVolumeSpecName "kube-api-access-p78kp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:33:05.354321 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:05.354300    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/067a1e78-106b-4bb5-8fd3-063b670292b4-util" (OuterVolumeSpecName: "util") pod "067a1e78-106b-4bb5-8fd3-063b670292b4" (UID: "067a1e78-106b-4bb5-8fd3-063b670292b4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:33:05.450207 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:05.450125    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p78kp\" (UniqueName: \"kubernetes.io/projected/067a1e78-106b-4bb5-8fd3-063b670292b4-kube-api-access-p78kp\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 24 23:33:05.450207 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:05.450152    2558 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/067a1e78-106b-4bb5-8fd3-063b670292b4-bundle\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 24 23:33:05.450207 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:05.450161    2558 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/067a1e78-106b-4bb5-8fd3-063b670292b4-util\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 24 23:33:06.164423 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:06.164395    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qtbs4"
Jul 24 23:33:06.164423 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:06.164400    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qtbs4" event={"ID":"067a1e78-106b-4bb5-8fd3-063b670292b4","Type":"ContainerDied","Data":"23f85a3c29a6dc1792373d8b537791d29f43310cb2266a715210672d36168254"}
Jul 24 23:33:06.164641 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:06.164441    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="23f85a3c29a6dc1792373d8b537791d29f43310cb2266a715210672d36168254"
Jul 24 23:33:11.112261 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:11.112229    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:33:11.908570 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:11.908503    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:33:13.053102 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:13.053072    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zvqrv"]
Jul 24 23:33:13.053483 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:13.053366    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="067a1e78-106b-4bb5-8fd3-063b670292b4" containerName="extract"
Jul 24 23:33:13.053483 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:13.053378    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="067a1e78-106b-4bb5-8fd3-063b670292b4" containerName="extract"
Jul 24 23:33:13.053483 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:13.053391    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="067a1e78-106b-4bb5-8fd3-063b670292b4" containerName="util"
Jul 24 23:33:13.053483 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:13.053396    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="067a1e78-106b-4bb5-8fd3-063b670292b4" containerName="util"
Jul 24 23:33:13.053483 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:13.053404    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="067a1e78-106b-4bb5-8fd3-063b670292b4" containerName="pull"
Jul 24 23:33:13.053483 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:13.053409    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="067a1e78-106b-4bb5-8fd3-063b670292b4" containerName="pull"
Jul 24 23:33:13.053483 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:13.053454    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="067a1e78-106b-4bb5-8fd3-063b670292b4" containerName="extract"
Jul 24 23:33:13.055284 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:13.055268    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zvqrv"
Jul 24 23:33:13.057593 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:13.057566    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 24 23:33:13.057718 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:13.057596    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-99bq5\""
Jul 24 23:33:13.057718 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:13.057629    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 24 23:33:13.070815 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:13.070789    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zvqrv"]
Jul 24 23:33:13.107078 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:13.107053    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4574g\" (UniqueName: \"kubernetes.io/projected/d8ca56e8-df79-4b95-a8aa-4ebbed001044-kube-api-access-4574g\") pod \"cert-manager-operator-controller-manager-55c444bf86-zvqrv\" (UID: \"d8ca56e8-df79-4b95-a8aa-4ebbed001044\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zvqrv"
Jul 24 23:33:13.107214 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:13.107106    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d8ca56e8-df79-4b95-a8aa-4ebbed001044-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-zvqrv\" (UID: \"d8ca56e8-df79-4b95-a8aa-4ebbed001044\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zvqrv"
Jul 24 23:33:13.208196 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:13.208169    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d8ca56e8-df79-4b95-a8aa-4ebbed001044-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-zvqrv\" (UID: \"d8ca56e8-df79-4b95-a8aa-4ebbed001044\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zvqrv"
Jul 24 23:33:13.208317 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:13.208219    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4574g\" (UniqueName: \"kubernetes.io/projected/d8ca56e8-df79-4b95-a8aa-4ebbed001044-kube-api-access-4574g\") pod \"cert-manager-operator-controller-manager-55c444bf86-zvqrv\" (UID: \"d8ca56e8-df79-4b95-a8aa-4ebbed001044\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zvqrv"
Jul 24 23:33:13.208553 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:13.208515    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d8ca56e8-df79-4b95-a8aa-4ebbed001044-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-zvqrv\" (UID: \"d8ca56e8-df79-4b95-a8aa-4ebbed001044\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zvqrv"
Jul 24 23:33:13.215926 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:13.215901    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4574g\" (UniqueName: \"kubernetes.io/projected/d8ca56e8-df79-4b95-a8aa-4ebbed001044-kube-api-access-4574g\") pod \"cert-manager-operator-controller-manager-55c444bf86-zvqrv\" (UID: \"d8ca56e8-df79-4b95-a8aa-4ebbed001044\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zvqrv"
Jul 24 23:33:13.364280 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:13.364187    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zvqrv"
Jul 24 23:33:13.490829 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:13.490797    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zvqrv"]
Jul 24 23:33:13.494158 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:33:13.494132    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8ca56e8_df79_4b95_a8aa_4ebbed001044.slice/crio-ce95ae1ec10e4bd58d464d4b17fe92dce6cafcd73cdb4e1c9b0cdac7934d904c WatchSource:0}: Error finding container ce95ae1ec10e4bd58d464d4b17fe92dce6cafcd73cdb4e1c9b0cdac7934d904c: Status 404 returned error can't find the container with id ce95ae1ec10e4bd58d464d4b17fe92dce6cafcd73cdb4e1c9b0cdac7934d904c
Jul 24 23:33:14.192393 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:14.192361    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zvqrv" event={"ID":"d8ca56e8-df79-4b95-a8aa-4ebbed001044","Type":"ContainerStarted","Data":"ce95ae1ec10e4bd58d464d4b17fe92dce6cafcd73cdb4e1c9b0cdac7934d904c"}
Jul 24 23:33:14.715259 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:14.715225    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:33:14.762037 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:14.762000    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:33:16.203623 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:16.203584    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zvqrv" event={"ID":"d8ca56e8-df79-4b95-a8aa-4ebbed001044","Type":"ContainerStarted","Data":"195047846fa3c1d27b2dfd6983943387128b37d55afd775be679958e6df6dc17"}
Jul 24 23:33:16.222996 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:16.222915    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zvqrv" podStartSLOduration=0.76296829 podStartE2EDuration="3.222899208s" totalImagesPullingTime="2.459930918s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:33:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:33:16.221100348 +0000 UTC m=+236.607531075" watchObservedRunningTime="2026-07-24 23:33:16.222899208 +0000 UTC m=+236.609329936"
Jul 24 23:33:16.579763 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:16.579729    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:33:16.699890 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:16.699848    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:33:16.872407 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:16.872301    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:33:16.895484 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:16.895445    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn4qnb"]
Jul 24 23:33:16.897872 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:16.897840    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn4qnb"
Jul 24 23:33:16.900603 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:16.900580    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 24 23:33:16.900722 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:16.900708    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 24 23:33:16.901605 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:16.901586    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-wtdzv\""
Jul 24 23:33:16.908175 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:16.908151    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn4qnb"]
Jul 24 23:33:16.937835 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:16.937802    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1ccf819e-a948-4382-8269-041db22c1ec1-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn4qnb\" (UID: \"1ccf819e-a948-4382-8269-041db22c1ec1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn4qnb"
Jul 24 23:33:16.937995 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:16.937847    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1ccf819e-a948-4382-8269-041db22c1ec1-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn4qnb\" (UID: \"1ccf819e-a948-4382-8269-041db22c1ec1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn4qnb"
Jul 24 23:33:16.937995 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:16.937874    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qz7x8\" (UniqueName: \"kubernetes.io/projected/1ccf819e-a948-4382-8269-041db22c1ec1-kube-api-access-qz7x8\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn4qnb\" (UID: \"1ccf819e-a948-4382-8269-041db22c1ec1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn4qnb"
Jul 24 23:33:16.962167 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:16.962140    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:33:17.039324 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:17.039223    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1ccf819e-a948-4382-8269-041db22c1ec1-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn4qnb\" (UID: \"1ccf819e-a948-4382-8269-041db22c1ec1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn4qnb"
Jul 24 23:33:17.039466 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:17.039366    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1ccf819e-a948-4382-8269-041db22c1ec1-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn4qnb\" (UID: \"1ccf819e-a948-4382-8269-041db22c1ec1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn4qnb"
Jul 24 23:33:17.039466 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:17.039395    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qz7x8\" (UniqueName: \"kubernetes.io/projected/1ccf819e-a948-4382-8269-041db22c1ec1-kube-api-access-qz7x8\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn4qnb\" (UID: \"1ccf819e-a948-4382-8269-041db22c1ec1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn4qnb"
Jul 24 23:33:17.039679 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:17.039654    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1ccf819e-a948-4382-8269-041db22c1ec1-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn4qnb\" (UID: \"1ccf819e-a948-4382-8269-041db22c1ec1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn4qnb"
Jul 24 23:33:17.039721 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:17.039687    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1ccf819e-a948-4382-8269-041db22c1ec1-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn4qnb\" (UID: \"1ccf819e-a948-4382-8269-041db22c1ec1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn4qnb"
Jul 24 23:33:17.049849 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:17.049818    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qz7x8\" (UniqueName: \"kubernetes.io/projected/1ccf819e-a948-4382-8269-041db22c1ec1-kube-api-access-qz7x8\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn4qnb\" (UID: \"1ccf819e-a948-4382-8269-041db22c1ec1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn4qnb"
Jul 24 23:33:17.206539 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:17.206445    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn4qnb"
Jul 24 23:33:17.334366 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:17.334305    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn4qnb"]
Jul 24 23:33:17.336868 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:33:17.336843    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ccf819e_a948_4382_8269_041db22c1ec1.slice/crio-7dbffe0ed8a768c93dd6fa9a1e193bb3810362f41289813b20a55bbaed969bc9 WatchSource:0}: Error finding container 7dbffe0ed8a768c93dd6fa9a1e193bb3810362f41289813b20a55bbaed969bc9: Status 404 returned error can't find the container with id 7dbffe0ed8a768c93dd6fa9a1e193bb3810362f41289813b20a55bbaed969bc9
Jul 24 23:33:18.029476 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:18.029443    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-k22c9"]
Jul 24 23:33:18.031963 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:18.031944    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-k22c9"
Jul 24 23:33:18.034300 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:18.034278    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-n7bxq\""
Jul 24 23:33:18.034300 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:18.034294    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 24 23:33:18.035168 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:18.035149    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 24 23:33:18.040625 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:18.040602    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-k22c9"]
Jul 24 23:33:18.149492 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:18.149446    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqbvj\" (UniqueName: \"kubernetes.io/projected/a42763b7-8dd0-431f-8c80-5e1d3b944376-kube-api-access-xqbvj\") pod \"cert-manager-webhook-6486ffd997-k22c9\" (UID: \"a42763b7-8dd0-431f-8c80-5e1d3b944376\") " pod="cert-manager/cert-manager-webhook-6486ffd997-k22c9"
Jul 24 23:33:18.149690 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:18.149505    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a42763b7-8dd0-431f-8c80-5e1d3b944376-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-k22c9\" (UID: \"a42763b7-8dd0-431f-8c80-5e1d3b944376\") " pod="cert-manager/cert-manager-webhook-6486ffd997-k22c9"
Jul 24 23:33:18.213125 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:18.213087    2558 generic.go:356] "Generic (PLEG): container finished" podID="1ccf819e-a948-4382-8269-041db22c1ec1" containerID="c2a1e44f3817d0326fe075986cc8602b48ec6c380a712557a8baa1de6e6434c4" exitCode=0
Jul 24 23:33:18.213553 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:18.213177    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn4qnb" event={"ID":"1ccf819e-a948-4382-8269-041db22c1ec1","Type":"ContainerDied","Data":"c2a1e44f3817d0326fe075986cc8602b48ec6c380a712557a8baa1de6e6434c4"}
Jul 24 23:33:18.213553 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:18.213237    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn4qnb" event={"ID":"1ccf819e-a948-4382-8269-041db22c1ec1","Type":"ContainerStarted","Data":"7dbffe0ed8a768c93dd6fa9a1e193bb3810362f41289813b20a55bbaed969bc9"}
Jul 24 23:33:18.250859 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:18.250825    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqbvj\" (UniqueName: \"kubernetes.io/projected/a42763b7-8dd0-431f-8c80-5e1d3b944376-kube-api-access-xqbvj\") pod \"cert-manager-webhook-6486ffd997-k22c9\" (UID: \"a42763b7-8dd0-431f-8c80-5e1d3b944376\") " pod="cert-manager/cert-manager-webhook-6486ffd997-k22c9"
Jul 24 23:33:18.251048 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:18.250866    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a42763b7-8dd0-431f-8c80-5e1d3b944376-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-k22c9\" (UID: \"a42763b7-8dd0-431f-8c80-5e1d3b944376\") " pod="cert-manager/cert-manager-webhook-6486ffd997-k22c9"
Jul 24 23:33:18.258399 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:18.258366    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a42763b7-8dd0-431f-8c80-5e1d3b944376-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-k22c9\" (UID: \"a42763b7-8dd0-431f-8c80-5e1d3b944376\") " pod="cert-manager/cert-manager-webhook-6486ffd997-k22c9"
Jul 24 23:33:18.258399 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:18.258384    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqbvj\" (UniqueName: \"kubernetes.io/projected/a42763b7-8dd0-431f-8c80-5e1d3b944376-kube-api-access-xqbvj\") pod \"cert-manager-webhook-6486ffd997-k22c9\" (UID: \"a42763b7-8dd0-431f-8c80-5e1d3b944376\") " pod="cert-manager/cert-manager-webhook-6486ffd997-k22c9"
Jul 24 23:33:18.346610 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:18.346465    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-k22c9"
Jul 24 23:33:18.470423 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:18.470384    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-k22c9"]
Jul 24 23:33:18.473408 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:33:18.473381    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda42763b7_8dd0_431f_8c80_5e1d3b944376.slice/crio-5512fde84bee4d301b03d6c37fb09c89c7b5d5f97e5549ee6e0d69a89f52cdce WatchSource:0}: Error finding container 5512fde84bee4d301b03d6c37fb09c89c7b5d5f97e5549ee6e0d69a89f52cdce: Status 404 returned error can't find the container with id 5512fde84bee4d301b03d6c37fb09c89c7b5d5f97e5549ee6e0d69a89f52cdce
Jul 24 23:33:19.222335 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:19.222270    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-k22c9" event={"ID":"a42763b7-8dd0-431f-8c80-5e1d3b944376","Type":"ContainerStarted","Data":"5512fde84bee4d301b03d6c37fb09c89c7b5d5f97e5549ee6e0d69a89f52cdce"}
Jul 24 23:33:19.243169 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:19.243115    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:33:19.659404 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:19.659339    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:33:19.756454 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:19.756407    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-d2svq"]
Jul 24 23:33:19.762972 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:19.762931    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-d2svq"
Jul 24 23:33:19.765374 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:19.765344    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-ps4gn\""
Jul 24 23:33:19.769205 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:19.769178    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-d2svq"]
Jul 24 23:33:19.864890 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:19.864848    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ad139f66-3154-465b-b3e5-8dff57da7ac9-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-d2svq\" (UID: \"ad139f66-3154-465b-b3e5-8dff57da7ac9\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-d2svq"
Jul 24 23:33:19.865080 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:19.864916    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kz8nd\" (UniqueName: \"kubernetes.io/projected/ad139f66-3154-465b-b3e5-8dff57da7ac9-kube-api-access-kz8nd\") pod \"cert-manager-cainjector-559cdd8c87-d2svq\" (UID: \"ad139f66-3154-465b-b3e5-8dff57da7ac9\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-d2svq"
Jul 24 23:33:19.965475 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:19.965377    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ad139f66-3154-465b-b3e5-8dff57da7ac9-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-d2svq\" (UID: \"ad139f66-3154-465b-b3e5-8dff57da7ac9\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-d2svq"
Jul 24 23:33:19.965475 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:19.965451    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kz8nd\" (UniqueName: \"kubernetes.io/projected/ad139f66-3154-465b-b3e5-8dff57da7ac9-kube-api-access-kz8nd\") pod \"cert-manager-cainjector-559cdd8c87-d2svq\" (UID: \"ad139f66-3154-465b-b3e5-8dff57da7ac9\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-d2svq"
Jul 24 23:33:19.973807 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:19.973773    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ad139f66-3154-465b-b3e5-8dff57da7ac9-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-d2svq\" (UID: \"ad139f66-3154-465b-b3e5-8dff57da7ac9\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-d2svq"
Jul 24 23:33:19.973942 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:19.973916    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kz8nd\" (UniqueName: \"kubernetes.io/projected/ad139f66-3154-465b-b3e5-8dff57da7ac9-kube-api-access-kz8nd\") pod \"cert-manager-cainjector-559cdd8c87-d2svq\" (UID: \"ad139f66-3154-465b-b3e5-8dff57da7ac9\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-d2svq"
Jul 24 23:33:20.078954 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:20.078920    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-ps4gn\""
Jul 24 23:33:20.086881 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:20.086839    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-d2svq"
Jul 24 23:33:21.368476 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:21.368257    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-d2svq"]
Jul 24 23:33:21.371710 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:33:21.371669    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podad139f66_3154_465b_b3e5_8dff57da7ac9.slice/crio-bb5fe0ee7c5ed573ea791ee4fc9251651d14164ddefae2c1909b90df4e1a228d WatchSource:0}: Error finding container bb5fe0ee7c5ed573ea791ee4fc9251651d14164ddefae2c1909b90df4e1a228d: Status 404 returned error can't find the container with id bb5fe0ee7c5ed573ea791ee4fc9251651d14164ddefae2c1909b90df4e1a228d
Jul 24 23:33:21.522196 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:21.522109    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:33:21.621927 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:21.621871    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:33:22.238776 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:22.238739    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-d2svq" event={"ID":"ad139f66-3154-465b-b3e5-8dff57da7ac9","Type":"ContainerStarted","Data":"6bd7c4e145f25dfd6e3501a9f11f347956463557919375992739184e721cd6bd"}
Jul 24 23:33:22.238927 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:22.238794    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-d2svq" event={"ID":"ad139f66-3154-465b-b3e5-8dff57da7ac9","Type":"ContainerStarted","Data":"bb5fe0ee7c5ed573ea791ee4fc9251651d14164ddefae2c1909b90df4e1a228d"}
Jul 24 23:33:22.240418 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:22.240394    2558 generic.go:356] "Generic (PLEG): container finished" podID="1ccf819e-a948-4382-8269-041db22c1ec1" containerID="b35dfbb23315cd3e793f96e605b90e1f19113a047c206a886332c04c09aa4f57" exitCode=0
Jul 24 23:33:22.240518 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:22.240483    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn4qnb" event={"ID":"1ccf819e-a948-4382-8269-041db22c1ec1","Type":"ContainerDied","Data":"b35dfbb23315cd3e793f96e605b90e1f19113a047c206a886332c04c09aa4f57"}
Jul 24 23:33:22.242009 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:22.241990    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-k22c9" event={"ID":"a42763b7-8dd0-431f-8c80-5e1d3b944376","Type":"ContainerStarted","Data":"9713f876f2413949309b2817e72b3cac06ed38d069fc51f8d616e174c54fce5f"}
Jul 24 23:33:22.242141 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:22.242126    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-k22c9"
Jul 24 23:33:22.254955 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:22.254909    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-d2svq" podStartSLOduration=3.254890439 podStartE2EDuration="3.254890439s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:33:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:33:22.253170055 +0000 UTC m=+242.639600798" watchObservedRunningTime="2026-07-24 23:33:22.254890439 +0000 UTC m=+242.641321170"
Jul 24 23:33:22.287233 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:22.287187    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-k22c9" podStartSLOduration=1.5232452109999999 podStartE2EDuration="4.287173027s" totalImagesPullingTime="2.763927816s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:33:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:33:22.285440325 +0000 UTC m=+242.671871053" watchObservedRunningTime="2026-07-24 23:33:22.287173027 +0000 UTC m=+242.673603755"
Jul 24 23:33:23.248420 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:23.248380    2558 generic.go:356] "Generic (PLEG): container finished" podID="1ccf819e-a948-4382-8269-041db22c1ec1" containerID="4059b0b5ead8690ef91614ab2f059fb196ffc2603787e7dba196c4cc957b66f0" exitCode=0
Jul 24 23:33:23.248801 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:23.248469    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn4qnb" event={"ID":"1ccf819e-a948-4382-8269-041db22c1ec1","Type":"ContainerDied","Data":"4059b0b5ead8690ef91614ab2f059fb196ffc2603787e7dba196c4cc957b66f0"}
Jul 24 23:33:24.376356 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:24.376332    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn4qnb"
Jul 24 23:33:24.504273 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:24.504190    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1ccf819e-a948-4382-8269-041db22c1ec1-bundle\") pod \"1ccf819e-a948-4382-8269-041db22c1ec1\" (UID: \"1ccf819e-a948-4382-8269-041db22c1ec1\") "
Jul 24 23:33:24.504273 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:24.504264    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1ccf819e-a948-4382-8269-041db22c1ec1-util\") pod \"1ccf819e-a948-4382-8269-041db22c1ec1\" (UID: \"1ccf819e-a948-4382-8269-041db22c1ec1\") "
Jul 24 23:33:24.504496 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:24.504304    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qz7x8\" (UniqueName: \"kubernetes.io/projected/1ccf819e-a948-4382-8269-041db22c1ec1-kube-api-access-qz7x8\") pod \"1ccf819e-a948-4382-8269-041db22c1ec1\" (UID: \"1ccf819e-a948-4382-8269-041db22c1ec1\") "
Jul 24 23:33:24.504612 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:24.504585    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ccf819e-a948-4382-8269-041db22c1ec1-bundle" (OuterVolumeSpecName: "bundle") pod "1ccf819e-a948-4382-8269-041db22c1ec1" (UID: "1ccf819e-a948-4382-8269-041db22c1ec1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:33:24.506444 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:24.506415    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ccf819e-a948-4382-8269-041db22c1ec1-kube-api-access-qz7x8" (OuterVolumeSpecName: "kube-api-access-qz7x8") pod "1ccf819e-a948-4382-8269-041db22c1ec1" (UID: "1ccf819e-a948-4382-8269-041db22c1ec1"). InnerVolumeSpecName "kube-api-access-qz7x8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:33:24.508949 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:24.508913    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ccf819e-a948-4382-8269-041db22c1ec1-util" (OuterVolumeSpecName: "util") pod "1ccf819e-a948-4382-8269-041db22c1ec1" (UID: "1ccf819e-a948-4382-8269-041db22c1ec1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:33:24.605575 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:24.605524    2558 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1ccf819e-a948-4382-8269-041db22c1ec1-bundle\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 24 23:33:24.605575 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:24.605571    2558 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1ccf819e-a948-4382-8269-041db22c1ec1-util\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 24 23:33:24.605575 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:24.605580    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qz7x8\" (UniqueName: \"kubernetes.io/projected/1ccf819e-a948-4382-8269-041db22c1ec1-kube-api-access-qz7x8\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 24 23:33:25.258047 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:25.258018    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn4qnb"
Jul 24 23:33:25.258047 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:25.258031    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn4qnb" event={"ID":"1ccf819e-a948-4382-8269-041db22c1ec1","Type":"ContainerDied","Data":"7dbffe0ed8a768c93dd6fa9a1e193bb3810362f41289813b20a55bbaed969bc9"}
Jul 24 23:33:25.258290 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:25.258070    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7dbffe0ed8a768c93dd6fa9a1e193bb3810362f41289813b20a55bbaed969bc9"
Jul 24 23:33:26.451832 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:26.451803    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:33:28.251079 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:28.251050    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-k22c9"
Jul 24 23:33:29.181842 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:29.181802    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:33:29.282723 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:29.282683    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:33:31.028856 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:31.028822    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kkhvt"]
Jul 24 23:33:31.029246 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:31.029095    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ccf819e-a948-4382-8269-041db22c1ec1" containerName="util"
Jul 24 23:33:31.029246 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:31.029106    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ccf819e-a948-4382-8269-041db22c1ec1" containerName="util"
Jul 24 23:33:31.029246 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:31.029118    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ccf819e-a948-4382-8269-041db22c1ec1" containerName="pull"
Jul 24 23:33:31.029246 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:31.029124    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ccf819e-a948-4382-8269-041db22c1ec1" containerName="pull"
Jul 24 23:33:31.029246 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:31.029132    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ccf819e-a948-4382-8269-041db22c1ec1" containerName="extract"
Jul 24 23:33:31.029246 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:31.029138    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ccf819e-a948-4382-8269-041db22c1ec1" containerName="extract"
Jul 24 23:33:31.029246 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:31.029185    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ccf819e-a948-4382-8269-041db22c1ec1" containerName="extract"
Jul 24 23:33:31.033461 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:31.033442    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kkhvt"
Jul 24 23:33:31.035709 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:31.035684    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 24 23:33:31.035811 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:31.035705    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 24 23:33:31.035811 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:31.035706    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-q94kv\""
Jul 24 23:33:31.038887 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:31.038864    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kkhvt"]
Jul 24 23:33:31.164051 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:31.164019    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6696c8f5-c130-4c9d-a1be-b7a12be36a83-tmp\") pod \"openshift-lws-operator-fd8ccff4c-kkhvt\" (UID: \"6696c8f5-c130-4c9d-a1be-b7a12be36a83\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kkhvt"
Jul 24 23:33:31.164244 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:31.164074    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8gj6\" (UniqueName: \"kubernetes.io/projected/6696c8f5-c130-4c9d-a1be-b7a12be36a83-kube-api-access-h8gj6\") pod \"openshift-lws-operator-fd8ccff4c-kkhvt\" (UID: \"6696c8f5-c130-4c9d-a1be-b7a12be36a83\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kkhvt"
Jul 24 23:33:31.265118 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:31.265076    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6696c8f5-c130-4c9d-a1be-b7a12be36a83-tmp\") pod \"openshift-lws-operator-fd8ccff4c-kkhvt\" (UID: \"6696c8f5-c130-4c9d-a1be-b7a12be36a83\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kkhvt"
Jul 24 23:33:31.265307 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:31.265145    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8gj6\" (UniqueName: \"kubernetes.io/projected/6696c8f5-c130-4c9d-a1be-b7a12be36a83-kube-api-access-h8gj6\") pod \"openshift-lws-operator-fd8ccff4c-kkhvt\" (UID: \"6696c8f5-c130-4c9d-a1be-b7a12be36a83\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kkhvt"
Jul 24 23:33:31.265511 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:31.265488    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6696c8f5-c130-4c9d-a1be-b7a12be36a83-tmp\") pod \"openshift-lws-operator-fd8ccff4c-kkhvt\" (UID: \"6696c8f5-c130-4c9d-a1be-b7a12be36a83\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kkhvt"
Jul 24 23:33:31.272457 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:31.272433    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8gj6\" (UniqueName: \"kubernetes.io/projected/6696c8f5-c130-4c9d-a1be-b7a12be36a83-kube-api-access-h8gj6\") pod \"openshift-lws-operator-fd8ccff4c-kkhvt\" (UID: \"6696c8f5-c130-4c9d-a1be-b7a12be36a83\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kkhvt"
Jul 24 23:33:31.343516 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:31.343448    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kkhvt"
Jul 24 23:33:31.484306 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:31.484278    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kkhvt"]
Jul 24 23:33:31.487127 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:33:31.487104    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6696c8f5_c130_4c9d_a1be_b7a12be36a83.slice/crio-7e4be437f86ffd46121a582703ec391a715c0cdc3c38397e9f70c27514451795 WatchSource:0}: Error finding container 7e4be437f86ffd46121a582703ec391a715c0cdc3c38397e9f70c27514451795: Status 404 returned error can't find the container with id 7e4be437f86ffd46121a582703ec391a715c0cdc3c38397e9f70c27514451795
Jul 24 23:33:32.284861 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:32.284822    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kkhvt" event={"ID":"6696c8f5-c130-4c9d-a1be-b7a12be36a83","Type":"ContainerStarted","Data":"7e4be437f86ffd46121a582703ec391a715c0cdc3c38397e9f70c27514451795"}
Jul 24 23:33:36.302713 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:36.302617    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kkhvt" event={"ID":"6696c8f5-c130-4c9d-a1be-b7a12be36a83","Type":"ContainerStarted","Data":"8f654d2fdf1239aaf6ea6890e9e87716e866fe7d79d803ff6a9cb3ca124f6893"}
Jul 24 23:33:36.319272 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:36.319220    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kkhvt" podStartSLOduration=0.783796306 podStartE2EDuration="5.319205436s" totalImagesPullingTime="4.53540913s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:33:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:33:36.317466893 +0000 UTC m=+256.703897622" watchObservedRunningTime="2026-07-24 23:33:36.319205436 +0000 UTC m=+256.705636164"
Jul 24 23:33:37.052261 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:37.052213    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:33:37.154287 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:37.154252    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:33:38.609437 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:38.609404    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8mf62"]
Jul 24 23:33:38.611724 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:38.611706    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8mf62"
Jul 24 23:33:38.614083 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:38.614061    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 24 23:33:38.614202 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:38.614097    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-wtdzv\""
Jul 24 23:33:38.614891 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:38.614876    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 24 23:33:38.620556 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:38.620515    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8mf62"]
Jul 24 23:33:38.727048 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:38.727010    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9r5xk\" (UniqueName: \"kubernetes.io/projected/3c9f3be9-c8a8-4d12-9369-5c10acdf514d-kube-api-access-9r5xk\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8mf62\" (UID: \"3c9f3be9-c8a8-4d12-9369-5c10acdf514d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8mf62"
Jul 24 23:33:38.727214 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:38.727080    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3c9f3be9-c8a8-4d12-9369-5c10acdf514d-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8mf62\" (UID: \"3c9f3be9-c8a8-4d12-9369-5c10acdf514d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8mf62"
Jul 24 23:33:38.727214 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:38.727111    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3c9f3be9-c8a8-4d12-9369-5c10acdf514d-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8mf62\" (UID: \"3c9f3be9-c8a8-4d12-9369-5c10acdf514d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8mf62"
Jul 24 23:33:38.828369 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:38.828331    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3c9f3be9-c8a8-4d12-9369-5c10acdf514d-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8mf62\" (UID: \"3c9f3be9-c8a8-4d12-9369-5c10acdf514d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8mf62"
Jul 24 23:33:38.828550 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:38.828381    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3c9f3be9-c8a8-4d12-9369-5c10acdf514d-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8mf62\" (UID: \"3c9f3be9-c8a8-4d12-9369-5c10acdf514d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8mf62"
Jul 24 23:33:38.828550 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:38.828442    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9r5xk\" (UniqueName: \"kubernetes.io/projected/3c9f3be9-c8a8-4d12-9369-5c10acdf514d-kube-api-access-9r5xk\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8mf62\" (UID: \"3c9f3be9-c8a8-4d12-9369-5c10acdf514d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8mf62"
Jul 24 23:33:38.828840 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:38.828823    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3c9f3be9-c8a8-4d12-9369-5c10acdf514d-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8mf62\" (UID: \"3c9f3be9-c8a8-4d12-9369-5c10acdf514d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8mf62"
Jul 24 23:33:38.828875 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:38.828824    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3c9f3be9-c8a8-4d12-9369-5c10acdf514d-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8mf62\" (UID: \"3c9f3be9-c8a8-4d12-9369-5c10acdf514d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8mf62"
Jul 24 23:33:38.837425 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:38.837403    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9r5xk\" (UniqueName: \"kubernetes.io/projected/3c9f3be9-c8a8-4d12-9369-5c10acdf514d-kube-api-access-9r5xk\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8mf62\" (UID: \"3c9f3be9-c8a8-4d12-9369-5c10acdf514d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8mf62"
Jul 24 23:33:38.921554 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:38.921445    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8mf62"
Jul 24 23:33:39.062778 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:39.062752    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8mf62"]
Jul 24 23:33:39.064928 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:33:39.064901    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c9f3be9_c8a8_4d12_9369_5c10acdf514d.slice/crio-332e1efb13683eac0eb9b5f1c26cc2bd41a64147e2b9d2fd9dee048402e07a82 WatchSource:0}: Error finding container 332e1efb13683eac0eb9b5f1c26cc2bd41a64147e2b9d2fd9dee048402e07a82: Status 404 returned error can't find the container with id 332e1efb13683eac0eb9b5f1c26cc2bd41a64147e2b9d2fd9dee048402e07a82
Jul 24 23:33:39.292315 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:39.292281    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:33:39.315852 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:39.315812    2558 generic.go:356] "Generic (PLEG): container finished" podID="3c9f3be9-c8a8-4d12-9369-5c10acdf514d" containerID="d13becf6efb683250ad5a25eb95689814fb7e8d73ce4634f3110319db85ae9d5" exitCode=0
Jul 24 23:33:39.316005 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:39.315901    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8mf62" event={"ID":"3c9f3be9-c8a8-4d12-9369-5c10acdf514d","Type":"ContainerDied","Data":"d13becf6efb683250ad5a25eb95689814fb7e8d73ce4634f3110319db85ae9d5"}
Jul 24 23:33:39.316005 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:39.315982    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8mf62" event={"ID":"3c9f3be9-c8a8-4d12-9369-5c10acdf514d","Type":"ContainerStarted","Data":"332e1efb13683eac0eb9b5f1c26cc2bd41a64147e2b9d2fd9dee048402e07a82"}
Jul 24 23:33:40.321302 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:40.321264    2558 generic.go:356] "Generic (PLEG): container finished" podID="3c9f3be9-c8a8-4d12-9369-5c10acdf514d" containerID="fd3dce6abefc19e6def374f9cdceac33989bc27e97d5a20df9bb63f4110a2012" exitCode=0
Jul 24 23:33:40.321745 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:40.321346    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8mf62" event={"ID":"3c9f3be9-c8a8-4d12-9369-5c10acdf514d","Type":"ContainerDied","Data":"fd3dce6abefc19e6def374f9cdceac33989bc27e97d5a20df9bb63f4110a2012"}
Jul 24 23:33:41.327725 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:41.327694    2558 generic.go:356] "Generic (PLEG): container finished" podID="3c9f3be9-c8a8-4d12-9369-5c10acdf514d" containerID="558b0ccfa92e6baeff4ccfa65399e660bf4caa38c20dfa94f6738bed2a9837fe" exitCode=0
Jul 24 23:33:41.328100 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:41.327760    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8mf62" event={"ID":"3c9f3be9-c8a8-4d12-9369-5c10acdf514d","Type":"ContainerDied","Data":"558b0ccfa92e6baeff4ccfa65399e660bf4caa38c20dfa94f6738bed2a9837fe"}
Jul 24 23:33:42.450249 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:42.450225    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8mf62"
Jul 24 23:33:42.557252 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:42.557219    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9r5xk\" (UniqueName: \"kubernetes.io/projected/3c9f3be9-c8a8-4d12-9369-5c10acdf514d-kube-api-access-9r5xk\") pod \"3c9f3be9-c8a8-4d12-9369-5c10acdf514d\" (UID: \"3c9f3be9-c8a8-4d12-9369-5c10acdf514d\") "
Jul 24 23:33:42.557399 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:42.557268    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3c9f3be9-c8a8-4d12-9369-5c10acdf514d-util\") pod \"3c9f3be9-c8a8-4d12-9369-5c10acdf514d\" (UID: \"3c9f3be9-c8a8-4d12-9369-5c10acdf514d\") "
Jul 24 23:33:42.557399 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:42.557302    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3c9f3be9-c8a8-4d12-9369-5c10acdf514d-bundle\") pod \"3c9f3be9-c8a8-4d12-9369-5c10acdf514d\" (UID: \"3c9f3be9-c8a8-4d12-9369-5c10acdf514d\") "
Jul 24 23:33:42.558068 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:42.558047    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c9f3be9-c8a8-4d12-9369-5c10acdf514d-bundle" (OuterVolumeSpecName: "bundle") pod "3c9f3be9-c8a8-4d12-9369-5c10acdf514d" (UID: "3c9f3be9-c8a8-4d12-9369-5c10acdf514d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:33:42.559338 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:42.559314    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c9f3be9-c8a8-4d12-9369-5c10acdf514d-kube-api-access-9r5xk" (OuterVolumeSpecName: "kube-api-access-9r5xk") pod "3c9f3be9-c8a8-4d12-9369-5c10acdf514d" (UID: "3c9f3be9-c8a8-4d12-9369-5c10acdf514d"). InnerVolumeSpecName "kube-api-access-9r5xk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:33:42.563155 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:42.563046    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c9f3be9-c8a8-4d12-9369-5c10acdf514d-util" (OuterVolumeSpecName: "util") pod "3c9f3be9-c8a8-4d12-9369-5c10acdf514d" (UID: "3c9f3be9-c8a8-4d12-9369-5c10acdf514d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:33:42.658277 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:42.658205    2558 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3c9f3be9-c8a8-4d12-9369-5c10acdf514d-bundle\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 24 23:33:42.658277 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:42.658230    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9r5xk\" (UniqueName: \"kubernetes.io/projected/3c9f3be9-c8a8-4d12-9369-5c10acdf514d-kube-api-access-9r5xk\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 24 23:33:42.658277 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:42.658239    2558 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3c9f3be9-c8a8-4d12-9369-5c10acdf514d-util\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 24 23:33:43.337433 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:43.337394    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8mf62" event={"ID":"3c9f3be9-c8a8-4d12-9369-5c10acdf514d","Type":"ContainerDied","Data":"332e1efb13683eac0eb9b5f1c26cc2bd41a64147e2b9d2fd9dee048402e07a82"}
Jul 24 23:33:43.337433 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:43.337412    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8mf62"
Jul 24 23:33:43.337433 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:43.337441    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="332e1efb13683eac0eb9b5f1c26cc2bd41a64147e2b9d2fd9dee048402e07a82"
Jul 24 23:33:51.758162 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:51.758101    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:33:51.846978 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:51.846937    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:33:52.393902 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:52.393851    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:33:52.588982 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:52.588944    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:33:52.693370 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:52.693281    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:33:52.719476 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:52.719438    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-69778f9898-b4rzt"]
Jul 24 23:33:52.719743 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:52.719730    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c9f3be9-c8a8-4d12-9369-5c10acdf514d" containerName="pull"
Jul 24 23:33:52.719743 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:52.719743    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c9f3be9-c8a8-4d12-9369-5c10acdf514d" containerName="pull"
Jul 24 23:33:52.719821 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:52.719753    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c9f3be9-c8a8-4d12-9369-5c10acdf514d" containerName="util"
Jul 24 23:33:52.719821 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:52.719759    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c9f3be9-c8a8-4d12-9369-5c10acdf514d" containerName="util"
Jul 24 23:33:52.719821 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:52.719766    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c9f3be9-c8a8-4d12-9369-5c10acdf514d" containerName="extract"
Jul 24 23:33:52.719821 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:52.719771    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c9f3be9-c8a8-4d12-9369-5c10acdf514d" containerName="extract"
Jul 24 23:33:52.719821 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:52.719817    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c9f3be9-c8a8-4d12-9369-5c10acdf514d" containerName="extract"
Jul 24 23:33:52.722662 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:52.722646    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-69778f9898-b4rzt"
Jul 24 23:33:52.725437 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:52.725409    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 24 23:33:52.725437 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:52.725438    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 24 23:33:52.725668 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:52.725466    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 24 23:33:52.725668 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:52.725484    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-r9vz8\""
Jul 24 23:33:52.725668 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:52.725488    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 24 23:33:52.738677 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:52.738592    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ffa2c446-0664-4bd8-be20-99c469570a69-webhook-cert\") pod \"opendatahub-operator-controller-manager-69778f9898-b4rzt\" (UID: \"ffa2c446-0664-4bd8-be20-99c469570a69\") " pod="opendatahub/opendatahub-operator-controller-manager-69778f9898-b4rzt"
Jul 24 23:33:52.738677 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:52.738633    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4r85\" (UniqueName: \"kubernetes.io/projected/ffa2c446-0664-4bd8-be20-99c469570a69-kube-api-access-w4r85\") pod \"opendatahub-operator-controller-manager-69778f9898-b4rzt\" (UID: \"ffa2c446-0664-4bd8-be20-99c469570a69\") " pod="opendatahub/opendatahub-operator-controller-manager-69778f9898-b4rzt"
Jul 24 23:33:52.738874 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:52.738698    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ffa2c446-0664-4bd8-be20-99c469570a69-apiservice-cert\") pod \"opendatahub-operator-controller-manager-69778f9898-b4rzt\" (UID: \"ffa2c446-0664-4bd8-be20-99c469570a69\") " pod="opendatahub/opendatahub-operator-controller-manager-69778f9898-b4rzt"
Jul 24 23:33:52.738874 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:52.738730    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ffa2c446-0664-4bd8-be20-99c469570a69-tmp\") pod \"opendatahub-operator-controller-manager-69778f9898-b4rzt\" (UID: \"ffa2c446-0664-4bd8-be20-99c469570a69\") " pod="opendatahub/opendatahub-operator-controller-manager-69778f9898-b4rzt"
Jul 24 23:33:52.738874 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:52.738783    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ffa2c446-0664-4bd8-be20-99c469570a69-manifests\") pod \"opendatahub-operator-controller-manager-69778f9898-b4rzt\" (UID: \"ffa2c446-0664-4bd8-be20-99c469570a69\") " pod="opendatahub/opendatahub-operator-controller-manager-69778f9898-b4rzt"
Jul 24 23:33:52.740423 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:52.740397    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-69778f9898-b4rzt"]
Jul 24 23:33:52.839142 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:52.839102    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ffa2c446-0664-4bd8-be20-99c469570a69-manifests\") pod \"opendatahub-operator-controller-manager-69778f9898-b4rzt\" (UID: \"ffa2c446-0664-4bd8-be20-99c469570a69\") " pod="opendatahub/opendatahub-operator-controller-manager-69778f9898-b4rzt"
Jul 24 23:33:52.839659 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:52.839173    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ffa2c446-0664-4bd8-be20-99c469570a69-webhook-cert\") pod \"opendatahub-operator-controller-manager-69778f9898-b4rzt\" (UID: \"ffa2c446-0664-4bd8-be20-99c469570a69\") " pod="opendatahub/opendatahub-operator-controller-manager-69778f9898-b4rzt"
Jul 24 23:33:52.839659 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:52.839203    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4r85\" (UniqueName: \"kubernetes.io/projected/ffa2c446-0664-4bd8-be20-99c469570a69-kube-api-access-w4r85\") pod \"opendatahub-operator-controller-manager-69778f9898-b4rzt\" (UID: \"ffa2c446-0664-4bd8-be20-99c469570a69\") " pod="opendatahub/opendatahub-operator-controller-manager-69778f9898-b4rzt"
Jul 24 23:33:52.839659 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:52.839239    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ffa2c446-0664-4bd8-be20-99c469570a69-apiservice-cert\") pod \"opendatahub-operator-controller-manager-69778f9898-b4rzt\" (UID: \"ffa2c446-0664-4bd8-be20-99c469570a69\") " pod="opendatahub/opendatahub-operator-controller-manager-69778f9898-b4rzt"
Jul 24 23:33:52.839659 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:52.839268    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ffa2c446-0664-4bd8-be20-99c469570a69-tmp\") pod \"opendatahub-operator-controller-manager-69778f9898-b4rzt\" (UID: \"ffa2c446-0664-4bd8-be20-99c469570a69\") " pod="opendatahub/opendatahub-operator-controller-manager-69778f9898-b4rzt"
Jul 24 23:33:52.839659 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:52.839499    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ffa2c446-0664-4bd8-be20-99c469570a69-manifests\") pod \"opendatahub-operator-controller-manager-69778f9898-b4rzt\" (UID: \"ffa2c446-0664-4bd8-be20-99c469570a69\") " pod="opendatahub/opendatahub-operator-controller-manager-69778f9898-b4rzt"
Jul 24 23:33:52.841793 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:52.841763    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ffa2c446-0664-4bd8-be20-99c469570a69-webhook-cert\") pod \"opendatahub-operator-controller-manager-69778f9898-b4rzt\" (UID: \"ffa2c446-0664-4bd8-be20-99c469570a69\") " pod="opendatahub/opendatahub-operator-controller-manager-69778f9898-b4rzt"
Jul 24 23:33:52.842001 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:52.841981    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ffa2c446-0664-4bd8-be20-99c469570a69-tmp\") pod \"opendatahub-operator-controller-manager-69778f9898-b4rzt\" (UID: \"ffa2c446-0664-4bd8-be20-99c469570a69\") " pod="opendatahub/opendatahub-operator-controller-manager-69778f9898-b4rzt"
Jul 24 23:33:52.842107 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:52.842090    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ffa2c446-0664-4bd8-be20-99c469570a69-apiservice-cert\") pod \"opendatahub-operator-controller-manager-69778f9898-b4rzt\" (UID: \"ffa2c446-0664-4bd8-be20-99c469570a69\") " pod="opendatahub/opendatahub-operator-controller-manager-69778f9898-b4rzt"
Jul 24 23:33:52.846713 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:52.846690    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4r85\" (UniqueName: \"kubernetes.io/projected/ffa2c446-0664-4bd8-be20-99c469570a69-kube-api-access-w4r85\") pod \"opendatahub-operator-controller-manager-69778f9898-b4rzt\" (UID: \"ffa2c446-0664-4bd8-be20-99c469570a69\") " pod="opendatahub/opendatahub-operator-controller-manager-69778f9898-b4rzt"
Jul 24 23:33:52.862652 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:52.862624    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:33:52.963584 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:52.963444    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:33:53.033652 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:53.033619    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-69778f9898-b4rzt"
Jul 24 23:33:53.065390 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:53.065355    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:33:53.169724 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:53.169119    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-69778f9898-b4rzt"]
Jul 24 23:33:53.171913 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:53.171881    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:33:53.174461 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:33:53.174434    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffa2c446_0664_4bd8_be20_99c469570a69.slice/crio-2830e4734f53f8a2280957d33d7739a67b626b5989f440b64bf24f39d00d8851 WatchSource:0}: Error finding container 2830e4734f53f8a2280957d33d7739a67b626b5989f440b64bf24f39d00d8851: Status 404 returned error can't find the container with id 2830e4734f53f8a2280957d33d7739a67b626b5989f440b64bf24f39d00d8851
Jul 24 23:33:53.274460 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:53.274431    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:33:53.379802 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:53.379759    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-69778f9898-b4rzt" event={"ID":"ffa2c446-0664-4bd8-be20-99c469570a69","Type":"ContainerStarted","Data":"2830e4734f53f8a2280957d33d7739a67b626b5989f440b64bf24f39d00d8851"}
Jul 24 23:33:55.058336 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:55.058299    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:33:55.164711 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:55.164672    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:33:56.397920 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:56.397884    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-69778f9898-b4rzt" event={"ID":"ffa2c446-0664-4bd8-be20-99c469570a69","Type":"ContainerStarted","Data":"9ca8122deda6c487772bc00675244e7957455be9f48f209a827efbdb78649f0b"}
Jul 24 23:33:57.408151 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:57.408109    2558 generic.go:356] "Generic (PLEG): container finished" podID="ffa2c446-0664-4bd8-be20-99c469570a69" containerID="9ca8122deda6c487772bc00675244e7957455be9f48f209a827efbdb78649f0b" exitCode=0
Jul 24 23:33:57.408633 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:57.408191    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-69778f9898-b4rzt" event={"ID":"ffa2c446-0664-4bd8-be20-99c469570a69","Type":"ContainerDied","Data":"9ca8122deda6c487772bc00675244e7957455be9f48f209a827efbdb78649f0b"}
Jul 24 23:33:58.414474 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:58.414437    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-69778f9898-b4rzt" event={"ID":"ffa2c446-0664-4bd8-be20-99c469570a69","Type":"ContainerStarted","Data":"007a8f7d9307edb7d35bbae9c142c17299fee71fa718e44543fffaae390e4d45"}
Jul 24 23:33:58.414856 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:58.414560    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-69778f9898-b4rzt"
Jul 24 23:33:58.437476 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:58.437427    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-69778f9898-b4rzt" podStartSLOduration=2.081734717 podStartE2EDuration="6.437410484s" totalImagesPullingTime="3.349493304s" totalInitContainerRuntime="1.006182463s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:33:52 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:33:58.436234063 +0000 UTC m=+278.822664791" watchObservedRunningTime="2026-07-24 23:33:58.437410484 +0000 UTC m=+278.823841212"
Jul 24 23:33:59.552436 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:59.552394    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:33:59.654567 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:33:59.654497    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:34:02.991986 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:02.991948    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:34:04.194989 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:04.194952    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:34:04.765019 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:04.764980    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:34:04.868779 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:04.868740    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:34:09.421122 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:09.421091    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-69778f9898-b4rzt"
Jul 24 23:34:11.088653 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:11.088614    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:34:11.177861 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:11.177825    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:34:11.777166 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:11.777125    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-s48c7"]
Jul 24 23:34:11.779266 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:11.779231    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-s48c7"
Jul 24 23:34:11.781771 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:11.781743    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 24 23:34:11.781921 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:11.781809    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-jlrnl\""
Jul 24 23:34:11.791082 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:11.791045    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-s48c7"]
Jul 24 23:34:11.817735 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:11.817701    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:34:11.877898 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:11.877852    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpchm\" (UniqueName: \"kubernetes.io/projected/1d634e98-014a-43c3-a840-efab61ea3a24-kube-api-access-kpchm\") pod \"odh-model-controller-7998c67dc4-s48c7\" (UID: \"1d634e98-014a-43c3-a840-efab61ea3a24\") " pod="opendatahub/odh-model-controller-7998c67dc4-s48c7"
Jul 24 23:34:11.878063 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:11.877961    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1d634e98-014a-43c3-a840-efab61ea3a24-cert\") pod \"odh-model-controller-7998c67dc4-s48c7\" (UID: \"1d634e98-014a-43c3-a840-efab61ea3a24\") " pod="opendatahub/odh-model-controller-7998c67dc4-s48c7"
Jul 24 23:34:11.926996 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:11.926957    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:34:11.978625 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:11.978588    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpchm\" (UniqueName: \"kubernetes.io/projected/1d634e98-014a-43c3-a840-efab61ea3a24-kube-api-access-kpchm\") pod \"odh-model-controller-7998c67dc4-s48c7\" (UID: \"1d634e98-014a-43c3-a840-efab61ea3a24\") " pod="opendatahub/odh-model-controller-7998c67dc4-s48c7"
Jul 24 23:34:11.978827 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:11.978662    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1d634e98-014a-43c3-a840-efab61ea3a24-cert\") pod \"odh-model-controller-7998c67dc4-s48c7\" (UID: \"1d634e98-014a-43c3-a840-efab61ea3a24\") " pod="opendatahub/odh-model-controller-7998c67dc4-s48c7"
Jul 24 23:34:11.978827 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:34:11.978769    2558 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 24 23:34:11.978930 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:34:11.978856    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d634e98-014a-43c3-a840-efab61ea3a24-cert podName:1d634e98-014a-43c3-a840-efab61ea3a24 nodeName:}" failed. No retries permitted until 2026-07-24 23:34:12.478834949 +0000 UTC m=+292.865265660 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/1d634e98-014a-43c3-a840-efab61ea3a24-cert") pod "odh-model-controller-7998c67dc4-s48c7" (UID: "1d634e98-014a-43c3-a840-efab61ea3a24") : secret "odh-model-controller-webhook-cert" not found
Jul 24 23:34:11.994269 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:11.994241    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpchm\" (UniqueName: \"kubernetes.io/projected/1d634e98-014a-43c3-a840-efab61ea3a24-kube-api-access-kpchm\") pod \"odh-model-controller-7998c67dc4-s48c7\" (UID: \"1d634e98-014a-43c3-a840-efab61ea3a24\") " pod="opendatahub/odh-model-controller-7998c67dc4-s48c7"
Jul 24 23:34:12.017799 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:12.017755    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:34:12.070193 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:12.070108    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-54fc5fd468-qpp94"]
Jul 24 23:34:12.072488 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:12.072461    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-54fc5fd468-qpp94"
Jul 24 23:34:12.074763 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:12.074735    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 24 23:34:12.074914 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:12.074833    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 24 23:34:12.074977 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:12.074933    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 24 23:34:12.075136 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:12.075090    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-9b9ls\""
Jul 24 23:34:12.078991 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:12.078970    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcbdn\" (UniqueName: \"kubernetes.io/projected/7ea2904c-c15e-4169-bbdd-bd4f3bf6956d-kube-api-access-gcbdn\") pod \"lws-controller-manager-54fc5fd468-qpp94\" (UID: \"7ea2904c-c15e-4169-bbdd-bd4f3bf6956d\") " pod="openshift-lws-operator/lws-controller-manager-54fc5fd468-qpp94"
Jul 24 23:34:12.079130 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:12.079019    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/7ea2904c-c15e-4169-bbdd-bd4f3bf6956d-metrics-cert\") pod \"lws-controller-manager-54fc5fd468-qpp94\" (UID: \"7ea2904c-c15e-4169-bbdd-bd4f3bf6956d\") " pod="openshift-lws-operator/lws-controller-manager-54fc5fd468-qpp94"
Jul 24 23:34:12.079307 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:12.079126    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/7ea2904c-c15e-4169-bbdd-bd4f3bf6956d-manager-config\") pod \"lws-controller-manager-54fc5fd468-qpp94\" (UID: \"7ea2904c-c15e-4169-bbdd-bd4f3bf6956d\") " pod="openshift-lws-operator/lws-controller-manager-54fc5fd468-qpp94"
Jul 24 23:34:12.079307 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:12.079211    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7ea2904c-c15e-4169-bbdd-bd4f3bf6956d-cert\") pod \"lws-controller-manager-54fc5fd468-qpp94\" (UID: \"7ea2904c-c15e-4169-bbdd-bd4f3bf6956d\") " pod="openshift-lws-operator/lws-controller-manager-54fc5fd468-qpp94"
Jul 24 23:34:12.083061 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:12.083040    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-54fc5fd468-qpp94"]
Jul 24 23:34:12.102343 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:12.102313    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:34:12.180480 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:12.180442    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcbdn\" (UniqueName: \"kubernetes.io/projected/7ea2904c-c15e-4169-bbdd-bd4f3bf6956d-kube-api-access-gcbdn\") pod \"lws-controller-manager-54fc5fd468-qpp94\" (UID: \"7ea2904c-c15e-4169-bbdd-bd4f3bf6956d\") " pod="openshift-lws-operator/lws-controller-manager-54fc5fd468-qpp94"
Jul 24 23:34:12.180699 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:12.180493    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/7ea2904c-c15e-4169-bbdd-bd4f3bf6956d-metrics-cert\") pod \"lws-controller-manager-54fc5fd468-qpp94\" (UID: \"7ea2904c-c15e-4169-bbdd-bd4f3bf6956d\") " pod="openshift-lws-operator/lws-controller-manager-54fc5fd468-qpp94"
Jul 24 23:34:12.180699 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:12.180569    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/7ea2904c-c15e-4169-bbdd-bd4f3bf6956d-manager-config\") pod \"lws-controller-manager-54fc5fd468-qpp94\" (UID: \"7ea2904c-c15e-4169-bbdd-bd4f3bf6956d\") " pod="openshift-lws-operator/lws-controller-manager-54fc5fd468-qpp94"
Jul 24 23:34:12.180699 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:12.180645    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7ea2904c-c15e-4169-bbdd-bd4f3bf6956d-cert\") pod \"lws-controller-manager-54fc5fd468-qpp94\" (UID: \"7ea2904c-c15e-4169-bbdd-bd4f3bf6956d\") " pod="openshift-lws-operator/lws-controller-manager-54fc5fd468-qpp94"
Jul 24 23:34:12.181259 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:12.181238    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/7ea2904c-c15e-4169-bbdd-bd4f3bf6956d-manager-config\") pod \"lws-controller-manager-54fc5fd468-qpp94\" (UID: \"7ea2904c-c15e-4169-bbdd-bd4f3bf6956d\") " pod="openshift-lws-operator/lws-controller-manager-54fc5fd468-qpp94"
Jul 24 23:34:12.182997 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:12.182975    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7ea2904c-c15e-4169-bbdd-bd4f3bf6956d-cert\") pod \"lws-controller-manager-54fc5fd468-qpp94\" (UID: \"7ea2904c-c15e-4169-bbdd-bd4f3bf6956d\") " pod="openshift-lws-operator/lws-controller-manager-54fc5fd468-qpp94"
Jul 24 23:34:12.183072 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:12.183030    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/7ea2904c-c15e-4169-bbdd-bd4f3bf6956d-metrics-cert\") pod \"lws-controller-manager-54fc5fd468-qpp94\" (UID: \"7ea2904c-c15e-4169-bbdd-bd4f3bf6956d\") " pod="openshift-lws-operator/lws-controller-manager-54fc5fd468-qpp94"
Jul 24 23:34:12.191241 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:12.191219    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcbdn\" (UniqueName: \"kubernetes.io/projected/7ea2904c-c15e-4169-bbdd-bd4f3bf6956d-kube-api-access-gcbdn\") pod \"lws-controller-manager-54fc5fd468-qpp94\" (UID: \"7ea2904c-c15e-4169-bbdd-bd4f3bf6956d\") " pod="openshift-lws-operator/lws-controller-manager-54fc5fd468-qpp94"
Jul 24 23:34:12.343634 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:12.343544    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:34:12.382035 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:12.382002    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-54fc5fd468-qpp94"
Jul 24 23:34:12.483741 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:12.483706    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1d634e98-014a-43c3-a840-efab61ea3a24-cert\") pod \"odh-model-controller-7998c67dc4-s48c7\" (UID: \"1d634e98-014a-43c3-a840-efab61ea3a24\") " pod="opendatahub/odh-model-controller-7998c67dc4-s48c7"
Jul 24 23:34:12.486117 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:12.486097    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1d634e98-014a-43c3-a840-efab61ea3a24-cert\") pod \"odh-model-controller-7998c67dc4-s48c7\" (UID: \"1d634e98-014a-43c3-a840-efab61ea3a24\") " pod="opendatahub/odh-model-controller-7998c67dc4-s48c7"
Jul 24 23:34:12.508253 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:12.508223    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-54fc5fd468-qpp94"]
Jul 24 23:34:12.510543 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:34:12.510507    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ea2904c_c15e_4169_bbdd_bd4f3bf6956d.slice/crio-690ae0f2e746321237d210cd5185bc789a044af94b584f0b6bdaa37998355b5e WatchSource:0}: Error finding container 690ae0f2e746321237d210cd5185bc789a044af94b584f0b6bdaa37998355b5e: Status 404 returned error can't find the container with id 690ae0f2e746321237d210cd5185bc789a044af94b584f0b6bdaa37998355b5e
Jul 24 23:34:12.690693 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:12.690606    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-s48c7"
Jul 24 23:34:12.771476 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:12.771443    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:34:12.818167 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:12.818139    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-s48c7"]
Jul 24 23:34:12.820138 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:34:12.820114    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d634e98_014a_43c3_a840_efab61ea3a24.slice/crio-c3f3bcb12ea1a50b43e1a7447cb25cdc8a29705e264c0bf7484e5dde428c18ab WatchSource:0}: Error finding container c3f3bcb12ea1a50b43e1a7447cb25cdc8a29705e264c0bf7484e5dde428c18ab: Status 404 returned error can't find the container with id c3f3bcb12ea1a50b43e1a7447cb25cdc8a29705e264c0bf7484e5dde428c18ab
Jul 24 23:34:13.485544 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:13.485490    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-s48c7" event={"ID":"1d634e98-014a-43c3-a840-efab61ea3a24","Type":"ContainerStarted","Data":"c3f3bcb12ea1a50b43e1a7447cb25cdc8a29705e264c0bf7484e5dde428c18ab"}
Jul 24 23:34:13.486495 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:13.486467    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-54fc5fd468-qpp94" event={"ID":"7ea2904c-c15e-4169-bbdd-bd4f3bf6956d","Type":"ContainerStarted","Data":"690ae0f2e746321237d210cd5185bc789a044af94b584f0b6bdaa37998355b5e"}
Jul 24 23:34:14.529193 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:14.529160    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:34:15.026316 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:15.026246    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:34:15.674933 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:15.674896    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:34:15.741768 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:15.741733    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:34:16.502600 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:16.502564    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-s48c7" event={"ID":"1d634e98-014a-43c3-a840-efab61ea3a24","Type":"ContainerStarted","Data":"411cec1c986f33b1ebfda85de90dfd594ca4d440c88c4fc13e809daefd7e1bba"}
Jul 24 23:34:16.502793 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:16.502647    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-s48c7"
Jul 24 23:34:16.504013 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:16.503991    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-54fc5fd468-qpp94" event={"ID":"7ea2904c-c15e-4169-bbdd-bd4f3bf6956d","Type":"ContainerStarted","Data":"243ae2dc876236e50f16139d16d9c6fe172e56132b2a4367758de7646a5eb127"}
Jul 24 23:34:16.504126 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:16.504111    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-54fc5fd468-qpp94"
Jul 24 23:34:16.521293 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:16.521246    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-s48c7" podStartSLOduration=2.216138349 podStartE2EDuration="5.521232281s" totalImagesPullingTime="3.305093932s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:34:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:34:16.519540835 +0000 UTC m=+296.905971554" watchObservedRunningTime="2026-07-24 23:34:16.521232281 +0000 UTC m=+296.907663022"
Jul 24 23:34:16.537547 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:16.537494    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-54fc5fd468-qpp94" podStartSLOduration=0.925566234 podStartE2EDuration="4.537476236s" totalImagesPullingTime="3.611910002s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:34:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:34:16.535818491 +0000 UTC m=+296.922249220" watchObservedRunningTime="2026-07-24 23:34:16.537476236 +0000 UTC m=+296.923907110"
Jul 24 23:34:16.701495 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:16.701457    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:34:19.644165 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:19.644123    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-mn8nv"]
Jul 24 23:34:19.647682 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:19.647660    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-mn8nv"
Jul 24 23:34:19.650422 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:19.650399    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 24 23:34:19.650676 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:19.650659    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 24 23:34:19.650766 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:19.650708    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-hpn6q\""
Jul 24 23:34:19.650766 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:19.650725    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 24 23:34:19.650766 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:19.650748    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 24 23:34:19.650766 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:19.650713    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 24 23:34:19.651406 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:19.651389    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 24 23:34:19.666968 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:19.666942    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-mn8nv"]
Jul 24 23:34:19.750057 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:19.750019    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/164984ed-ef05-4e6d-860b-38908ef71961-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-mn8nv\" (UID: \"164984ed-ef05-4e6d-860b-38908ef71961\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-mn8nv"
Jul 24 23:34:19.750220 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:19.750078    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmtrp\" (UniqueName: \"kubernetes.io/projected/164984ed-ef05-4e6d-860b-38908ef71961-kube-api-access-dmtrp\") pod \"istiod-openshift-gateway-5677ccb49b-mn8nv\" (UID: \"164984ed-ef05-4e6d-860b-38908ef71961\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-mn8nv"
Jul 24 23:34:19.750220 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:19.750102    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/164984ed-ef05-4e6d-860b-38908ef71961-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-mn8nv\" (UID: \"164984ed-ef05-4e6d-860b-38908ef71961\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-mn8nv"
Jul 24 23:34:19.750220 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:19.750121    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/164984ed-ef05-4e6d-860b-38908ef71961-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-mn8nv\" (UID: \"164984ed-ef05-4e6d-860b-38908ef71961\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-mn8nv"
Jul 24 23:34:19.750321 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:19.750218    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/164984ed-ef05-4e6d-860b-38908ef71961-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-mn8nv\" (UID: \"164984ed-ef05-4e6d-860b-38908ef71961\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-mn8nv"
Jul 24 23:34:19.750321 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:19.750242    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/164984ed-ef05-4e6d-860b-38908ef71961-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-mn8nv\" (UID: \"164984ed-ef05-4e6d-860b-38908ef71961\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-mn8nv"
Jul 24 23:34:19.750321 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:19.750268    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/164984ed-ef05-4e6d-860b-38908ef71961-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-mn8nv\" (UID: \"164984ed-ef05-4e6d-860b-38908ef71961\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-mn8nv"
Jul 24 23:34:19.851083 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:19.851042    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/164984ed-ef05-4e6d-860b-38908ef71961-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-mn8nv\" (UID: \"164984ed-ef05-4e6d-860b-38908ef71961\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-mn8nv"
Jul 24 23:34:19.851298 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:19.851099    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmtrp\" (UniqueName: \"kubernetes.io/projected/164984ed-ef05-4e6d-860b-38908ef71961-kube-api-access-dmtrp\") pod \"istiod-openshift-gateway-5677ccb49b-mn8nv\" (UID: \"164984ed-ef05-4e6d-860b-38908ef71961\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-mn8nv"
Jul 24 23:34:19.851298 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:19.851120    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/164984ed-ef05-4e6d-860b-38908ef71961-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-mn8nv\" (UID: \"164984ed-ef05-4e6d-860b-38908ef71961\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-mn8nv"
Jul 24 23:34:19.851298 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:19.851226    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/164984ed-ef05-4e6d-860b-38908ef71961-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-mn8nv\" (UID: \"164984ed-ef05-4e6d-860b-38908ef71961\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-mn8nv"
Jul 24 23:34:19.851298 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:19.851291    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/164984ed-ef05-4e6d-860b-38908ef71961-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-mn8nv\" (UID: \"164984ed-ef05-4e6d-860b-38908ef71961\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-mn8nv"
Jul 24 23:34:19.851514 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:19.851309    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/164984ed-ef05-4e6d-860b-38908ef71961-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-mn8nv\" (UID: \"164984ed-ef05-4e6d-860b-38908ef71961\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-mn8nv"
Jul 24 23:34:19.851514 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:19.851336    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/164984ed-ef05-4e6d-860b-38908ef71961-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-mn8nv\" (UID: \"164984ed-ef05-4e6d-860b-38908ef71961\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-mn8nv"
Jul 24 23:34:19.851845 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:19.851822    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/164984ed-ef05-4e6d-860b-38908ef71961-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-mn8nv\" (UID: \"164984ed-ef05-4e6d-860b-38908ef71961\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-mn8nv"
Jul 24 23:34:19.853446 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:19.853421    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/164984ed-ef05-4e6d-860b-38908ef71961-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-mn8nv\" (UID: \"164984ed-ef05-4e6d-860b-38908ef71961\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-mn8nv"
Jul 24 23:34:19.853612 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:19.853596    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/164984ed-ef05-4e6d-860b-38908ef71961-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-mn8nv\" (UID: \"164984ed-ef05-4e6d-860b-38908ef71961\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-mn8nv"
Jul 24 23:34:19.853956 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:19.853940    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/164984ed-ef05-4e6d-860b-38908ef71961-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-mn8nv\" (UID: \"164984ed-ef05-4e6d-860b-38908ef71961\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-mn8nv"
Jul 24 23:34:19.854017 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:19.853978    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/164984ed-ef05-4e6d-860b-38908ef71961-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-mn8nv\" (UID: \"164984ed-ef05-4e6d-860b-38908ef71961\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-mn8nv"
Jul 24 23:34:19.861972 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:19.861951    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/164984ed-ef05-4e6d-860b-38908ef71961-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-mn8nv\" (UID: \"164984ed-ef05-4e6d-860b-38908ef71961\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-mn8nv"
Jul 24 23:34:19.862511 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:19.862493    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmtrp\" (UniqueName: \"kubernetes.io/projected/164984ed-ef05-4e6d-860b-38908ef71961-kube-api-access-dmtrp\") pod \"istiod-openshift-gateway-5677ccb49b-mn8nv\" (UID: \"164984ed-ef05-4e6d-860b-38908ef71961\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-mn8nv"
Jul 24 23:34:19.941462 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:19.941359    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:34:19.957506 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:19.957475    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-mn8nv"
Jul 24 23:34:20.039968 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:20.039870    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:34:20.068985 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:20.068944    2558 kubelet.go:1648] "Image garbage collection succeeded"
Jul 24 23:34:20.132124 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:20.130628    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-mn8nv"]
Jul 24 23:34:20.145638 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:34:20.136122    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod164984ed_ef05_4e6d_860b_38908ef71961.slice/crio-de71c89481412d00cdfb7f7a48846915c487dd5017b1970386fbb6b01c860871 WatchSource:0}: Error finding container de71c89481412d00cdfb7f7a48846915c487dd5017b1970386fbb6b01c860871: Status 404 returned error can't find the container with id de71c89481412d00cdfb7f7a48846915c487dd5017b1970386fbb6b01c860871
Jul 24 23:34:20.522725 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:20.522680    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-mn8nv" event={"ID":"164984ed-ef05-4e6d-860b-38908ef71961","Type":"ContainerStarted","Data":"de71c89481412d00cdfb7f7a48846915c487dd5017b1970386fbb6b01c860871"}
Jul 24 23:34:20.702425 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:20.702381    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:34:20.859487 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:20.859392    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:34:20.961501 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:20.961438    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:34:22.438736 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:22.438666    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:34:22.535923 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:22.535884    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:34:22.773468 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:22.773433    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:34:22.923197 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:22.923139    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:34:23.708076 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:23.708030    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:34:23.805949 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:23.805916    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:34:23.869310 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:23.869272    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:34:23.871898 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:23.871854    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Jul 24 23:34:23.871991 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:23.871935    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Jul 24 23:34:24.543639 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:24.543601    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-mn8nv" event={"ID":"164984ed-ef05-4e6d-860b-38908ef71961","Type":"ContainerStarted","Data":"f5dcc152c6514a0ea6b1d3ada21561f2a550c244620e3e8eea2a6c81b36e3241"}
Jul 24 23:34:24.543837 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:24.543744    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-mn8nv"
Jul 24 23:34:24.545330 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:24.545304    2558 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-mn8nv container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 24 23:34:24.545456 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:24.545370    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-mn8nv" podUID="164984ed-ef05-4e6d-860b-38908ef71961" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 24 23:34:24.570305 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:24.570255    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-mn8nv" podStartSLOduration=1.836651416 podStartE2EDuration="5.570240627s" totalImagesPullingTime="3.733589211s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:34:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:34:24.567640738 +0000 UTC m=+304.954071468" watchObservedRunningTime="2026-07-24 23:34:24.570240627 +0000 UTC m=+304.956671356"
Jul 24 23:34:25.549387 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:25.549357    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-mn8nv"
Jul 24 23:34:27.510188 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:27.510156    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-s48c7"
Jul 24 23:34:27.510677 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:27.510239    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-54fc5fd468-qpp94"
Jul 24 23:34:28.596992 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:28.596957    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:34:28.688947 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:28.688909    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:34:35.110151 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:35.110111    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:34:35.164379 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:35.164332    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:34:36.965705 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:36.965666    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:34:37.025988 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:37.025949    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:34:47.180113 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:47.180059    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:34:47.278305 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:47.278270    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:34:53.831423 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:34:53.831380    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:35:12.615911 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:12.615795    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:35:12.744647 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:12.744610    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:35:12.852839 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:12.852799    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:35:27.017602 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:27.017563    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kqwqt"]
Jul 24 23:35:27.021158 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:27.021138    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kqwqt"
Jul 24 23:35:27.023743 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:27.023722    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 24 23:35:27.023743 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:27.023732    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-2bx2t\""
Jul 24 23:35:27.023904 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:27.023759    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 24 23:35:27.030471 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:27.030445    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kqwqt"]
Jul 24 23:35:27.128867 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:27.128824    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/650df1f9-9ba5-44bc-82a9-07cc28c371d2-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kqwqt\" (UID: \"650df1f9-9ba5-44bc-82a9-07cc28c371d2\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kqwqt"
Jul 24 23:35:27.129052 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:27.128878    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/650df1f9-9ba5-44bc-82a9-07cc28c371d2-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kqwqt\" (UID: \"650df1f9-9ba5-44bc-82a9-07cc28c371d2\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kqwqt"
Jul 24 23:35:27.129052 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:27.128911    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fcccl\" (UniqueName: \"kubernetes.io/projected/650df1f9-9ba5-44bc-82a9-07cc28c371d2-kube-api-access-fcccl\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kqwqt\" (UID: \"650df1f9-9ba5-44bc-82a9-07cc28c371d2\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kqwqt"
Jul 24 23:35:27.230139 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:27.230106    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/650df1f9-9ba5-44bc-82a9-07cc28c371d2-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kqwqt\" (UID: \"650df1f9-9ba5-44bc-82a9-07cc28c371d2\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kqwqt"
Jul 24 23:35:27.230308 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:27.230150    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/650df1f9-9ba5-44bc-82a9-07cc28c371d2-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kqwqt\" (UID: \"650df1f9-9ba5-44bc-82a9-07cc28c371d2\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kqwqt"
Jul 24 23:35:27.230308 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:27.230172    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fcccl\" (UniqueName: \"kubernetes.io/projected/650df1f9-9ba5-44bc-82a9-07cc28c371d2-kube-api-access-fcccl\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kqwqt\" (UID: \"650df1f9-9ba5-44bc-82a9-07cc28c371d2\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kqwqt"
Jul 24 23:35:27.230518 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:27.230493    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/650df1f9-9ba5-44bc-82a9-07cc28c371d2-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kqwqt\" (UID: \"650df1f9-9ba5-44bc-82a9-07cc28c371d2\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kqwqt"
Jul 24 23:35:27.230625 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:27.230574    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/650df1f9-9ba5-44bc-82a9-07cc28c371d2-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kqwqt\" (UID: \"650df1f9-9ba5-44bc-82a9-07cc28c371d2\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kqwqt"
Jul 24 23:35:27.238319 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:27.238296    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fcccl\" (UniqueName: \"kubernetes.io/projected/650df1f9-9ba5-44bc-82a9-07cc28c371d2-kube-api-access-fcccl\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kqwqt\" (UID: \"650df1f9-9ba5-44bc-82a9-07cc28c371d2\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kqwqt"
Jul 24 23:35:27.331214 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:27.331117    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kqwqt"
Jul 24 23:35:27.461465 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:27.461440    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kqwqt"]
Jul 24 23:35:27.463146 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:35:27.463110    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod650df1f9_9ba5_44bc_82a9_07cc28c371d2.slice/crio-07a142aab71055e17eeea9869d09a6ae924fd4c45748e7ec33b62bbdb8197596 WatchSource:0}: Error finding container 07a142aab71055e17eeea9869d09a6ae924fd4c45748e7ec33b62bbdb8197596: Status 404 returned error can't find the container with id 07a142aab71055e17eeea9869d09a6ae924fd4c45748e7ec33b62bbdb8197596
Jul 24 23:35:27.610254 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:27.610177    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75999hgt"]
Jul 24 23:35:27.613375 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:27.613359    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75999hgt"
Jul 24 23:35:27.622161 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:27.622129    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75999hgt"]
Jul 24 23:35:27.735392 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:27.735353    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bad54421-36a4-4e6f-be54-59d837f047c4-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75999hgt\" (UID: \"bad54421-36a4-4e6f-be54-59d837f047c4\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75999hgt"
Jul 24 23:35:27.735603 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:27.735400    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ft28f\" (UniqueName: \"kubernetes.io/projected/bad54421-36a4-4e6f-be54-59d837f047c4-kube-api-access-ft28f\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75999hgt\" (UID: \"bad54421-36a4-4e6f-be54-59d837f047c4\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75999hgt"
Jul 24 23:35:27.735603 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:27.735502    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bad54421-36a4-4e6f-be54-59d837f047c4-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75999hgt\" (UID: \"bad54421-36a4-4e6f-be54-59d837f047c4\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75999hgt"
Jul 24 23:35:27.824545 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:27.824508    2558 generic.go:356] "Generic (PLEG): container finished" podID="650df1f9-9ba5-44bc-82a9-07cc28c371d2" containerID="1606d85cdc122d637dfa4e10124c4870f413dfb87b1d14efa9bcf8f6c3c57581" exitCode=0
Jul 24 23:35:27.824734 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:27.824609    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kqwqt" event={"ID":"650df1f9-9ba5-44bc-82a9-07cc28c371d2","Type":"ContainerDied","Data":"1606d85cdc122d637dfa4e10124c4870f413dfb87b1d14efa9bcf8f6c3c57581"}
Jul 24 23:35:27.824734 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:27.824671    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kqwqt" event={"ID":"650df1f9-9ba5-44bc-82a9-07cc28c371d2","Type":"ContainerStarted","Data":"07a142aab71055e17eeea9869d09a6ae924fd4c45748e7ec33b62bbdb8197596"}
Jul 24 23:35:27.825073 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:27.825056    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 23:35:27.836420 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:27.836392    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ft28f\" (UniqueName: \"kubernetes.io/projected/bad54421-36a4-4e6f-be54-59d837f047c4-kube-api-access-ft28f\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75999hgt\" (UID: \"bad54421-36a4-4e6f-be54-59d837f047c4\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75999hgt"
Jul 24 23:35:27.836524 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:27.836492    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bad54421-36a4-4e6f-be54-59d837f047c4-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75999hgt\" (UID: \"bad54421-36a4-4e6f-be54-59d837f047c4\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75999hgt"
Jul 24 23:35:27.836592 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:27.836565    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bad54421-36a4-4e6f-be54-59d837f047c4-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75999hgt\" (UID: \"bad54421-36a4-4e6f-be54-59d837f047c4\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75999hgt"
Jul 24 23:35:27.836841 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:27.836825    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bad54421-36a4-4e6f-be54-59d837f047c4-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75999hgt\" (UID: \"bad54421-36a4-4e6f-be54-59d837f047c4\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75999hgt"
Jul 24 23:35:27.836946 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:27.836927    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bad54421-36a4-4e6f-be54-59d837f047c4-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75999hgt\" (UID: \"bad54421-36a4-4e6f-be54-59d837f047c4\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75999hgt"
Jul 24 23:35:27.844832 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:27.844805    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ft28f\" (UniqueName: \"kubernetes.io/projected/bad54421-36a4-4e6f-be54-59d837f047c4-kube-api-access-ft28f\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75999hgt\" (UID: \"bad54421-36a4-4e6f-be54-59d837f047c4\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75999hgt"
Jul 24 23:35:27.929909 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:27.929806    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75999hgt"
Jul 24 23:35:28.058266 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:28.058241    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75999hgt"]
Jul 24 23:35:28.060138 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:35:28.060113    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbad54421_36a4_4e6f_be54_59d837f047c4.slice/crio-2aa932ecb0896bff288d7e00f7ae4aba75610ef8c9dbf71cab56e1b06580b483 WatchSource:0}: Error finding container 2aa932ecb0896bff288d7e00f7ae4aba75610ef8c9dbf71cab56e1b06580b483: Status 404 returned error can't find the container with id 2aa932ecb0896bff288d7e00f7ae4aba75610ef8c9dbf71cab56e1b06580b483
Jul 24 23:35:28.209980 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:28.209898    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7394m4w"]
Jul 24 23:35:28.214906 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:28.214884    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7394m4w"
Jul 24 23:35:28.221729 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:28.221461    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7394m4w"]
Jul 24 23:35:28.339970 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:28.339935    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvfmx\" (UniqueName: \"kubernetes.io/projected/8db0cdbd-5d8c-4213-8aa2-cb0d25ffb425-kube-api-access-vvfmx\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7394m4w\" (UID: \"8db0cdbd-5d8c-4213-8aa2-cb0d25ffb425\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7394m4w"
Jul 24 23:35:28.340156 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:28.340045    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8db0cdbd-5d8c-4213-8aa2-cb0d25ffb425-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7394m4w\" (UID: \"8db0cdbd-5d8c-4213-8aa2-cb0d25ffb425\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7394m4w"
Jul 24 23:35:28.340156 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:28.340075    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8db0cdbd-5d8c-4213-8aa2-cb0d25ffb425-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7394m4w\" (UID: \"8db0cdbd-5d8c-4213-8aa2-cb0d25ffb425\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7394m4w"
Jul 24 23:35:28.441175 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:28.441142    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvfmx\" (UniqueName: \"kubernetes.io/projected/8db0cdbd-5d8c-4213-8aa2-cb0d25ffb425-kube-api-access-vvfmx\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7394m4w\" (UID: \"8db0cdbd-5d8c-4213-8aa2-cb0d25ffb425\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7394m4w"
Jul 24 23:35:28.441319 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:28.441188    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8db0cdbd-5d8c-4213-8aa2-cb0d25ffb425-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7394m4w\" (UID: \"8db0cdbd-5d8c-4213-8aa2-cb0d25ffb425\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7394m4w"
Jul 24 23:35:28.441319 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:28.441213    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8db0cdbd-5d8c-4213-8aa2-cb0d25ffb425-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7394m4w\" (UID: \"8db0cdbd-5d8c-4213-8aa2-cb0d25ffb425\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7394m4w"
Jul 24 23:35:28.441586 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:28.441565    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8db0cdbd-5d8c-4213-8aa2-cb0d25ffb425-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7394m4w\" (UID: \"8db0cdbd-5d8c-4213-8aa2-cb0d25ffb425\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7394m4w"
Jul 24 23:35:28.441629 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:28.441598    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8db0cdbd-5d8c-4213-8aa2-cb0d25ffb425-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7394m4w\" (UID: \"8db0cdbd-5d8c-4213-8aa2-cb0d25ffb425\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7394m4w"
Jul 24 23:35:28.448788 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:28.448770    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvfmx\" (UniqueName: \"kubernetes.io/projected/8db0cdbd-5d8c-4213-8aa2-cb0d25ffb425-kube-api-access-vvfmx\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7394m4w\" (UID: \"8db0cdbd-5d8c-4213-8aa2-cb0d25ffb425\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7394m4w"
Jul 24 23:35:28.525082 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:28.525034    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7394m4w"
Jul 24 23:35:28.615637 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:28.615607    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jp4jc"]
Jul 24 23:35:28.618810 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:28.618792    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jp4jc"
Jul 24 23:35:28.628461 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:28.628431    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jp4jc"]
Jul 24 23:35:28.729786 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:28.729758    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7394m4w"]
Jul 24 23:35:28.731657 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:35:28.731628    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8db0cdbd_5d8c_4213_8aa2_cb0d25ffb425.slice/crio-6bc1e284413b5513f1bc2cb6ae18d2baa6a799594fb4fc36e6a029c1280b36e6 WatchSource:0}: Error finding container 6bc1e284413b5513f1bc2cb6ae18d2baa6a799594fb4fc36e6a029c1280b36e6: Status 404 returned error can't find the container with id 6bc1e284413b5513f1bc2cb6ae18d2baa6a799594fb4fc36e6a029c1280b36e6
Jul 24 23:35:28.745009 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:28.744986    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/33ce1323-16e5-4514-8a41-26d0b876bb91-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jp4jc\" (UID: \"33ce1323-16e5-4514-8a41-26d0b876bb91\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jp4jc"
Jul 24 23:35:28.745097 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:28.745021    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/33ce1323-16e5-4514-8a41-26d0b876bb91-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jp4jc\" (UID: \"33ce1323-16e5-4514-8a41-26d0b876bb91\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jp4jc"
Jul 24 23:35:28.745097 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:28.745039    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npjds\" (UniqueName: \"kubernetes.io/projected/33ce1323-16e5-4514-8a41-26d0b876bb91-kube-api-access-npjds\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jp4jc\" (UID: \"33ce1323-16e5-4514-8a41-26d0b876bb91\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jp4jc"
Jul 24 23:35:28.831164 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:28.831133    2558 generic.go:356] "Generic (PLEG): container finished" podID="650df1f9-9ba5-44bc-82a9-07cc28c371d2" containerID="d906f6d82a5489cf49b4fdeb05c6ab22e2b8ba6b455b18c85110d50afe431c6b" exitCode=0
Jul 24 23:35:28.831320 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:28.831219    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kqwqt" event={"ID":"650df1f9-9ba5-44bc-82a9-07cc28c371d2","Type":"ContainerDied","Data":"d906f6d82a5489cf49b4fdeb05c6ab22e2b8ba6b455b18c85110d50afe431c6b"}
Jul 24 23:35:28.832923 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:28.832903    2558 generic.go:356] "Generic (PLEG): container finished" podID="bad54421-36a4-4e6f-be54-59d837f047c4" containerID="43705e0001c18fc55dd9043e4fbad908c57abd294b74a102709034583a59f6ee" exitCode=0
Jul 24 23:35:28.833011 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:28.832966    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75999hgt" event={"ID":"bad54421-36a4-4e6f-be54-59d837f047c4","Type":"ContainerDied","Data":"43705e0001c18fc55dd9043e4fbad908c57abd294b74a102709034583a59f6ee"}
Jul 24 23:35:28.833071 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:28.833014    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75999hgt" event={"ID":"bad54421-36a4-4e6f-be54-59d837f047c4","Type":"ContainerStarted","Data":"2aa932ecb0896bff288d7e00f7ae4aba75610ef8c9dbf71cab56e1b06580b483"}
Jul 24 23:35:28.834620 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:28.834595    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7394m4w" event={"ID":"8db0cdbd-5d8c-4213-8aa2-cb0d25ffb425","Type":"ContainerStarted","Data":"2e3429aee630397efa35f532171b05d9526e05be7075b7a62d9857b00654d1a9"}
Jul 24 23:35:28.834716 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:28.834643    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7394m4w" event={"ID":"8db0cdbd-5d8c-4213-8aa2-cb0d25ffb425","Type":"ContainerStarted","Data":"6bc1e284413b5513f1bc2cb6ae18d2baa6a799594fb4fc36e6a029c1280b36e6"}
Jul 24 23:35:28.845906 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:28.845867    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/33ce1323-16e5-4514-8a41-26d0b876bb91-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jp4jc\" (UID: \"33ce1323-16e5-4514-8a41-26d0b876bb91\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jp4jc"
Jul 24 23:35:28.846043 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:28.845913    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/33ce1323-16e5-4514-8a41-26d0b876bb91-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jp4jc\" (UID: \"33ce1323-16e5-4514-8a41-26d0b876bb91\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jp4jc"
Jul 24 23:35:28.846043 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:28.845930    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npjds\" (UniqueName: \"kubernetes.io/projected/33ce1323-16e5-4514-8a41-26d0b876bb91-kube-api-access-npjds\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jp4jc\" (UID: \"33ce1323-16e5-4514-8a41-26d0b876bb91\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jp4jc"
Jul 24 23:35:28.846342 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:28.846321    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/33ce1323-16e5-4514-8a41-26d0b876bb91-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jp4jc\" (UID: \"33ce1323-16e5-4514-8a41-26d0b876bb91\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jp4jc"
Jul 24 23:35:28.846436 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:28.846358    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/33ce1323-16e5-4514-8a41-26d0b876bb91-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jp4jc\" (UID: \"33ce1323-16e5-4514-8a41-26d0b876bb91\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jp4jc"
Jul 24 23:35:28.855355 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:28.855325    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-npjds\" (UniqueName: \"kubernetes.io/projected/33ce1323-16e5-4514-8a41-26d0b876bb91-kube-api-access-npjds\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jp4jc\" (UID: \"33ce1323-16e5-4514-8a41-26d0b876bb91\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jp4jc"
Jul 24 23:35:28.931873 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:28.931847    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jp4jc"
Jul 24 23:35:29.064410 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:29.064379    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jp4jc"]
Jul 24 23:35:29.066285 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:35:29.066255    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33ce1323_16e5_4514_8a41_26d0b876bb91.slice/crio-d6aecaabf032d5775e325b33e448883dd5b117d4a5c71767ff7fae432c6efc28 WatchSource:0}: Error finding container d6aecaabf032d5775e325b33e448883dd5b117d4a5c71767ff7fae432c6efc28: Status 404 returned error can't find the container with id d6aecaabf032d5775e325b33e448883dd5b117d4a5c71767ff7fae432c6efc28
Jul 24 23:35:29.841309 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:29.841274    2558 generic.go:356] "Generic (PLEG): container finished" podID="8db0cdbd-5d8c-4213-8aa2-cb0d25ffb425" containerID="2e3429aee630397efa35f532171b05d9526e05be7075b7a62d9857b00654d1a9" exitCode=0
Jul 24 23:35:29.841482 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:29.841359    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7394m4w" event={"ID":"8db0cdbd-5d8c-4213-8aa2-cb0d25ffb425","Type":"ContainerDied","Data":"2e3429aee630397efa35f532171b05d9526e05be7075b7a62d9857b00654d1a9"}
Jul 24 23:35:29.843482 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:29.843460    2558 generic.go:356] "Generic (PLEG): container finished" podID="650df1f9-9ba5-44bc-82a9-07cc28c371d2" containerID="aa81f275c9deebdbef3078e8765307bcc9106867519a19f1baa65e2424e25ce6" exitCode=0
Jul 24 23:35:29.843616 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:29.843551    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kqwqt" event={"ID":"650df1f9-9ba5-44bc-82a9-07cc28c371d2","Type":"ContainerDied","Data":"aa81f275c9deebdbef3078e8765307bcc9106867519a19f1baa65e2424e25ce6"}
Jul 24 23:35:29.845406 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:29.845385    2558 generic.go:356] "Generic (PLEG): container finished" podID="bad54421-36a4-4e6f-be54-59d837f047c4" containerID="f7390ed9c84d84c858b360d56d1f025734b7a33e1a39cd79f18d49a1b8f0d68e" exitCode=0
Jul 24 23:35:29.845515 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:29.845410    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75999hgt" event={"ID":"bad54421-36a4-4e6f-be54-59d837f047c4","Type":"ContainerDied","Data":"f7390ed9c84d84c858b360d56d1f025734b7a33e1a39cd79f18d49a1b8f0d68e"}
Jul 24 23:35:29.847056 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:29.847036    2558 generic.go:356] "Generic (PLEG): container finished" podID="33ce1323-16e5-4514-8a41-26d0b876bb91" containerID="14ab87caffbc791d9b46562686943c8d8f4c25dff3090bb2e07d70539ce0b5d2" exitCode=0
Jul 24 23:35:29.847126 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:29.847082    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jp4jc" event={"ID":"33ce1323-16e5-4514-8a41-26d0b876bb91","Type":"ContainerDied","Data":"14ab87caffbc791d9b46562686943c8d8f4c25dff3090bb2e07d70539ce0b5d2"}
Jul 24 23:35:29.847126 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:29.847117    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jp4jc" event={"ID":"33ce1323-16e5-4514-8a41-26d0b876bb91","Type":"ContainerStarted","Data":"d6aecaabf032d5775e325b33e448883dd5b117d4a5c71767ff7fae432c6efc28"}
Jul 24 23:35:30.853770 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:30.853727    2558 generic.go:356] "Generic (PLEG): container finished" podID="bad54421-36a4-4e6f-be54-59d837f047c4" containerID="6a3e519fc96a5114a0bf355ba4f2c08f81434d097065268f18bba3f03c9c8d35" exitCode=0
Jul 24 23:35:30.854157 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:30.853815    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75999hgt" event={"ID":"bad54421-36a4-4e6f-be54-59d837f047c4","Type":"ContainerDied","Data":"6a3e519fc96a5114a0bf355ba4f2c08f81434d097065268f18bba3f03c9c8d35"}
Jul 24 23:35:30.855630 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:30.855604    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jp4jc" event={"ID":"33ce1323-16e5-4514-8a41-26d0b876bb91","Type":"ContainerDied","Data":"864a17bc5854429d6110c9bbd3f397778a91513dd90ca819ba02df4645b8a031"}
Jul 24 23:35:30.855753 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:30.855645    2558 generic.go:356] "Generic (PLEG): container finished" podID="33ce1323-16e5-4514-8a41-26d0b876bb91" containerID="864a17bc5854429d6110c9bbd3f397778a91513dd90ca819ba02df4645b8a031" exitCode=0
Jul 24 23:35:30.857855 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:30.857833    2558 generic.go:356] "Generic (PLEG): container finished" podID="8db0cdbd-5d8c-4213-8aa2-cb0d25ffb425" containerID="7633cd00f69f7b5fd636b243dd10114a5012257a5a60801825b3c15fbac1f26a" exitCode=0
Jul 24 23:35:30.857991 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:30.857934    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7394m4w" event={"ID":"8db0cdbd-5d8c-4213-8aa2-cb0d25ffb425","Type":"ContainerDied","Data":"7633cd00f69f7b5fd636b243dd10114a5012257a5a60801825b3c15fbac1f26a"}
Jul 24 23:35:30.994288 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:30.994261    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kqwqt"
Jul 24 23:35:31.062851 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:31.062822    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/650df1f9-9ba5-44bc-82a9-07cc28c371d2-bundle\") pod \"650df1f9-9ba5-44bc-82a9-07cc28c371d2\" (UID: \"650df1f9-9ba5-44bc-82a9-07cc28c371d2\") "
Jul 24 23:35:31.062993 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:31.062976    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fcccl\" (UniqueName: \"kubernetes.io/projected/650df1f9-9ba5-44bc-82a9-07cc28c371d2-kube-api-access-fcccl\") pod \"650df1f9-9ba5-44bc-82a9-07cc28c371d2\" (UID: \"650df1f9-9ba5-44bc-82a9-07cc28c371d2\") "
Jul 24 23:35:31.063056 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:31.063014    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/650df1f9-9ba5-44bc-82a9-07cc28c371d2-util\") pod \"650df1f9-9ba5-44bc-82a9-07cc28c371d2\" (UID: \"650df1f9-9ba5-44bc-82a9-07cc28c371d2\") "
Jul 24 23:35:31.063366 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:31.063340    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/650df1f9-9ba5-44bc-82a9-07cc28c371d2-bundle" (OuterVolumeSpecName: "bundle") pod "650df1f9-9ba5-44bc-82a9-07cc28c371d2" (UID: "650df1f9-9ba5-44bc-82a9-07cc28c371d2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:35:31.065179 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:31.065149    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/650df1f9-9ba5-44bc-82a9-07cc28c371d2-kube-api-access-fcccl" (OuterVolumeSpecName: "kube-api-access-fcccl") pod "650df1f9-9ba5-44bc-82a9-07cc28c371d2" (UID: "650df1f9-9ba5-44bc-82a9-07cc28c371d2"). InnerVolumeSpecName "kube-api-access-fcccl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:35:31.069281 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:31.069250    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/650df1f9-9ba5-44bc-82a9-07cc28c371d2-util" (OuterVolumeSpecName: "util") pod "650df1f9-9ba5-44bc-82a9-07cc28c371d2" (UID: "650df1f9-9ba5-44bc-82a9-07cc28c371d2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:35:31.163952 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:31.163854    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fcccl\" (UniqueName: \"kubernetes.io/projected/650df1f9-9ba5-44bc-82a9-07cc28c371d2-kube-api-access-fcccl\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 24 23:35:31.163952 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:31.163900    2558 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/650df1f9-9ba5-44bc-82a9-07cc28c371d2-util\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 24 23:35:31.163952 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:31.163910    2558 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/650df1f9-9ba5-44bc-82a9-07cc28c371d2-bundle\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 24 23:35:31.864419 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:31.864383    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kqwqt"
Jul 24 23:35:31.864419 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:31.864413    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kqwqt" event={"ID":"650df1f9-9ba5-44bc-82a9-07cc28c371d2","Type":"ContainerDied","Data":"07a142aab71055e17eeea9869d09a6ae924fd4c45748e7ec33b62bbdb8197596"}
Jul 24 23:35:31.864891 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:31.864463    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="07a142aab71055e17eeea9869d09a6ae924fd4c45748e7ec33b62bbdb8197596"
Jul 24 23:35:31.866617 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:31.866594    2558 generic.go:356] "Generic (PLEG): container finished" podID="33ce1323-16e5-4514-8a41-26d0b876bb91" containerID="f95df75258609e2234c999db3b71e0a578081046bda88aa36f70af8a842a77ae" exitCode=0
Jul 24 23:35:31.866744 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:31.866681    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jp4jc" event={"ID":"33ce1323-16e5-4514-8a41-26d0b876bb91","Type":"ContainerDied","Data":"f95df75258609e2234c999db3b71e0a578081046bda88aa36f70af8a842a77ae"}
Jul 24 23:35:31.868578 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:31.868556    2558 generic.go:356] "Generic (PLEG): container finished" podID="8db0cdbd-5d8c-4213-8aa2-cb0d25ffb425" containerID="f86ddbd55f817ff4534620f4e552b404b5ae6afa93ef7759292c099a3d4fa65b" exitCode=0
Jul 24 23:35:31.868661 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:31.868638    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7394m4w" event={"ID":"8db0cdbd-5d8c-4213-8aa2-cb0d25ffb425","Type":"ContainerDied","Data":"f86ddbd55f817ff4534620f4e552b404b5ae6afa93ef7759292c099a3d4fa65b"}
Jul 24 23:35:31.996132 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:31.996107    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75999hgt"
Jul 24 23:35:32.072051 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:32.072018    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bad54421-36a4-4e6f-be54-59d837f047c4-util\") pod \"bad54421-36a4-4e6f-be54-59d837f047c4\" (UID: \"bad54421-36a4-4e6f-be54-59d837f047c4\") "
Jul 24 23:35:32.072235 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:32.072076    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ft28f\" (UniqueName: \"kubernetes.io/projected/bad54421-36a4-4e6f-be54-59d837f047c4-kube-api-access-ft28f\") pod \"bad54421-36a4-4e6f-be54-59d837f047c4\" (UID: \"bad54421-36a4-4e6f-be54-59d837f047c4\") "
Jul 24 23:35:32.072235 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:32.072135    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bad54421-36a4-4e6f-be54-59d837f047c4-bundle\") pod \"bad54421-36a4-4e6f-be54-59d837f047c4\" (UID: \"bad54421-36a4-4e6f-be54-59d837f047c4\") "
Jul 24 23:35:32.072645 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:32.072618    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bad54421-36a4-4e6f-be54-59d837f047c4-bundle" (OuterVolumeSpecName: "bundle") pod "bad54421-36a4-4e6f-be54-59d837f047c4" (UID: "bad54421-36a4-4e6f-be54-59d837f047c4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:35:32.074174 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:32.074144    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bad54421-36a4-4e6f-be54-59d837f047c4-kube-api-access-ft28f" (OuterVolumeSpecName: "kube-api-access-ft28f") pod "bad54421-36a4-4e6f-be54-59d837f047c4" (UID: "bad54421-36a4-4e6f-be54-59d837f047c4"). InnerVolumeSpecName "kube-api-access-ft28f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:35:32.077332 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:32.077294    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bad54421-36a4-4e6f-be54-59d837f047c4-util" (OuterVolumeSpecName: "util") pod "bad54421-36a4-4e6f-be54-59d837f047c4" (UID: "bad54421-36a4-4e6f-be54-59d837f047c4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:35:32.173204 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:32.173168    2558 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bad54421-36a4-4e6f-be54-59d837f047c4-util\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 24 23:35:32.173204 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:32.173197    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ft28f\" (UniqueName: \"kubernetes.io/projected/bad54421-36a4-4e6f-be54-59d837f047c4-kube-api-access-ft28f\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 24 23:35:32.173204 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:32.173208    2558 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bad54421-36a4-4e6f-be54-59d837f047c4-bundle\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 24 23:35:32.875545 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:32.875508    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75999hgt"
Jul 24 23:35:32.875974 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:32.875546    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75999hgt" event={"ID":"bad54421-36a4-4e6f-be54-59d837f047c4","Type":"ContainerDied","Data":"2aa932ecb0896bff288d7e00f7ae4aba75610ef8c9dbf71cab56e1b06580b483"}
Jul 24 23:35:32.875974 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:32.875592    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2aa932ecb0896bff288d7e00f7ae4aba75610ef8c9dbf71cab56e1b06580b483"
Jul 24 23:35:33.006574 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:33.006549    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7394m4w"
Jul 24 23:35:33.037822 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:33.037791    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jp4jc"
Jul 24 23:35:33.081122 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:33.081088    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8db0cdbd-5d8c-4213-8aa2-cb0d25ffb425-bundle\") pod \"8db0cdbd-5d8c-4213-8aa2-cb0d25ffb425\" (UID: \"8db0cdbd-5d8c-4213-8aa2-cb0d25ffb425\") "
Jul 24 23:35:33.081277 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:33.081168    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8db0cdbd-5d8c-4213-8aa2-cb0d25ffb425-util\") pod \"8db0cdbd-5d8c-4213-8aa2-cb0d25ffb425\" (UID: \"8db0cdbd-5d8c-4213-8aa2-cb0d25ffb425\") "
Jul 24 23:35:33.081277 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:33.081192    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vvfmx\" (UniqueName: \"kubernetes.io/projected/8db0cdbd-5d8c-4213-8aa2-cb0d25ffb425-kube-api-access-vvfmx\") pod \"8db0cdbd-5d8c-4213-8aa2-cb0d25ffb425\" (UID: \"8db0cdbd-5d8c-4213-8aa2-cb0d25ffb425\") "
Jul 24 23:35:33.081876 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:33.081845    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8db0cdbd-5d8c-4213-8aa2-cb0d25ffb425-bundle" (OuterVolumeSpecName: "bundle") pod "8db0cdbd-5d8c-4213-8aa2-cb0d25ffb425" (UID: "8db0cdbd-5d8c-4213-8aa2-cb0d25ffb425"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:35:33.083319 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:33.083269    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8db0cdbd-5d8c-4213-8aa2-cb0d25ffb425-kube-api-access-vvfmx" (OuterVolumeSpecName: "kube-api-access-vvfmx") pod "8db0cdbd-5d8c-4213-8aa2-cb0d25ffb425" (UID: "8db0cdbd-5d8c-4213-8aa2-cb0d25ffb425"). InnerVolumeSpecName "kube-api-access-vvfmx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:35:33.086517 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:33.086493    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8db0cdbd-5d8c-4213-8aa2-cb0d25ffb425-util" (OuterVolumeSpecName: "util") pod "8db0cdbd-5d8c-4213-8aa2-cb0d25ffb425" (UID: "8db0cdbd-5d8c-4213-8aa2-cb0d25ffb425"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:35:33.181678 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:33.181603    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-npjds\" (UniqueName: \"kubernetes.io/projected/33ce1323-16e5-4514-8a41-26d0b876bb91-kube-api-access-npjds\") pod \"33ce1323-16e5-4514-8a41-26d0b876bb91\" (UID: \"33ce1323-16e5-4514-8a41-26d0b876bb91\") "
Jul 24 23:35:33.181813 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:33.181680    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/33ce1323-16e5-4514-8a41-26d0b876bb91-bundle\") pod \"33ce1323-16e5-4514-8a41-26d0b876bb91\" (UID: \"33ce1323-16e5-4514-8a41-26d0b876bb91\") "
Jul 24 23:35:33.181813 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:33.181769    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/33ce1323-16e5-4514-8a41-26d0b876bb91-util\") pod \"33ce1323-16e5-4514-8a41-26d0b876bb91\" (UID: \"33ce1323-16e5-4514-8a41-26d0b876bb91\") "
Jul 24 23:35:33.182018 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:33.182000    2558 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8db0cdbd-5d8c-4213-8aa2-cb0d25ffb425-bundle\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 24 23:35:33.182062 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:33.182024    2558 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8db0cdbd-5d8c-4213-8aa2-cb0d25ffb425-util\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 24 23:35:33.182062 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:33.182037    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vvfmx\" (UniqueName: \"kubernetes.io/projected/8db0cdbd-5d8c-4213-8aa2-cb0d25ffb425-kube-api-access-vvfmx\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 24 23:35:33.182163 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:33.182129    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33ce1323-16e5-4514-8a41-26d0b876bb91-bundle" (OuterVolumeSpecName: "bundle") pod "33ce1323-16e5-4514-8a41-26d0b876bb91" (UID: "33ce1323-16e5-4514-8a41-26d0b876bb91"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:35:33.183843 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:33.183823    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33ce1323-16e5-4514-8a41-26d0b876bb91-kube-api-access-npjds" (OuterVolumeSpecName: "kube-api-access-npjds") pod "33ce1323-16e5-4514-8a41-26d0b876bb91" (UID: "33ce1323-16e5-4514-8a41-26d0b876bb91"). InnerVolumeSpecName "kube-api-access-npjds". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:35:33.187094 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:33.187062    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33ce1323-16e5-4514-8a41-26d0b876bb91-util" (OuterVolumeSpecName: "util") pod "33ce1323-16e5-4514-8a41-26d0b876bb91" (UID: "33ce1323-16e5-4514-8a41-26d0b876bb91"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:35:33.283327 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:33.283291    2558 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/33ce1323-16e5-4514-8a41-26d0b876bb91-util\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 24 23:35:33.283327 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:33.283321    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-npjds\" (UniqueName: \"kubernetes.io/projected/33ce1323-16e5-4514-8a41-26d0b876bb91-kube-api-access-npjds\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 24 23:35:33.283327 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:33.283332    2558 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/33ce1323-16e5-4514-8a41-26d0b876bb91-bundle\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 24 23:35:33.882457 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:33.882430    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jp4jc"
Jul 24 23:35:33.882896 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:33.882427    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jp4jc" event={"ID":"33ce1323-16e5-4514-8a41-26d0b876bb91","Type":"ContainerDied","Data":"d6aecaabf032d5775e325b33e448883dd5b117d4a5c71767ff7fae432c6efc28"}
Jul 24 23:35:33.882896 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:33.882584    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d6aecaabf032d5775e325b33e448883dd5b117d4a5c71767ff7fae432c6efc28"
Jul 24 23:35:33.884392 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:33.884365    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7394m4w" event={"ID":"8db0cdbd-5d8c-4213-8aa2-cb0d25ffb425","Type":"ContainerDied","Data":"6bc1e284413b5513f1bc2cb6ae18d2baa6a799594fb4fc36e6a029c1280b36e6"}
Jul 24 23:35:33.884548 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:33.884391    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7394m4w"
Jul 24 23:35:33.884548 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:33.884407    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6bc1e284413b5513f1bc2cb6ae18d2baa6a799594fb4fc36e6a029c1280b36e6"
Jul 24 23:35:44.526754 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:44.526716    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:35:48.525957 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:48.525917    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:35:50.561234 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:50.561196    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:35:50.661503 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:50.661463    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:35:50.765511 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:50.765473    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:35:50.869056 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:50.868969    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:35:51.315058 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:51.315025    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:35:51.366722 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:51.366690    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:35:53.195298 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:53.195257    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:35:53.755397 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:53.755349    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:35:54.175147 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:54.175040    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:35:55.391361 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:55.391322    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:35:55.996128 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:55.996087    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:35:57.524269 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:57.524229    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:35:57.627589 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:57.627554    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:35:58.133930 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:35:58.133861    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:36:00.020549 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:00.020489    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:36:00.154158 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:00.154120    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:36:00.579743 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:00.579704    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:36:00.719146 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:00.719112    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:36:01.199933 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:01.199888    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:36:01.260069 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:01.260037    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:36:01.339239 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:01.339202    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:36:01.731435 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:01.731400    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:36:01.830109 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:01.830071    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:36:05.332228 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:05.332192    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:36:05.410415 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:05.410377    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:36:13.932326 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:13.932281    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:36:14.032027 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:14.031976    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:36:16.922610 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:16.922570    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:36:17.026277 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:17.026243    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:36:19.522012 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:19.521944    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:36:19.623481 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:19.623435    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:36:23.331503 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:23.331464    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:36:23.429799 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:23.429761    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:36:33.531340 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:33.531300    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:36:33.631670 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:33.631633    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:36:36.527908 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:36.527864    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:36:36.648963 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:36.648927    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:36:40.514357 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:40.514315    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:36:40.900578 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:40.900482    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:36:42.272592 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:42.272493    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:36:42.276303 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:42.276278    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8278x"]
Jul 24 23:36:42.276621 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:42.276607    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bad54421-36a4-4e6f-be54-59d837f047c4" containerName="extract"
Jul 24 23:36:42.276676 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:42.276622    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="bad54421-36a4-4e6f-be54-59d837f047c4" containerName="extract"
Jul 24 23:36:42.276676 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:42.276636    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="650df1f9-9ba5-44bc-82a9-07cc28c371d2" containerName="util"
Jul 24 23:36:42.276676 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:42.276644    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="650df1f9-9ba5-44bc-82a9-07cc28c371d2" containerName="util"
Jul 24 23:36:42.276676 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:42.276655    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8db0cdbd-5d8c-4213-8aa2-cb0d25ffb425" containerName="extract"
Jul 24 23:36:42.276676 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:42.276663    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="8db0cdbd-5d8c-4213-8aa2-cb0d25ffb425" containerName="extract"
Jul 24 23:36:42.276849 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:42.276680    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33ce1323-16e5-4514-8a41-26d0b876bb91" containerName="util"
Jul 24 23:36:42.276849 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:42.276689    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="33ce1323-16e5-4514-8a41-26d0b876bb91" containerName="util"
Jul 24 23:36:42.276849 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:42.276701    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8db0cdbd-5d8c-4213-8aa2-cb0d25ffb425" containerName="util"
Jul 24 23:36:42.276849 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:42.276709    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="8db0cdbd-5d8c-4213-8aa2-cb0d25ffb425" containerName="util"
Jul 24 23:36:42.276849 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:42.276718    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33ce1323-16e5-4514-8a41-26d0b876bb91" containerName="pull"
Jul 24 23:36:42.276849 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:42.276728    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="33ce1323-16e5-4514-8a41-26d0b876bb91" containerName="pull"
Jul 24 23:36:42.276849 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:42.276741    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bad54421-36a4-4e6f-be54-59d837f047c4" containerName="util"
Jul 24 23:36:42.276849 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:42.276750    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="bad54421-36a4-4e6f-be54-59d837f047c4" containerName="util"
Jul 24 23:36:42.276849 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:42.276757    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="650df1f9-9ba5-44bc-82a9-07cc28c371d2" containerName="pull"
Jul 24 23:36:42.276849 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:42.276762    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="650df1f9-9ba5-44bc-82a9-07cc28c371d2" containerName="pull"
Jul 24 23:36:42.276849 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:42.276771    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="650df1f9-9ba5-44bc-82a9-07cc28c371d2" containerName="extract"
Jul 24 23:36:42.276849 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:42.276776    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="650df1f9-9ba5-44bc-82a9-07cc28c371d2" containerName="extract"
Jul 24 23:36:42.276849 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:42.276782    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8db0cdbd-5d8c-4213-8aa2-cb0d25ffb425" containerName="pull"
Jul 24 23:36:42.276849 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:42.276787    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="8db0cdbd-5d8c-4213-8aa2-cb0d25ffb425" containerName="pull"
Jul 24 23:36:42.276849 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:42.276796    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bad54421-36a4-4e6f-be54-59d837f047c4" containerName="pull"
Jul 24 23:36:42.276849 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:42.276801    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="bad54421-36a4-4e6f-be54-59d837f047c4" containerName="pull"
Jul 24 23:36:42.276849 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:42.276806    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33ce1323-16e5-4514-8a41-26d0b876bb91" containerName="extract"
Jul 24 23:36:42.276849 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:42.276811    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="33ce1323-16e5-4514-8a41-26d0b876bb91" containerName="extract"
Jul 24 23:36:42.277453 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:42.276882    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="650df1f9-9ba5-44bc-82a9-07cc28c371d2" containerName="extract"
Jul 24 23:36:42.277453 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:42.276892    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="33ce1323-16e5-4514-8a41-26d0b876bb91" containerName="extract"
Jul 24 23:36:42.277453 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:42.276898    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="bad54421-36a4-4e6f-be54-59d837f047c4" containerName="extract"
Jul 24 23:36:42.277453 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:42.276904    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="8db0cdbd-5d8c-4213-8aa2-cb0d25ffb425" containerName="extract"
Jul 24 23:36:42.279898 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:42.279881    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-8278x"
Jul 24 23:36:42.282355 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:42.282333    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 24 23:36:42.282487 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:42.282367    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 24 23:36:42.282487 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:42.282368    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 24 23:36:42.283329 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:42.283312    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-2bx2t\""
Jul 24 23:36:42.289056 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:42.289034    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8278x"]
Jul 24 23:36:42.368203 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:42.368165    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvwcl\" (UniqueName: \"kubernetes.io/projected/e80c28af-7aa8-4467-b5de-78663c533a10-kube-api-access-fvwcl\") pod \"limitador-limitador-64c94fd64f-8278x\" (UID: \"e80c28af-7aa8-4467-b5de-78663c533a10\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8278x"
Jul 24 23:36:42.368367 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:42.368240    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e80c28af-7aa8-4467-b5de-78663c533a10-config-file\") pod \"limitador-limitador-64c94fd64f-8278x\" (UID: \"e80c28af-7aa8-4467-b5de-78663c533a10\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8278x"
Jul 24 23:36:42.378924 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:42.378888    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:36:42.379463 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:42.379441    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8278x"]
Jul 24 23:36:42.469621 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:42.469589    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvwcl\" (UniqueName: \"kubernetes.io/projected/e80c28af-7aa8-4467-b5de-78663c533a10-kube-api-access-fvwcl\") pod \"limitador-limitador-64c94fd64f-8278x\" (UID: \"e80c28af-7aa8-4467-b5de-78663c533a10\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8278x"
Jul 24 23:36:42.469789 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:42.469642    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e80c28af-7aa8-4467-b5de-78663c533a10-config-file\") pod \"limitador-limitador-64c94fd64f-8278x\" (UID: \"e80c28af-7aa8-4467-b5de-78663c533a10\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8278x"
Jul 24 23:36:42.470205 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:42.470186    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e80c28af-7aa8-4467-b5de-78663c533a10-config-file\") pod \"limitador-limitador-64c94fd64f-8278x\" (UID: \"e80c28af-7aa8-4467-b5de-78663c533a10\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8278x"
Jul 24 23:36:42.477085 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:42.477056    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:36:42.478868 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:42.478836    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvwcl\" (UniqueName: \"kubernetes.io/projected/e80c28af-7aa8-4467-b5de-78663c533a10-kube-api-access-fvwcl\") pod \"limitador-limitador-64c94fd64f-8278x\" (UID: \"e80c28af-7aa8-4467-b5de-78663c533a10\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8278x"
Jul 24 23:36:42.590848 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:42.590767    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-8278x"
Jul 24 23:36:42.723414 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:42.723376    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8278x"]
Jul 24 23:36:42.725342 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:36:42.725314    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode80c28af_7aa8_4467_b5de_78663c533a10.slice/crio-a5f0b8078d8b5bd4853dbd6b7544adec4dbfc243a3a25ec55e87b92327a576a1 WatchSource:0}: Error finding container a5f0b8078d8b5bd4853dbd6b7544adec4dbfc243a3a25ec55e87b92327a576a1: Status 404 returned error can't find the container with id a5f0b8078d8b5bd4853dbd6b7544adec4dbfc243a3a25ec55e87b92327a576a1
Jul 24 23:36:43.085773 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:43.085735    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:36:43.140763 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:43.140728    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:36:43.219844 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:43.219809    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-8278x" event={"ID":"e80c28af-7aa8-4467-b5de-78663c533a10","Type":"ContainerStarted","Data":"a5f0b8078d8b5bd4853dbd6b7544adec4dbfc243a3a25ec55e87b92327a576a1"}
Jul 24 23:36:46.238718 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:46.238681    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-8278x" event={"ID":"e80c28af-7aa8-4467-b5de-78663c533a10","Type":"ContainerStarted","Data":"a2ad4ca99da33661ca14db2ed8b3c86f7b8a8d9ad0bb1c2eecab40e0e9a3db05"}
Jul 24 23:36:46.239210 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:46.238796    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-8278x"
Jul 24 23:36:46.255817 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:46.255766    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-8278x" podStartSLOduration=1.693394242 podStartE2EDuration="4.2557513s" totalImagesPullingTime="2.562357058s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:36:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:36:46.253547807 +0000 UTC m=+446.639978526" watchObservedRunningTime="2026-07-24 23:36:46.2557513 +0000 UTC m=+446.642182083"
Jul 24 23:36:57.244350 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:57.244312    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-8278x"
Jul 24 23:36:59.329632 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:59.329598    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8278x"]
Jul 24 23:36:59.330083 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:59.329829    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-8278x" podUID="e80c28af-7aa8-4467-b5de-78663c533a10" containerName="limitador" containerID="cri-o://a2ad4ca99da33661ca14db2ed8b3c86f7b8a8d9ad0bb1c2eecab40e0e9a3db05" gracePeriod=30
Jul 24 23:36:59.881126 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:59.881098    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-8278x"
Jul 24 23:36:59.914021 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:59.913946    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e80c28af-7aa8-4467-b5de-78663c533a10-config-file\") pod \"e80c28af-7aa8-4467-b5de-78663c533a10\" (UID: \"e80c28af-7aa8-4467-b5de-78663c533a10\") "
Jul 24 23:36:59.914175 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:59.914042    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fvwcl\" (UniqueName: \"kubernetes.io/projected/e80c28af-7aa8-4467-b5de-78663c533a10-kube-api-access-fvwcl\") pod \"e80c28af-7aa8-4467-b5de-78663c533a10\" (UID: \"e80c28af-7aa8-4467-b5de-78663c533a10\") "
Jul 24 23:36:59.914345 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:59.914322    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e80c28af-7aa8-4467-b5de-78663c533a10-config-file" (OuterVolumeSpecName: "config-file") pod "e80c28af-7aa8-4467-b5de-78663c533a10" (UID: "e80c28af-7aa8-4467-b5de-78663c533a10"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:36:59.916069 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:36:59.916044    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e80c28af-7aa8-4467-b5de-78663c533a10-kube-api-access-fvwcl" (OuterVolumeSpecName: "kube-api-access-fvwcl") pod "e80c28af-7aa8-4467-b5de-78663c533a10" (UID: "e80c28af-7aa8-4467-b5de-78663c533a10"). InnerVolumeSpecName "kube-api-access-fvwcl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:37:00.015210 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:00.015170    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fvwcl\" (UniqueName: \"kubernetes.io/projected/e80c28af-7aa8-4467-b5de-78663c533a10-kube-api-access-fvwcl\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 24 23:37:00.015210 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:00.015203    2558 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e80c28af-7aa8-4467-b5de-78663c533a10-config-file\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 24 23:37:00.304810 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:00.304777    2558 generic.go:356] "Generic (PLEG): container finished" podID="e80c28af-7aa8-4467-b5de-78663c533a10" containerID="a2ad4ca99da33661ca14db2ed8b3c86f7b8a8d9ad0bb1c2eecab40e0e9a3db05" exitCode=0
Jul 24 23:37:00.305001 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:00.304824    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-8278x"
Jul 24 23:37:00.305001 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:00.304875    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-8278x" event={"ID":"e80c28af-7aa8-4467-b5de-78663c533a10","Type":"ContainerDied","Data":"a2ad4ca99da33661ca14db2ed8b3c86f7b8a8d9ad0bb1c2eecab40e0e9a3db05"}
Jul 24 23:37:00.305001 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:00.304924    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-8278x" event={"ID":"e80c28af-7aa8-4467-b5de-78663c533a10","Type":"ContainerDied","Data":"a5f0b8078d8b5bd4853dbd6b7544adec4dbfc243a3a25ec55e87b92327a576a1"}
Jul 24 23:37:00.305001 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:00.304947    2558 scope.go:117] "RemoveContainer" containerID="a2ad4ca99da33661ca14db2ed8b3c86f7b8a8d9ad0bb1c2eecab40e0e9a3db05"
Jul 24 23:37:00.314094 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:00.314075    2558 scope.go:117] "RemoveContainer" containerID="a2ad4ca99da33661ca14db2ed8b3c86f7b8a8d9ad0bb1c2eecab40e0e9a3db05"
Jul 24 23:37:00.314345 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:37:00.314325    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2ad4ca99da33661ca14db2ed8b3c86f7b8a8d9ad0bb1c2eecab40e0e9a3db05\": container with ID starting with a2ad4ca99da33661ca14db2ed8b3c86f7b8a8d9ad0bb1c2eecab40e0e9a3db05 not found: ID does not exist" containerID="a2ad4ca99da33661ca14db2ed8b3c86f7b8a8d9ad0bb1c2eecab40e0e9a3db05"
Jul 24 23:37:00.314392 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:00.314354    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2ad4ca99da33661ca14db2ed8b3c86f7b8a8d9ad0bb1c2eecab40e0e9a3db05"} err="failed to get container status \"a2ad4ca99da33661ca14db2ed8b3c86f7b8a8d9ad0bb1c2eecab40e0e9a3db05\": rpc error: code = NotFound desc = could not find container \"a2ad4ca99da33661ca14db2ed8b3c86f7b8a8d9ad0bb1c2eecab40e0e9a3db05\": container with ID starting with a2ad4ca99da33661ca14db2ed8b3c86f7b8a8d9ad0bb1c2eecab40e0e9a3db05 not found: ID does not exist"
Jul 24 23:37:00.321030 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:00.321003    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8278x"]
Jul 24 23:37:00.324662 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:00.324639    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8278x"]
Jul 24 23:37:02.203796 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:02.203760    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e80c28af-7aa8-4467-b5de-78663c533a10" path="/var/lib/kubelet/pods/e80c28af-7aa8-4467-b5de-78663c533a10/volumes"
Jul 24 23:37:03.624265 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:03.624227    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:37:03.718810 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:03.718777    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:37:04.117516 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:04.117480    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-w9zls"]
Jul 24 23:37:04.117951 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:04.117930    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e80c28af-7aa8-4467-b5de-78663c533a10" containerName="limitador"
Jul 24 23:37:04.117951 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:04.117951    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="e80c28af-7aa8-4467-b5de-78663c533a10" containerName="limitador"
Jul 24 23:37:04.118107 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:04.118015    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="e80c28af-7aa8-4467-b5de-78663c533a10" containerName="limitador"
Jul 24 23:37:04.122400 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:04.122378    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-w9zls"
Jul 24 23:37:04.126132 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:04.126105    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 24 23:37:04.126298 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:04.126279    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 24 23:37:04.126762 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:04.126741    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 23:37:04.140810 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:04.140770    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-w9zls"]
Jul 24 23:37:04.141297 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:04.141273    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-6j2n2\""
Jul 24 23:37:04.199695 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:04.199654    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:37:04.250906 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:04.250871    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tl7p\" (UniqueName: \"kubernetes.io/projected/73160574-b79c-4c1e-8f5f-8a11c814f3fe-kube-api-access-5tl7p\") pod \"postgres-69fb4594cf-w9zls\" (UID: \"73160574-b79c-4c1e-8f5f-8a11c814f3fe\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-w9zls"
Jul 24 23:37:04.251127 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:04.250931    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/73160574-b79c-4c1e-8f5f-8a11c814f3fe-data\") pod \"postgres-69fb4594cf-w9zls\" (UID: \"73160574-b79c-4c1e-8f5f-8a11c814f3fe\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-w9zls"
Jul 24 23:37:04.352185 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:04.352146    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/73160574-b79c-4c1e-8f5f-8a11c814f3fe-data\") pod \"postgres-69fb4594cf-w9zls\" (UID: \"73160574-b79c-4c1e-8f5f-8a11c814f3fe\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-w9zls"
Jul 24 23:37:04.352432 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:04.352212    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tl7p\" (UniqueName: \"kubernetes.io/projected/73160574-b79c-4c1e-8f5f-8a11c814f3fe-kube-api-access-5tl7p\") pod \"postgres-69fb4594cf-w9zls\" (UID: \"73160574-b79c-4c1e-8f5f-8a11c814f3fe\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-w9zls"
Jul 24 23:37:04.352563 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:04.352519    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/73160574-b79c-4c1e-8f5f-8a11c814f3fe-data\") pod \"postgres-69fb4594cf-w9zls\" (UID: \"73160574-b79c-4c1e-8f5f-8a11c814f3fe\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-w9zls"
Jul 24 23:37:04.360575 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:04.360558    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tl7p\" (UniqueName: \"kubernetes.io/projected/73160574-b79c-4c1e-8f5f-8a11c814f3fe-kube-api-access-5tl7p\") pod \"postgres-69fb4594cf-w9zls\" (UID: \"73160574-b79c-4c1e-8f5f-8a11c814f3fe\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-w9zls"
Jul 24 23:37:04.437242 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:04.437158    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-w9zls"
Jul 24 23:37:04.561762 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:04.561736    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-w9zls"]
Jul 24 23:37:04.564301 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:37:04.564268    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73160574_b79c_4c1e_8f5f_8a11c814f3fe.slice/crio-790b218dd6a873cb1df4d8dda33ac3882b9358cfe2fb9ace764812a30d7b390f WatchSource:0}: Error finding container 790b218dd6a873cb1df4d8dda33ac3882b9358cfe2fb9ace764812a30d7b390f: Status 404 returned error can't find the container with id 790b218dd6a873cb1df4d8dda33ac3882b9358cfe2fb9ace764812a30d7b390f
Jul 24 23:37:05.334455 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:05.334377    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-w9zls" event={"ID":"73160574-b79c-4c1e-8f5f-8a11c814f3fe","Type":"ContainerStarted","Data":"790b218dd6a873cb1df4d8dda33ac3882b9358cfe2fb9ace764812a30d7b390f"}
Jul 24 23:37:10.363096 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:10.363062    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-w9zls" event={"ID":"73160574-b79c-4c1e-8f5f-8a11c814f3fe","Type":"ContainerStarted","Data":"060affb7642633e8b08eb65c4ecd532ef54d8d80e458becf98277cfd807e4eab"}
Jul 24 23:37:10.363486 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:10.363233    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-w9zls"
Jul 24 23:37:10.380016 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:10.379955    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-w9zls" podStartSLOduration=1.433229738 podStartE2EDuration="6.3799353s" totalImagesPullingTime="4.946705562s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:37:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:37:10.377009791 +0000 UTC m=+470.763440520" watchObservedRunningTime="2026-07-24 23:37:10.3799353 +0000 UTC m=+470.766366029"
Jul 24 23:37:12.805818 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:12.805785    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:37:12.858750 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:12.858710    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:37:16.396961 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:16.396928    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-w9zls"
Jul 24 23:37:16.726970 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:16.726932    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:37:16.828601 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:16.828554    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:37:23.254321 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:23.254276    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:37:23.274157 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:23.274126    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-86d4999dfb-l8bcr"]
Jul 24 23:37:23.292193 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:23.292163    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-86d4999dfb-l8bcr"]
Jul 24 23:37:23.292352 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:23.292295    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-86d4999dfb-l8bcr"
Jul 24 23:37:23.294784 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:23.294761    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 24 23:37:23.294918 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:23.294790    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-l7fhg\""
Jul 24 23:37:23.294918 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:23.294760    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 24 23:37:23.353544 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:23.353496    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:37:23.421785 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:23.421747    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqdnz\" (UniqueName: \"kubernetes.io/projected/6c8e3ad6-74dd-43b0-b674-9995ef3c14ca-kube-api-access-sqdnz\") pod \"maas-controller-86d4999dfb-l8bcr\" (UID: \"6c8e3ad6-74dd-43b0-b674-9995ef3c14ca\") " pod="opendatahub/maas-controller-86d4999dfb-l8bcr"
Jul 24 23:37:23.421957 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:23.421861    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6c8e3ad6-74dd-43b0-b674-9995ef3c14ca-webhook-cert\") pod \"maas-controller-86d4999dfb-l8bcr\" (UID: \"6c8e3ad6-74dd-43b0-b674-9995ef3c14ca\") " pod="opendatahub/maas-controller-86d4999dfb-l8bcr"
Jul 24 23:37:23.455245 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:23.455210    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:37:23.456453 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:23.456429    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-59bb55c686-gnnsj"]
Jul 24 23:37:23.458803 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:23.458785    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-59bb55c686-gnnsj"
Jul 24 23:37:23.468151 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:23.468130    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-59bb55c686-gnnsj"]
Jul 24 23:37:23.523225 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:23.523139    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqdnz\" (UniqueName: \"kubernetes.io/projected/6c8e3ad6-74dd-43b0-b674-9995ef3c14ca-kube-api-access-sqdnz\") pod \"maas-controller-86d4999dfb-l8bcr\" (UID: \"6c8e3ad6-74dd-43b0-b674-9995ef3c14ca\") " pod="opendatahub/maas-controller-86d4999dfb-l8bcr"
Jul 24 23:37:23.523225 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:23.523202    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6c8e3ad6-74dd-43b0-b674-9995ef3c14ca-webhook-cert\") pod \"maas-controller-86d4999dfb-l8bcr\" (UID: \"6c8e3ad6-74dd-43b0-b674-9995ef3c14ca\") " pod="opendatahub/maas-controller-86d4999dfb-l8bcr"
Jul 24 23:37:23.523418 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:37:23.523294    2558 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Jul 24 23:37:23.523418 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:37:23.523353    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c8e3ad6-74dd-43b0-b674-9995ef3c14ca-webhook-cert podName:6c8e3ad6-74dd-43b0-b674-9995ef3c14ca nodeName:}" failed. No retries permitted until 2026-07-24 23:37:24.023335741 +0000 UTC m=+484.409766448 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/6c8e3ad6-74dd-43b0-b674-9995ef3c14ca-webhook-cert") pod "maas-controller-86d4999dfb-l8bcr" (UID: "6c8e3ad6-74dd-43b0-b674-9995ef3c14ca") : secret "maas-controller-webhook-cert" not found
Jul 24 23:37:23.531650 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:23.531625    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqdnz\" (UniqueName: \"kubernetes.io/projected/6c8e3ad6-74dd-43b0-b674-9995ef3c14ca-kube-api-access-sqdnz\") pod \"maas-controller-86d4999dfb-l8bcr\" (UID: \"6c8e3ad6-74dd-43b0-b674-9995ef3c14ca\") " pod="opendatahub/maas-controller-86d4999dfb-l8bcr"
Jul 24 23:37:23.624178 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:23.624145    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jqx2\" (UniqueName: \"kubernetes.io/projected/4df5609d-267e-4e54-bda3-ce7f3435cd2c-kube-api-access-2jqx2\") pod \"maas-controller-59bb55c686-gnnsj\" (UID: \"4df5609d-267e-4e54-bda3-ce7f3435cd2c\") " pod="opendatahub/maas-controller-59bb55c686-gnnsj"
Jul 24 23:37:23.624328 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:23.624240    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4df5609d-267e-4e54-bda3-ce7f3435cd2c-webhook-cert\") pod \"maas-controller-59bb55c686-gnnsj\" (UID: \"4df5609d-267e-4e54-bda3-ce7f3435cd2c\") " pod="opendatahub/maas-controller-59bb55c686-gnnsj"
Jul 24 23:37:23.685991 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:23.685952    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:37:23.725110 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:23.725077    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4df5609d-267e-4e54-bda3-ce7f3435cd2c-webhook-cert\") pod \"maas-controller-59bb55c686-gnnsj\" (UID: \"4df5609d-267e-4e54-bda3-ce7f3435cd2c\") " pod="opendatahub/maas-controller-59bb55c686-gnnsj"
Jul 24 23:37:23.725290 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:23.725132    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jqx2\" (UniqueName: \"kubernetes.io/projected/4df5609d-267e-4e54-bda3-ce7f3435cd2c-kube-api-access-2jqx2\") pod \"maas-controller-59bb55c686-gnnsj\" (UID: \"4df5609d-267e-4e54-bda3-ce7f3435cd2c\") " pod="opendatahub/maas-controller-59bb55c686-gnnsj"
Jul 24 23:37:23.727705 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:23.727681    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4df5609d-267e-4e54-bda3-ce7f3435cd2c-webhook-cert\") pod \"maas-controller-59bb55c686-gnnsj\" (UID: \"4df5609d-267e-4e54-bda3-ce7f3435cd2c\") " pod="opendatahub/maas-controller-59bb55c686-gnnsj"
Jul 24 23:37:23.732664 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:23.732638    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jqx2\" (UniqueName: \"kubernetes.io/projected/4df5609d-267e-4e54-bda3-ce7f3435cd2c-kube-api-access-2jqx2\") pod \"maas-controller-59bb55c686-gnnsj\" (UID: \"4df5609d-267e-4e54-bda3-ce7f3435cd2c\") " pod="opendatahub/maas-controller-59bb55c686-gnnsj"
Jul 24 23:37:23.769266 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:23.769243    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-59bb55c686-gnnsj"
Jul 24 23:37:23.896636 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:23.896609    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-59bb55c686-gnnsj"]
Jul 24 23:37:23.898365 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:37:23.898337    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4df5609d_267e_4e54_bda3_ce7f3435cd2c.slice/crio-e1f9a1a246232a13941db25c5e114a5e6ed2dec7ba4c25136108eba3eb0dd199 WatchSource:0}: Error finding container e1f9a1a246232a13941db25c5e114a5e6ed2dec7ba4c25136108eba3eb0dd199: Status 404 returned error can't find the container with id e1f9a1a246232a13941db25c5e114a5e6ed2dec7ba4c25136108eba3eb0dd199
Jul 24 23:37:24.028733 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:24.028645    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6c8e3ad6-74dd-43b0-b674-9995ef3c14ca-webhook-cert\") pod \"maas-controller-86d4999dfb-l8bcr\" (UID: \"6c8e3ad6-74dd-43b0-b674-9995ef3c14ca\") " pod="opendatahub/maas-controller-86d4999dfb-l8bcr"
Jul 24 23:37:24.031090 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:24.031069    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6c8e3ad6-74dd-43b0-b674-9995ef3c14ca-webhook-cert\") pod \"maas-controller-86d4999dfb-l8bcr\" (UID: \"6c8e3ad6-74dd-43b0-b674-9995ef3c14ca\") " pod="opendatahub/maas-controller-86d4999dfb-l8bcr"
Jul 24 23:37:24.202891 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:24.202861    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-86d4999dfb-l8bcr"
Jul 24 23:37:24.332996 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:24.332968    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-86d4999dfb-l8bcr"]
Jul 24 23:37:24.334417 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:37:24.334389    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c8e3ad6_74dd_43b0_b674_9995ef3c14ca.slice/crio-5ea4e602befe5da66ba36029a0651139ea1a922097606cd0570f0152eebd2690 WatchSource:0}: Error finding container 5ea4e602befe5da66ba36029a0651139ea1a922097606cd0570f0152eebd2690: Status 404 returned error can't find the container with id 5ea4e602befe5da66ba36029a0651139ea1a922097606cd0570f0152eebd2690
Jul 24 23:37:24.437090 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:24.437049    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-59bb55c686-gnnsj" event={"ID":"4df5609d-267e-4e54-bda3-ce7f3435cd2c","Type":"ContainerStarted","Data":"e1f9a1a246232a13941db25c5e114a5e6ed2dec7ba4c25136108eba3eb0dd199"}
Jul 24 23:37:24.438164 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:24.438138    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-86d4999dfb-l8bcr" event={"ID":"6c8e3ad6-74dd-43b0-b674-9995ef3c14ca","Type":"ContainerStarted","Data":"5ea4e602befe5da66ba36029a0651139ea1a922097606cd0570f0152eebd2690"}
Jul 24 23:37:26.819952 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:26.819908    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:37:26.927684 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:26.927602    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:37:27.456795 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:27.456756    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-59bb55c686-gnnsj" event={"ID":"4df5609d-267e-4e54-bda3-ce7f3435cd2c","Type":"ContainerStarted","Data":"8af74588e0b337aea9ab901439208157f56c93bf4a8128c77875655ef0cf5226"}
Jul 24 23:37:27.456979 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:27.456893    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-59bb55c686-gnnsj"
Jul 24 23:37:27.458384 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:27.458357    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-86d4999dfb-l8bcr" event={"ID":"6c8e3ad6-74dd-43b0-b674-9995ef3c14ca","Type":"ContainerStarted","Data":"a46f7ac585d1c99977c7f78d3f4ad16d50a9bf677cc6b16941aae29f6711a6b8"}
Jul 24 23:37:27.458514 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:27.458497    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-86d4999dfb-l8bcr"
Jul 24 23:37:27.476681 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:27.476632    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-59bb55c686-gnnsj" podStartSLOduration=1.712888595 podStartE2EDuration="4.476616523s" totalImagesPullingTime="2.763727928s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:37:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:37:27.475864557 +0000 UTC m=+487.862295286" watchObservedRunningTime="2026-07-24 23:37:27.476616523 +0000 UTC m=+487.863047266"
Jul 24 23:37:27.493443 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:27.493393    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-86d4999dfb-l8bcr" podStartSLOduration=2.174229671 podStartE2EDuration="4.493375371s" totalImagesPullingTime="2.3191457s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:37:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:37:27.491852679 +0000 UTC m=+487.878283408" watchObservedRunningTime="2026-07-24 23:37:27.493375371 +0000 UTC m=+487.879806098"
Jul 24 23:37:38.466629 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:38.466593    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-86d4999dfb-l8bcr"
Jul 24 23:37:38.467066 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:38.466942    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-59bb55c686-gnnsj"
Jul 24 23:37:38.532164 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:38.527452    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-86d4999dfb-l8bcr"]
Jul 24 23:37:38.533159 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:38.533096    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-86d4999dfb-l8bcr" podUID="6c8e3ad6-74dd-43b0-b674-9995ef3c14ca" containerName="manager" containerID="cri-o://a46f7ac585d1c99977c7f78d3f4ad16d50a9bf677cc6b16941aae29f6711a6b8" gracePeriod=30
Jul 24 23:37:38.775109 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:38.775085    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-86d4999dfb-l8bcr"
Jul 24 23:37:38.861929 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:38.861897    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6c8e3ad6-74dd-43b0-b674-9995ef3c14ca-webhook-cert\") pod \"6c8e3ad6-74dd-43b0-b674-9995ef3c14ca\" (UID: \"6c8e3ad6-74dd-43b0-b674-9995ef3c14ca\") "
Jul 24 23:37:38.862133 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:38.861973    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sqdnz\" (UniqueName: \"kubernetes.io/projected/6c8e3ad6-74dd-43b0-b674-9995ef3c14ca-kube-api-access-sqdnz\") pod \"6c8e3ad6-74dd-43b0-b674-9995ef3c14ca\" (UID: \"6c8e3ad6-74dd-43b0-b674-9995ef3c14ca\") "
Jul 24 23:37:38.864066 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:38.864036    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6c8e3ad6-74dd-43b0-b674-9995ef3c14ca-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "6c8e3ad6-74dd-43b0-b674-9995ef3c14ca" (UID: "6c8e3ad6-74dd-43b0-b674-9995ef3c14ca"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 23:37:38.864190 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:38.864167    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c8e3ad6-74dd-43b0-b674-9995ef3c14ca-kube-api-access-sqdnz" (OuterVolumeSpecName: "kube-api-access-sqdnz") pod "6c8e3ad6-74dd-43b0-b674-9995ef3c14ca" (UID: "6c8e3ad6-74dd-43b0-b674-9995ef3c14ca"). InnerVolumeSpecName "kube-api-access-sqdnz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:37:38.963196 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:38.963168    2558 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6c8e3ad6-74dd-43b0-b674-9995ef3c14ca-webhook-cert\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 24 23:37:38.963196 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:38.963194    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sqdnz\" (UniqueName: \"kubernetes.io/projected/6c8e3ad6-74dd-43b0-b674-9995ef3c14ca-kube-api-access-sqdnz\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 24 23:37:39.519374 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:39.519336    2558 generic.go:356] "Generic (PLEG): container finished" podID="6c8e3ad6-74dd-43b0-b674-9995ef3c14ca" containerID="a46f7ac585d1c99977c7f78d3f4ad16d50a9bf677cc6b16941aae29f6711a6b8" exitCode=0
Jul 24 23:37:39.519811 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:39.519384    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-86d4999dfb-l8bcr"
Jul 24 23:37:39.519811 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:39.519398    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-86d4999dfb-l8bcr" event={"ID":"6c8e3ad6-74dd-43b0-b674-9995ef3c14ca","Type":"ContainerDied","Data":"a46f7ac585d1c99977c7f78d3f4ad16d50a9bf677cc6b16941aae29f6711a6b8"}
Jul 24 23:37:39.519811 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:39.519450    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-86d4999dfb-l8bcr" event={"ID":"6c8e3ad6-74dd-43b0-b674-9995ef3c14ca","Type":"ContainerDied","Data":"5ea4e602befe5da66ba36029a0651139ea1a922097606cd0570f0152eebd2690"}
Jul 24 23:37:39.519811 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:39.519480    2558 scope.go:117] "RemoveContainer" containerID="a46f7ac585d1c99977c7f78d3f4ad16d50a9bf677cc6b16941aae29f6711a6b8"
Jul 24 23:37:39.529299 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:39.529280    2558 scope.go:117] "RemoveContainer" containerID="a46f7ac585d1c99977c7f78d3f4ad16d50a9bf677cc6b16941aae29f6711a6b8"
Jul 24 23:37:39.529594 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:37:39.529568    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a46f7ac585d1c99977c7f78d3f4ad16d50a9bf677cc6b16941aae29f6711a6b8\": container with ID starting with a46f7ac585d1c99977c7f78d3f4ad16d50a9bf677cc6b16941aae29f6711a6b8 not found: ID does not exist" containerID="a46f7ac585d1c99977c7f78d3f4ad16d50a9bf677cc6b16941aae29f6711a6b8"
Jul 24 23:37:39.529698 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:39.529600    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a46f7ac585d1c99977c7f78d3f4ad16d50a9bf677cc6b16941aae29f6711a6b8"} err="failed to get container status \"a46f7ac585d1c99977c7f78d3f4ad16d50a9bf677cc6b16941aae29f6711a6b8\": rpc error: code = NotFound desc = could not find container \"a46f7ac585d1c99977c7f78d3f4ad16d50a9bf677cc6b16941aae29f6711a6b8\": container with ID starting with a46f7ac585d1c99977c7f78d3f4ad16d50a9bf677cc6b16941aae29f6711a6b8 not found: ID does not exist"
Jul 24 23:37:39.541486 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:39.541459    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-86d4999dfb-l8bcr"]
Jul 24 23:37:39.545101 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:39.545077    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-86d4999dfb-l8bcr"]
Jul 24 23:37:40.204343 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:40.204310    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6c8e3ad6-74dd-43b0-b674-9995ef3c14ca" path="/var/lib/kubelet/pods/6c8e3ad6-74dd-43b0-b674-9995ef3c14ca/volumes"
Jul 24 23:37:57.784632 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:57.784582    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:37:57.885587 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:57.885547    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:37:58.332304 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:58.332268    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:37:58.400675 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:58.400644    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-587494447-w87vs"]
Jul 24 23:37:58.401022 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:58.401008    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c8e3ad6-74dd-43b0-b674-9995ef3c14ca" containerName="manager"
Jul 24 23:37:58.401022 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:58.401022    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c8e3ad6-74dd-43b0-b674-9995ef3c14ca" containerName="manager"
Jul 24 23:37:58.401143 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:58.401087    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c8e3ad6-74dd-43b0-b674-9995ef3c14ca" containerName="manager"
Jul 24 23:37:58.409496 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:58.409471    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-587494447-w87vs"
Jul 24 23:37:58.411932 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:58.411900    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 24 23:37:58.412076 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:58.411949    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-m8p8g\""
Jul 24 23:37:58.415868 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:58.415841    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-587494447-w87vs"]
Jul 24 23:37:58.536579 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:58.536521    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8d30495b-e008-4ea6-951f-d1c97c6fbadf-maas-api-tls\") pod \"maas-api-587494447-w87vs\" (UID: \"8d30495b-e008-4ea6-951f-d1c97c6fbadf\") " pod="odh-ai-gateway-infra/maas-api-587494447-w87vs"
Jul 24 23:37:58.536752 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:58.536592    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d30495b-e008-4ea6-951f-d1c97c6fbadf-openshift-service-ca\") pod \"maas-api-587494447-w87vs\" (UID: \"8d30495b-e008-4ea6-951f-d1c97c6fbadf\") " pod="odh-ai-gateway-infra/maas-api-587494447-w87vs"
Jul 24 23:37:58.536752 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:58.536616    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sc88b\" (UniqueName: \"kubernetes.io/projected/8d30495b-e008-4ea6-951f-d1c97c6fbadf-kube-api-access-sc88b\") pod \"maas-api-587494447-w87vs\" (UID: \"8d30495b-e008-4ea6-951f-d1c97c6fbadf\") " pod="odh-ai-gateway-infra/maas-api-587494447-w87vs"
Jul 24 23:37:58.637063 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:58.636967    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d30495b-e008-4ea6-951f-d1c97c6fbadf-openshift-service-ca\") pod \"maas-api-587494447-w87vs\" (UID: \"8d30495b-e008-4ea6-951f-d1c97c6fbadf\") " pod="odh-ai-gateway-infra/maas-api-587494447-w87vs"
Jul 24 23:37:58.637063 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:58.637006    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sc88b\" (UniqueName: \"kubernetes.io/projected/8d30495b-e008-4ea6-951f-d1c97c6fbadf-kube-api-access-sc88b\") pod \"maas-api-587494447-w87vs\" (UID: \"8d30495b-e008-4ea6-951f-d1c97c6fbadf\") " pod="odh-ai-gateway-infra/maas-api-587494447-w87vs"
Jul 24 23:37:58.637063 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:58.637053    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8d30495b-e008-4ea6-951f-d1c97c6fbadf-maas-api-tls\") pod \"maas-api-587494447-w87vs\" (UID: \"8d30495b-e008-4ea6-951f-d1c97c6fbadf\") " pod="odh-ai-gateway-infra/maas-api-587494447-w87vs"
Jul 24 23:37:58.637350 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:37:58.637150    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found
Jul 24 23:37:58.637350 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:37:58.637212    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d30495b-e008-4ea6-951f-d1c97c6fbadf-maas-api-tls podName:8d30495b-e008-4ea6-951f-d1c97c6fbadf nodeName:}" failed. No retries permitted until 2026-07-24 23:37:59.137195134 +0000 UTC m=+519.523625840 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8d30495b-e008-4ea6-951f-d1c97c6fbadf-maas-api-tls") pod "maas-api-587494447-w87vs" (UID: "8d30495b-e008-4ea6-951f-d1c97c6fbadf") : secret "maas-api-serving-cert" not found
Jul 24 23:37:58.637617 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:58.637599    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d30495b-e008-4ea6-951f-d1c97c6fbadf-openshift-service-ca\") pod \"maas-api-587494447-w87vs\" (UID: \"8d30495b-e008-4ea6-951f-d1c97c6fbadf\") " pod="odh-ai-gateway-infra/maas-api-587494447-w87vs"
Jul 24 23:37:58.646045 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:58.646019    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sc88b\" (UniqueName: \"kubernetes.io/projected/8d30495b-e008-4ea6-951f-d1c97c6fbadf-kube-api-access-sc88b\") pod \"maas-api-587494447-w87vs\" (UID: \"8d30495b-e008-4ea6-951f-d1c97c6fbadf\") " pod="odh-ai-gateway-infra/maas-api-587494447-w87vs"
Jul 24 23:37:58.708926 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:58.708890    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:37:58.947764 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:58.947672    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:37:59.042341 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:59.042302    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:37:59.076763 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:59.076724    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-8ddmv"]
Jul 24 23:37:59.081150 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:59.081125    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-8ddmv"
Jul 24 23:37:59.083848 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:59.083821    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 24 23:37:59.083972 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:59.083871    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-tjx4m\""
Jul 24 23:37:59.089063 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:59.088929    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-8ddmv"]
Jul 24 23:37:59.142277 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:59.142246    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpkdv\" (UniqueName: \"kubernetes.io/projected/180a4542-a43a-4d03-bd07-9380a02de282-kube-api-access-cpkdv\") pod \"payload-processing-5cc6cd444b-8ddmv\" (UID: \"180a4542-a43a-4d03-bd07-9380a02de282\") " pod="openshift-ingress/payload-processing-5cc6cd444b-8ddmv"
Jul 24 23:37:59.142438 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:59.142287    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/180a4542-a43a-4d03-bd07-9380a02de282-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-8ddmv\" (UID: \"180a4542-a43a-4d03-bd07-9380a02de282\") " pod="openshift-ingress/payload-processing-5cc6cd444b-8ddmv"
Jul 24 23:37:59.142438 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:59.142364    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8d30495b-e008-4ea6-951f-d1c97c6fbadf-maas-api-tls\") pod \"maas-api-587494447-w87vs\" (UID: \"8d30495b-e008-4ea6-951f-d1c97c6fbadf\") " pod="odh-ai-gateway-infra/maas-api-587494447-w87vs"
Jul 24 23:37:59.144910 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:59.144881    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8d30495b-e008-4ea6-951f-d1c97c6fbadf-maas-api-tls\") pod \"maas-api-587494447-w87vs\" (UID: \"8d30495b-e008-4ea6-951f-d1c97c6fbadf\") " pod="odh-ai-gateway-infra/maas-api-587494447-w87vs"
Jul 24 23:37:59.243404 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:59.243365    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpkdv\" (UniqueName: \"kubernetes.io/projected/180a4542-a43a-4d03-bd07-9380a02de282-kube-api-access-cpkdv\") pod \"payload-processing-5cc6cd444b-8ddmv\" (UID: \"180a4542-a43a-4d03-bd07-9380a02de282\") " pod="openshift-ingress/payload-processing-5cc6cd444b-8ddmv"
Jul 24 23:37:59.243622 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:59.243427    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/180a4542-a43a-4d03-bd07-9380a02de282-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-8ddmv\" (UID: \"180a4542-a43a-4d03-bd07-9380a02de282\") " pod="openshift-ingress/payload-processing-5cc6cd444b-8ddmv"
Jul 24 23:37:59.244190 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:59.244162    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/180a4542-a43a-4d03-bd07-9380a02de282-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-8ddmv\" (UID: \"180a4542-a43a-4d03-bd07-9380a02de282\") " pod="openshift-ingress/payload-processing-5cc6cd444b-8ddmv"
Jul 24 23:37:59.251355 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:59.251332    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpkdv\" (UniqueName: \"kubernetes.io/projected/180a4542-a43a-4d03-bd07-9380a02de282-kube-api-access-cpkdv\") pod \"payload-processing-5cc6cd444b-8ddmv\" (UID: \"180a4542-a43a-4d03-bd07-9380a02de282\") " pod="openshift-ingress/payload-processing-5cc6cd444b-8ddmv"
Jul 24 23:37:59.322715 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:59.322683    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-587494447-w87vs"
Jul 24 23:37:59.393353 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:59.393319    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-8ddmv"
Jul 24 23:37:59.466270 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:59.466111    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-587494447-w87vs"]
Jul 24 23:37:59.469441 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:37:59.469414    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d30495b_e008_4ea6_951f_d1c97c6fbadf.slice/crio-3cce69bf8285eb1ba3d003a178887a0e803dfe84f9809036cf9eb08e985d6948 WatchSource:0}: Error finding container 3cce69bf8285eb1ba3d003a178887a0e803dfe84f9809036cf9eb08e985d6948: Status 404 returned error can't find the container with id 3cce69bf8285eb1ba3d003a178887a0e803dfe84f9809036cf9eb08e985d6948
Jul 24 23:37:59.537145 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:59.537114    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-8ddmv"]
Jul 24 23:37:59.538183 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:37:59.538159    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod180a4542_a43a_4d03_bd07_9380a02de282.slice/crio-dbfb70b96018b54d956afd5d2017f56718eae1a39b24366035c556dac4702308 WatchSource:0}: Error finding container dbfb70b96018b54d956afd5d2017f56718eae1a39b24366035c556dac4702308: Status 404 returned error can't find the container with id dbfb70b96018b54d956afd5d2017f56718eae1a39b24366035c556dac4702308
Jul 24 23:37:59.618638 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:59.618600    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-587494447-w87vs" event={"ID":"8d30495b-e008-4ea6-951f-d1c97c6fbadf","Type":"ContainerStarted","Data":"3cce69bf8285eb1ba3d003a178887a0e803dfe84f9809036cf9eb08e985d6948"}
Jul 24 23:37:59.619819 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:37:59.619791    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-8ddmv" event={"ID":"180a4542-a43a-4d03-bd07-9380a02de282","Type":"ContainerStarted","Data":"dbfb70b96018b54d956afd5d2017f56718eae1a39b24366035c556dac4702308"}
Jul 24 23:38:02.174553 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:38:02.174495    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:38:02.227908 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:38:02.227876    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:38:02.639826 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:38:02.639781    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-8ddmv" event={"ID":"180a4542-a43a-4d03-bd07-9380a02de282","Type":"ContainerStarted","Data":"4dbd6184d86b61fe41f073898ad73a9485f9ffdcdd3ee412403280aee714ef3d"}
Jul 24 23:38:02.640034 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:38:02.639853    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5cc6cd444b-8ddmv"
Jul 24 23:38:02.641506 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:38:02.641482    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-587494447-w87vs" event={"ID":"8d30495b-e008-4ea6-951f-d1c97c6fbadf","Type":"ContainerStarted","Data":"060d3a0e0e67fab09eb33fdc62e909beb8615de8dbe09db57c190f31335c9bdb"}
Jul 24 23:38:02.641676 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:38:02.641562    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-587494447-w87vs"
Jul 24 23:38:02.657493 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:38:02.657444    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5cc6cd444b-8ddmv" podStartSLOduration=1.593599518 podStartE2EDuration="3.657428767s" totalImagesPullingTime="2.063829249s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:37:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:38:02.656116976 +0000 UTC m=+523.042547707" watchObservedRunningTime="2026-07-24 23:38:02.657428767 +0000 UTC m=+523.043859495"
Jul 24 23:38:02.672902 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:38:02.672849    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-587494447-w87vs" podStartSLOduration=2.539949334 podStartE2EDuration="4.672833318s" totalImagesPullingTime="2.132883984s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:37:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:38:02.671879971 +0000 UTC m=+523.058310724" watchObservedRunningTime="2026-07-24 23:38:02.672833318 +0000 UTC m=+523.059264045"
Jul 24 23:38:08.652679 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:38:08.652651    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-587494447-w87vs"
Jul 24 23:38:09.666237 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:38:09.666204    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:38:09.766198 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:38:09.766161    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:38:13.649350 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:38:13.649269    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5cc6cd444b-8ddmv"
Jul 24 23:38:15.305300 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:38:15.305259    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:38:23.989370 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:38:23.989333    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-qtt6g"]
Jul 24 23:38:23.994418 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:38:23.994392    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qtt6g"
Jul 24 23:38:23.996798 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:38:23.996765    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 24 23:38:23.997844 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:38:23.997823    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 24 23:38:23.997903 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:38:23.997855    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-n27r8\""
Jul 24 23:38:23.997960 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:38:23.997947    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 24 23:38:24.002978 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:38:24.002956    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-qtt6g"]
Jul 24 23:38:24.017882 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:38:24.017842    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:38:24.058702 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:38:24.058670    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ebada3cb-11ae-457a-a140-b40306cbc4bf-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qtt6g\" (UID: \"ebada3cb-11ae-457a-a140-b40306cbc4bf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qtt6g"
Jul 24 23:38:24.058854 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:38:24.058708    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8gq2\" (UniqueName: \"kubernetes.io/projected/ebada3cb-11ae-457a-a140-b40306cbc4bf-kube-api-access-v8gq2\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qtt6g\" (UID: \"ebada3cb-11ae-457a-a140-b40306cbc4bf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qtt6g"
Jul 24 23:38:24.058854 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:38:24.058731    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ebada3cb-11ae-457a-a140-b40306cbc4bf-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qtt6g\" (UID: \"ebada3cb-11ae-457a-a140-b40306cbc4bf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qtt6g"
Jul 24 23:38:24.058854 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:38:24.058762    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ebada3cb-11ae-457a-a140-b40306cbc4bf-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qtt6g\" (UID: \"ebada3cb-11ae-457a-a140-b40306cbc4bf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qtt6g"
Jul 24 23:38:24.058854 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:38:24.058832    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ebada3cb-11ae-457a-a140-b40306cbc4bf-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qtt6g\" (UID: \"ebada3cb-11ae-457a-a140-b40306cbc4bf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qtt6g"
Jul 24 23:38:24.059004 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:38:24.058859    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ebada3cb-11ae-457a-a140-b40306cbc4bf-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qtt6g\" (UID: \"ebada3cb-11ae-457a-a140-b40306cbc4bf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qtt6g"
Jul 24 23:38:24.160108 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:38:24.160064    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ebada3cb-11ae-457a-a140-b40306cbc4bf-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qtt6g\" (UID: \"ebada3cb-11ae-457a-a140-b40306cbc4bf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qtt6g"
Jul 24 23:38:24.160108 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:38:24.160108    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ebada3cb-11ae-457a-a140-b40306cbc4bf-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qtt6g\" (UID: \"ebada3cb-11ae-457a-a140-b40306cbc4bf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qtt6g"
Jul 24 23:38:24.160355 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:38:24.160221    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ebada3cb-11ae-457a-a140-b40306cbc4bf-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qtt6g\" (UID: \"ebada3cb-11ae-457a-a140-b40306cbc4bf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qtt6g"
Jul 24 23:38:24.160355 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:38:24.160253    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8gq2\" (UniqueName: \"kubernetes.io/projected/ebada3cb-11ae-457a-a140-b40306cbc4bf-kube-api-access-v8gq2\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qtt6g\" (UID: \"ebada3cb-11ae-457a-a140-b40306cbc4bf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qtt6g"
Jul 24 23:38:24.160355 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:38:24.160289    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ebada3cb-11ae-457a-a140-b40306cbc4bf-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qtt6g\" (UID: \"ebada3cb-11ae-457a-a140-b40306cbc4bf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qtt6g"
Jul 24 23:38:24.160355 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:38:24.160316    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ebada3cb-11ae-457a-a140-b40306cbc4bf-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qtt6g\" (UID: \"ebada3cb-11ae-457a-a140-b40306cbc4bf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qtt6g"
Jul 24 23:38:24.160914 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:38:24.160565    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ebada3cb-11ae-457a-a140-b40306cbc4bf-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qtt6g\" (UID: \"ebada3cb-11ae-457a-a140-b40306cbc4bf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qtt6g"
Jul 24 23:38:24.160914 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:38:24.160661    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ebada3cb-11ae-457a-a140-b40306cbc4bf-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qtt6g\" (UID: \"ebada3cb-11ae-457a-a140-b40306cbc4bf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qtt6g"
Jul 24 23:38:24.160914 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:38:24.160891    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ebada3cb-11ae-457a-a140-b40306cbc4bf-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qtt6g\" (UID: \"ebada3cb-11ae-457a-a140-b40306cbc4bf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qtt6g"
Jul 24 23:38:24.162625 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:38:24.162601    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ebada3cb-11ae-457a-a140-b40306cbc4bf-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qtt6g\" (UID: \"ebada3cb-11ae-457a-a140-b40306cbc4bf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qtt6g"
Jul 24 23:38:24.163234 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:38:24.163214    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ebada3cb-11ae-457a-a140-b40306cbc4bf-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qtt6g\" (UID: \"ebada3cb-11ae-457a-a140-b40306cbc4bf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qtt6g"
Jul 24 23:38:24.169157 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:38:24.169131    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8gq2\" (UniqueName: \"kubernetes.io/projected/ebada3cb-11ae-457a-a140-b40306cbc4bf-kube-api-access-v8gq2\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qtt6g\" (UID: \"ebada3cb-11ae-457a-a140-b40306cbc4bf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qtt6g"
Jul 24 23:38:24.307456 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:38:24.307346    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qtt6g"
Jul 24 23:38:24.438652 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:38:24.438624    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-qtt6g"]
Jul 24 23:38:24.440419 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:38:24.440391    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebada3cb_11ae_457a_a140_b40306cbc4bf.slice/crio-467fea60240c3eca06e32bc4265c8a0140f8a037192719d59b84c3ecc9ef0f63 WatchSource:0}: Error finding container 467fea60240c3eca06e32bc4265c8a0140f8a037192719d59b84c3ecc9ef0f63: Status 404 returned error can't find the container with id 467fea60240c3eca06e32bc4265c8a0140f8a037192719d59b84c3ecc9ef0f63
Jul 24 23:38:24.759243 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:38:24.759204    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qtt6g" event={"ID":"ebada3cb-11ae-457a-a140-b40306cbc4bf","Type":"ContainerStarted","Data":"467fea60240c3eca06e32bc4265c8a0140f8a037192719d59b84c3ecc9ef0f63"}
Jul 24 23:38:25.765456 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:38:25.765423    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qtt6g" event={"ID":"ebada3cb-11ae-457a-a140-b40306cbc4bf","Type":"ContainerStarted","Data":"de30044dbad012c01a07a68c49eefe3587b71b632fa81e93f1f7b6ef6cb95bb4"}
Jul 24 23:38:25.784332 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:38:25.784278    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qtt6g" podStartSLOduration=1.845528834 podStartE2EDuration="2.784261461s" totalImagesPullingTime="938.732627ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:38:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:38:25.781763727 +0000 UTC m=+546.168194461" watchObservedRunningTime="2026-07-24 23:38:25.784261461 +0000 UTC m=+546.170692188"
Jul 24 23:38:34.307936 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:38:34.307879    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qtt6g"
Jul 24 23:38:34.308411 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:38:34.307968    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qtt6g"
Jul 24 23:38:34.320755 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:38:34.320725    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qtt6g"
Jul 24 23:38:34.820319 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:38:34.820290    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qtt6g"
Jul 24 23:38:47.509988 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:38:47.509949    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:38:59.894665 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:38:59.894625    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbpwhk"]
Jul 24 23:38:59.899072 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:38:59.899054    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbpwhk"
Jul 24 23:38:59.901386 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:38:59.901360    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 24 23:38:59.905547 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:38:59.905510    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbpwhk"]
Jul 24 23:38:59.919930 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:38:59.919898    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:39:00.082689 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:39:00.082640    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f5d3e73-8c38-4b2f-acd4-c91e986baaa6-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbpwhk\" (UID: \"7f5d3e73-8c38-4b2f-acd4-c91e986baaa6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbpwhk"
Jul 24 23:39:00.083069 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:39:00.083022    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8hvc\" (UniqueName: \"kubernetes.io/projected/7f5d3e73-8c38-4b2f-acd4-c91e986baaa6-kube-api-access-g8hvc\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbpwhk\" (UID: \"7f5d3e73-8c38-4b2f-acd4-c91e986baaa6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbpwhk"
Jul 24 23:39:00.083429 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:39:00.083407    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7f5d3e73-8c38-4b2f-acd4-c91e986baaa6-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbpwhk\" (UID: \"7f5d3e73-8c38-4b2f-acd4-c91e986baaa6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbpwhk"
Jul 24 23:39:00.083600 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:39:00.083583    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f5d3e73-8c38-4b2f-acd4-c91e986baaa6-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbpwhk\" (UID: \"7f5d3e73-8c38-4b2f-acd4-c91e986baaa6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbpwhk"
Jul 24 23:39:00.083773 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:39:00.083757    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f5d3e73-8c38-4b2f-acd4-c91e986baaa6-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbpwhk\" (UID: \"7f5d3e73-8c38-4b2f-acd4-c91e986baaa6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbpwhk"
Jul 24 23:39:00.083907 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:39:00.083895    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f5d3e73-8c38-4b2f-acd4-c91e986baaa6-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbpwhk\" (UID: \"7f5d3e73-8c38-4b2f-acd4-c91e986baaa6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbpwhk"
Jul 24 23:39:00.185012 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:39:00.184908    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f5d3e73-8c38-4b2f-acd4-c91e986baaa6-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbpwhk\" (UID: \"7f5d3e73-8c38-4b2f-acd4-c91e986baaa6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbpwhk"
Jul 24 23:39:00.185012 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:39:00.184972    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8hvc\" (UniqueName: \"kubernetes.io/projected/7f5d3e73-8c38-4b2f-acd4-c91e986baaa6-kube-api-access-g8hvc\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbpwhk\" (UID: \"7f5d3e73-8c38-4b2f-acd4-c91e986baaa6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbpwhk"
Jul 24 23:39:00.185012 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:39:00.185004    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7f5d3e73-8c38-4b2f-acd4-c91e986baaa6-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbpwhk\" (UID: \"7f5d3e73-8c38-4b2f-acd4-c91e986baaa6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbpwhk"
Jul 24 23:39:00.185298 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:39:00.185024    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f5d3e73-8c38-4b2f-acd4-c91e986baaa6-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbpwhk\" (UID: \"7f5d3e73-8c38-4b2f-acd4-c91e986baaa6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbpwhk"
Jul 24 23:39:00.185298 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:39:00.185046    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f5d3e73-8c38-4b2f-acd4-c91e986baaa6-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbpwhk\" (UID: \"7f5d3e73-8c38-4b2f-acd4-c91e986baaa6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbpwhk"
Jul 24 23:39:00.185298 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:39:00.185069    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f5d3e73-8c38-4b2f-acd4-c91e986baaa6-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbpwhk\" (UID: \"7f5d3e73-8c38-4b2f-acd4-c91e986baaa6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbpwhk"
Jul 24 23:39:00.185435 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:39:00.185399    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7f5d3e73-8c38-4b2f-acd4-c91e986baaa6-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbpwhk\" (UID: \"7f5d3e73-8c38-4b2f-acd4-c91e986baaa6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbpwhk"
Jul 24 23:39:00.185507 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:39:00.185480    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f5d3e73-8c38-4b2f-acd4-c91e986baaa6-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbpwhk\" (UID: \"7f5d3e73-8c38-4b2f-acd4-c91e986baaa6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbpwhk"
Jul 24 23:39:00.185600 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:39:00.185583    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f5d3e73-8c38-4b2f-acd4-c91e986baaa6-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbpwhk\" (UID: \"7f5d3e73-8c38-4b2f-acd4-c91e986baaa6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbpwhk"
Jul 24 23:39:00.187348 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:39:00.187326    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f5d3e73-8c38-4b2f-acd4-c91e986baaa6-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbpwhk\" (UID: \"7f5d3e73-8c38-4b2f-acd4-c91e986baaa6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbpwhk"
Jul 24 23:39:00.187517 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:39:00.187497    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f5d3e73-8c38-4b2f-acd4-c91e986baaa6-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbpwhk\" (UID: \"7f5d3e73-8c38-4b2f-acd4-c91e986baaa6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbpwhk"
Jul 24 23:39:00.192756 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:39:00.192728    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8hvc\" (UniqueName: \"kubernetes.io/projected/7f5d3e73-8c38-4b2f-acd4-c91e986baaa6-kube-api-access-g8hvc\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbpwhk\" (UID: \"7f5d3e73-8c38-4b2f-acd4-c91e986baaa6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbpwhk"
Jul 24 23:39:00.210366 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:39:00.210340    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbpwhk"
Jul 24 23:39:00.345927 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:39:00.345901    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbpwhk"]
Jul 24 23:39:00.347854 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:39:00.347825    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f5d3e73_8c38_4b2f_acd4_c91e986baaa6.slice/crio-cbd850a33719176d110539343606a446ccb25d0dccaeaea89a25158d33e497fc WatchSource:0}: Error finding container cbd850a33719176d110539343606a446ccb25d0dccaeaea89a25158d33e497fc: Status 404 returned error can't find the container with id cbd850a33719176d110539343606a446ccb25d0dccaeaea89a25158d33e497fc
Jul 24 23:39:00.947725 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:39:00.947634    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbpwhk" event={"ID":"7f5d3e73-8c38-4b2f-acd4-c91e986baaa6","Type":"ContainerStarted","Data":"198cdac4033cc78227c467b3ca42215ca16b82c9846893a765e3a7147e514f8b"}
Jul 24 23:39:00.947725 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:39:00.947689    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbpwhk" event={"ID":"7f5d3e73-8c38-4b2f-acd4-c91e986baaa6","Type":"ContainerStarted","Data":"cbd850a33719176d110539343606a446ccb25d0dccaeaea89a25158d33e497fc"}
Jul 24 23:39:00.966128 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:39:00.966078    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbpwhk" podStartSLOduration=1.762114057 podStartE2EDuration="1.966061428s" totalImagesPullingTime="203.947371ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:38:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:39:00.963921179 +0000 UTC m=+581.350351934" watchObservedRunningTime="2026-07-24 23:39:00.966061428 +0000 UTC m=+581.352492157"
Jul 24 23:39:10.211326 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:39:10.211290    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbpwhk"
Jul 24 23:39:10.211811 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:39:10.211351    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbpwhk"
Jul 24 23:39:10.223949 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:39:10.223924    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbpwhk"
Jul 24 23:39:11.009886 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:39:11.009857    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbpwhk"
Jul 24 23:39:15.885691 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:39:15.885654    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:39:27.803101 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:39:27.803061    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:39:27.854948 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:39:27.854909    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:39:28.507299 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:39:28.507257    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:40:21.960168 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:40:21.960125    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:40:38.930291 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:40:38.930252    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:40:51.836023 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:40:51.835982    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:41:01.790476 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:41:01.790439    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:41:01.894149 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:41:01.894114    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:41:02.977904 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:41:02.977865    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:41:03.078761 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:41:03.078718    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:41:49.561509 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:41:49.561462    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-59bb55c686-gnnsj"]
Jul 24 23:41:49.562140 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:41:49.561727    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-59bb55c686-gnnsj" podUID="4df5609d-267e-4e54-bda3-ce7f3435cd2c" containerName="manager" containerID="cri-o://8af74588e0b337aea9ab901439208157f56c93bf4a8128c77875655ef0cf5226" gracePeriod=30
Jul 24 23:41:49.791682 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:41:49.791649    2558 generic.go:356] "Generic (PLEG): container finished" podID="4df5609d-267e-4e54-bda3-ce7f3435cd2c" containerID="8af74588e0b337aea9ab901439208157f56c93bf4a8128c77875655ef0cf5226" exitCode=0
Jul 24 23:41:49.791805 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:41:49.791716    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-59bb55c686-gnnsj" event={"ID":"4df5609d-267e-4e54-bda3-ce7f3435cd2c","Type":"ContainerDied","Data":"8af74588e0b337aea9ab901439208157f56c93bf4a8128c77875655ef0cf5226"}
Jul 24 23:41:49.809994 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:41:49.809959    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-59bb55c686-gnnsj"
Jul 24 23:41:49.911555 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:41:49.911439    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4df5609d-267e-4e54-bda3-ce7f3435cd2c-webhook-cert\") pod \"4df5609d-267e-4e54-bda3-ce7f3435cd2c\" (UID: \"4df5609d-267e-4e54-bda3-ce7f3435cd2c\") "
Jul 24 23:41:49.911705 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:41:49.911568    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2jqx2\" (UniqueName: \"kubernetes.io/projected/4df5609d-267e-4e54-bda3-ce7f3435cd2c-kube-api-access-2jqx2\") pod \"4df5609d-267e-4e54-bda3-ce7f3435cd2c\" (UID: \"4df5609d-267e-4e54-bda3-ce7f3435cd2c\") "
Jul 24 23:41:49.913563 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:41:49.913518    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4df5609d-267e-4e54-bda3-ce7f3435cd2c-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "4df5609d-267e-4e54-bda3-ce7f3435cd2c" (UID: "4df5609d-267e-4e54-bda3-ce7f3435cd2c"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 23:41:49.913644 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:41:49.913617    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4df5609d-267e-4e54-bda3-ce7f3435cd2c-kube-api-access-2jqx2" (OuterVolumeSpecName: "kube-api-access-2jqx2") pod "4df5609d-267e-4e54-bda3-ce7f3435cd2c" (UID: "4df5609d-267e-4e54-bda3-ce7f3435cd2c"). InnerVolumeSpecName "kube-api-access-2jqx2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:41:50.012803 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:41:50.012761    2558 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4df5609d-267e-4e54-bda3-ce7f3435cd2c-webhook-cert\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 24 23:41:50.012803 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:41:50.012793    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2jqx2\" (UniqueName: \"kubernetes.io/projected/4df5609d-267e-4e54-bda3-ce7f3435cd2c-kube-api-access-2jqx2\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 24 23:41:50.797464 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:41:50.797435    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-59bb55c686-gnnsj"
Jul 24 23:41:50.797900 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:41:50.797483    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-59bb55c686-gnnsj" event={"ID":"4df5609d-267e-4e54-bda3-ce7f3435cd2c","Type":"ContainerDied","Data":"e1f9a1a246232a13941db25c5e114a5e6ed2dec7ba4c25136108eba3eb0dd199"}
Jul 24 23:41:50.797900 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:41:50.797562    2558 scope.go:117] "RemoveContainer" containerID="8af74588e0b337aea9ab901439208157f56c93bf4a8128c77875655ef0cf5226"
Jul 24 23:41:50.813649 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:41:50.813618    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-59bb55c686-gnnsj"]
Jul 24 23:41:50.816906 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:41:50.816881    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-59bb55c686-gnnsj"]
Jul 24 23:41:52.204467 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:41:52.204429    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4df5609d-267e-4e54-bda3-ce7f3435cd2c" path="/var/lib/kubelet/pods/4df5609d-267e-4e54-bda3-ce7f3435cd2c/volumes"
Jul 24 23:42:05.537340 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:42:05.537296    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:42:51.651657 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:42:51.651569    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-59bb55c686-jbsq7"]
Jul 24 23:42:51.652123 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:42:51.651952    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4df5609d-267e-4e54-bda3-ce7f3435cd2c" containerName="manager"
Jul 24 23:42:51.652123 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:42:51.651964    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="4df5609d-267e-4e54-bda3-ce7f3435cd2c" containerName="manager"
Jul 24 23:42:51.652123 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:42:51.652024    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="4df5609d-267e-4e54-bda3-ce7f3435cd2c" containerName="manager"
Jul 24 23:42:51.654929 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:42:51.654909    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-59bb55c686-jbsq7"
Jul 24 23:42:51.658164 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:42:51.658128    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-l7fhg\""
Jul 24 23:42:51.658164 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:42:51.658145    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 24 23:42:51.658385 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:42:51.658145    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 24 23:42:51.663598 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:42:51.663568    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-59bb55c686-jbsq7"]
Jul 24 23:42:51.749784 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:42:51.749746    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/93768090-a58e-457d-91f7-caeb814c088f-webhook-cert\") pod \"maas-controller-59bb55c686-jbsq7\" (UID: \"93768090-a58e-457d-91f7-caeb814c088f\") " pod="opendatahub/maas-controller-59bb55c686-jbsq7"
Jul 24 23:42:51.749784 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:42:51.749786    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k22fr\" (UniqueName: \"kubernetes.io/projected/93768090-a58e-457d-91f7-caeb814c088f-kube-api-access-k22fr\") pod \"maas-controller-59bb55c686-jbsq7\" (UID: \"93768090-a58e-457d-91f7-caeb814c088f\") " pod="opendatahub/maas-controller-59bb55c686-jbsq7"
Jul 24 23:42:51.850709 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:42:51.850673    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/93768090-a58e-457d-91f7-caeb814c088f-webhook-cert\") pod \"maas-controller-59bb55c686-jbsq7\" (UID: \"93768090-a58e-457d-91f7-caeb814c088f\") " pod="opendatahub/maas-controller-59bb55c686-jbsq7"
Jul 24 23:42:51.850709 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:42:51.850708    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k22fr\" (UniqueName: \"kubernetes.io/projected/93768090-a58e-457d-91f7-caeb814c088f-kube-api-access-k22fr\") pod \"maas-controller-59bb55c686-jbsq7\" (UID: \"93768090-a58e-457d-91f7-caeb814c088f\") " pod="opendatahub/maas-controller-59bb55c686-jbsq7"
Jul 24 23:42:51.853148 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:42:51.853123    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/93768090-a58e-457d-91f7-caeb814c088f-webhook-cert\") pod \"maas-controller-59bb55c686-jbsq7\" (UID: \"93768090-a58e-457d-91f7-caeb814c088f\") " pod="opendatahub/maas-controller-59bb55c686-jbsq7"
Jul 24 23:42:51.858177 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:42:51.858155    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k22fr\" (UniqueName: \"kubernetes.io/projected/93768090-a58e-457d-91f7-caeb814c088f-kube-api-access-k22fr\") pod \"maas-controller-59bb55c686-jbsq7\" (UID: \"93768090-a58e-457d-91f7-caeb814c088f\") " pod="opendatahub/maas-controller-59bb55c686-jbsq7"
Jul 24 23:42:51.966686 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:42:51.966590    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-59bb55c686-jbsq7"
Jul 24 23:42:52.101754 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:42:52.101728    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-59bb55c686-jbsq7"]
Jul 24 23:42:52.103834 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:42:52.103808    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93768090_a58e_457d_91f7_caeb814c088f.slice/crio-5664e2cacb9e7b3b2c6ce51788380ff07a86d2d6d134bd93de299b31050b301e WatchSource:0}: Error finding container 5664e2cacb9e7b3b2c6ce51788380ff07a86d2d6d134bd93de299b31050b301e: Status 404 returned error can't find the container with id 5664e2cacb9e7b3b2c6ce51788380ff07a86d2d6d134bd93de299b31050b301e
Jul 24 23:42:52.105737 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:42:52.105718    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 23:42:52.122689 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:42:52.122658    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-59bb55c686-jbsq7" event={"ID":"93768090-a58e-457d-91f7-caeb814c088f","Type":"ContainerStarted","Data":"5664e2cacb9e7b3b2c6ce51788380ff07a86d2d6d134bd93de299b31050b301e"}
Jul 24 23:42:53.129044 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:42:53.129000    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-59bb55c686-jbsq7" event={"ID":"93768090-a58e-457d-91f7-caeb814c088f","Type":"ContainerStarted","Data":"4f9a6b74459c83575dc746d21e9107aa633df620e1acc8fa4694a1cad5205445"}
Jul 24 23:42:53.129551 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:42:53.129079    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-59bb55c686-jbsq7"
Jul 24 23:42:53.145687 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:42:53.145638    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-59bb55c686-jbsq7" podStartSLOduration=1.643294079 podStartE2EDuration="2.145620247s" totalImagesPullingTime="502.326168ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:42:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:42:53.144205831 +0000 UTC m=+813.530636560" watchObservedRunningTime="2026-07-24 23:42:53.145620247 +0000 UTC m=+813.532050977"
Jul 24 23:43:04.136258 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:43:04.136222    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-59bb55c686-jbsq7"
Jul 24 23:43:08.691661 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:43:08.691620    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:43:08.800641 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:43:08.800603    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-59bb55c686-jbsq7"]
Jul 24 23:43:08.800850 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:43:08.800825    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-59bb55c686-jbsq7" podUID="93768090-a58e-457d-91f7-caeb814c088f" containerName="manager" containerID="cri-o://4f9a6b74459c83575dc746d21e9107aa633df620e1acc8fa4694a1cad5205445" gracePeriod=30
Jul 24 23:43:09.150070 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:43:09.150043    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-59bb55c686-jbsq7"
Jul 24 23:43:09.202330 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:43:09.202299    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/93768090-a58e-457d-91f7-caeb814c088f-webhook-cert\") pod \"93768090-a58e-457d-91f7-caeb814c088f\" (UID: \"93768090-a58e-457d-91f7-caeb814c088f\") "
Jul 24 23:43:09.202505 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:43:09.202411    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k22fr\" (UniqueName: \"kubernetes.io/projected/93768090-a58e-457d-91f7-caeb814c088f-kube-api-access-k22fr\") pod \"93768090-a58e-457d-91f7-caeb814c088f\" (UID: \"93768090-a58e-457d-91f7-caeb814c088f\") "
Jul 24 23:43:09.204505 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:43:09.204430    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93768090-a58e-457d-91f7-caeb814c088f-kube-api-access-k22fr" (OuterVolumeSpecName: "kube-api-access-k22fr") pod "93768090-a58e-457d-91f7-caeb814c088f" (UID: "93768090-a58e-457d-91f7-caeb814c088f"). InnerVolumeSpecName "kube-api-access-k22fr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:43:09.204505 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:43:09.204471    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93768090-a58e-457d-91f7-caeb814c088f-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "93768090-a58e-457d-91f7-caeb814c088f" (UID: "93768090-a58e-457d-91f7-caeb814c088f"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 23:43:09.216345 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:43:09.216313    2558 generic.go:356] "Generic (PLEG): container finished" podID="93768090-a58e-457d-91f7-caeb814c088f" containerID="4f9a6b74459c83575dc746d21e9107aa633df620e1acc8fa4694a1cad5205445" exitCode=0
Jul 24 23:43:09.216477 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:43:09.216363    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-59bb55c686-jbsq7"
Jul 24 23:43:09.216477 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:43:09.216364    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-59bb55c686-jbsq7" event={"ID":"93768090-a58e-457d-91f7-caeb814c088f","Type":"ContainerDied","Data":"4f9a6b74459c83575dc746d21e9107aa633df620e1acc8fa4694a1cad5205445"}
Jul 24 23:43:09.216586 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:43:09.216473    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-59bb55c686-jbsq7" event={"ID":"93768090-a58e-457d-91f7-caeb814c088f","Type":"ContainerDied","Data":"5664e2cacb9e7b3b2c6ce51788380ff07a86d2d6d134bd93de299b31050b301e"}
Jul 24 23:43:09.216586 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:43:09.216501    2558 scope.go:117] "RemoveContainer" containerID="4f9a6b74459c83575dc746d21e9107aa633df620e1acc8fa4694a1cad5205445"
Jul 24 23:43:09.225435 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:43:09.225409    2558 scope.go:117] "RemoveContainer" containerID="4f9a6b74459c83575dc746d21e9107aa633df620e1acc8fa4694a1cad5205445"
Jul 24 23:43:09.225753 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:43:09.225733    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f9a6b74459c83575dc746d21e9107aa633df620e1acc8fa4694a1cad5205445\": container with ID starting with 4f9a6b74459c83575dc746d21e9107aa633df620e1acc8fa4694a1cad5205445 not found: ID does not exist" containerID="4f9a6b74459c83575dc746d21e9107aa633df620e1acc8fa4694a1cad5205445"
Jul 24 23:43:09.225835 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:43:09.225767    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f9a6b74459c83575dc746d21e9107aa633df620e1acc8fa4694a1cad5205445"} err="failed to get container status \"4f9a6b74459c83575dc746d21e9107aa633df620e1acc8fa4694a1cad5205445\": rpc error: code = NotFound desc = could not find container \"4f9a6b74459c83575dc746d21e9107aa633df620e1acc8fa4694a1cad5205445\": container with ID starting with 4f9a6b74459c83575dc746d21e9107aa633df620e1acc8fa4694a1cad5205445 not found: ID does not exist"
Jul 24 23:43:09.237934 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:43:09.237904    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-59bb55c686-jbsq7"]
Jul 24 23:43:09.239830 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:43:09.239808    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-59bb55c686-jbsq7"]
Jul 24 23:43:09.303391 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:43:09.303360    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k22fr\" (UniqueName: \"kubernetes.io/projected/93768090-a58e-457d-91f7-caeb814c088f-kube-api-access-k22fr\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 24 23:43:09.303391 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:43:09.303386    2558 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/93768090-a58e-457d-91f7-caeb814c088f-webhook-cert\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 24 23:43:09.931042 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:43:09.931002    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:43:10.059461 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:43:10.059424    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-59bb55c686-v7p6f"]
Jul 24 23:43:10.059793 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:43:10.059780    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93768090-a58e-457d-91f7-caeb814c088f" containerName="manager"
Jul 24 23:43:10.059836 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:43:10.059794    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="93768090-a58e-457d-91f7-caeb814c088f" containerName="manager"
Jul 24 23:43:10.059874 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:43:10.059864    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="93768090-a58e-457d-91f7-caeb814c088f" containerName="manager"
Jul 24 23:43:10.064114 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:43:10.064086    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-59bb55c686-v7p6f"
Jul 24 23:43:10.066517 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:43:10.066494    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-l7fhg\""
Jul 24 23:43:10.067350 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:43:10.067332    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 24 23:43:10.067461 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:43:10.067444    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 24 23:43:10.071713 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:43:10.071695    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-59bb55c686-v7p6f"]
Jul 24 23:43:10.116044 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:43:10.116009    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e3d3d419-d577-41e6-952f-032abd72c77c-webhook-cert\") pod \"maas-controller-59bb55c686-v7p6f\" (UID: \"e3d3d419-d577-41e6-952f-032abd72c77c\") " pod="opendatahub/maas-controller-59bb55c686-v7p6f"
Jul 24 23:43:10.116221 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:43:10.116164    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6frb5\" (UniqueName: \"kubernetes.io/projected/e3d3d419-d577-41e6-952f-032abd72c77c-kube-api-access-6frb5\") pod \"maas-controller-59bb55c686-v7p6f\" (UID: \"e3d3d419-d577-41e6-952f-032abd72c77c\") " pod="opendatahub/maas-controller-59bb55c686-v7p6f"
Jul 24 23:43:10.204690 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:43:10.204604    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93768090-a58e-457d-91f7-caeb814c088f" path="/var/lib/kubelet/pods/93768090-a58e-457d-91f7-caeb814c088f/volumes"
Jul 24 23:43:10.217558 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:43:10.217516    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e3d3d419-d577-41e6-952f-032abd72c77c-webhook-cert\") pod \"maas-controller-59bb55c686-v7p6f\" (UID: \"e3d3d419-d577-41e6-952f-032abd72c77c\") " pod="opendatahub/maas-controller-59bb55c686-v7p6f"
Jul 24 23:43:10.217745 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:43:10.217648    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6frb5\" (UniqueName: \"kubernetes.io/projected/e3d3d419-d577-41e6-952f-032abd72c77c-kube-api-access-6frb5\") pod \"maas-controller-59bb55c686-v7p6f\" (UID: \"e3d3d419-d577-41e6-952f-032abd72c77c\") " pod="opendatahub/maas-controller-59bb55c686-v7p6f"
Jul 24 23:43:10.220026 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:43:10.220000    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e3d3d419-d577-41e6-952f-032abd72c77c-webhook-cert\") pod \"maas-controller-59bb55c686-v7p6f\" (UID: \"e3d3d419-d577-41e6-952f-032abd72c77c\") " pod="opendatahub/maas-controller-59bb55c686-v7p6f"
Jul 24 23:43:10.225033 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:43:10.225004    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6frb5\" (UniqueName: \"kubernetes.io/projected/e3d3d419-d577-41e6-952f-032abd72c77c-kube-api-access-6frb5\") pod \"maas-controller-59bb55c686-v7p6f\" (UID: \"e3d3d419-d577-41e6-952f-032abd72c77c\") " pod="opendatahub/maas-controller-59bb55c686-v7p6f"
Jul 24 23:43:10.375976 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:43:10.375937    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-59bb55c686-v7p6f"
Jul 24 23:43:10.512485 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:43:10.512457    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-59bb55c686-v7p6f"]
Jul 24 23:43:10.514933 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:43:10.514899    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3d3d419_d577_41e6_952f_032abd72c77c.slice/crio-5e9b16fd1c1940d66ee0fe3d58fead473b2ce6014c26f4b77b6f6137b8faf589 WatchSource:0}: Error finding container 5e9b16fd1c1940d66ee0fe3d58fead473b2ce6014c26f4b77b6f6137b8faf589: Status 404 returned error can't find the container with id 5e9b16fd1c1940d66ee0fe3d58fead473b2ce6014c26f4b77b6f6137b8faf589
Jul 24 23:43:11.231052 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:43:11.231010    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-59bb55c686-v7p6f" event={"ID":"e3d3d419-d577-41e6-952f-032abd72c77c","Type":"ContainerStarted","Data":"a7fc0904b5e926c95eadcf8194ac8f4be8cff1708b5eb400a50757b974b3b98e"}
Jul 24 23:43:11.231513 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:43:11.231077    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-59bb55c686-v7p6f" event={"ID":"e3d3d419-d577-41e6-952f-032abd72c77c","Type":"ContainerStarted","Data":"5e9b16fd1c1940d66ee0fe3d58fead473b2ce6014c26f4b77b6f6137b8faf589"}
Jul 24 23:43:11.231513 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:43:11.231116    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-59bb55c686-v7p6f"
Jul 24 23:43:11.263012 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:43:11.262904    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-59bb55c686-v7p6f" podStartSLOduration=0.766256938 podStartE2EDuration="1.262885858s" totalImagesPullingTime="496.62892ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:43:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:43:11.260976724 +0000 UTC m=+831.647407447" watchObservedRunningTime="2026-07-24 23:43:11.262885858 +0000 UTC m=+831.649316586"
Jul 24 23:43:22.239404 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:43:22.239373    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-59bb55c686-v7p6f"
Jul 24 23:43:48.719944 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:43:48.719906    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:43:48.816937 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:43:48.816902    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:43:48.954353 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:43:48.954318    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:43:54.511511 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:43:54.511464    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:43:54.610800 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:43:54.610765    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:43:54.709683 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:43:54.709644    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:43:54.873855 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:43:54.873759    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:44:00.406005 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:44:00.405962    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:44:00.507165 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:44:00.507121    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:44:03.248131 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:44:03.248093    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:47:03.299981 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:47:03.299946    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:47:04.811336 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:47:04.811294    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:51:39.443801 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:51:39.443764    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:51:48.319017 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:51:48.318929    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:52:18.859478 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:52:18.859434    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:52:19.854365 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:52:19.854327    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:52:25.180573 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:52:25.180526    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:52:26.487749 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:52:26.487715    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:52:50.195344 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:52:50.195300    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:55:29.851382 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:55:29.851345    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:55:34.634118 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:55:34.634088    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:55:34.683235 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:55:34.683201    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:55:50.491206 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:55:50.491172    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:55:53.639331 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:55:53.639292    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:55:53.739779 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:55:53.739738    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:55:54.097589 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:55:54.097557    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:55:59.720590 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:55:59.720551    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:55:59.780913 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:55:59.780879    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:55:59.940268 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:55:59.940234    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:56:00.251264 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:00.251230    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:56:00.443788 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:00.443755    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:56:05.247855 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:05.247819    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:56:05.347864 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:05.347825    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:56:05.496432 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:05.496399    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-90db14c6-745474dd7f-6q8kg"]
Jul 24 23:56:05.500013 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:05.499949    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-90db14c6-745474dd7f-6q8kg"
Jul 24 23:56:05.502282 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:05.502258    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-90db14c6-serving-cert\""
Jul 24 23:56:05.510808 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:05.510782    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-90db14c6-745474dd7f-6q8kg"]
Jul 24 23:56:05.534570 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:05.534543    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:56:05.586970 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:05.586939    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07ff6d8b-dd97-4f34-8cff-dd0f16dd8e5d-maas-api-tls\") pod \"maas-api-e2e-ait-90db14c6-745474dd7f-6q8kg\" (UID: \"07ff6d8b-dd97-4f34-8cff-dd0f16dd8e5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-90db14c6-745474dd7f-6q8kg"
Jul 24 23:56:05.587123 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:05.586973    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2c7v\" (UniqueName: \"kubernetes.io/projected/07ff6d8b-dd97-4f34-8cff-dd0f16dd8e5d-kube-api-access-f2c7v\") pod \"maas-api-e2e-ait-90db14c6-745474dd7f-6q8kg\" (UID: \"07ff6d8b-dd97-4f34-8cff-dd0f16dd8e5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-90db14c6-745474dd7f-6q8kg"
Jul 24 23:56:05.587123 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:05.587064    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/07ff6d8b-dd97-4f34-8cff-dd0f16dd8e5d-openshift-service-ca\") pod \"maas-api-e2e-ait-90db14c6-745474dd7f-6q8kg\" (UID: \"07ff6d8b-dd97-4f34-8cff-dd0f16dd8e5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-90db14c6-745474dd7f-6q8kg"
Jul 24 23:56:05.687897 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:05.687863    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07ff6d8b-dd97-4f34-8cff-dd0f16dd8e5d-maas-api-tls\") pod \"maas-api-e2e-ait-90db14c6-745474dd7f-6q8kg\" (UID: \"07ff6d8b-dd97-4f34-8cff-dd0f16dd8e5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-90db14c6-745474dd7f-6q8kg"
Jul 24 23:56:05.687897 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:05.687897    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2c7v\" (UniqueName: \"kubernetes.io/projected/07ff6d8b-dd97-4f34-8cff-dd0f16dd8e5d-kube-api-access-f2c7v\") pod \"maas-api-e2e-ait-90db14c6-745474dd7f-6q8kg\" (UID: \"07ff6d8b-dd97-4f34-8cff-dd0f16dd8e5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-90db14c6-745474dd7f-6q8kg"
Jul 24 23:56:05.688165 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:05.687930    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/07ff6d8b-dd97-4f34-8cff-dd0f16dd8e5d-openshift-service-ca\") pod \"maas-api-e2e-ait-90db14c6-745474dd7f-6q8kg\" (UID: \"07ff6d8b-dd97-4f34-8cff-dd0f16dd8e5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-90db14c6-745474dd7f-6q8kg"
Jul 24 23:56:05.688165 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:56:05.688037    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-90db14c6-serving-cert: secret "maas-api-e2e-ait-90db14c6-serving-cert" not found
Jul 24 23:56:05.688165 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:56:05.688131    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07ff6d8b-dd97-4f34-8cff-dd0f16dd8e5d-maas-api-tls podName:07ff6d8b-dd97-4f34-8cff-dd0f16dd8e5d nodeName:}" failed. No retries permitted until 2026-07-24 23:56:06.188109048 +0000 UTC m=+1606.574539768 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/07ff6d8b-dd97-4f34-8cff-dd0f16dd8e5d-maas-api-tls") pod "maas-api-e2e-ait-90db14c6-745474dd7f-6q8kg" (UID: "07ff6d8b-dd97-4f34-8cff-dd0f16dd8e5d") : secret "maas-api-e2e-ait-90db14c6-serving-cert" not found
Jul 24 23:56:05.688589 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:05.688522    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/07ff6d8b-dd97-4f34-8cff-dd0f16dd8e5d-openshift-service-ca\") pod \"maas-api-e2e-ait-90db14c6-745474dd7f-6q8kg\" (UID: \"07ff6d8b-dd97-4f34-8cff-dd0f16dd8e5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-90db14c6-745474dd7f-6q8kg"
Jul 24 23:56:05.696823 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:05.696803    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2c7v\" (UniqueName: \"kubernetes.io/projected/07ff6d8b-dd97-4f34-8cff-dd0f16dd8e5d-kube-api-access-f2c7v\") pod \"maas-api-e2e-ait-90db14c6-745474dd7f-6q8kg\" (UID: \"07ff6d8b-dd97-4f34-8cff-dd0f16dd8e5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-90db14c6-745474dd7f-6q8kg"
Jul 24 23:56:06.023594 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:06.023559    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:56:06.110268 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:06.110238    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:56:06.160426 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:06.160392    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-90db14c6-868dd9cc7d-pw9dt"]
Jul 24 23:56:06.164020 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:06.164002    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-90db14c6-868dd9cc7d-pw9dt"
Jul 24 23:56:06.166490 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:06.166466    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-90db14c6-dockercfg-pp5vk\""
Jul 24 23:56:06.166490 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:06.166487    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-90db14c6\""
Jul 24 23:56:06.173950 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:06.173914    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-90db14c6-868dd9cc7d-pw9dt"]
Jul 24 23:56:06.193566 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:06.193509    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07ff6d8b-dd97-4f34-8cff-dd0f16dd8e5d-maas-api-tls\") pod \"maas-api-e2e-ait-90db14c6-745474dd7f-6q8kg\" (UID: \"07ff6d8b-dd97-4f34-8cff-dd0f16dd8e5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-90db14c6-745474dd7f-6q8kg"
Jul 24 23:56:06.193731 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:56:06.193653    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-90db14c6-serving-cert: secret "maas-api-e2e-ait-90db14c6-serving-cert" not found
Jul 24 23:56:06.193731 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:56:06.193714    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07ff6d8b-dd97-4f34-8cff-dd0f16dd8e5d-maas-api-tls podName:07ff6d8b-dd97-4f34-8cff-dd0f16dd8e5d nodeName:}" failed. No retries permitted until 2026-07-24 23:56:07.193700086 +0000 UTC m=+1607.580130792 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/07ff6d8b-dd97-4f34-8cff-dd0f16dd8e5d-maas-api-tls") pod "maas-api-e2e-ait-90db14c6-745474dd7f-6q8kg" (UID: "07ff6d8b-dd97-4f34-8cff-dd0f16dd8e5d") : secret "maas-api-e2e-ait-90db14c6-serving-cert" not found
Jul 24 23:56:06.295068 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:06.294975    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zwsd\" (UniqueName: \"kubernetes.io/projected/8f7b7592-adfd-44d3-85a6-fba7f525d6ac-kube-api-access-6zwsd\") pod \"payload-processing-e2e-ait-90db14c6-868dd9cc7d-pw9dt\" (UID: \"8f7b7592-adfd-44d3-85a6-fba7f525d6ac\") " pod="openshift-ingress/payload-processing-e2e-ait-90db14c6-868dd9cc7d-pw9dt"
Jul 24 23:56:06.295068 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:06.295029    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f7b7592-adfd-44d3-85a6-fba7f525d6ac-plugins-config-volume\") pod \"payload-processing-e2e-ait-90db14c6-868dd9cc7d-pw9dt\" (UID: \"8f7b7592-adfd-44d3-85a6-fba7f525d6ac\") " pod="openshift-ingress/payload-processing-e2e-ait-90db14c6-868dd9cc7d-pw9dt"
Jul 24 23:56:06.396605 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:06.396570    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f7b7592-adfd-44d3-85a6-fba7f525d6ac-plugins-config-volume\") pod \"payload-processing-e2e-ait-90db14c6-868dd9cc7d-pw9dt\" (UID: \"8f7b7592-adfd-44d3-85a6-fba7f525d6ac\") " pod="openshift-ingress/payload-processing-e2e-ait-90db14c6-868dd9cc7d-pw9dt"
Jul 24 23:56:06.396780 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:06.396675    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zwsd\" (UniqueName: \"kubernetes.io/projected/8f7b7592-adfd-44d3-85a6-fba7f525d6ac-kube-api-access-6zwsd\") pod \"payload-processing-e2e-ait-90db14c6-868dd9cc7d-pw9dt\" (UID: \"8f7b7592-adfd-44d3-85a6-fba7f525d6ac\") " pod="openshift-ingress/payload-processing-e2e-ait-90db14c6-868dd9cc7d-pw9dt"
Jul 24 23:56:06.397245 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:06.397221    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f7b7592-adfd-44d3-85a6-fba7f525d6ac-plugins-config-volume\") pod \"payload-processing-e2e-ait-90db14c6-868dd9cc7d-pw9dt\" (UID: \"8f7b7592-adfd-44d3-85a6-fba7f525d6ac\") " pod="openshift-ingress/payload-processing-e2e-ait-90db14c6-868dd9cc7d-pw9dt"
Jul 24 23:56:06.404049 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:06.404024    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zwsd\" (UniqueName: \"kubernetes.io/projected/8f7b7592-adfd-44d3-85a6-fba7f525d6ac-kube-api-access-6zwsd\") pod \"payload-processing-e2e-ait-90db14c6-868dd9cc7d-pw9dt\" (UID: \"8f7b7592-adfd-44d3-85a6-fba7f525d6ac\") " pod="openshift-ingress/payload-processing-e2e-ait-90db14c6-868dd9cc7d-pw9dt"
Jul 24 23:56:06.476221 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:06.476181    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-90db14c6-868dd9cc7d-pw9dt"
Jul 24 23:56:06.523213 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:06.523183    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:56:06.615848 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:06.615814    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-90db14c6-868dd9cc7d-pw9dt"]
Jul 24 23:56:06.617906 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:56:06.617871    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f7b7592_adfd_44d3_85a6_fba7f525d6ac.slice/crio-a109bc2aba968b8a2df797c9d34e29759a18004ae1ce70257a5be1facdabef22 WatchSource:0}: Error finding container a109bc2aba968b8a2df797c9d34e29759a18004ae1ce70257a5be1facdabef22: Status 404 returned error can't find the container with id a109bc2aba968b8a2df797c9d34e29759a18004ae1ce70257a5be1facdabef22
Jul 24 23:56:07.154423 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:07.154387    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-90db14c6-868dd9cc7d-pw9dt" event={"ID":"8f7b7592-adfd-44d3-85a6-fba7f525d6ac","Type":"ContainerStarted","Data":"4856fc8ca96d44301739a70f27231153eed9956e54ad2688fd8867c76823177e"}
Jul 24 23:56:07.154652 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:07.154445    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-90db14c6-868dd9cc7d-pw9dt" event={"ID":"8f7b7592-adfd-44d3-85a6-fba7f525d6ac","Type":"ContainerStarted","Data":"a109bc2aba968b8a2df797c9d34e29759a18004ae1ce70257a5be1facdabef22"}
Jul 24 23:56:07.154652 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:07.154625    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-90db14c6-868dd9cc7d-pw9dt"
Jul 24 23:56:07.171290 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:07.171235    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-90db14c6-868dd9cc7d-pw9dt" podStartSLOduration=1.171220195 podStartE2EDuration="1.171220195s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:56:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:56:07.168685176 +0000 UTC m=+1607.555115940" watchObservedRunningTime="2026-07-24 23:56:07.171220195 +0000 UTC m=+1607.557650922"
Jul 24 23:56:07.205034 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:07.204996    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07ff6d8b-dd97-4f34-8cff-dd0f16dd8e5d-maas-api-tls\") pod \"maas-api-e2e-ait-90db14c6-745474dd7f-6q8kg\" (UID: \"07ff6d8b-dd97-4f34-8cff-dd0f16dd8e5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-90db14c6-745474dd7f-6q8kg"
Jul 24 23:56:07.207385 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:07.207359    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07ff6d8b-dd97-4f34-8cff-dd0f16dd8e5d-maas-api-tls\") pod \"maas-api-e2e-ait-90db14c6-745474dd7f-6q8kg\" (UID: \"07ff6d8b-dd97-4f34-8cff-dd0f16dd8e5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-90db14c6-745474dd7f-6q8kg"
Jul 24 23:56:07.312190 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:07.312156    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-90db14c6-745474dd7f-6q8kg"
Jul 24 23:56:07.448203 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:07.448175    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-90db14c6-745474dd7f-6q8kg"]
Jul 24 23:56:07.449343 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:56:07.449309    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07ff6d8b_dd97_4f34_8cff_dd0f16dd8e5d.slice/crio-3b4ba30f97918bd77f0a5cf36b9278ae40794a6f4f2b5b5f433c446e0e7ae5fe WatchSource:0}: Error finding container 3b4ba30f97918bd77f0a5cf36b9278ae40794a6f4f2b5b5f433c446e0e7ae5fe: Status 404 returned error can't find the container with id 3b4ba30f97918bd77f0a5cf36b9278ae40794a6f4f2b5b5f433c446e0e7ae5fe
Jul 24 23:56:07.450873 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:07.450856    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 23:56:08.162272 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:08.162182    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-90db14c6-745474dd7f-6q8kg" event={"ID":"07ff6d8b-dd97-4f34-8cff-dd0f16dd8e5d","Type":"ContainerStarted","Data":"e3e4f42d1d1926c6508b957151871cf0a1e832927603aa1f25a7b2cd379c7bf1"}
Jul 24 23:56:08.162272 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:08.162246    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-90db14c6-745474dd7f-6q8kg" event={"ID":"07ff6d8b-dd97-4f34-8cff-dd0f16dd8e5d","Type":"ContainerStarted","Data":"3b4ba30f97918bd77f0a5cf36b9278ae40794a6f4f2b5b5f433c446e0e7ae5fe"}
Jul 24 23:56:08.162454 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:08.162332    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-90db14c6-745474dd7f-6q8kg"
Jul 24 23:56:08.178854 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:08.178802    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-90db14c6-745474dd7f-6q8kg" podStartSLOduration=2.732412101 podStartE2EDuration="3.17878577s" totalImagesPullingTime="446.373669ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:56:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:56:08.176786501 +0000 UTC m=+1608.563217230" watchObservedRunningTime="2026-07-24 23:56:08.17878577 +0000 UTC m=+1608.565216498"
Jul 24 23:56:14.172150 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:14.172070    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-90db14c6-745474dd7f-6q8kg"
Jul 24 23:56:18.165094 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:18.165063    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-90db14c6-868dd9cc7d-pw9dt"
Jul 24 23:56:41.250950 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:41.250914    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-90db14c6-745474dd7f-6q8kg"]
Jul 24 23:56:41.253461 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:41.251129    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-90db14c6-745474dd7f-6q8kg" podUID="07ff6d8b-dd97-4f34-8cff-dd0f16dd8e5d" containerName="maas-api" containerID="cri-o://e3e4f42d1d1926c6508b957151871cf0a1e832927603aa1f25a7b2cd379c7bf1" gracePeriod=30
Jul 24 23:56:41.270113 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:41.270088    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:56:41.295341 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:41.295313    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-90db14c6-868dd9cc7d-pw9dt"]
Jul 24 23:56:41.295553 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:41.295510    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-90db14c6-868dd9cc7d-pw9dt" podUID="8f7b7592-adfd-44d3-85a6-fba7f525d6ac" containerName="payload-processing" containerID="cri-o://4856fc8ca96d44301739a70f27231153eed9956e54ad2688fd8867c76823177e" gracePeriod=30
Jul 24 23:56:41.370146 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:41.370111    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:56:41.470891 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:41.470857    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:56:41.538645 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:41.538623    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-90db14c6-745474dd7f-6q8kg"
Jul 24 23:56:41.569334 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:41.569309    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-90db14c6-868dd9cc7d-pw9dt"
Jul 24 23:56:41.600754 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:41.600724    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f2c7v\" (UniqueName: \"kubernetes.io/projected/07ff6d8b-dd97-4f34-8cff-dd0f16dd8e5d-kube-api-access-f2c7v\") pod \"07ff6d8b-dd97-4f34-8cff-dd0f16dd8e5d\" (UID: \"07ff6d8b-dd97-4f34-8cff-dd0f16dd8e5d\") "
Jul 24 23:56:41.600903 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:41.600810    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/07ff6d8b-dd97-4f34-8cff-dd0f16dd8e5d-openshift-service-ca\") pod \"07ff6d8b-dd97-4f34-8cff-dd0f16dd8e5d\" (UID: \"07ff6d8b-dd97-4f34-8cff-dd0f16dd8e5d\") "
Jul 24 23:56:41.600903 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:41.600837    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6zwsd\" (UniqueName: \"kubernetes.io/projected/8f7b7592-adfd-44d3-85a6-fba7f525d6ac-kube-api-access-6zwsd\") pod \"8f7b7592-adfd-44d3-85a6-fba7f525d6ac\" (UID: \"8f7b7592-adfd-44d3-85a6-fba7f525d6ac\") "
Jul 24 23:56:41.600903 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:41.600870    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07ff6d8b-dd97-4f34-8cff-dd0f16dd8e5d-maas-api-tls\") pod \"07ff6d8b-dd97-4f34-8cff-dd0f16dd8e5d\" (UID: \"07ff6d8b-dd97-4f34-8cff-dd0f16dd8e5d\") "
Jul 24 23:56:41.601052 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:41.600928    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f7b7592-adfd-44d3-85a6-fba7f525d6ac-plugins-config-volume\") pod \"8f7b7592-adfd-44d3-85a6-fba7f525d6ac\" (UID: \"8f7b7592-adfd-44d3-85a6-fba7f525d6ac\") "
Jul 24 23:56:41.601268 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:41.601240    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07ff6d8b-dd97-4f34-8cff-dd0f16dd8e5d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "07ff6d8b-dd97-4f34-8cff-dd0f16dd8e5d" (UID: "07ff6d8b-dd97-4f34-8cff-dd0f16dd8e5d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:56:41.601399 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:41.601379    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f7b7592-adfd-44d3-85a6-fba7f525d6ac-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8f7b7592-adfd-44d3-85a6-fba7f525d6ac" (UID: "8f7b7592-adfd-44d3-85a6-fba7f525d6ac"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:56:41.603137 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:41.603113    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f7b7592-adfd-44d3-85a6-fba7f525d6ac-kube-api-access-6zwsd" (OuterVolumeSpecName: "kube-api-access-6zwsd") pod "8f7b7592-adfd-44d3-85a6-fba7f525d6ac" (UID: "8f7b7592-adfd-44d3-85a6-fba7f525d6ac"). InnerVolumeSpecName "kube-api-access-6zwsd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:56:41.603137 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:41.603125    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07ff6d8b-dd97-4f34-8cff-dd0f16dd8e5d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "07ff6d8b-dd97-4f34-8cff-dd0f16dd8e5d" (UID: "07ff6d8b-dd97-4f34-8cff-dd0f16dd8e5d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 23:56:41.603272 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:41.603258    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07ff6d8b-dd97-4f34-8cff-dd0f16dd8e5d-kube-api-access-f2c7v" (OuterVolumeSpecName: "kube-api-access-f2c7v") pod "07ff6d8b-dd97-4f34-8cff-dd0f16dd8e5d" (UID: "07ff6d8b-dd97-4f34-8cff-dd0f16dd8e5d"). InnerVolumeSpecName "kube-api-access-f2c7v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:56:41.701477 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:41.701443    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f7b7592-adfd-44d3-85a6-fba7f525d6ac-plugins-config-volume\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 24 23:56:41.701477 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:41.701470    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f2c7v\" (UniqueName: \"kubernetes.io/projected/07ff6d8b-dd97-4f34-8cff-dd0f16dd8e5d-kube-api-access-f2c7v\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 24 23:56:41.701477 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:41.701479    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/07ff6d8b-dd97-4f34-8cff-dd0f16dd8e5d-openshift-service-ca\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 24 23:56:41.701728 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:41.701488    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6zwsd\" (UniqueName: \"kubernetes.io/projected/8f7b7592-adfd-44d3-85a6-fba7f525d6ac-kube-api-access-6zwsd\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 24 23:56:41.701728 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:41.701498    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07ff6d8b-dd97-4f34-8cff-dd0f16dd8e5d-maas-api-tls\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 24 23:56:42.336918 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:42.336879    2558 generic.go:356] "Generic (PLEG): container finished" podID="8f7b7592-adfd-44d3-85a6-fba7f525d6ac" containerID="4856fc8ca96d44301739a70f27231153eed9956e54ad2688fd8867c76823177e" exitCode=0
Jul 24 23:56:42.336918 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:42.336919    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-90db14c6-868dd9cc7d-pw9dt"
Jul 24 23:56:42.337437 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:42.336963    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-90db14c6-868dd9cc7d-pw9dt" event={"ID":"8f7b7592-adfd-44d3-85a6-fba7f525d6ac","Type":"ContainerDied","Data":"4856fc8ca96d44301739a70f27231153eed9956e54ad2688fd8867c76823177e"}
Jul 24 23:56:42.337437 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:42.337022    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-90db14c6-868dd9cc7d-pw9dt" event={"ID":"8f7b7592-adfd-44d3-85a6-fba7f525d6ac","Type":"ContainerDied","Data":"a109bc2aba968b8a2df797c9d34e29759a18004ae1ce70257a5be1facdabef22"}
Jul 24 23:56:42.337437 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:42.337056    2558 scope.go:117] "RemoveContainer" containerID="4856fc8ca96d44301739a70f27231153eed9956e54ad2688fd8867c76823177e"
Jul 24 23:56:42.338386 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:42.338271    2558 generic.go:356] "Generic (PLEG): container finished" podID="07ff6d8b-dd97-4f34-8cff-dd0f16dd8e5d" containerID="e3e4f42d1d1926c6508b957151871cf0a1e832927603aa1f25a7b2cd379c7bf1" exitCode=0
Jul 24 23:56:42.338386 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:42.338321    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-90db14c6-745474dd7f-6q8kg"
Jul 24 23:56:42.338386 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:42.338361    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-90db14c6-745474dd7f-6q8kg" event={"ID":"07ff6d8b-dd97-4f34-8cff-dd0f16dd8e5d","Type":"ContainerDied","Data":"e3e4f42d1d1926c6508b957151871cf0a1e832927603aa1f25a7b2cd379c7bf1"}
Jul 24 23:56:42.338563 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:42.338425    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-90db14c6-745474dd7f-6q8kg" event={"ID":"07ff6d8b-dd97-4f34-8cff-dd0f16dd8e5d","Type":"ContainerDied","Data":"3b4ba30f97918bd77f0a5cf36b9278ae40794a6f4f2b5b5f433c446e0e7ae5fe"}
Jul 24 23:56:42.346247 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:42.346231    2558 scope.go:117] "RemoveContainer" containerID="4856fc8ca96d44301739a70f27231153eed9956e54ad2688fd8867c76823177e"
Jul 24 23:56:42.346514 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:56:42.346493    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4856fc8ca96d44301739a70f27231153eed9956e54ad2688fd8867c76823177e\": container with ID starting with 4856fc8ca96d44301739a70f27231153eed9956e54ad2688fd8867c76823177e not found: ID does not exist" containerID="4856fc8ca96d44301739a70f27231153eed9956e54ad2688fd8867c76823177e"
Jul 24 23:56:42.346691 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:42.346523    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4856fc8ca96d44301739a70f27231153eed9956e54ad2688fd8867c76823177e"} err="failed to get container status \"4856fc8ca96d44301739a70f27231153eed9956e54ad2688fd8867c76823177e\": rpc error: code = NotFound desc = could not find container \"4856fc8ca96d44301739a70f27231153eed9956e54ad2688fd8867c76823177e\": container with ID starting with 4856fc8ca96d44301739a70f27231153eed9956e54ad2688fd8867c76823177e not found: ID does not exist"
Jul 24 23:56:42.346755 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:42.346695    2558 scope.go:117] "RemoveContainer" containerID="e3e4f42d1d1926c6508b957151871cf0a1e832927603aa1f25a7b2cd379c7bf1"
Jul 24 23:56:42.356069 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:42.356041    2558 scope.go:117] "RemoveContainer" containerID="e3e4f42d1d1926c6508b957151871cf0a1e832927603aa1f25a7b2cd379c7bf1"
Jul 24 23:56:42.356749 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:56:42.356517    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3e4f42d1d1926c6508b957151871cf0a1e832927603aa1f25a7b2cd379c7bf1\": container with ID starting with e3e4f42d1d1926c6508b957151871cf0a1e832927603aa1f25a7b2cd379c7bf1 not found: ID does not exist" containerID="e3e4f42d1d1926c6508b957151871cf0a1e832927603aa1f25a7b2cd379c7bf1"
Jul 24 23:56:42.356749 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:42.356582    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3e4f42d1d1926c6508b957151871cf0a1e832927603aa1f25a7b2cd379c7bf1"} err="failed to get container status \"e3e4f42d1d1926c6508b957151871cf0a1e832927603aa1f25a7b2cd379c7bf1\": rpc error: code = NotFound desc = could not find container \"e3e4f42d1d1926c6508b957151871cf0a1e832927603aa1f25a7b2cd379c7bf1\": container with ID starting with e3e4f42d1d1926c6508b957151871cf0a1e832927603aa1f25a7b2cd379c7bf1 not found: ID does not exist"
Jul 24 23:56:42.359355 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:42.359329    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-90db14c6-868dd9cc7d-pw9dt"]
Jul 24 23:56:42.363088 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:42.363069    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-90db14c6-868dd9cc7d-pw9dt"]
Jul 24 23:56:42.368126 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:42.368105    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-90db14c6-745474dd7f-6q8kg"]
Jul 24 23:56:42.371164 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:42.371144    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-90db14c6-745474dd7f-6q8kg"]
Jul 24 23:56:44.203970 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:44.203937    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07ff6d8b-dd97-4f34-8cff-dd0f16dd8e5d" path="/var/lib/kubelet/pods/07ff6d8b-dd97-4f34-8cff-dd0f16dd8e5d/volumes"
Jul 24 23:56:44.204377 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:44.204291    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f7b7592-adfd-44d3-85a6-fba7f525d6ac" path="/var/lib/kubelet/pods/8f7b7592-adfd-44d3-85a6-fba7f525d6ac/volumes"
Jul 24 23:56:45.911348 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:45.911310    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:56:46.081696 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:46.081658    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:56:46.325554 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:46.325501    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:56:46.590826 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:46.590736    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:56:46.691680 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:46.691641    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:56:46.793377 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:46.793337    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:56:47.221343 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:47.221301    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:56:47.386009 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:47.385975    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:56:49.616441 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:49.616381    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:56:49.696838 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:49.696798    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-25f28971-59c48c7485-xfxsl"]
Jul 24 23:56:49.697342 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:49.697321    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07ff6d8b-dd97-4f34-8cff-dd0f16dd8e5d" containerName="maas-api"
Jul 24 23:56:49.697386 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:49.697348    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="07ff6d8b-dd97-4f34-8cff-dd0f16dd8e5d" containerName="maas-api"
Jul 24 23:56:49.697386 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:49.697379    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f7b7592-adfd-44d3-85a6-fba7f525d6ac" containerName="payload-processing"
Jul 24 23:56:49.697468 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:49.697389    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f7b7592-adfd-44d3-85a6-fba7f525d6ac" containerName="payload-processing"
Jul 24 23:56:49.697542 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:49.697490    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f7b7592-adfd-44d3-85a6-fba7f525d6ac" containerName="payload-processing"
Jul 24 23:56:49.697542 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:49.697502    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="07ff6d8b-dd97-4f34-8cff-dd0f16dd8e5d" containerName="maas-api"
Jul 24 23:56:49.702159 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:49.702132    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-25f28971-59c48c7485-xfxsl"
Jul 24 23:56:49.704628 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:49.704610    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-25f28971-serving-cert\""
Jul 24 23:56:49.710731 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:49.710707    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-25f28971-59c48c7485-xfxsl"]
Jul 24 23:56:49.723799 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:49.723766    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:56:49.772468 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:49.772431    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46a34598-2429-409c-8ba3-59f9d2f78e64-openshift-service-ca\") pod \"maas-api-e2e-migrate-25f28971-59c48c7485-xfxsl\" (UID: \"46a34598-2429-409c-8ba3-59f9d2f78e64\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-25f28971-59c48c7485-xfxsl"
Jul 24 23:56:49.772677 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:49.772497    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46a34598-2429-409c-8ba3-59f9d2f78e64-maas-api-tls\") pod \"maas-api-e2e-migrate-25f28971-59c48c7485-xfxsl\" (UID: \"46a34598-2429-409c-8ba3-59f9d2f78e64\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-25f28971-59c48c7485-xfxsl"
Jul 24 23:56:49.772677 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:49.772573    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tscvb\" (UniqueName: \"kubernetes.io/projected/46a34598-2429-409c-8ba3-59f9d2f78e64-kube-api-access-tscvb\") pod \"maas-api-e2e-migrate-25f28971-59c48c7485-xfxsl\" (UID: \"46a34598-2429-409c-8ba3-59f9d2f78e64\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-25f28971-59c48c7485-xfxsl"
Jul 24 23:56:49.873894 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:49.873793    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tscvb\" (UniqueName: \"kubernetes.io/projected/46a34598-2429-409c-8ba3-59f9d2f78e64-kube-api-access-tscvb\") pod \"maas-api-e2e-migrate-25f28971-59c48c7485-xfxsl\" (UID: \"46a34598-2429-409c-8ba3-59f9d2f78e64\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-25f28971-59c48c7485-xfxsl"
Jul 24 23:56:49.874082 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:49.873947    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46a34598-2429-409c-8ba3-59f9d2f78e64-openshift-service-ca\") pod \"maas-api-e2e-migrate-25f28971-59c48c7485-xfxsl\" (UID: \"46a34598-2429-409c-8ba3-59f9d2f78e64\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-25f28971-59c48c7485-xfxsl"
Jul 24 23:56:49.874082 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:49.873998    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46a34598-2429-409c-8ba3-59f9d2f78e64-maas-api-tls\") pod \"maas-api-e2e-migrate-25f28971-59c48c7485-xfxsl\" (UID: \"46a34598-2429-409c-8ba3-59f9d2f78e64\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-25f28971-59c48c7485-xfxsl"
Jul 24 23:56:49.874201 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:56:49.874099    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-25f28971-serving-cert: secret "maas-api-e2e-migrate-25f28971-serving-cert" not found
Jul 24 23:56:49.874201 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:56:49.874160    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46a34598-2429-409c-8ba3-59f9d2f78e64-maas-api-tls podName:46a34598-2429-409c-8ba3-59f9d2f78e64 nodeName:}" failed. No retries permitted until 2026-07-24 23:56:50.374145176 +0000 UTC m=+1650.760575882 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/46a34598-2429-409c-8ba3-59f9d2f78e64-maas-api-tls") pod "maas-api-e2e-migrate-25f28971-59c48c7485-xfxsl" (UID: "46a34598-2429-409c-8ba3-59f9d2f78e64") : secret "maas-api-e2e-migrate-25f28971-serving-cert" not found
Jul 24 23:56:49.874604 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:49.874584    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46a34598-2429-409c-8ba3-59f9d2f78e64-openshift-service-ca\") pod \"maas-api-e2e-migrate-25f28971-59c48c7485-xfxsl\" (UID: \"46a34598-2429-409c-8ba3-59f9d2f78e64\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-25f28971-59c48c7485-xfxsl"
Jul 24 23:56:49.882066 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:49.882035    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tscvb\" (UniqueName: \"kubernetes.io/projected/46a34598-2429-409c-8ba3-59f9d2f78e64-kube-api-access-tscvb\") pod \"maas-api-e2e-migrate-25f28971-59c48c7485-xfxsl\" (UID: \"46a34598-2429-409c-8ba3-59f9d2f78e64\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-25f28971-59c48c7485-xfxsl"
Jul 24 23:56:49.960881 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:49.960847    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:56:50.305344 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:50.305304    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:56:50.378334 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:50.378298    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46a34598-2429-409c-8ba3-59f9d2f78e64-maas-api-tls\") pod \"maas-api-e2e-migrate-25f28971-59c48c7485-xfxsl\" (UID: \"46a34598-2429-409c-8ba3-59f9d2f78e64\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-25f28971-59c48c7485-xfxsl"
Jul 24 23:56:50.380900 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:50.380873    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46a34598-2429-409c-8ba3-59f9d2f78e64-maas-api-tls\") pod \"maas-api-e2e-migrate-25f28971-59c48c7485-xfxsl\" (UID: \"46a34598-2429-409c-8ba3-59f9d2f78e64\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-25f28971-59c48c7485-xfxsl"
Jul 24 23:56:50.396908 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:50.396871    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:56:50.397311 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:50.397290    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-25f28971-54cb9f97b-fvzlc"]
Jul 24 23:56:50.400812 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:50.400789    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-25f28971-54cb9f97b-fvzlc"
Jul 24 23:56:50.403206 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:50.403181    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-25f28971\""
Jul 24 23:56:50.409930 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:50.409907    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-25f28971-54cb9f97b-fvzlc"]
Jul 24 23:56:50.479120 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:50.479076    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5d165735-118a-4f85-939a-951526aab2f1-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-25f28971-54cb9f97b-fvzlc\" (UID: \"5d165735-118a-4f85-939a-951526aab2f1\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-25f28971-54cb9f97b-fvzlc"
Jul 24 23:56:50.479297 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:50.479140    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jcjz\" (UniqueName: \"kubernetes.io/projected/5d165735-118a-4f85-939a-951526aab2f1-kube-api-access-7jcjz\") pod \"payload-pre-processing-e2e-migrate-25f28971-54cb9f97b-fvzlc\" (UID: \"5d165735-118a-4f85-939a-951526aab2f1\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-25f28971-54cb9f97b-fvzlc"
Jul 24 23:56:50.580060 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:50.579973    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5d165735-118a-4f85-939a-951526aab2f1-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-25f28971-54cb9f97b-fvzlc\" (UID: \"5d165735-118a-4f85-939a-951526aab2f1\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-25f28971-54cb9f97b-fvzlc"
Jul 24 23:56:50.580060 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:50.580026    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jcjz\" (UniqueName: \"kubernetes.io/projected/5d165735-118a-4f85-939a-951526aab2f1-kube-api-access-7jcjz\") pod \"payload-pre-processing-e2e-migrate-25f28971-54cb9f97b-fvzlc\" (UID: \"5d165735-118a-4f85-939a-951526aab2f1\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-25f28971-54cb9f97b-fvzlc"
Jul 24 23:56:50.580812 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:50.580787    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5d165735-118a-4f85-939a-951526aab2f1-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-25f28971-54cb9f97b-fvzlc\" (UID: \"5d165735-118a-4f85-939a-951526aab2f1\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-25f28971-54cb9f97b-fvzlc"
Jul 24 23:56:50.587552 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:50.587517    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jcjz\" (UniqueName: \"kubernetes.io/projected/5d165735-118a-4f85-939a-951526aab2f1-kube-api-access-7jcjz\") pod \"payload-pre-processing-e2e-migrate-25f28971-54cb9f97b-fvzlc\" (UID: \"5d165735-118a-4f85-939a-951526aab2f1\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-25f28971-54cb9f97b-fvzlc"
Jul 24 23:56:50.613933 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:50.613900    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-25f28971-59c48c7485-xfxsl"
Jul 24 23:56:50.712636 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:50.712575    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-25f28971-54cb9f97b-fvzlc"
Jul 24 23:56:50.754191 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:50.754164    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-25f28971-59c48c7485-xfxsl"]
Jul 24 23:56:50.755736 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:56:50.755697    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46a34598_2429_409c_8ba3_59f9d2f78e64.slice/crio-29420b52f6f493cc0539faddff3e1d8dae5692f6af92490ddfa5c3f67eab339d WatchSource:0}: Error finding container 29420b52f6f493cc0539faddff3e1d8dae5692f6af92490ddfa5c3f67eab339d: Status 404 returned error can't find the container with id 29420b52f6f493cc0539faddff3e1d8dae5692f6af92490ddfa5c3f67eab339d
Jul 24 23:56:50.853360 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:50.853324    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-25f28971-54cb9f97b-fvzlc"]
Jul 24 23:56:50.855161 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:56:50.855132    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d165735_118a_4f85_939a_951526aab2f1.slice/crio-35abe52527905ed47cecfc8b219dda94730b399bb45b6669ea0004b9a4a5c9d3 WatchSource:0}: Error finding container 35abe52527905ed47cecfc8b219dda94730b399bb45b6669ea0004b9a4a5c9d3: Status 404 returned error can't find the container with id 35abe52527905ed47cecfc8b219dda94730b399bb45b6669ea0004b9a4a5c9d3
Jul 24 23:56:51.391025 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:51.390988    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-25f28971-54cb9f97b-fvzlc" event={"ID":"5d165735-118a-4f85-939a-951526aab2f1","Type":"ContainerStarted","Data":"bfc7d94c076871d719a75ddc26e14f34f90bcfd83aad0885a728cbc375608684"}
Jul 24 23:56:51.391228 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:51.391048    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-25f28971-54cb9f97b-fvzlc" event={"ID":"5d165735-118a-4f85-939a-951526aab2f1","Type":"ContainerStarted","Data":"35abe52527905ed47cecfc8b219dda94730b399bb45b6669ea0004b9a4a5c9d3"}
Jul 24 23:56:51.391228 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:51.391108    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-25f28971-54cb9f97b-fvzlc"
Jul 24 23:56:51.392874 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:51.392838    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-25f28971-59c48c7485-xfxsl" event={"ID":"46a34598-2429-409c-8ba3-59f9d2f78e64","Type":"ContainerStarted","Data":"120a912a77145eea5a195ae311ce508c953591b11271a75c0c99088fba53bc0a"}
Jul 24 23:56:51.392985 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:51.392899    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-25f28971-59c48c7485-xfxsl"
Jul 24 23:56:51.392985 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:51.392929    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-25f28971-59c48c7485-xfxsl" event={"ID":"46a34598-2429-409c-8ba3-59f9d2f78e64","Type":"ContainerStarted","Data":"29420b52f6f493cc0539faddff3e1d8dae5692f6af92490ddfa5c3f67eab339d"}
Jul 24 23:56:51.406155 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:51.406069    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-25f28971-54cb9f97b-fvzlc" podStartSLOduration=1.4060530820000001 podStartE2EDuration="1.406053082s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:56:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:56:51.404040782 +0000 UTC m=+1651.790471512" watchObservedRunningTime="2026-07-24 23:56:51.406053082 +0000 UTC m=+1651.792483809"
Jul 24 23:56:51.420498 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:51.420449    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-25f28971-59c48c7485-xfxsl" podStartSLOduration=2.023666143 podStartE2EDuration="2.420434159s" totalImagesPullingTime="396.768016ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:56:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:56:51.418040328 +0000 UTC m=+1651.804471055" watchObservedRunningTime="2026-07-24 23:56:51.420434159 +0000 UTC m=+1651.806864887"
Jul 24 23:56:57.403404 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:56:57.403371    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-25f28971-59c48c7485-xfxsl"
Jul 24 23:57:02.400433 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:02.400399    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-25f28971-54cb9f97b-fvzlc"
Jul 24 23:57:14.666462 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:14.666419    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-25f28971-fbnp8"]
Jul 24 23:57:14.670466 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:14.670445    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-25f28971-fbnp8"
Jul 24 23:57:14.673161 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:14.673134    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bxr46\""
Jul 24 23:57:14.676309 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:14.676283    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-25f28971-fbnp8"]
Jul 24 23:57:14.811159 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:14.811125    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/46911e98-766c-4579-9690-65092c21aecc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-25f28971-fbnp8\" (UID: \"46911e98-766c-4579-9690-65092c21aecc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-25f28971-fbnp8"
Jul 24 23:57:14.912180 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:14.912144    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/46911e98-766c-4579-9690-65092c21aecc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-25f28971-fbnp8\" (UID: \"46911e98-766c-4579-9690-65092c21aecc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-25f28971-fbnp8"
Jul 24 23:57:14.912869 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:14.912844    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/46911e98-766c-4579-9690-65092c21aecc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-25f28971-fbnp8\" (UID: \"46911e98-766c-4579-9690-65092c21aecc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-25f28971-fbnp8"
Jul 24 23:57:14.982491 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:14.982461    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-25f28971-fbnp8"
Jul 24 23:57:15.116397 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:15.116365    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-25f28971-fbnp8"]
Jul 24 23:57:15.117675 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:57:15.117641    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46911e98_766c_4579_9690_65092c21aecc.slice/crio-9ed8e44f5a95859ccabd58fb9d082cc7702ce6fb9370a6b79df028dbcd26bd86 WatchSource:0}: Error finding container 9ed8e44f5a95859ccabd58fb9d082cc7702ce6fb9370a6b79df028dbcd26bd86: Status 404 returned error can't find the container with id 9ed8e44f5a95859ccabd58fb9d082cc7702ce6fb9370a6b79df028dbcd26bd86
Jul 24 23:57:15.528099 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:15.528061    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-25f28971-fbnp8" event={"ID":"46911e98-766c-4579-9690-65092c21aecc","Type":"ContainerStarted","Data":"9ed8e44f5a95859ccabd58fb9d082cc7702ce6fb9370a6b79df028dbcd26bd86"}
Jul 24 23:57:18.546975 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:18.546881    2558 generic.go:356] "Generic (PLEG): container finished" podID="46911e98-766c-4579-9690-65092c21aecc" containerID="843234444c4bc6d98104b0158965a4ddbb6f99a054ce43a4ca2c023560c015a5" exitCode=0
Jul 24 23:57:18.547392 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:18.546972    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-25f28971-fbnp8" event={"ID":"46911e98-766c-4579-9690-65092c21aecc","Type":"ContainerDied","Data":"843234444c4bc6d98104b0158965a4ddbb6f99a054ce43a4ca2c023560c015a5"}
Jul 24 23:57:19.679186 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:19.679160    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-25f28971-fbnp8"
Jul 24 23:57:19.853688 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:19.853600    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/46911e98-766c-4579-9690-65092c21aecc-maas-api-service-ca\") pod \"46911e98-766c-4579-9690-65092c21aecc\" (UID: \"46911e98-766c-4579-9690-65092c21aecc\") "
Jul 24 23:57:19.853964 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:19.853941    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46911e98-766c-4579-9690-65092c21aecc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "46911e98-766c-4579-9690-65092c21aecc" (UID: "46911e98-766c-4579-9690-65092c21aecc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:57:19.955113 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:19.955076    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/46911e98-766c-4579-9690-65092c21aecc-maas-api-service-ca\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 24 23:57:20.557341 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:20.557309    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-25f28971-fbnp8"
Jul 24 23:57:20.557520 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:20.557349    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-25f28971-fbnp8" event={"ID":"46911e98-766c-4579-9690-65092c21aecc","Type":"ContainerDied","Data":"9ed8e44f5a95859ccabd58fb9d082cc7702ce6fb9370a6b79df028dbcd26bd86"}
Jul 24 23:57:20.557520 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:20.557434    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9ed8e44f5a95859ccabd58fb9d082cc7702ce6fb9370a6b79df028dbcd26bd86"
Jul 24 23:57:25.196771 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:25.196736    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-25f28971-59c48c7485-xfxsl"]
Jul 24 23:57:25.197266 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:25.197044    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-25f28971-59c48c7485-xfxsl" podUID="46a34598-2429-409c-8ba3-59f9d2f78e64" containerName="maas-api" containerID="cri-o://120a912a77145eea5a195ae311ce508c953591b11271a75c0c99088fba53bc0a" gracePeriod=30
Jul 24 23:57:25.218506 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:25.218477    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:57:25.259389 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:25.259334    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-25f28971-54cb9f97b-fvzlc"]
Jul 24 23:57:25.259647 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:25.259621    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-25f28971-54cb9f97b-fvzlc" podUID="5d165735-118a-4f85-939a-951526aab2f1" containerName="payload-pre-processing" containerID="cri-o://bfc7d94c076871d719a75ddc26e14f34f90bcfd83aad0885a728cbc375608684" gracePeriod=30
Jul 24 23:57:25.506423 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:25.506398    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-25f28971-59c48c7485-xfxsl"
Jul 24 23:57:25.507483 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:25.507462    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46a34598-2429-409c-8ba3-59f9d2f78e64-openshift-service-ca\") pod \"46a34598-2429-409c-8ba3-59f9d2f78e64\" (UID: \"46a34598-2429-409c-8ba3-59f9d2f78e64\") "
Jul 24 23:57:25.507552 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:25.507506    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46a34598-2429-409c-8ba3-59f9d2f78e64-maas-api-tls\") pod \"46a34598-2429-409c-8ba3-59f9d2f78e64\" (UID: \"46a34598-2429-409c-8ba3-59f9d2f78e64\") "
Jul 24 23:57:25.507613 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:25.507598    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tscvb\" (UniqueName: \"kubernetes.io/projected/46a34598-2429-409c-8ba3-59f9d2f78e64-kube-api-access-tscvb\") pod \"46a34598-2429-409c-8ba3-59f9d2f78e64\" (UID: \"46a34598-2429-409c-8ba3-59f9d2f78e64\") "
Jul 24 23:57:25.507904 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:25.507872    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46a34598-2429-409c-8ba3-59f9d2f78e64-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "46a34598-2429-409c-8ba3-59f9d2f78e64" (UID: "46a34598-2429-409c-8ba3-59f9d2f78e64"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:57:25.510837 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:25.510812    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46a34598-2429-409c-8ba3-59f9d2f78e64-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "46a34598-2429-409c-8ba3-59f9d2f78e64" (UID: "46a34598-2429-409c-8ba3-59f9d2f78e64"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 23:57:25.510959 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:25.510938    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46a34598-2429-409c-8ba3-59f9d2f78e64-kube-api-access-tscvb" (OuterVolumeSpecName: "kube-api-access-tscvb") pod "46a34598-2429-409c-8ba3-59f9d2f78e64" (UID: "46a34598-2429-409c-8ba3-59f9d2f78e64"). InnerVolumeSpecName "kube-api-access-tscvb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:57:25.529663 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:25.529639    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-25f28971-54cb9f97b-fvzlc"
Jul 24 23:57:25.585407 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:25.585375    2558 generic.go:356] "Generic (PLEG): container finished" podID="5d165735-118a-4f85-939a-951526aab2f1" containerID="bfc7d94c076871d719a75ddc26e14f34f90bcfd83aad0885a728cbc375608684" exitCode=0
Jul 24 23:57:25.585686 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:25.585421    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-25f28971-54cb9f97b-fvzlc"
Jul 24 23:57:25.585686 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:25.585450    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-25f28971-54cb9f97b-fvzlc" event={"ID":"5d165735-118a-4f85-939a-951526aab2f1","Type":"ContainerDied","Data":"bfc7d94c076871d719a75ddc26e14f34f90bcfd83aad0885a728cbc375608684"}
Jul 24 23:57:25.585686 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:25.585501    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-25f28971-54cb9f97b-fvzlc" event={"ID":"5d165735-118a-4f85-939a-951526aab2f1","Type":"ContainerDied","Data":"35abe52527905ed47cecfc8b219dda94730b399bb45b6669ea0004b9a4a5c9d3"}
Jul 24 23:57:25.585686 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:25.585544    2558 scope.go:117] "RemoveContainer" containerID="bfc7d94c076871d719a75ddc26e14f34f90bcfd83aad0885a728cbc375608684"
Jul 24 23:57:25.586770 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:25.586747    2558 generic.go:356] "Generic (PLEG): container finished" podID="46a34598-2429-409c-8ba3-59f9d2f78e64" containerID="120a912a77145eea5a195ae311ce508c953591b11271a75c0c99088fba53bc0a" exitCode=0
Jul 24 23:57:25.586848 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:25.586787    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-25f28971-59c48c7485-xfxsl"
Jul 24 23:57:25.586896 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:25.586871    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-25f28971-59c48c7485-xfxsl" event={"ID":"46a34598-2429-409c-8ba3-59f9d2f78e64","Type":"ContainerDied","Data":"120a912a77145eea5a195ae311ce508c953591b11271a75c0c99088fba53bc0a"}
Jul 24 23:57:25.586939 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:25.586925    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-25f28971-59c48c7485-xfxsl" event={"ID":"46a34598-2429-409c-8ba3-59f9d2f78e64","Type":"ContainerDied","Data":"29420b52f6f493cc0539faddff3e1d8dae5692f6af92490ddfa5c3f67eab339d"}
Jul 24 23:57:25.595252 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:25.595170    2558 scope.go:117] "RemoveContainer" containerID="bfc7d94c076871d719a75ddc26e14f34f90bcfd83aad0885a728cbc375608684"
Jul 24 23:57:25.595503 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:57:25.595464    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bfc7d94c076871d719a75ddc26e14f34f90bcfd83aad0885a728cbc375608684\": container with ID starting with bfc7d94c076871d719a75ddc26e14f34f90bcfd83aad0885a728cbc375608684 not found: ID does not exist" containerID="bfc7d94c076871d719a75ddc26e14f34f90bcfd83aad0885a728cbc375608684"
Jul 24 23:57:25.595619 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:25.595503    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bfc7d94c076871d719a75ddc26e14f34f90bcfd83aad0885a728cbc375608684"} err="failed to get container status \"bfc7d94c076871d719a75ddc26e14f34f90bcfd83aad0885a728cbc375608684\": rpc error: code = NotFound desc = could not find container \"bfc7d94c076871d719a75ddc26e14f34f90bcfd83aad0885a728cbc375608684\": container with ID starting with bfc7d94c076871d719a75ddc26e14f34f90bcfd83aad0885a728cbc375608684 not found: ID does not exist"
Jul 24 23:57:25.595619 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:25.595526    2558 scope.go:117] "RemoveContainer" containerID="120a912a77145eea5a195ae311ce508c953591b11271a75c0c99088fba53bc0a"
Jul 24 23:57:25.603577 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:25.603555    2558 scope.go:117] "RemoveContainer" containerID="120a912a77145eea5a195ae311ce508c953591b11271a75c0c99088fba53bc0a"
Jul 24 23:57:25.603839 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:57:25.603821    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"120a912a77145eea5a195ae311ce508c953591b11271a75c0c99088fba53bc0a\": container with ID starting with 120a912a77145eea5a195ae311ce508c953591b11271a75c0c99088fba53bc0a not found: ID does not exist" containerID="120a912a77145eea5a195ae311ce508c953591b11271a75c0c99088fba53bc0a"
Jul 24 23:57:25.603902 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:25.603850    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"120a912a77145eea5a195ae311ce508c953591b11271a75c0c99088fba53bc0a"} err="failed to get container status \"120a912a77145eea5a195ae311ce508c953591b11271a75c0c99088fba53bc0a\": rpc error: code = NotFound desc = could not find container \"120a912a77145eea5a195ae311ce508c953591b11271a75c0c99088fba53bc0a\": container with ID starting with 120a912a77145eea5a195ae311ce508c953591b11271a75c0c99088fba53bc0a not found: ID does not exist"
Jul 24 23:57:25.608506 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:25.608479    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-25f28971-59c48c7485-xfxsl"]
Jul 24 23:57:25.608755 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:25.608737    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5d165735-118a-4f85-939a-951526aab2f1-plugins-config-volume\") pod \"5d165735-118a-4f85-939a-951526aab2f1\" (UID: \"5d165735-118a-4f85-939a-951526aab2f1\") "
Jul 24 23:57:25.608809 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:25.608797    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7jcjz\" (UniqueName: \"kubernetes.io/projected/5d165735-118a-4f85-939a-951526aab2f1-kube-api-access-7jcjz\") pod \"5d165735-118a-4f85-939a-951526aab2f1\" (UID: \"5d165735-118a-4f85-939a-951526aab2f1\") "
Jul 24 23:57:25.609046 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:25.609027    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46a34598-2429-409c-8ba3-59f9d2f78e64-openshift-service-ca\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 24 23:57:25.609118 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:25.609052    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46a34598-2429-409c-8ba3-59f9d2f78e64-maas-api-tls\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 24 23:57:25.609118 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:25.609067    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tscvb\" (UniqueName: \"kubernetes.io/projected/46a34598-2429-409c-8ba3-59f9d2f78e64-kube-api-access-tscvb\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 24 23:57:25.609118 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:25.609091    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d165735-118a-4f85-939a-951526aab2f1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5d165735-118a-4f85-939a-951526aab2f1" (UID: "5d165735-118a-4f85-939a-951526aab2f1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:57:25.611020 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:25.610999    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d165735-118a-4f85-939a-951526aab2f1-kube-api-access-7jcjz" (OuterVolumeSpecName: "kube-api-access-7jcjz") pod "5d165735-118a-4f85-939a-951526aab2f1" (UID: "5d165735-118a-4f85-939a-951526aab2f1"). InnerVolumeSpecName "kube-api-access-7jcjz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:57:25.612616 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:25.612599    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-25f28971-59c48c7485-xfxsl"]
Jul 24 23:57:25.709681 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:25.709643    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7jcjz\" (UniqueName: \"kubernetes.io/projected/5d165735-118a-4f85-939a-951526aab2f1-kube-api-access-7jcjz\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 24 23:57:25.709681 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:25.709673    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5d165735-118a-4f85-939a-951526aab2f1-plugins-config-volume\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 24 23:57:25.907077 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:25.907047    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-25f28971-54cb9f97b-fvzlc"]
Jul 24 23:57:25.909235 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:25.909216    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-25f28971-54cb9f97b-fvzlc"]
Jul 24 23:57:26.124853 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:26.124822    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:57:26.205485 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:26.205394    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="46a34598-2429-409c-8ba3-59f9d2f78e64" path="/var/lib/kubelet/pods/46a34598-2429-409c-8ba3-59f9d2f78e64/volumes"
Jul 24 23:57:26.205966 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:26.205945    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d165735-118a-4f85-939a-951526aab2f1" path="/var/lib/kubelet/pods/5d165735-118a-4f85-939a-951526aab2f1/volumes"
Jul 24 23:57:29.737131 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:29.737087    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-25f28971-fbnp8"]
Jul 24 23:57:29.739275 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:29.739253    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-25f28971-fbnp8"]
Jul 24 23:57:29.751779 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:29.751752    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:57:29.892663 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:29.892626    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:57:30.154507 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:30.154414    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:57:30.204447 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:30.204403    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="46911e98-766c-4579-9690-65092c21aecc" path="/var/lib/kubelet/pods/46911e98-766c-4579-9690-65092c21aecc/volumes"
Jul 24 23:57:30.411800 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:30.411701    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:57:30.739089 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:30.739054    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:57:31.075929 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:31.075822    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:57:32.762074 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:32.762038    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:57:32.848786 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:32.848749    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:57:32.852987 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:32.852963    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0c7d785d-7cf6797964-55t9p"]
Jul 24 23:57:32.853427 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:32.853410    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46a34598-2429-409c-8ba3-59f9d2f78e64" containerName="maas-api"
Jul 24 23:57:32.853472 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:32.853431    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="46a34598-2429-409c-8ba3-59f9d2f78e64" containerName="maas-api"
Jul 24 23:57:32.853472 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:32.853443    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46911e98-766c-4579-9690-65092c21aecc" containerName="revoke-keys"
Jul 24 23:57:32.853472 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:32.853448    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="46911e98-766c-4579-9690-65092c21aecc" containerName="revoke-keys"
Jul 24 23:57:32.853472 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:32.853457    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d165735-118a-4f85-939a-951526aab2f1" containerName="payload-pre-processing"
Jul 24 23:57:32.853472 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:32.853463    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d165735-118a-4f85-939a-951526aab2f1" containerName="payload-pre-processing"
Jul 24 23:57:32.853644 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:32.853519    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d165735-118a-4f85-939a-951526aab2f1" containerName="payload-pre-processing"
Jul 24 23:57:32.853644 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:32.853544    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="46a34598-2429-409c-8ba3-59f9d2f78e64" containerName="maas-api"
Jul 24 23:57:32.853644 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:32.853556    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="46911e98-766c-4579-9690-65092c21aecc" containerName="revoke-keys"
Jul 24 23:57:32.857906 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:32.857882    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0c7d785d-7cf6797964-55t9p"
Jul 24 23:57:32.859961 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:32.859942    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-0c7d785d-serving-cert\""
Jul 24 23:57:32.867902 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:32.867878    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0c7d785d-7cf6797964-55t9p"]
Jul 24 23:57:32.869219 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:32.869195    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/602ed1b8-ecb8-46b4-a607-8b47e633deff-maas-api-tls\") pod \"maas-api-e2e-ait-0c7d785d-7cf6797964-55t9p\" (UID: \"602ed1b8-ecb8-46b4-a607-8b47e633deff\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0c7d785d-7cf6797964-55t9p"
Jul 24 23:57:32.869338 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:32.869260    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8llbt\" (UniqueName: \"kubernetes.io/projected/602ed1b8-ecb8-46b4-a607-8b47e633deff-kube-api-access-8llbt\") pod \"maas-api-e2e-ait-0c7d785d-7cf6797964-55t9p\" (UID: \"602ed1b8-ecb8-46b4-a607-8b47e633deff\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0c7d785d-7cf6797964-55t9p"
Jul 24 23:57:32.869338 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:32.869312    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/602ed1b8-ecb8-46b4-a607-8b47e633deff-openshift-service-ca\") pod \"maas-api-e2e-ait-0c7d785d-7cf6797964-55t9p\" (UID: \"602ed1b8-ecb8-46b4-a607-8b47e633deff\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0c7d785d-7cf6797964-55t9p"
Jul 24 23:57:32.909714 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:32.909666    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:57:32.970288 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:32.970258    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/602ed1b8-ecb8-46b4-a607-8b47e633deff-openshift-service-ca\") pod \"maas-api-e2e-ait-0c7d785d-7cf6797964-55t9p\" (UID: \"602ed1b8-ecb8-46b4-a607-8b47e633deff\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0c7d785d-7cf6797964-55t9p"
Jul 24 23:57:32.970474 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:32.970327    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/602ed1b8-ecb8-46b4-a607-8b47e633deff-maas-api-tls\") pod \"maas-api-e2e-ait-0c7d785d-7cf6797964-55t9p\" (UID: \"602ed1b8-ecb8-46b4-a607-8b47e633deff\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0c7d785d-7cf6797964-55t9p"
Jul 24 23:57:32.970474 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:32.970387    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8llbt\" (UniqueName: \"kubernetes.io/projected/602ed1b8-ecb8-46b4-a607-8b47e633deff-kube-api-access-8llbt\") pod \"maas-api-e2e-ait-0c7d785d-7cf6797964-55t9p\" (UID: \"602ed1b8-ecb8-46b4-a607-8b47e633deff\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0c7d785d-7cf6797964-55t9p"
Jul 24 23:57:32.970952 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:32.970927    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/602ed1b8-ecb8-46b4-a607-8b47e633deff-openshift-service-ca\") pod \"maas-api-e2e-ait-0c7d785d-7cf6797964-55t9p\" (UID: \"602ed1b8-ecb8-46b4-a607-8b47e633deff\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0c7d785d-7cf6797964-55t9p"
Jul 24 23:57:32.972755 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:32.972733    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/602ed1b8-ecb8-46b4-a607-8b47e633deff-maas-api-tls\") pod \"maas-api-e2e-ait-0c7d785d-7cf6797964-55t9p\" (UID: \"602ed1b8-ecb8-46b4-a607-8b47e633deff\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0c7d785d-7cf6797964-55t9p"
Jul 24 23:57:32.978677 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:32.978652    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8llbt\" (UniqueName: \"kubernetes.io/projected/602ed1b8-ecb8-46b4-a607-8b47e633deff-kube-api-access-8llbt\") pod \"maas-api-e2e-ait-0c7d785d-7cf6797964-55t9p\" (UID: \"602ed1b8-ecb8-46b4-a607-8b47e633deff\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0c7d785d-7cf6797964-55t9p"
Jul 24 23:57:33.009737 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:33.009694    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:57:33.169762 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:33.169664    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0c7d785d-7cf6797964-55t9p"
Jul 24 23:57:33.312170 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:33.312134    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0c7d785d-7cf6797964-55t9p"]
Jul 24 23:57:33.313800 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:57:33.313773    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod602ed1b8_ecb8_46b4_a607_8b47e633deff.slice/crio-0efb483b0b934232eaf3b3c51661777f5f6e8100898a1f220b45282ed84437b0 WatchSource:0}: Error finding container 0efb483b0b934232eaf3b3c51661777f5f6e8100898a1f220b45282ed84437b0: Status 404 returned error can't find the container with id 0efb483b0b934232eaf3b3c51661777f5f6e8100898a1f220b45282ed84437b0
Jul 24 23:57:33.509561 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:33.509512    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:57:33.610192 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:33.610160    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:57:33.621965 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:33.619225    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-0c7d785d-848b6c7f5b-8pt7f"]
Jul 24 23:57:33.625199 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:33.625168    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-0c7d785d-848b6c7f5b-8pt7f"
Jul 24 23:57:33.627648 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:33.627616    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-0c7d785d-dockercfg-lrqlg\""
Jul 24 23:57:33.627818 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:33.627616    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-0c7d785d\""
Jul 24 23:57:33.631032 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:33.631006    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-0c7d785d-848b6c7f5b-8pt7f"]
Jul 24 23:57:33.633027 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:33.633004    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0c7d785d-7cf6797964-55t9p" event={"ID":"602ed1b8-ecb8-46b4-a607-8b47e633deff","Type":"ContainerStarted","Data":"0efb483b0b934232eaf3b3c51661777f5f6e8100898a1f220b45282ed84437b0"}
Jul 24 23:57:33.678190 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:33.678155    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2w7q\" (UniqueName: \"kubernetes.io/projected/1c4a828f-8fb5-46e0-ba14-9a85df6838e5-kube-api-access-j2w7q\") pod \"payload-pre-processing-e2e-ait-0c7d785d-848b6c7f5b-8pt7f\" (UID: \"1c4a828f-8fb5-46e0-ba14-9a85df6838e5\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-0c7d785d-848b6c7f5b-8pt7f"
Jul 24 23:57:33.678370 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:33.678209    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c4a828f-8fb5-46e0-ba14-9a85df6838e5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-0c7d785d-848b6c7f5b-8pt7f\" (UID: \"1c4a828f-8fb5-46e0-ba14-9a85df6838e5\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-0c7d785d-848b6c7f5b-8pt7f"
Jul 24 23:57:33.778907 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:33.778823    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c4a828f-8fb5-46e0-ba14-9a85df6838e5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-0c7d785d-848b6c7f5b-8pt7f\" (UID: \"1c4a828f-8fb5-46e0-ba14-9a85df6838e5\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-0c7d785d-848b6c7f5b-8pt7f"
Jul 24 23:57:33.779291 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:33.778946    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2w7q\" (UniqueName: \"kubernetes.io/projected/1c4a828f-8fb5-46e0-ba14-9a85df6838e5-kube-api-access-j2w7q\") pod \"payload-pre-processing-e2e-ait-0c7d785d-848b6c7f5b-8pt7f\" (UID: \"1c4a828f-8fb5-46e0-ba14-9a85df6838e5\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-0c7d785d-848b6c7f5b-8pt7f"
Jul 24 23:57:33.779519 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:33.779497    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c4a828f-8fb5-46e0-ba14-9a85df6838e5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-0c7d785d-848b6c7f5b-8pt7f\" (UID: \"1c4a828f-8fb5-46e0-ba14-9a85df6838e5\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-0c7d785d-848b6c7f5b-8pt7f"
Jul 24 23:57:33.790304 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:33.790277    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2w7q\" (UniqueName: \"kubernetes.io/projected/1c4a828f-8fb5-46e0-ba14-9a85df6838e5-kube-api-access-j2w7q\") pod \"payload-pre-processing-e2e-ait-0c7d785d-848b6c7f5b-8pt7f\" (UID: \"1c4a828f-8fb5-46e0-ba14-9a85df6838e5\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-0c7d785d-848b6c7f5b-8pt7f"
Jul 24 23:57:33.939917 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:33.939875    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-0c7d785d-848b6c7f5b-8pt7f"
Jul 24 23:57:34.074618 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:34.074588    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-0c7d785d-848b6c7f5b-8pt7f"]
Jul 24 23:57:34.076255 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:57:34.076213    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c4a828f_8fb5_46e0_ba14_9a85df6838e5.slice/crio-9418615d946f6401ec8598974046a4ed26d8373f5bcde52a034c6107292a827d WatchSource:0}: Error finding container 9418615d946f6401ec8598974046a4ed26d8373f5bcde52a034c6107292a827d: Status 404 returned error can't find the container with id 9418615d946f6401ec8598974046a4ed26d8373f5bcde52a034c6107292a827d
Jul 24 23:57:34.640318 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:34.640272    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0c7d785d-7cf6797964-55t9p" event={"ID":"602ed1b8-ecb8-46b4-a607-8b47e633deff","Type":"ContainerStarted","Data":"41db9a78a020e328cc0e129332b9cb213c93b88dcc36431f610bef42e050882f"}
Jul 24 23:57:34.640500 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:34.640390    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0c7d785d-7cf6797964-55t9p"
Jul 24 23:57:34.641852 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:34.641829    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-0c7d785d-848b6c7f5b-8pt7f" event={"ID":"1c4a828f-8fb5-46e0-ba14-9a85df6838e5","Type":"ContainerStarted","Data":"bb3ce745dd9515a6f8d52e91b408f5fd4da9c32f269be8cc79fdd6b779bce27e"}
Jul 24 23:57:34.641961 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:34.641863    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-0c7d785d-848b6c7f5b-8pt7f" event={"ID":"1c4a828f-8fb5-46e0-ba14-9a85df6838e5","Type":"ContainerStarted","Data":"9418615d946f6401ec8598974046a4ed26d8373f5bcde52a034c6107292a827d"}
Jul 24 23:57:34.641961 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:34.641892    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-0c7d785d-848b6c7f5b-8pt7f"
Jul 24 23:57:34.656926 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:34.656881    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0c7d785d-7cf6797964-55t9p" podStartSLOduration=2.227534481 podStartE2EDuration="2.656867496s" totalImagesPullingTime="429.333015ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:57:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:57:34.656004832 +0000 UTC m=+1695.042435562" watchObservedRunningTime="2026-07-24 23:57:34.656867496 +0000 UTC m=+1695.043298224"
Jul 24 23:57:34.672835 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:34.672787    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-0c7d785d-848b6c7f5b-8pt7f" podStartSLOduration=1.6727715459999999 podStartE2EDuration="1.672771546s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:57:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:57:34.672111582 +0000 UTC m=+1695.058542310" watchObservedRunningTime="2026-07-24 23:57:34.672771546 +0000 UTC m=+1695.059202274"
Jul 24 23:57:40.652946 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:40.652915    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0c7d785d-7cf6797964-55t9p"
Jul 24 23:57:45.649676 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:45.649594    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-0c7d785d-848b6c7f5b-8pt7f"
Jul 24 23:57:58.696256 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:58.696219    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c7d785d-l9f7z"]
Jul 24 23:57:58.699746 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:58.699727    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c7d785d-l9f7z"
Jul 24 23:57:58.702031 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:58.702007    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bxr46\""
Jul 24 23:57:58.706662 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:58.706639    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c7d785d-l9f7z"]
Jul 24 23:57:58.801631 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:58.801592    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/71edc1ef-1e4f-4f27-b482-50c3159b714b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0c7d785d-l9f7z\" (UID: \"71edc1ef-1e4f-4f27-b482-50c3159b714b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c7d785d-l9f7z"
Jul 24 23:57:58.902596 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:58.902560    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/71edc1ef-1e4f-4f27-b482-50c3159b714b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0c7d785d-l9f7z\" (UID: \"71edc1ef-1e4f-4f27-b482-50c3159b714b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c7d785d-l9f7z"
Jul 24 23:57:58.903265 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:58.903240    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/71edc1ef-1e4f-4f27-b482-50c3159b714b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0c7d785d-l9f7z\" (UID: \"71edc1ef-1e4f-4f27-b482-50c3159b714b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c7d785d-l9f7z"
Jul 24 23:57:59.012320 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:59.012284    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c7d785d-l9f7z"
Jul 24 23:57:59.147129 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:59.147099    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c7d785d-l9f7z"]
Jul 24 23:57:59.148799 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:57:59.148766    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71edc1ef_1e4f_4f27_b482_50c3159b714b.slice/crio-fec7be14b9221d4588bb5e4cd1362f19541ac03e8122290b1478165ad973a0c2 WatchSource:0}: Error finding container fec7be14b9221d4588bb5e4cd1362f19541ac03e8122290b1478165ad973a0c2: Status 404 returned error can't find the container with id fec7be14b9221d4588bb5e4cd1362f19541ac03e8122290b1478165ad973a0c2
Jul 24 23:57:59.781351 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:59.781311    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c7d785d-l9f7z" event={"ID":"71edc1ef-1e4f-4f27-b482-50c3159b714b","Type":"ContainerStarted","Data":"84db80ba39035df403d0de589328fddc88a45b5cdf08c13f632c31290f73be59"}
Jul 24 23:57:59.781783 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:59.781375    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c7d785d-l9f7z" event={"ID":"71edc1ef-1e4f-4f27-b482-50c3159b714b","Type":"ContainerStarted","Data":"fec7be14b9221d4588bb5e4cd1362f19541ac03e8122290b1478165ad973a0c2"}
Jul 24 23:57:59.796205 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:57:59.796153    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c7d785d-l9f7z" podStartSLOduration=1.796136626 podStartE2EDuration="1.796136626s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:57:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:57:59.794727287 +0000 UTC m=+1720.181158016" watchObservedRunningTime="2026-07-24 23:57:59.796136626 +0000 UTC m=+1720.182567353"
Jul 24 23:58:00.788421 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:00.788379    2558 generic.go:356] "Generic (PLEG): container finished" podID="71edc1ef-1e4f-4f27-b482-50c3159b714b" containerID="84db80ba39035df403d0de589328fddc88a45b5cdf08c13f632c31290f73be59" exitCode=0
Jul 24 23:58:00.788867 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:00.788434    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c7d785d-l9f7z" event={"ID":"71edc1ef-1e4f-4f27-b482-50c3159b714b","Type":"ContainerDied","Data":"84db80ba39035df403d0de589328fddc88a45b5cdf08c13f632c31290f73be59"}
Jul 24 23:58:01.926847 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:01.926818    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c7d785d-l9f7z"
Jul 24 23:58:02.031698 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:02.031649    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/71edc1ef-1e4f-4f27-b482-50c3159b714b-maas-api-service-ca\") pod \"71edc1ef-1e4f-4f27-b482-50c3159b714b\" (UID: \"71edc1ef-1e4f-4f27-b482-50c3159b714b\") "
Jul 24 23:58:02.032081 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:02.032046    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71edc1ef-1e4f-4f27-b482-50c3159b714b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "71edc1ef-1e4f-4f27-b482-50c3159b714b" (UID: "71edc1ef-1e4f-4f27-b482-50c3159b714b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:58:02.133331 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:02.133239    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/71edc1ef-1e4f-4f27-b482-50c3159b714b-maas-api-service-ca\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 24 23:58:02.800688 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:02.800650    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c7d785d-l9f7z"
Jul 24 23:58:02.800688 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:02.800662    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c7d785d-l9f7z" event={"ID":"71edc1ef-1e4f-4f27-b482-50c3159b714b","Type":"ContainerDied","Data":"fec7be14b9221d4588bb5e4cd1362f19541ac03e8122290b1478165ad973a0c2"}
Jul 24 23:58:02.800909 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:02.800718    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fec7be14b9221d4588bb5e4cd1362f19541ac03e8122290b1478165ad973a0c2"
Jul 24 23:58:09.205115 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:09.205082    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0c7d785d-7cf6797964-55t9p"]
Jul 24 23:58:09.205523 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:09.205258    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0c7d785d-7cf6797964-55t9p" podUID="602ed1b8-ecb8-46b4-a607-8b47e633deff" containerName="maas-api" containerID="cri-o://41db9a78a020e328cc0e129332b9cb213c93b88dcc36431f610bef42e050882f" gracePeriod=30
Jul 24 23:58:09.214855 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:09.214802    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:58:09.257160 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:09.257116    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-0c7d785d-848b6c7f5b-8pt7f"]
Jul 24 23:58:09.257451 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:09.257418    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-0c7d785d-848b6c7f5b-8pt7f" podUID="1c4a828f-8fb5-46e0-ba14-9a85df6838e5" containerName="payload-pre-processing" containerID="cri-o://bb3ce745dd9515a6f8d52e91b408f5fd4da9c32f269be8cc79fdd6b779bce27e" gracePeriod=30
Jul 24 23:58:09.315212 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:09.315159    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:58:09.415388 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:09.415107    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:58:09.510654 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:09.510629    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0c7d785d-7cf6797964-55t9p"
Jul 24 23:58:09.538277 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:09.538254    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-0c7d785d-848b6c7f5b-8pt7f"
Jul 24 23:58:09.601729 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:09.601696    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/602ed1b8-ecb8-46b4-a607-8b47e633deff-maas-api-tls\") pod \"602ed1b8-ecb8-46b4-a607-8b47e633deff\" (UID: \"602ed1b8-ecb8-46b4-a607-8b47e633deff\") "
Jul 24 23:58:09.601891 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:09.601744    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/602ed1b8-ecb8-46b4-a607-8b47e633deff-openshift-service-ca\") pod \"602ed1b8-ecb8-46b4-a607-8b47e633deff\" (UID: \"602ed1b8-ecb8-46b4-a607-8b47e633deff\") "
Jul 24 23:58:09.601891 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:09.601782    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8llbt\" (UniqueName: \"kubernetes.io/projected/602ed1b8-ecb8-46b4-a607-8b47e633deff-kube-api-access-8llbt\") pod \"602ed1b8-ecb8-46b4-a607-8b47e633deff\" (UID: \"602ed1b8-ecb8-46b4-a607-8b47e633deff\") "
Jul 24 23:58:09.602135 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:09.602107    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/602ed1b8-ecb8-46b4-a607-8b47e633deff-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "602ed1b8-ecb8-46b4-a607-8b47e633deff" (UID: "602ed1b8-ecb8-46b4-a607-8b47e633deff"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:58:09.603848 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:09.603822    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/602ed1b8-ecb8-46b4-a607-8b47e633deff-kube-api-access-8llbt" (OuterVolumeSpecName: "kube-api-access-8llbt") pod "602ed1b8-ecb8-46b4-a607-8b47e633deff" (UID: "602ed1b8-ecb8-46b4-a607-8b47e633deff"). InnerVolumeSpecName "kube-api-access-8llbt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:58:09.603848 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:09.603830    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/602ed1b8-ecb8-46b4-a607-8b47e633deff-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "602ed1b8-ecb8-46b4-a607-8b47e633deff" (UID: "602ed1b8-ecb8-46b4-a607-8b47e633deff"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 23:58:09.702932 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:09.702902    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j2w7q\" (UniqueName: \"kubernetes.io/projected/1c4a828f-8fb5-46e0-ba14-9a85df6838e5-kube-api-access-j2w7q\") pod \"1c4a828f-8fb5-46e0-ba14-9a85df6838e5\" (UID: \"1c4a828f-8fb5-46e0-ba14-9a85df6838e5\") "
Jul 24 23:58:09.703064 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:09.702976    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c4a828f-8fb5-46e0-ba14-9a85df6838e5-plugins-config-volume\") pod \"1c4a828f-8fb5-46e0-ba14-9a85df6838e5\" (UID: \"1c4a828f-8fb5-46e0-ba14-9a85df6838e5\") "
Jul 24 23:58:09.703178 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:09.703166    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/602ed1b8-ecb8-46b4-a607-8b47e633deff-maas-api-tls\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 24 23:58:09.703217 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:09.703183    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/602ed1b8-ecb8-46b4-a607-8b47e633deff-openshift-service-ca\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 24 23:58:09.703217 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:09.703198    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8llbt\" (UniqueName: \"kubernetes.io/projected/602ed1b8-ecb8-46b4-a607-8b47e633deff-kube-api-access-8llbt\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 24 23:58:09.703312 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:09.703291    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c4a828f-8fb5-46e0-ba14-9a85df6838e5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1c4a828f-8fb5-46e0-ba14-9a85df6838e5" (UID: "1c4a828f-8fb5-46e0-ba14-9a85df6838e5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:58:09.704951 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:09.704932    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c4a828f-8fb5-46e0-ba14-9a85df6838e5-kube-api-access-j2w7q" (OuterVolumeSpecName: "kube-api-access-j2w7q") pod "1c4a828f-8fb5-46e0-ba14-9a85df6838e5" (UID: "1c4a828f-8fb5-46e0-ba14-9a85df6838e5"). InnerVolumeSpecName "kube-api-access-j2w7q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:58:09.804569 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:09.804515    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j2w7q\" (UniqueName: \"kubernetes.io/projected/1c4a828f-8fb5-46e0-ba14-9a85df6838e5-kube-api-access-j2w7q\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 24 23:58:09.804569 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:09.804571    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c4a828f-8fb5-46e0-ba14-9a85df6838e5-plugins-config-volume\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 24 23:58:09.841413 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:09.841382    2558 generic.go:356] "Generic (PLEG): container finished" podID="602ed1b8-ecb8-46b4-a607-8b47e633deff" containerID="41db9a78a020e328cc0e129332b9cb213c93b88dcc36431f610bef42e050882f" exitCode=0
Jul 24 23:58:09.841604 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:09.841432    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0c7d785d-7cf6797964-55t9p"
Jul 24 23:58:09.841604 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:09.841456    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0c7d785d-7cf6797964-55t9p" event={"ID":"602ed1b8-ecb8-46b4-a607-8b47e633deff","Type":"ContainerDied","Data":"41db9a78a020e328cc0e129332b9cb213c93b88dcc36431f610bef42e050882f"}
Jul 24 23:58:09.841604 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:09.841507    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0c7d785d-7cf6797964-55t9p" event={"ID":"602ed1b8-ecb8-46b4-a607-8b47e633deff","Type":"ContainerDied","Data":"0efb483b0b934232eaf3b3c51661777f5f6e8100898a1f220b45282ed84437b0"}
Jul 24 23:58:09.841604 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:09.841568    2558 scope.go:117] "RemoveContainer" containerID="41db9a78a020e328cc0e129332b9cb213c93b88dcc36431f610bef42e050882f"
Jul 24 23:58:09.842829 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:09.842804    2558 generic.go:356] "Generic (PLEG): container finished" podID="1c4a828f-8fb5-46e0-ba14-9a85df6838e5" containerID="bb3ce745dd9515a6f8d52e91b408f5fd4da9c32f269be8cc79fdd6b779bce27e" exitCode=0
Jul 24 23:58:09.842961 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:09.842842    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-0c7d785d-848b6c7f5b-8pt7f" event={"ID":"1c4a828f-8fb5-46e0-ba14-9a85df6838e5","Type":"ContainerDied","Data":"bb3ce745dd9515a6f8d52e91b408f5fd4da9c32f269be8cc79fdd6b779bce27e"}
Jul 24 23:58:09.842961 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:09.842860    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-0c7d785d-848b6c7f5b-8pt7f"
Jul 24 23:58:09.842961 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:09.842897    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-0c7d785d-848b6c7f5b-8pt7f" event={"ID":"1c4a828f-8fb5-46e0-ba14-9a85df6838e5","Type":"ContainerDied","Data":"9418615d946f6401ec8598974046a4ed26d8373f5bcde52a034c6107292a827d"}
Jul 24 23:58:09.852656 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:09.852640    2558 scope.go:117] "RemoveContainer" containerID="41db9a78a020e328cc0e129332b9cb213c93b88dcc36431f610bef42e050882f"
Jul 24 23:58:09.852989 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:58:09.852962    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41db9a78a020e328cc0e129332b9cb213c93b88dcc36431f610bef42e050882f\": container with ID starting with 41db9a78a020e328cc0e129332b9cb213c93b88dcc36431f610bef42e050882f not found: ID does not exist" containerID="41db9a78a020e328cc0e129332b9cb213c93b88dcc36431f610bef42e050882f"
Jul 24 23:58:09.853077 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:09.853002    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41db9a78a020e328cc0e129332b9cb213c93b88dcc36431f610bef42e050882f"} err="failed to get container status \"41db9a78a020e328cc0e129332b9cb213c93b88dcc36431f610bef42e050882f\": rpc error: code = NotFound desc = could not find container \"41db9a78a020e328cc0e129332b9cb213c93b88dcc36431f610bef42e050882f\": container with ID starting with 41db9a78a020e328cc0e129332b9cb213c93b88dcc36431f610bef42e050882f not found: ID does not exist"
Jul 24 23:58:09.853077 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:09.853026    2558 scope.go:117] "RemoveContainer" containerID="bb3ce745dd9515a6f8d52e91b408f5fd4da9c32f269be8cc79fdd6b779bce27e"
Jul 24 23:58:09.860992 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:09.860973    2558 scope.go:117] "RemoveContainer" containerID="bb3ce745dd9515a6f8d52e91b408f5fd4da9c32f269be8cc79fdd6b779bce27e"
Jul 24 23:58:09.861242 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:58:09.861222    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb3ce745dd9515a6f8d52e91b408f5fd4da9c32f269be8cc79fdd6b779bce27e\": container with ID starting with bb3ce745dd9515a6f8d52e91b408f5fd4da9c32f269be8cc79fdd6b779bce27e not found: ID does not exist" containerID="bb3ce745dd9515a6f8d52e91b408f5fd4da9c32f269be8cc79fdd6b779bce27e"
Jul 24 23:58:09.861302 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:09.861252    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb3ce745dd9515a6f8d52e91b408f5fd4da9c32f269be8cc79fdd6b779bce27e"} err="failed to get container status \"bb3ce745dd9515a6f8d52e91b408f5fd4da9c32f269be8cc79fdd6b779bce27e\": rpc error: code = NotFound desc = could not find container \"bb3ce745dd9515a6f8d52e91b408f5fd4da9c32f269be8cc79fdd6b779bce27e\": container with ID starting with bb3ce745dd9515a6f8d52e91b408f5fd4da9c32f269be8cc79fdd6b779bce27e not found: ID does not exist"
Jul 24 23:58:09.865626 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:09.865603    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0c7d785d-7cf6797964-55t9p"]
Jul 24 23:58:09.869127 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:09.869108    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0c7d785d-7cf6797964-55t9p"]
Jul 24 23:58:09.877853 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:09.877829    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-0c7d785d-848b6c7f5b-8pt7f"]
Jul 24 23:58:09.881143 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:09.881124    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-0c7d785d-848b6c7f5b-8pt7f"]
Jul 24 23:58:10.204451 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:10.204374    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c4a828f-8fb5-46e0-ba14-9a85df6838e5" path="/var/lib/kubelet/pods/1c4a828f-8fb5-46e0-ba14-9a85df6838e5/volumes"
Jul 24 23:58:10.204796 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:10.204781    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="602ed1b8-ecb8-46b4-a607-8b47e633deff" path="/var/lib/kubelet/pods/602ed1b8-ecb8-46b4-a607-8b47e633deff/volumes"
Jul 24 23:58:13.759454 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:13.759411    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:58:13.764470 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:13.764429    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c7d785d-l9f7z"]
Jul 24 23:58:13.768683 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:13.768660    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0c7d785d-l9f7z"]
Jul 24 23:58:14.204092 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:14.204017    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71edc1ef-1e4f-4f27-b482-50c3159b714b" path="/var/lib/kubelet/pods/71edc1ef-1e4f-4f27-b482-50c3159b714b/volumes"
Jul 24 23:58:15.015966 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:15.015929    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:58:15.244884 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:15.244846    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:58:15.475024 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:15.474937    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:58:15.803079 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:15.803044    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:58:15.926273 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:15.926243    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:58:18.005566 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:18.005502    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:58:18.093731 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:18.093695    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:58:18.093927 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:18.093896    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-9ffa79f3-59d65f9b8d-mjwnm"]
Jul 24 23:58:18.094294 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:18.094278    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71edc1ef-1e4f-4f27-b482-50c3159b714b" containerName="revoke-keys"
Jul 24 23:58:18.094294 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:18.094296    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="71edc1ef-1e4f-4f27-b482-50c3159b714b" containerName="revoke-keys"
Jul 24 23:58:18.094439 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:18.094323    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c4a828f-8fb5-46e0-ba14-9a85df6838e5" containerName="payload-pre-processing"
Jul 24 23:58:18.094439 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:18.094332    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c4a828f-8fb5-46e0-ba14-9a85df6838e5" containerName="payload-pre-processing"
Jul 24 23:58:18.094439 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:18.094365    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="602ed1b8-ecb8-46b4-a607-8b47e633deff" containerName="maas-api"
Jul 24 23:58:18.094439 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:18.094373    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="602ed1b8-ecb8-46b4-a607-8b47e633deff" containerName="maas-api"
Jul 24 23:58:18.094663 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:18.094456    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="602ed1b8-ecb8-46b4-a607-8b47e633deff" containerName="maas-api"
Jul 24 23:58:18.094663 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:18.094473    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c4a828f-8fb5-46e0-ba14-9a85df6838e5" containerName="payload-pre-processing"
Jul 24 23:58:18.094663 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:18.094485    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="71edc1ef-1e4f-4f27-b482-50c3159b714b" containerName="revoke-keys"
Jul 24 23:58:18.099095 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:18.099076    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9ffa79f3-59d65f9b8d-mjwnm"
Jul 24 23:58:18.101332 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:18.101311    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-9ffa79f3-serving-cert\""
Jul 24 23:58:18.108267 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:18.108243    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-9ffa79f3-59d65f9b8d-mjwnm"]
Jul 24 23:58:18.151666 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:18.151628    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:58:18.173772 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:18.173733    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w27sq\" (UniqueName: \"kubernetes.io/projected/354fc7be-3fcd-4fe8-acfe-bf279071c654-kube-api-access-w27sq\") pod \"maas-api-e2e-derive-9ffa79f3-59d65f9b8d-mjwnm\" (UID: \"354fc7be-3fcd-4fe8-acfe-bf279071c654\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9ffa79f3-59d65f9b8d-mjwnm"
Jul 24 23:58:18.173936 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:18.173811    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/354fc7be-3fcd-4fe8-acfe-bf279071c654-maas-api-tls\") pod \"maas-api-e2e-derive-9ffa79f3-59d65f9b8d-mjwnm\" (UID: \"354fc7be-3fcd-4fe8-acfe-bf279071c654\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9ffa79f3-59d65f9b8d-mjwnm"
Jul 24 23:58:18.173936 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:18.173872    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/354fc7be-3fcd-4fe8-acfe-bf279071c654-openshift-service-ca\") pod \"maas-api-e2e-derive-9ffa79f3-59d65f9b8d-mjwnm\" (UID: \"354fc7be-3fcd-4fe8-acfe-bf279071c654\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9ffa79f3-59d65f9b8d-mjwnm"
Jul 24 23:58:18.275112 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:18.275025    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w27sq\" (UniqueName: \"kubernetes.io/projected/354fc7be-3fcd-4fe8-acfe-bf279071c654-kube-api-access-w27sq\") pod \"maas-api-e2e-derive-9ffa79f3-59d65f9b8d-mjwnm\" (UID: \"354fc7be-3fcd-4fe8-acfe-bf279071c654\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9ffa79f3-59d65f9b8d-mjwnm"
Jul 24 23:58:18.275112 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:18.275076    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/354fc7be-3fcd-4fe8-acfe-bf279071c654-maas-api-tls\") pod \"maas-api-e2e-derive-9ffa79f3-59d65f9b8d-mjwnm\" (UID: \"354fc7be-3fcd-4fe8-acfe-bf279071c654\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9ffa79f3-59d65f9b8d-mjwnm"
Jul 24 23:58:18.275291 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:18.275118    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/354fc7be-3fcd-4fe8-acfe-bf279071c654-openshift-service-ca\") pod \"maas-api-e2e-derive-9ffa79f3-59d65f9b8d-mjwnm\" (UID: \"354fc7be-3fcd-4fe8-acfe-bf279071c654\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9ffa79f3-59d65f9b8d-mjwnm"
Jul 24 23:58:18.275291 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:58:18.275245    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-9ffa79f3-serving-cert: secret "maas-api-e2e-derive-9ffa79f3-serving-cert" not found
Jul 24 23:58:18.275371 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:58:18.275310    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/354fc7be-3fcd-4fe8-acfe-bf279071c654-maas-api-tls podName:354fc7be-3fcd-4fe8-acfe-bf279071c654 nodeName:}" failed. No retries permitted until 2026-07-24 23:58:18.77529295 +0000 UTC m=+1739.161723868 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/354fc7be-3fcd-4fe8-acfe-bf279071c654-maas-api-tls") pod "maas-api-e2e-derive-9ffa79f3-59d65f9b8d-mjwnm" (UID: "354fc7be-3fcd-4fe8-acfe-bf279071c654") : secret "maas-api-e2e-derive-9ffa79f3-serving-cert" not found
Jul 24 23:58:18.275718 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:18.275700    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/354fc7be-3fcd-4fe8-acfe-bf279071c654-openshift-service-ca\") pod \"maas-api-e2e-derive-9ffa79f3-59d65f9b8d-mjwnm\" (UID: \"354fc7be-3fcd-4fe8-acfe-bf279071c654\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9ffa79f3-59d65f9b8d-mjwnm"
Jul 24 23:58:18.284506 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:18.284478    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w27sq\" (UniqueName: \"kubernetes.io/projected/354fc7be-3fcd-4fe8-acfe-bf279071c654-kube-api-access-w27sq\") pod \"maas-api-e2e-derive-9ffa79f3-59d65f9b8d-mjwnm\" (UID: \"354fc7be-3fcd-4fe8-acfe-bf279071c654\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9ffa79f3-59d65f9b8d-mjwnm"
Jul 24 23:58:18.347871 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:18.347841    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:58:18.780299 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:18.780268    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/354fc7be-3fcd-4fe8-acfe-bf279071c654-maas-api-tls\") pod \"maas-api-e2e-derive-9ffa79f3-59d65f9b8d-mjwnm\" (UID: \"354fc7be-3fcd-4fe8-acfe-bf279071c654\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9ffa79f3-59d65f9b8d-mjwnm"
Jul 24 23:58:18.782676 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:18.782655    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/354fc7be-3fcd-4fe8-acfe-bf279071c654-maas-api-tls\") pod \"maas-api-e2e-derive-9ffa79f3-59d65f9b8d-mjwnm\" (UID: \"354fc7be-3fcd-4fe8-acfe-bf279071c654\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9ffa79f3-59d65f9b8d-mjwnm"
Jul 24 23:58:18.858948 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:18.858919    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:58:18.959521 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:18.959486    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:58:18.960940 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:18.960918    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-9ffa79f3-666fb9f8d8-bqvrk"]
Jul 24 23:58:18.964494 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:18.964472    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-9ffa79f3-666fb9f8d8-bqvrk"
Jul 24 23:58:18.966757 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:18.966733    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-9ffa79f3-dockercfg-w62mg\""
Jul 24 23:58:18.966895 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:18.966776    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-9ffa79f3\""
Jul 24 23:58:18.973491 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:18.973464    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-9ffa79f3-666fb9f8d8-bqvrk"]
Jul 24 23:58:18.983089 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:18.983057    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/afee1ac5-e711-4129-92cc-1e41a54b846d-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-9ffa79f3-666fb9f8d8-bqvrk\" (UID: \"afee1ac5-e711-4129-92cc-1e41a54b846d\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-9ffa79f3-666fb9f8d8-bqvrk"
Jul 24 23:58:18.983213 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:18.983099    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbh62\" (UniqueName: \"kubernetes.io/projected/afee1ac5-e711-4129-92cc-1e41a54b846d-kube-api-access-cbh62\") pod \"payload-pre-processing-e2e-derive-9ffa79f3-666fb9f8d8-bqvrk\" (UID: \"afee1ac5-e711-4129-92cc-1e41a54b846d\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-9ffa79f3-666fb9f8d8-bqvrk"
Jul 24 23:58:18.994967 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:18.994940    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-9ffa79f3-7d879dbdb6-bcfnt"]
Jul 24 23:58:18.999113 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:18.999092    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-9ffa79f3-7d879dbdb6-bcfnt"
Jul 24 23:58:19.006439 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:19.006411    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-9ffa79f3-7d879dbdb6-bcfnt"]
Jul 24 23:58:19.010359 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:19.010337    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9ffa79f3-59d65f9b8d-mjwnm"
Jul 24 23:58:19.087029 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:19.086974    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9s7q8\" (UniqueName: \"kubernetes.io/projected/f1edcf2a-75ec-4715-97e6-a42a56d96c7e-kube-api-access-9s7q8\") pod \"payload-processing-e2e-derive-9ffa79f3-7d879dbdb6-bcfnt\" (UID: \"f1edcf2a-75ec-4715-97e6-a42a56d96c7e\") " pod="openshift-ingress/payload-processing-e2e-derive-9ffa79f3-7d879dbdb6-bcfnt"
Jul 24 23:58:19.087029 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:19.087020    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/afee1ac5-e711-4129-92cc-1e41a54b846d-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-9ffa79f3-666fb9f8d8-bqvrk\" (UID: \"afee1ac5-e711-4129-92cc-1e41a54b846d\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-9ffa79f3-666fb9f8d8-bqvrk"
Jul 24 23:58:19.087271 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:19.087038    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbh62\" (UniqueName: \"kubernetes.io/projected/afee1ac5-e711-4129-92cc-1e41a54b846d-kube-api-access-cbh62\") pod \"payload-pre-processing-e2e-derive-9ffa79f3-666fb9f8d8-bqvrk\" (UID: \"afee1ac5-e711-4129-92cc-1e41a54b846d\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-9ffa79f3-666fb9f8d8-bqvrk"
Jul 24 23:58:19.087271 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:19.087106    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1edcf2a-75ec-4715-97e6-a42a56d96c7e-plugins-config-volume\") pod \"payload-processing-e2e-derive-9ffa79f3-7d879dbdb6-bcfnt\" (UID: \"f1edcf2a-75ec-4715-97e6-a42a56d96c7e\") " pod="openshift-ingress/payload-processing-e2e-derive-9ffa79f3-7d879dbdb6-bcfnt"
Jul 24 23:58:19.088408 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:19.088383    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/afee1ac5-e711-4129-92cc-1e41a54b846d-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-9ffa79f3-666fb9f8d8-bqvrk\" (UID: \"afee1ac5-e711-4129-92cc-1e41a54b846d\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-9ffa79f3-666fb9f8d8-bqvrk"
Jul 24 23:58:19.095148 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:19.095123    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbh62\" (UniqueName: \"kubernetes.io/projected/afee1ac5-e711-4129-92cc-1e41a54b846d-kube-api-access-cbh62\") pod \"payload-pre-processing-e2e-derive-9ffa79f3-666fb9f8d8-bqvrk\" (UID: \"afee1ac5-e711-4129-92cc-1e41a54b846d\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-9ffa79f3-666fb9f8d8-bqvrk"
Jul 24 23:58:19.157796 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:19.157766    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-9ffa79f3-59d65f9b8d-mjwnm"]
Jul 24 23:58:19.159208 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:58:19.159181    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod354fc7be_3fcd_4fe8_acfe_bf279071c654.slice/crio-9456db7f62b64600e61f8176e2c0f07ad8109cbfc9d856fb452c03fe2bd09648 WatchSource:0}: Error finding container 9456db7f62b64600e61f8176e2c0f07ad8109cbfc9d856fb452c03fe2bd09648: Status 404 returned error can't find the container with id 9456db7f62b64600e61f8176e2c0f07ad8109cbfc9d856fb452c03fe2bd09648
Jul 24 23:58:19.188638 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:19.188600    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1edcf2a-75ec-4715-97e6-a42a56d96c7e-plugins-config-volume\") pod \"payload-processing-e2e-derive-9ffa79f3-7d879dbdb6-bcfnt\" (UID: \"f1edcf2a-75ec-4715-97e6-a42a56d96c7e\") " pod="openshift-ingress/payload-processing-e2e-derive-9ffa79f3-7d879dbdb6-bcfnt"
Jul 24 23:58:19.188791 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:19.188734    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9s7q8\" (UniqueName: \"kubernetes.io/projected/f1edcf2a-75ec-4715-97e6-a42a56d96c7e-kube-api-access-9s7q8\") pod \"payload-processing-e2e-derive-9ffa79f3-7d879dbdb6-bcfnt\" (UID: \"f1edcf2a-75ec-4715-97e6-a42a56d96c7e\") " pod="openshift-ingress/payload-processing-e2e-derive-9ffa79f3-7d879dbdb6-bcfnt"
Jul 24 23:58:19.189336 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:19.189320    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1edcf2a-75ec-4715-97e6-a42a56d96c7e-plugins-config-volume\") pod \"payload-processing-e2e-derive-9ffa79f3-7d879dbdb6-bcfnt\" (UID: \"f1edcf2a-75ec-4715-97e6-a42a56d96c7e\") " pod="openshift-ingress/payload-processing-e2e-derive-9ffa79f3-7d879dbdb6-bcfnt"
Jul 24 23:58:19.196288 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:19.196257    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9s7q8\" (UniqueName: \"kubernetes.io/projected/f1edcf2a-75ec-4715-97e6-a42a56d96c7e-kube-api-access-9s7q8\") pod \"payload-processing-e2e-derive-9ffa79f3-7d879dbdb6-bcfnt\" (UID: \"f1edcf2a-75ec-4715-97e6-a42a56d96c7e\") " pod="openshift-ingress/payload-processing-e2e-derive-9ffa79f3-7d879dbdb6-bcfnt"
Jul 24 23:58:19.277248 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:19.277208    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-9ffa79f3-666fb9f8d8-bqvrk"
Jul 24 23:58:19.312457 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:19.312378    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-9ffa79f3-7d879dbdb6-bcfnt"
Jul 24 23:58:19.431511 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:19.431480    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-9ffa79f3-666fb9f8d8-bqvrk"]
Jul 24 23:58:19.433088 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:58:19.433049    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafee1ac5_e711_4129_92cc_1e41a54b846d.slice/crio-dc191a7f539c56ef8fc06023e72d4ae382edf9b295b6cce3ae9c3f852621b54f WatchSource:0}: Error finding container dc191a7f539c56ef8fc06023e72d4ae382edf9b295b6cce3ae9c3f852621b54f: Status 404 returned error can't find the container with id dc191a7f539c56ef8fc06023e72d4ae382edf9b295b6cce3ae9c3f852621b54f
Jul 24 23:58:19.472070 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:19.472041    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-9ffa79f3-7d879dbdb6-bcfnt"]
Jul 24 23:58:19.473080 ip-10-0-142-133 kubenswrapper[2558]: W0724 23:58:19.473054    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1edcf2a_75ec_4715_97e6_a42a56d96c7e.slice/crio-4e025097c8806a453302e63760c6d6cf052f0cccd25e1ac3c1d11ac836481e93 WatchSource:0}: Error finding container 4e025097c8806a453302e63760c6d6cf052f0cccd25e1ac3c1d11ac836481e93: Status 404 returned error can't find the container with id 4e025097c8806a453302e63760c6d6cf052f0cccd25e1ac3c1d11ac836481e93
Jul 24 23:58:19.898950 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:19.898848    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-9ffa79f3-666fb9f8d8-bqvrk" event={"ID":"afee1ac5-e711-4129-92cc-1e41a54b846d","Type":"ContainerStarted","Data":"4efd2f3ec0c1d58b381c0266074d3eb0e55a4a3ca4b9adb1725d879f29db6049"}
Jul 24 23:58:19.898950 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:19.898944    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-9ffa79f3-666fb9f8d8-bqvrk" event={"ID":"afee1ac5-e711-4129-92cc-1e41a54b846d","Type":"ContainerStarted","Data":"dc191a7f539c56ef8fc06023e72d4ae382edf9b295b6cce3ae9c3f852621b54f"}
Jul 24 23:58:19.899166 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:19.898974    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-9ffa79f3-666fb9f8d8-bqvrk"
Jul 24 23:58:19.900386 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:19.900356    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9ffa79f3-59d65f9b8d-mjwnm" event={"ID":"354fc7be-3fcd-4fe8-acfe-bf279071c654","Type":"ContainerStarted","Data":"5b624ff10d3c659ac416b3e7aece6e2da5f8ea292f60616d8c7573ac2b6c3af9"}
Jul 24 23:58:19.900756 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:19.900407    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9ffa79f3-59d65f9b8d-mjwnm" event={"ID":"354fc7be-3fcd-4fe8-acfe-bf279071c654","Type":"ContainerStarted","Data":"9456db7f62b64600e61f8176e2c0f07ad8109cbfc9d856fb452c03fe2bd09648"}
Jul 24 23:58:19.900756 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:19.900449    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9ffa79f3-59d65f9b8d-mjwnm"
Jul 24 23:58:19.901961 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:19.901938    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-9ffa79f3-7d879dbdb6-bcfnt" event={"ID":"f1edcf2a-75ec-4715-97e6-a42a56d96c7e","Type":"ContainerStarted","Data":"f7330438de82d2471db1728658a9fc74b1c14b6b3d51952eb0ac6931cb197080"}
Jul 24 23:58:19.902078 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:19.901977    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-9ffa79f3-7d879dbdb6-bcfnt" event={"ID":"f1edcf2a-75ec-4715-97e6-a42a56d96c7e","Type":"ContainerStarted","Data":"4e025097c8806a453302e63760c6d6cf052f0cccd25e1ac3c1d11ac836481e93"}
Jul 24 23:58:19.902078 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:19.902017    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-9ffa79f3-7d879dbdb6-bcfnt"
Jul 24 23:58:19.914430 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:19.914388    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-9ffa79f3-666fb9f8d8-bqvrk" podStartSLOduration=1.91437631 podStartE2EDuration="1.91437631s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:58:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:58:19.91297937 +0000 UTC m=+1740.299410097" watchObservedRunningTime="2026-07-24 23:58:19.91437631 +0000 UTC m=+1740.300807038"
Jul 24 23:58:19.930677 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:19.930641    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-9ffa79f3-7d879dbdb6-bcfnt" podStartSLOduration=1.9306294510000002 podStartE2EDuration="1.930629451s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:58:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:58:19.92845211 +0000 UTC m=+1740.314882839" watchObservedRunningTime="2026-07-24 23:58:19.930629451 +0000 UTC m=+1740.317060179"
Jul 24 23:58:19.945677 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:19.945636    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9ffa79f3-59d65f9b8d-mjwnm" podStartSLOduration=1.5710880170000001 podStartE2EDuration="1.945619276s" totalImagesPullingTime="374.531259ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:58:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:58:19.942177529 +0000 UTC m=+1740.328608258" watchObservedRunningTime="2026-07-24 23:58:19.945619276 +0000 UTC m=+1740.332050005"
Jul 24 23:58:25.912062 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:25.912030    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9ffa79f3-59d65f9b8d-mjwnm"
Jul 24 23:58:30.909672 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:30.909640    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-9ffa79f3-7d879dbdb6-bcfnt"
Jul 24 23:58:30.910164 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:30.909734    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-9ffa79f3-666fb9f8d8-bqvrk"
Jul 24 23:58:53.404227 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:53.404196    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-9ffa79f3-59d65f9b8d-mjwnm"]
Jul 24 23:58:53.404725 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:53.404433    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9ffa79f3-59d65f9b8d-mjwnm" podUID="354fc7be-3fcd-4fe8-acfe-bf279071c654" containerName="maas-api" containerID="cri-o://5b624ff10d3c659ac416b3e7aece6e2da5f8ea292f60616d8c7573ac2b6c3af9" gracePeriod=30
Jul 24 23:58:53.430519 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:53.430489    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:58:53.456849 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:53.456808    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-9ffa79f3-7d879dbdb6-bcfnt"]
Jul 24 23:58:53.457095 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:53.457064    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-9ffa79f3-7d879dbdb6-bcfnt" podUID="f1edcf2a-75ec-4715-97e6-a42a56d96c7e" containerName="payload-processing" containerID="cri-o://f7330438de82d2471db1728658a9fc74b1c14b6b3d51952eb0ac6931cb197080" gracePeriod=30
Jul 24 23:58:53.466296 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:53.466270    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-9ffa79f3-666fb9f8d8-bqvrk"]
Jul 24 23:58:53.466518 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:53.466490    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-9ffa79f3-666fb9f8d8-bqvrk" podUID="afee1ac5-e711-4129-92cc-1e41a54b846d" containerName="payload-pre-processing" containerID="cri-o://4efd2f3ec0c1d58b381c0266074d3eb0e55a4a3ca4b9adb1725d879f29db6049" gracePeriod=30
Jul 24 23:58:53.492517 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:53.492485    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:58:53.550593 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:53.550559    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:58:53.719109 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:53.719081    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-9ffa79f3-7d879dbdb6-bcfnt"
Jul 24 23:58:53.747756 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:53.747727    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9ffa79f3-59d65f9b8d-mjwnm"
Jul 24 23:58:53.759317 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:53.759295    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-9ffa79f3-666fb9f8d8-bqvrk"
Jul 24 23:58:53.896483 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:53.896447    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/afee1ac5-e711-4129-92cc-1e41a54b846d-plugins-config-volume\") pod \"afee1ac5-e711-4129-92cc-1e41a54b846d\" (UID: \"afee1ac5-e711-4129-92cc-1e41a54b846d\") "
Jul 24 23:58:53.896713 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:53.896489    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9s7q8\" (UniqueName: \"kubernetes.io/projected/f1edcf2a-75ec-4715-97e6-a42a56d96c7e-kube-api-access-9s7q8\") pod \"f1edcf2a-75ec-4715-97e6-a42a56d96c7e\" (UID: \"f1edcf2a-75ec-4715-97e6-a42a56d96c7e\") "
Jul 24 23:58:53.896713 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:53.896519    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/354fc7be-3fcd-4fe8-acfe-bf279071c654-openshift-service-ca\") pod \"354fc7be-3fcd-4fe8-acfe-bf279071c654\" (UID: \"354fc7be-3fcd-4fe8-acfe-bf279071c654\") "
Jul 24 23:58:53.896713 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:53.896563    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1edcf2a-75ec-4715-97e6-a42a56d96c7e-plugins-config-volume\") pod \"f1edcf2a-75ec-4715-97e6-a42a56d96c7e\" (UID: \"f1edcf2a-75ec-4715-97e6-a42a56d96c7e\") "
Jul 24 23:58:53.896713 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:53.896600    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w27sq\" (UniqueName: \"kubernetes.io/projected/354fc7be-3fcd-4fe8-acfe-bf279071c654-kube-api-access-w27sq\") pod \"354fc7be-3fcd-4fe8-acfe-bf279071c654\" (UID: \"354fc7be-3fcd-4fe8-acfe-bf279071c654\") "
Jul 24 23:58:53.896713 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:53.896664    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/354fc7be-3fcd-4fe8-acfe-bf279071c654-maas-api-tls\") pod \"354fc7be-3fcd-4fe8-acfe-bf279071c654\" (UID: \"354fc7be-3fcd-4fe8-acfe-bf279071c654\") "
Jul 24 23:58:53.896713 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:53.896695    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cbh62\" (UniqueName: \"kubernetes.io/projected/afee1ac5-e711-4129-92cc-1e41a54b846d-kube-api-access-cbh62\") pod \"afee1ac5-e711-4129-92cc-1e41a54b846d\" (UID: \"afee1ac5-e711-4129-92cc-1e41a54b846d\") "
Jul 24 23:58:53.897034 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:53.896923    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/afee1ac5-e711-4129-92cc-1e41a54b846d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "afee1ac5-e711-4129-92cc-1e41a54b846d" (UID: "afee1ac5-e711-4129-92cc-1e41a54b846d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:58:53.897034 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:53.896940    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/354fc7be-3fcd-4fe8-acfe-bf279071c654-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "354fc7be-3fcd-4fe8-acfe-bf279071c654" (UID: "354fc7be-3fcd-4fe8-acfe-bf279071c654"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:58:53.897034 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:53.896998    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1edcf2a-75ec-4715-97e6-a42a56d96c7e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f1edcf2a-75ec-4715-97e6-a42a56d96c7e" (UID: "f1edcf2a-75ec-4715-97e6-a42a56d96c7e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:58:53.898847 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:53.898821    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/354fc7be-3fcd-4fe8-acfe-bf279071c654-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "354fc7be-3fcd-4fe8-acfe-bf279071c654" (UID: "354fc7be-3fcd-4fe8-acfe-bf279071c654"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 23:58:53.898950 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:53.898853    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1edcf2a-75ec-4715-97e6-a42a56d96c7e-kube-api-access-9s7q8" (OuterVolumeSpecName: "kube-api-access-9s7q8") pod "f1edcf2a-75ec-4715-97e6-a42a56d96c7e" (UID: "f1edcf2a-75ec-4715-97e6-a42a56d96c7e"). InnerVolumeSpecName "kube-api-access-9s7q8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:58:53.899139 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:53.899117    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/afee1ac5-e711-4129-92cc-1e41a54b846d-kube-api-access-cbh62" (OuterVolumeSpecName: "kube-api-access-cbh62") pod "afee1ac5-e711-4129-92cc-1e41a54b846d" (UID: "afee1ac5-e711-4129-92cc-1e41a54b846d"). InnerVolumeSpecName "kube-api-access-cbh62". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:58:53.899221 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:53.899121    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/354fc7be-3fcd-4fe8-acfe-bf279071c654-kube-api-access-w27sq" (OuterVolumeSpecName: "kube-api-access-w27sq") pod "354fc7be-3fcd-4fe8-acfe-bf279071c654" (UID: "354fc7be-3fcd-4fe8-acfe-bf279071c654"). InnerVolumeSpecName "kube-api-access-w27sq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:58:53.998178 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:53.998153    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/afee1ac5-e711-4129-92cc-1e41a54b846d-plugins-config-volume\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 24 23:58:53.998332 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:53.998180    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9s7q8\" (UniqueName: \"kubernetes.io/projected/f1edcf2a-75ec-4715-97e6-a42a56d96c7e-kube-api-access-9s7q8\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 24 23:58:53.998332 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:53.998191    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/354fc7be-3fcd-4fe8-acfe-bf279071c654-openshift-service-ca\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 24 23:58:53.998332 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:53.998200    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1edcf2a-75ec-4715-97e6-a42a56d96c7e-plugins-config-volume\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 24 23:58:53.998332 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:53.998210    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w27sq\" (UniqueName: \"kubernetes.io/projected/354fc7be-3fcd-4fe8-acfe-bf279071c654-kube-api-access-w27sq\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 24 23:58:53.998332 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:53.998219    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/354fc7be-3fcd-4fe8-acfe-bf279071c654-maas-api-tls\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 24 23:58:53.998332 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:53.998228    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cbh62\" (UniqueName: \"kubernetes.io/projected/afee1ac5-e711-4129-92cc-1e41a54b846d-kube-api-access-cbh62\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 24 23:58:54.083636 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:54.083599    2558 generic.go:356] "Generic (PLEG): container finished" podID="f1edcf2a-75ec-4715-97e6-a42a56d96c7e" containerID="f7330438de82d2471db1728658a9fc74b1c14b6b3d51952eb0ac6931cb197080" exitCode=0
Jul 24 23:58:54.083824 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:54.083648    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-9ffa79f3-7d879dbdb6-bcfnt"
Jul 24 23:58:54.083824 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:54.083691    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-9ffa79f3-7d879dbdb6-bcfnt" event={"ID":"f1edcf2a-75ec-4715-97e6-a42a56d96c7e","Type":"ContainerDied","Data":"f7330438de82d2471db1728658a9fc74b1c14b6b3d51952eb0ac6931cb197080"}
Jul 24 23:58:54.083824 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:54.083755    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-9ffa79f3-7d879dbdb6-bcfnt" event={"ID":"f1edcf2a-75ec-4715-97e6-a42a56d96c7e","Type":"ContainerDied","Data":"4e025097c8806a453302e63760c6d6cf052f0cccd25e1ac3c1d11ac836481e93"}
Jul 24 23:58:54.083824 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:54.083786    2558 scope.go:117] "RemoveContainer" containerID="f7330438de82d2471db1728658a9fc74b1c14b6b3d51952eb0ac6931cb197080"
Jul 24 23:58:54.085004 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:54.084980    2558 generic.go:356] "Generic (PLEG): container finished" podID="afee1ac5-e711-4129-92cc-1e41a54b846d" containerID="4efd2f3ec0c1d58b381c0266074d3eb0e55a4a3ca4b9adb1725d879f29db6049" exitCode=0
Jul 24 23:58:54.085071 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:54.085016    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-9ffa79f3-666fb9f8d8-bqvrk" event={"ID":"afee1ac5-e711-4129-92cc-1e41a54b846d","Type":"ContainerDied","Data":"4efd2f3ec0c1d58b381c0266074d3eb0e55a4a3ca4b9adb1725d879f29db6049"}
Jul 24 23:58:54.085071 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:54.085026    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-9ffa79f3-666fb9f8d8-bqvrk"
Jul 24 23:58:54.085179 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:54.085076    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-9ffa79f3-666fb9f8d8-bqvrk" event={"ID":"afee1ac5-e711-4129-92cc-1e41a54b846d","Type":"ContainerDied","Data":"dc191a7f539c56ef8fc06023e72d4ae382edf9b295b6cce3ae9c3f852621b54f"}
Jul 24 23:58:54.086514 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:54.086493    2558 generic.go:356] "Generic (PLEG): container finished" podID="354fc7be-3fcd-4fe8-acfe-bf279071c654" containerID="5b624ff10d3c659ac416b3e7aece6e2da5f8ea292f60616d8c7573ac2b6c3af9" exitCode=0
Jul 24 23:58:54.086650 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:54.086557    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9ffa79f3-59d65f9b8d-mjwnm"
Jul 24 23:58:54.086650 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:54.086525    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9ffa79f3-59d65f9b8d-mjwnm" event={"ID":"354fc7be-3fcd-4fe8-acfe-bf279071c654","Type":"ContainerDied","Data":"5b624ff10d3c659ac416b3e7aece6e2da5f8ea292f60616d8c7573ac2b6c3af9"}
Jul 24 23:58:54.086650 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:54.086616    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9ffa79f3-59d65f9b8d-mjwnm" event={"ID":"354fc7be-3fcd-4fe8-acfe-bf279071c654","Type":"ContainerDied","Data":"9456db7f62b64600e61f8176e2c0f07ad8109cbfc9d856fb452c03fe2bd09648"}
Jul 24 23:58:54.096695 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:54.096670    2558 scope.go:117] "RemoveContainer" containerID="f7330438de82d2471db1728658a9fc74b1c14b6b3d51952eb0ac6931cb197080"
Jul 24 23:58:54.097381 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:58:54.097353    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7330438de82d2471db1728658a9fc74b1c14b6b3d51952eb0ac6931cb197080\": container with ID starting with f7330438de82d2471db1728658a9fc74b1c14b6b3d51952eb0ac6931cb197080 not found: ID does not exist" containerID="f7330438de82d2471db1728658a9fc74b1c14b6b3d51952eb0ac6931cb197080"
Jul 24 23:58:54.097484 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:54.097393    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7330438de82d2471db1728658a9fc74b1c14b6b3d51952eb0ac6931cb197080"} err="failed to get container status \"f7330438de82d2471db1728658a9fc74b1c14b6b3d51952eb0ac6931cb197080\": rpc error: code = NotFound desc = could not find container \"f7330438de82d2471db1728658a9fc74b1c14b6b3d51952eb0ac6931cb197080\": container with ID starting with f7330438de82d2471db1728658a9fc74b1c14b6b3d51952eb0ac6931cb197080 not found: ID does not exist"
Jul 24 23:58:54.097484 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:54.097415    2558 scope.go:117] "RemoveContainer" containerID="4efd2f3ec0c1d58b381c0266074d3eb0e55a4a3ca4b9adb1725d879f29db6049"
Jul 24 23:58:54.105647 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:54.105626    2558 scope.go:117] "RemoveContainer" containerID="4efd2f3ec0c1d58b381c0266074d3eb0e55a4a3ca4b9adb1725d879f29db6049"
Jul 24 23:58:54.105920 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:58:54.105898    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4efd2f3ec0c1d58b381c0266074d3eb0e55a4a3ca4b9adb1725d879f29db6049\": container with ID starting with 4efd2f3ec0c1d58b381c0266074d3eb0e55a4a3ca4b9adb1725d879f29db6049 not found: ID does not exist" containerID="4efd2f3ec0c1d58b381c0266074d3eb0e55a4a3ca4b9adb1725d879f29db6049"
Jul 24 23:58:54.105996 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:54.105921    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4efd2f3ec0c1d58b381c0266074d3eb0e55a4a3ca4b9adb1725d879f29db6049"} err="failed to get container status \"4efd2f3ec0c1d58b381c0266074d3eb0e55a4a3ca4b9adb1725d879f29db6049\": rpc error: code = NotFound desc = could not find container \"4efd2f3ec0c1d58b381c0266074d3eb0e55a4a3ca4b9adb1725d879f29db6049\": container with ID starting with 4efd2f3ec0c1d58b381c0266074d3eb0e55a4a3ca4b9adb1725d879f29db6049 not found: ID does not exist"
Jul 24 23:58:54.105996 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:54.105938    2558 scope.go:117] "RemoveContainer" containerID="5b624ff10d3c659ac416b3e7aece6e2da5f8ea292f60616d8c7573ac2b6c3af9"
Jul 24 23:58:54.111462 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:54.111396    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-9ffa79f3-7d879dbdb6-bcfnt"]
Jul 24 23:58:54.115103 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:54.115080    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-9ffa79f3-7d879dbdb6-bcfnt"]
Jul 24 23:58:54.120974 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:54.120956    2558 scope.go:117] "RemoveContainer" containerID="5b624ff10d3c659ac416b3e7aece6e2da5f8ea292f60616d8c7573ac2b6c3af9"
Jul 24 23:58:54.121226 ip-10-0-142-133 kubenswrapper[2558]: E0724 23:58:54.121208    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b624ff10d3c659ac416b3e7aece6e2da5f8ea292f60616d8c7573ac2b6c3af9\": container with ID starting with 5b624ff10d3c659ac416b3e7aece6e2da5f8ea292f60616d8c7573ac2b6c3af9 not found: ID does not exist" containerID="5b624ff10d3c659ac416b3e7aece6e2da5f8ea292f60616d8c7573ac2b6c3af9"
Jul 24 23:58:54.121270 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:54.121232    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b624ff10d3c659ac416b3e7aece6e2da5f8ea292f60616d8c7573ac2b6c3af9"} err="failed to get container status \"5b624ff10d3c659ac416b3e7aece6e2da5f8ea292f60616d8c7573ac2b6c3af9\": rpc error: code = NotFound desc = could not find container \"5b624ff10d3c659ac416b3e7aece6e2da5f8ea292f60616d8c7573ac2b6c3af9\": container with ID starting with 5b624ff10d3c659ac416b3e7aece6e2da5f8ea292f60616d8c7573ac2b6c3af9 not found: ID does not exist"
Jul 24 23:58:54.122108 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:54.122091    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-9ffa79f3-666fb9f8d8-bqvrk"]
Jul 24 23:58:54.126796 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:54.126779    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-9ffa79f3-666fb9f8d8-bqvrk"]
Jul 24 23:58:54.136664 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:54.136647    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-9ffa79f3-59d65f9b8d-mjwnm"]
Jul 24 23:58:54.140359 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:54.140339    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-9ffa79f3-59d65f9b8d-mjwnm"]
Jul 24 23:58:54.204236 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:54.204206    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="354fc7be-3fcd-4fe8-acfe-bf279071c654" path="/var/lib/kubelet/pods/354fc7be-3fcd-4fe8-acfe-bf279071c654/volumes"
Jul 24 23:58:54.204596 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:54.204582    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="afee1ac5-e711-4129-92cc-1e41a54b846d" path="/var/lib/kubelet/pods/afee1ac5-e711-4129-92cc-1e41a54b846d/volumes"
Jul 24 23:58:54.204904 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:54.204893    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1edcf2a-75ec-4715-97e6-a42a56d96c7e" path="/var/lib/kubelet/pods/f1edcf2a-75ec-4715-97e6-a42a56d96c7e/volumes"
Jul 24 23:58:58.005543 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:58.005496    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:58:58.166436 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:58.166401    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:58:58.421737 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:58.421661    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:58:58.950596 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:58.950559    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:58:59.047237 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:59.047205    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:58:59.147618 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:58:59.147584    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:59:00.599267 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:59:00.599231    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:59:06.340033 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:59:06.340003    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:59:06.448747 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:59:06.448710    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:59:06.540244 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:59:06.540212    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:59:06.643569 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:59:06.643464    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:59:07.054103 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:59:07.054067    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:59:33.068153 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:59:33.068122    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:59:39.354708 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:59:39.354676    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:59:39.459656 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:59:39.459603    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:59:39.553831 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:59:39.553792    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 24 23:59:55.518234 ip-10-0-142-133 kubenswrapper[2558]: I0724 23:59:55.518199    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:00:00.233851 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:00.233812    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748960-9n6jt"]
Jul 25 00:00:00.234264 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:00.234176    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1edcf2a-75ec-4715-97e6-a42a56d96c7e" containerName="payload-processing"
Jul 25 00:00:00.234264 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:00.234190    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1edcf2a-75ec-4715-97e6-a42a56d96c7e" containerName="payload-processing"
Jul 25 00:00:00.234264 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:00.234205    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="354fc7be-3fcd-4fe8-acfe-bf279071c654" containerName="maas-api"
Jul 25 00:00:00.234264 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:00.234212    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="354fc7be-3fcd-4fe8-acfe-bf279071c654" containerName="maas-api"
Jul 25 00:00:00.234264 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:00.234223    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="afee1ac5-e711-4129-92cc-1e41a54b846d" containerName="payload-pre-processing"
Jul 25 00:00:00.234264 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:00.234228    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="afee1ac5-e711-4129-92cc-1e41a54b846d" containerName="payload-pre-processing"
Jul 25 00:00:00.234457 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:00.234287    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="afee1ac5-e711-4129-92cc-1e41a54b846d" containerName="payload-pre-processing"
Jul 25 00:00:00.234457 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:00.234297    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1edcf2a-75ec-4715-97e6-a42a56d96c7e" containerName="payload-processing"
Jul 25 00:00:00.234457 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:00.234305    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="354fc7be-3fcd-4fe8-acfe-bf279071c654" containerName="maas-api"
Jul 25 00:00:00.237376 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:00.237359    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748960-9n6jt"
Jul 25 00:00:00.239398 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:00.239378    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bxr46\""
Jul 25 00:00:00.243990 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:00.243966    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748960-9n6jt"]
Jul 25 00:00:00.267117 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:00.267087    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748960-9n6jt"
Jul 25 00:00:00.401459 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:00.401434    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748960-9n6jt"]
Jul 25 00:00:00.403522 ip-10-0-142-133 kubenswrapper[2558]: W0725 00:00:00.403493    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83c97281_d4ed_4be8_a187_776def664053.slice/crio-f35f9234e28328cfb866a7905ef762d25f2fa220b83bc2737f763687410df03f WatchSource:0}: Error finding container f35f9234e28328cfb866a7905ef762d25f2fa220b83bc2737f763687410df03f: Status 404 returned error can't find the container with id f35f9234e28328cfb866a7905ef762d25f2fa220b83bc2737f763687410df03f
Jul 25 00:00:00.426691 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:00.426658    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748960-9n6jt" event={"ID":"83c97281-d4ed-4be8-a187-776def664053","Type":"ContainerStarted","Data":"f35f9234e28328cfb866a7905ef762d25f2fa220b83bc2737f763687410df03f"}
Jul 25 00:00:01.207773 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:01.207735    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:00:01.311887 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:01.311829    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:00:01.410376 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:01.410333    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:00:01.432873 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:01.432842    2558 generic.go:356] "Generic (PLEG): container finished" podID="83c97281-d4ed-4be8-a187-776def664053" containerID="bb1ec6a801c416295439efdfd1177f332303e475c82e7f4062e46ee878e57e49" exitCode=0
Jul 25 00:00:01.433014 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:01.432916    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748960-9n6jt" event={"ID":"83c97281-d4ed-4be8-a187-776def664053","Type":"ContainerDied","Data":"bb1ec6a801c416295439efdfd1177f332303e475c82e7f4062e46ee878e57e49"}
Jul 25 00:00:02.558451 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:02.558424    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748960-9n6jt"
Jul 25 00:00:03.444614 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:03.444584    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748960-9n6jt"
Jul 25 00:00:03.444799 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:03.444583    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748960-9n6jt" event={"ID":"83c97281-d4ed-4be8-a187-776def664053","Type":"ContainerDied","Data":"f35f9234e28328cfb866a7905ef762d25f2fa220b83bc2737f763687410df03f"}
Jul 25 00:00:03.444799 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:03.444711    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f35f9234e28328cfb866a7905ef762d25f2fa220b83bc2737f763687410df03f"
Jul 25 00:00:11.910671 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:11.910631    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:00:12.789207 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:12.789174    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:00:19.274682 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:19.274651    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:00:19.378041 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:19.378003    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:00:19.475938 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:19.475905    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:00:19.575506 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:19.575430    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:00:20.388740 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:20.388705    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:00:20.510147 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:20.510114    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:00:20.611513 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:20.611478    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:00:22.269778 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:22.269735    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:00:23.777927 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:23.777880    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:00:28.452788 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:28.452745    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:00:28.552878 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:28.552839    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:00:29.229076 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:29.229042    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:00:34.791836 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:34.791798    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:00:34.891984 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:34.891949    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:00:35.044001 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:35.043919    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:00:35.376319 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:35.376247    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:00:35.682171 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:35.682062    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:00:38.461809 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:38.461749    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:00:38.561398 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:38.561362    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:00:38.690805 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:38.690774    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-438308e8-75f6dc978b-7bbmf"]
Jul 25 00:00:38.691273 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:38.691251    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83c97281-d4ed-4be8-a187-776def664053" containerName="cleanup"
Jul 25 00:00:38.691411 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:38.691375    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="83c97281-d4ed-4be8-a187-776def664053" containerName="cleanup"
Jul 25 00:00:38.691521 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:38.691505    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="83c97281-d4ed-4be8-a187-776def664053" containerName="cleanup"
Jul 25 00:00:38.694672 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:38.694651    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-438308e8-75f6dc978b-7bbmf"
Jul 25 00:00:38.697168 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:38.697149    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-438308e8-serving-cert\""
Jul 25 00:00:38.705364 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:38.705341    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-438308e8-75f6dc978b-7bbmf"]
Jul 25 00:00:38.739567 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:38.739524    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:00:38.776771 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:38.776738    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cw6q6\" (UniqueName: \"kubernetes.io/projected/612ec524-b9db-4ff7-94ae-98e0bb33e686-kube-api-access-cw6q6\") pod \"maas-api-e2e-mt-438308e8-75f6dc978b-7bbmf\" (UID: \"612ec524-b9db-4ff7-94ae-98e0bb33e686\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-438308e8-75f6dc978b-7bbmf"
Jul 25 00:00:38.776932 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:38.776809    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/612ec524-b9db-4ff7-94ae-98e0bb33e686-openshift-service-ca\") pod \"maas-api-e2e-mt-438308e8-75f6dc978b-7bbmf\" (UID: \"612ec524-b9db-4ff7-94ae-98e0bb33e686\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-438308e8-75f6dc978b-7bbmf"
Jul 25 00:00:38.776932 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:38.776868    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/612ec524-b9db-4ff7-94ae-98e0bb33e686-maas-api-tls\") pod \"maas-api-e2e-mt-438308e8-75f6dc978b-7bbmf\" (UID: \"612ec524-b9db-4ff7-94ae-98e0bb33e686\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-438308e8-75f6dc978b-7bbmf"
Jul 25 00:00:38.844793 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:38.844765    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:00:38.878168 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:38.878140    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cw6q6\" (UniqueName: \"kubernetes.io/projected/612ec524-b9db-4ff7-94ae-98e0bb33e686-kube-api-access-cw6q6\") pod \"maas-api-e2e-mt-438308e8-75f6dc978b-7bbmf\" (UID: \"612ec524-b9db-4ff7-94ae-98e0bb33e686\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-438308e8-75f6dc978b-7bbmf"
Jul 25 00:00:38.878291 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:38.878187    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/612ec524-b9db-4ff7-94ae-98e0bb33e686-openshift-service-ca\") pod \"maas-api-e2e-mt-438308e8-75f6dc978b-7bbmf\" (UID: \"612ec524-b9db-4ff7-94ae-98e0bb33e686\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-438308e8-75f6dc978b-7bbmf"
Jul 25 00:00:38.878291 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:38.878230    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/612ec524-b9db-4ff7-94ae-98e0bb33e686-maas-api-tls\") pod \"maas-api-e2e-mt-438308e8-75f6dc978b-7bbmf\" (UID: \"612ec524-b9db-4ff7-94ae-98e0bb33e686\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-438308e8-75f6dc978b-7bbmf"
Jul 25 00:00:38.878960 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:38.878939    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/612ec524-b9db-4ff7-94ae-98e0bb33e686-openshift-service-ca\") pod \"maas-api-e2e-mt-438308e8-75f6dc978b-7bbmf\" (UID: \"612ec524-b9db-4ff7-94ae-98e0bb33e686\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-438308e8-75f6dc978b-7bbmf"
Jul 25 00:00:38.880705 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:38.880686    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/612ec524-b9db-4ff7-94ae-98e0bb33e686-maas-api-tls\") pod \"maas-api-e2e-mt-438308e8-75f6dc978b-7bbmf\" (UID: \"612ec524-b9db-4ff7-94ae-98e0bb33e686\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-438308e8-75f6dc978b-7bbmf"
Jul 25 00:00:38.886652 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:38.886622    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cw6q6\" (UniqueName: \"kubernetes.io/projected/612ec524-b9db-4ff7-94ae-98e0bb33e686-kube-api-access-cw6q6\") pod \"maas-api-e2e-mt-438308e8-75f6dc978b-7bbmf\" (UID: \"612ec524-b9db-4ff7-94ae-98e0bb33e686\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-438308e8-75f6dc978b-7bbmf"
Jul 25 00:00:39.008198 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:39.008117    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-438308e8-75f6dc978b-7bbmf"
Jul 25 00:00:39.139862 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:39.139825    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-438308e8-75f6dc978b-7bbmf"]
Jul 25 00:00:39.140605 ip-10-0-142-133 kubenswrapper[2558]: W0725 00:00:39.140580    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod612ec524_b9db_4ff7_94ae_98e0bb33e686.slice/crio-1a3d7f7bf7bff48b5ba5b4c3dbe9cfd5747225ad6618641058704e92f75188a3 WatchSource:0}: Error finding container 1a3d7f7bf7bff48b5ba5b4c3dbe9cfd5747225ad6618641058704e92f75188a3: Status 404 returned error can't find the container with id 1a3d7f7bf7bff48b5ba5b4c3dbe9cfd5747225ad6618641058704e92f75188a3
Jul 25 00:00:39.264407 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:39.264331    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:00:39.374138 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:39.374110    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:00:39.636492 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:39.636408    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-438308e8-75f6dc978b-7bbmf" event={"ID":"612ec524-b9db-4ff7-94ae-98e0bb33e686","Type":"ContainerStarted","Data":"1a3d7f7bf7bff48b5ba5b4c3dbe9cfd5747225ad6618641058704e92f75188a3"}
Jul 25 00:00:40.642871 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:40.642837    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-438308e8-75f6dc978b-7bbmf" event={"ID":"612ec524-b9db-4ff7-94ae-98e0bb33e686","Type":"ContainerStarted","Data":"6eb6a07a8f946f4b8b1a2277f09dd814a4c6df55aa4957f5cd2473b57a80c01b"}
Jul 25 00:00:40.643325 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:40.642915    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-438308e8-75f6dc978b-7bbmf"
Jul 25 00:00:40.664597 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:40.664548    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-438308e8-75f6dc978b-7bbmf" podStartSLOduration=1.597923376 podStartE2EDuration="2.664520702s" totalImagesPullingTime="1.066597326s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:00:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:00:40.66224725 +0000 UTC m=+1881.048677977" watchObservedRunningTime="2026-07-25 00:00:40.664520702 +0000 UTC m=+1881.050951430"
Jul 25 00:00:42.924156 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:42.924078    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:00:43.600938 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:43.600904    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:00:43.696526 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:43.696491    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:00:44.048779 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:44.048738    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:00:46.653798 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:46.653771    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-438308e8-75f6dc978b-7bbmf"
Jul 25 00:00:49.598343 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:49.598302    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:00:49.653975 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:49.653944    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:00:49.965873 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:49.965782    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-438308e8-n9rn2"]
Jul 25 00:00:49.969448 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:49.969424    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-438308e8-n9rn2"
Jul 25 00:00:49.971747 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:49.971727    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bxr46\""
Jul 25 00:00:49.976273 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:49.976250    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-438308e8-n9rn2"]
Jul 25 00:00:50.068958 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:50.068923    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f555b938-f372-4cef-ac2f-ff6d3b225994-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-438308e8-n9rn2\" (UID: \"f555b938-f372-4cef-ac2f-ff6d3b225994\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-438308e8-n9rn2"
Jul 25 00:00:50.170497 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:50.170328    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f555b938-f372-4cef-ac2f-ff6d3b225994-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-438308e8-n9rn2\" (UID: \"f555b938-f372-4cef-ac2f-ff6d3b225994\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-438308e8-n9rn2"
Jul 25 00:00:50.171042 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:50.171020    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f555b938-f372-4cef-ac2f-ff6d3b225994-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-438308e8-n9rn2\" (UID: \"f555b938-f372-4cef-ac2f-ff6d3b225994\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-438308e8-n9rn2"
Jul 25 00:00:50.283393 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:50.283351    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-438308e8-n9rn2"
Jul 25 00:00:50.421015 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:50.420986    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-438308e8-n9rn2"]
Jul 25 00:00:50.422105 ip-10-0-142-133 kubenswrapper[2558]: W0725 00:00:50.422073    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf555b938_f372_4cef_ac2f_ff6d3b225994.slice/crio-c61724a82f6edbb5d3f11ba9a9532a5fb0b26e8716d29d3cc3170a17fd3dd073 WatchSource:0}: Error finding container c61724a82f6edbb5d3f11ba9a9532a5fb0b26e8716d29d3cc3170a17fd3dd073: Status 404 returned error can't find the container with id c61724a82f6edbb5d3f11ba9a9532a5fb0b26e8716d29d3cc3170a17fd3dd073
Jul 25 00:00:50.693825 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:50.693791    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-438308e8-n9rn2" event={"ID":"f555b938-f372-4cef-ac2f-ff6d3b225994","Type":"ContainerStarted","Data":"c61724a82f6edbb5d3f11ba9a9532a5fb0b26e8716d29d3cc3170a17fd3dd073"}
Jul 25 00:00:51.700172 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:51.700130    2558 generic.go:356] "Generic (PLEG): container finished" podID="f555b938-f372-4cef-ac2f-ff6d3b225994" containerID="2f61ad50527631baf70d6f36a1fbdb42fa5e7bd5a38a524ce3979eba30a68bba" exitCode=0
Jul 25 00:00:51.700592 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:51.700190    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-438308e8-n9rn2" event={"ID":"f555b938-f372-4cef-ac2f-ff6d3b225994","Type":"ContainerDied","Data":"2f61ad50527631baf70d6f36a1fbdb42fa5e7bd5a38a524ce3979eba30a68bba"}
Jul 25 00:00:52.830677 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:52.830653    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-438308e8-n9rn2"
Jul 25 00:00:52.993884 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:52.993854    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f555b938-f372-4cef-ac2f-ff6d3b225994-maas-api-service-ca\") pod \"f555b938-f372-4cef-ac2f-ff6d3b225994\" (UID: \"f555b938-f372-4cef-ac2f-ff6d3b225994\") "
Jul 25 00:00:52.994215 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:52.994192    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f555b938-f372-4cef-ac2f-ff6d3b225994-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f555b938-f372-4cef-ac2f-ff6d3b225994" (UID: "f555b938-f372-4cef-ac2f-ff6d3b225994"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 00:00:53.094846 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:53.094799    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f555b938-f372-4cef-ac2f-ff6d3b225994-maas-api-service-ca\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 25 00:00:53.711323 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:53.711290    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-438308e8-n9rn2" event={"ID":"f555b938-f372-4cef-ac2f-ff6d3b225994","Type":"ContainerDied","Data":"c61724a82f6edbb5d3f11ba9a9532a5fb0b26e8716d29d3cc3170a17fd3dd073"}
Jul 25 00:00:53.711323 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:53.711314    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-438308e8-n9rn2"
Jul 25 00:00:53.711552 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:00:53.711339    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c61724a82f6edbb5d3f11ba9a9532a5fb0b26e8716d29d3cc3170a17fd3dd073"
Jul 25 00:01:00.402826 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:00.402789    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-438308e8-75f6dc978b-7bbmf"]
Jul 25 00:01:00.403759 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:00.403721    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-438308e8-75f6dc978b-7bbmf" podUID="612ec524-b9db-4ff7-94ae-98e0bb33e686" containerName="maas-api" containerID="cri-o://6eb6a07a8f946f4b8b1a2277f09dd814a4c6df55aa4957f5cd2473b57a80c01b" gracePeriod=30
Jul 25 00:01:00.430749 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:00.430717    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:01:00.660494 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:00.660424    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-438308e8-75f6dc978b-7bbmf"
Jul 25 00:01:00.743769 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:00.743731    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:01:00.751195 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:00.751165    2558 generic.go:356] "Generic (PLEG): container finished" podID="612ec524-b9db-4ff7-94ae-98e0bb33e686" containerID="6eb6a07a8f946f4b8b1a2277f09dd814a4c6df55aa4957f5cd2473b57a80c01b" exitCode=0
Jul 25 00:01:00.751365 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:00.751204    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-438308e8-75f6dc978b-7bbmf"
Jul 25 00:01:00.751365 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:00.751251    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-438308e8-75f6dc978b-7bbmf" event={"ID":"612ec524-b9db-4ff7-94ae-98e0bb33e686","Type":"ContainerDied","Data":"6eb6a07a8f946f4b8b1a2277f09dd814a4c6df55aa4957f5cd2473b57a80c01b"}
Jul 25 00:01:00.751365 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:00.751326    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-438308e8-75f6dc978b-7bbmf" event={"ID":"612ec524-b9db-4ff7-94ae-98e0bb33e686","Type":"ContainerDied","Data":"1a3d7f7bf7bff48b5ba5b4c3dbe9cfd5747225ad6618641058704e92f75188a3"}
Jul 25 00:01:00.751365 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:00.751349    2558 scope.go:117] "RemoveContainer" containerID="6eb6a07a8f946f4b8b1a2277f09dd814a4c6df55aa4957f5cd2473b57a80c01b"
Jul 25 00:01:00.760637 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:00.760610    2558 scope.go:117] "RemoveContainer" containerID="6eb6a07a8f946f4b8b1a2277f09dd814a4c6df55aa4957f5cd2473b57a80c01b"
Jul 25 00:01:00.760842 ip-10-0-142-133 kubenswrapper[2558]: E0725 00:01:00.760821    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6eb6a07a8f946f4b8b1a2277f09dd814a4c6df55aa4957f5cd2473b57a80c01b\": container with ID starting with 6eb6a07a8f946f4b8b1a2277f09dd814a4c6df55aa4957f5cd2473b57a80c01b not found: ID does not exist" containerID="6eb6a07a8f946f4b8b1a2277f09dd814a4c6df55aa4957f5cd2473b57a80c01b"
Jul 25 00:01:00.760920 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:00.760851    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6eb6a07a8f946f4b8b1a2277f09dd814a4c6df55aa4957f5cd2473b57a80c01b"} err="failed to get container status \"6eb6a07a8f946f4b8b1a2277f09dd814a4c6df55aa4957f5cd2473b57a80c01b\": rpc error: code = NotFound desc = could not find container \"6eb6a07a8f946f4b8b1a2277f09dd814a4c6df55aa4957f5cd2473b57a80c01b\": container with ID starting with 6eb6a07a8f946f4b8b1a2277f09dd814a4c6df55aa4957f5cd2473b57a80c01b not found: ID does not exist"
Jul 25 00:01:00.762459 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:00.762438    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/612ec524-b9db-4ff7-94ae-98e0bb33e686-maas-api-tls\") pod \"612ec524-b9db-4ff7-94ae-98e0bb33e686\" (UID: \"612ec524-b9db-4ff7-94ae-98e0bb33e686\") "
Jul 25 00:01:00.762608 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:00.762502    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/612ec524-b9db-4ff7-94ae-98e0bb33e686-openshift-service-ca\") pod \"612ec524-b9db-4ff7-94ae-98e0bb33e686\" (UID: \"612ec524-b9db-4ff7-94ae-98e0bb33e686\") "
Jul 25 00:01:00.762608 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:00.762588    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cw6q6\" (UniqueName: \"kubernetes.io/projected/612ec524-b9db-4ff7-94ae-98e0bb33e686-kube-api-access-cw6q6\") pod \"612ec524-b9db-4ff7-94ae-98e0bb33e686\" (UID: \"612ec524-b9db-4ff7-94ae-98e0bb33e686\") "
Jul 25 00:01:00.762896 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:00.762876    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/612ec524-b9db-4ff7-94ae-98e0bb33e686-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "612ec524-b9db-4ff7-94ae-98e0bb33e686" (UID: "612ec524-b9db-4ff7-94ae-98e0bb33e686"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 00:01:00.764790 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:00.764756    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/612ec524-b9db-4ff7-94ae-98e0bb33e686-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "612ec524-b9db-4ff7-94ae-98e0bb33e686" (UID: "612ec524-b9db-4ff7-94ae-98e0bb33e686"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 00:01:00.764998 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:00.764964    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/612ec524-b9db-4ff7-94ae-98e0bb33e686-kube-api-access-cw6q6" (OuterVolumeSpecName: "kube-api-access-cw6q6") pod "612ec524-b9db-4ff7-94ae-98e0bb33e686" (UID: "612ec524-b9db-4ff7-94ae-98e0bb33e686"). InnerVolumeSpecName "kube-api-access-cw6q6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 00:01:00.802141 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:00.802107    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:01:00.863428 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:00.863397    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/612ec524-b9db-4ff7-94ae-98e0bb33e686-maas-api-tls\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 25 00:01:00.863428 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:00.863425    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/612ec524-b9db-4ff7-94ae-98e0bb33e686-openshift-service-ca\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 25 00:01:00.863428 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:00.863436    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cw6q6\" (UniqueName: \"kubernetes.io/projected/612ec524-b9db-4ff7-94ae-98e0bb33e686-kube-api-access-cw6q6\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 25 00:01:01.072708 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:01.072676    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-438308e8-75f6dc978b-7bbmf"]
Jul 25 00:01:01.076015 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:01.075991    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-438308e8-75f6dc978b-7bbmf"]
Jul 25 00:01:02.204028 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:02.203997    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="612ec524-b9db-4ff7-94ae-98e0bb33e686" path="/var/lib/kubelet/pods/612ec524-b9db-4ff7-94ae-98e0bb33e686/volumes"
Jul 25 00:01:05.115374 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:05.115336    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:01:05.124333 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:05.124303    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-438308e8-n9rn2"]
Jul 25 00:01:05.126346 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:05.126321    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-438308e8-n9rn2"]
Jul 25 00:01:05.527560 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:05.527511    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:01:06.204608 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:06.204577    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f555b938-f372-4cef-ac2f-ff6d3b225994" path="/var/lib/kubelet/pods/f555b938-f372-4cef-ac2f-ff6d3b225994/volumes"
Jul 25 00:01:11.164785 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:11.164747    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:01:11.265955 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:11.265901    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:01:11.453213 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:11.453129    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:01:11.666836 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:11.666806    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:01:11.803245 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:11.803214    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:01:11.902798 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:11.902743    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:01:12.041919 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:12.041887    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:01:17.685228 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:17.685193    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:01:17.791659 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:17.791615    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:01:17.885611 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:17.885579    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:01:18.078435 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:18.078403    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:01:23.713390 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:23.713351    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:01:23.780352 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:23.780325    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:01:30.627279 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:30.627245    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:01:30.956016 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:30.955935    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:01:31.220786 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:31.220670    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:01:33.673892 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:33.673840    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:01:33.735396 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:33.735354    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-292ac4d2-858cccdb94-jz2gc"]
Jul 25 00:01:33.735907 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:33.735888    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="612ec524-b9db-4ff7-94ae-98e0bb33e686" containerName="maas-api"
Jul 25 00:01:33.735958 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:33.735910    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="612ec524-b9db-4ff7-94ae-98e0bb33e686" containerName="maas-api"
Jul 25 00:01:33.735958 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:33.735935    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f555b938-f372-4cef-ac2f-ff6d3b225994" containerName="revoke-keys"
Jul 25 00:01:33.735958 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:33.735944    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="f555b938-f372-4cef-ac2f-ff6d3b225994" containerName="revoke-keys"
Jul 25 00:01:33.736051 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:33.736039    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="f555b938-f372-4cef-ac2f-ff6d3b225994" containerName="revoke-keys"
Jul 25 00:01:33.736085 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:33.736055    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="612ec524-b9db-4ff7-94ae-98e0bb33e686" containerName="maas-api"
Jul 25 00:01:33.739569 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:33.739545    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-292ac4d2-858cccdb94-jz2gc"
Jul 25 00:01:33.743253 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:33.743231    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-292ac4d2-serving-cert\""
Jul 25 00:01:33.750328 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:33.750263    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-292ac4d2-858cccdb94-jz2gc"]
Jul 25 00:01:33.760904 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:33.760874    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:01:33.821923 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:33.821894    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:01:33.845142 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:33.845103    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b45771b6-e23b-49bf-b2bc-f898498abe90-openshift-service-ca\") pod \"maas-api-e2e-mt-292ac4d2-858cccdb94-jz2gc\" (UID: \"b45771b6-e23b-49bf-b2bc-f898498abe90\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-292ac4d2-858cccdb94-jz2gc"
Jul 25 00:01:33.845306 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:33.845157    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqm8x\" (UniqueName: \"kubernetes.io/projected/b45771b6-e23b-49bf-b2bc-f898498abe90-kube-api-access-tqm8x\") pod \"maas-api-e2e-mt-292ac4d2-858cccdb94-jz2gc\" (UID: \"b45771b6-e23b-49bf-b2bc-f898498abe90\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-292ac4d2-858cccdb94-jz2gc"
Jul 25 00:01:33.845306 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:33.845238    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b45771b6-e23b-49bf-b2bc-f898498abe90-maas-api-tls\") pod \"maas-api-e2e-mt-292ac4d2-858cccdb94-jz2gc\" (UID: \"b45771b6-e23b-49bf-b2bc-f898498abe90\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-292ac4d2-858cccdb94-jz2gc"
Jul 25 00:01:33.946179 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:33.946094    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b45771b6-e23b-49bf-b2bc-f898498abe90-openshift-service-ca\") pod \"maas-api-e2e-mt-292ac4d2-858cccdb94-jz2gc\" (UID: \"b45771b6-e23b-49bf-b2bc-f898498abe90\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-292ac4d2-858cccdb94-jz2gc"
Jul 25 00:01:33.946179 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:33.946138    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqm8x\" (UniqueName: \"kubernetes.io/projected/b45771b6-e23b-49bf-b2bc-f898498abe90-kube-api-access-tqm8x\") pod \"maas-api-e2e-mt-292ac4d2-858cccdb94-jz2gc\" (UID: \"b45771b6-e23b-49bf-b2bc-f898498abe90\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-292ac4d2-858cccdb94-jz2gc"
Jul 25 00:01:33.946401 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:33.946261    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b45771b6-e23b-49bf-b2bc-f898498abe90-maas-api-tls\") pod \"maas-api-e2e-mt-292ac4d2-858cccdb94-jz2gc\" (UID: \"b45771b6-e23b-49bf-b2bc-f898498abe90\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-292ac4d2-858cccdb94-jz2gc"
Jul 25 00:01:33.946467 ip-10-0-142-133 kubenswrapper[2558]: E0725 00:01:33.946399    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-292ac4d2-serving-cert: secret "maas-api-e2e-mt-292ac4d2-serving-cert" not found
Jul 25 00:01:33.946521 ip-10-0-142-133 kubenswrapper[2558]: E0725 00:01:33.946484    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b45771b6-e23b-49bf-b2bc-f898498abe90-maas-api-tls podName:b45771b6-e23b-49bf-b2bc-f898498abe90 nodeName:}" failed. No retries permitted until 2026-07-25 00:01:34.44646101 +0000 UTC m=+1934.832891730 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b45771b6-e23b-49bf-b2bc-f898498abe90-maas-api-tls") pod "maas-api-e2e-mt-292ac4d2-858cccdb94-jz2gc" (UID: "b45771b6-e23b-49bf-b2bc-f898498abe90") : secret "maas-api-e2e-mt-292ac4d2-serving-cert" not found
Jul 25 00:01:33.946738 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:33.946719    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b45771b6-e23b-49bf-b2bc-f898498abe90-openshift-service-ca\") pod \"maas-api-e2e-mt-292ac4d2-858cccdb94-jz2gc\" (UID: \"b45771b6-e23b-49bf-b2bc-f898498abe90\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-292ac4d2-858cccdb94-jz2gc"
Jul 25 00:01:33.954046 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:33.954025    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqm8x\" (UniqueName: \"kubernetes.io/projected/b45771b6-e23b-49bf-b2bc-f898498abe90-kube-api-access-tqm8x\") pod \"maas-api-e2e-mt-292ac4d2-858cccdb94-jz2gc\" (UID: \"b45771b6-e23b-49bf-b2bc-f898498abe90\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-292ac4d2-858cccdb94-jz2gc"
Jul 25 00:01:34.196442 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:34.196363    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:01:34.323880 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:34.323850    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:01:34.432631 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:34.432597    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-292ac4d2-ffdf5cb-xpdc8"]
Jul 25 00:01:34.436691 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:34.436667    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-292ac4d2-ffdf5cb-xpdc8"
Jul 25 00:01:34.439323 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:34.439302    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-292ac4d2\""
Jul 25 00:01:34.439442 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:34.439362    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-292ac4d2-dockercfg-5qg5b\""
Jul 25 00:01:34.444387 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:34.444364    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-292ac4d2-ffdf5cb-xpdc8"]
Jul 25 00:01:34.452087 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:34.452024    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b45771b6-e23b-49bf-b2bc-f898498abe90-maas-api-tls\") pod \"maas-api-e2e-mt-292ac4d2-858cccdb94-jz2gc\" (UID: \"b45771b6-e23b-49bf-b2bc-f898498abe90\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-292ac4d2-858cccdb94-jz2gc"
Jul 25 00:01:34.454271 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:34.454248    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b45771b6-e23b-49bf-b2bc-f898498abe90-maas-api-tls\") pod \"maas-api-e2e-mt-292ac4d2-858cccdb94-jz2gc\" (UID: \"b45771b6-e23b-49bf-b2bc-f898498abe90\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-292ac4d2-858cccdb94-jz2gc"
Jul 25 00:01:34.552826 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:34.552792    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/43aaea7f-ff29-49c8-ba36-05503d6e1e37-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-292ac4d2-ffdf5cb-xpdc8\" (UID: \"43aaea7f-ff29-49c8-ba36-05503d6e1e37\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-292ac4d2-ffdf5cb-xpdc8"
Jul 25 00:01:34.553033 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:34.552832    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prm6w\" (UniqueName: \"kubernetes.io/projected/43aaea7f-ff29-49c8-ba36-05503d6e1e37-kube-api-access-prm6w\") pod \"payload-pre-processing-e2e-mt-292ac4d2-ffdf5cb-xpdc8\" (UID: \"43aaea7f-ff29-49c8-ba36-05503d6e1e37\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-292ac4d2-ffdf5cb-xpdc8"
Jul 25 00:01:34.653742 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:34.653704    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-292ac4d2-858cccdb94-jz2gc"
Jul 25 00:01:34.653938 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:34.653885    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/43aaea7f-ff29-49c8-ba36-05503d6e1e37-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-292ac4d2-ffdf5cb-xpdc8\" (UID: \"43aaea7f-ff29-49c8-ba36-05503d6e1e37\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-292ac4d2-ffdf5cb-xpdc8"
Jul 25 00:01:34.653938 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:34.653931    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prm6w\" (UniqueName: \"kubernetes.io/projected/43aaea7f-ff29-49c8-ba36-05503d6e1e37-kube-api-access-prm6w\") pod \"payload-pre-processing-e2e-mt-292ac4d2-ffdf5cb-xpdc8\" (UID: \"43aaea7f-ff29-49c8-ba36-05503d6e1e37\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-292ac4d2-ffdf5cb-xpdc8"
Jul 25 00:01:34.654599 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:34.654581    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/43aaea7f-ff29-49c8-ba36-05503d6e1e37-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-292ac4d2-ffdf5cb-xpdc8\" (UID: \"43aaea7f-ff29-49c8-ba36-05503d6e1e37\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-292ac4d2-ffdf5cb-xpdc8"
Jul 25 00:01:34.662348 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:34.662320    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prm6w\" (UniqueName: \"kubernetes.io/projected/43aaea7f-ff29-49c8-ba36-05503d6e1e37-kube-api-access-prm6w\") pod \"payload-pre-processing-e2e-mt-292ac4d2-ffdf5cb-xpdc8\" (UID: \"43aaea7f-ff29-49c8-ba36-05503d6e1e37\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-292ac4d2-ffdf5cb-xpdc8"
Jul 25 00:01:34.748715 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:34.748669    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-292ac4d2-ffdf5cb-xpdc8"
Jul 25 00:01:34.886077 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:34.886052    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-292ac4d2-ffdf5cb-xpdc8"]
Jul 25 00:01:34.887402 ip-10-0-142-133 kubenswrapper[2558]: W0725 00:01:34.887374    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43aaea7f_ff29_49c8_ba36_05503d6e1e37.slice/crio-e0ddb54ffe7b1af869c94ecdeefdf92599b8035a588b5ec0598a5b479e851026 WatchSource:0}: Error finding container e0ddb54ffe7b1af869c94ecdeefdf92599b8035a588b5ec0598a5b479e851026: Status 404 returned error can't find the container with id e0ddb54ffe7b1af869c94ecdeefdf92599b8035a588b5ec0598a5b479e851026
Jul 25 00:01:34.929041 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:34.928999    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-292ac4d2-ffdf5cb-xpdc8" event={"ID":"43aaea7f-ff29-49c8-ba36-05503d6e1e37","Type":"ContainerStarted","Data":"e0ddb54ffe7b1af869c94ecdeefdf92599b8035a588b5ec0598a5b479e851026"}
Jul 25 00:01:34.993862 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:34.993838    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-292ac4d2-858cccdb94-jz2gc"]
Jul 25 00:01:34.995375 ip-10-0-142-133 kubenswrapper[2558]: W0725 00:01:34.995349    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb45771b6_e23b_49bf_b2bc_f898498abe90.slice/crio-b6ea80106f4dae144293ee8f5bbe08e34583d117607b209af06c5f492681ecde WatchSource:0}: Error finding container b6ea80106f4dae144293ee8f5bbe08e34583d117607b209af06c5f492681ecde: Status 404 returned error can't find the container with id b6ea80106f4dae144293ee8f5bbe08e34583d117607b209af06c5f492681ecde
Jul 25 00:01:34.996993 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:34.996972    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 25 00:01:35.588684 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:35.588657    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:01:35.935823 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:35.935782    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-292ac4d2-ffdf5cb-xpdc8" event={"ID":"43aaea7f-ff29-49c8-ba36-05503d6e1e37","Type":"ContainerStarted","Data":"70949d0128b371609024b7326a99e607e10f094a04474e34b851f6fe5f156e9d"}
Jul 25 00:01:35.936271 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:35.935872    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-292ac4d2-ffdf5cb-xpdc8"
Jul 25 00:01:35.937419 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:35.937392    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-292ac4d2-858cccdb94-jz2gc" event={"ID":"b45771b6-e23b-49bf-b2bc-f898498abe90","Type":"ContainerStarted","Data":"25a4b8fb1d074e4138e7d03a4e1a31e6896aa0c6f4c312a3b1b8f6d22affd851"}
Jul 25 00:01:35.937559 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:35.937443    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-292ac4d2-858cccdb94-jz2gc" event={"ID":"b45771b6-e23b-49bf-b2bc-f898498abe90","Type":"ContainerStarted","Data":"b6ea80106f4dae144293ee8f5bbe08e34583d117607b209af06c5f492681ecde"}
Jul 25 00:01:35.937559 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:35.937470    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-292ac4d2-858cccdb94-jz2gc"
Jul 25 00:01:35.950455 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:35.950409    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-292ac4d2-ffdf5cb-xpdc8" podStartSLOduration=1.950392457 podStartE2EDuration="1.950392457s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:01:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:01:35.949944074 +0000 UTC m=+1936.336374802" watchObservedRunningTime="2026-07-25 00:01:35.950392457 +0000 UTC m=+1936.336823187"
Jul 25 00:01:35.967449 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:35.967398    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-292ac4d2-858cccdb94-jz2gc" podStartSLOduration=2.582917603 podStartE2EDuration="2.967384815s" totalImagesPullingTime="384.467212ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:01:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:01:35.96474023 +0000 UTC m=+1936.351170971" watchObservedRunningTime="2026-07-25 00:01:35.967384815 +0000 UTC m=+1936.353815570"
Jul 25 00:01:38.187004 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:38.186971    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:01:38.781125 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:38.781093    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:01:41.948381 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:41.948349    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-292ac4d2-858cccdb94-jz2gc"
Jul 25 00:01:46.944963 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:46.944929    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-292ac4d2-ffdf5cb-xpdc8"
Jul 25 00:01:56.726707 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:01:56.726668    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:02:00.754203 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:00.754167    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:02:00.854250 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:00.854190    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:02:08.608642 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:08.608591    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:02:23.065657 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:23.065558    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-292ac4d2-fkrkn"]
Jul 25 00:02:23.069725 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:23.069703    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-292ac4d2-fkrkn"
Jul 25 00:02:23.072164 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:23.072137    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bxr46\""
Jul 25 00:02:23.073550 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:23.073506    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-292ac4d2-fkrkn"]
Jul 25 00:02:23.174680 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:23.174647    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ee2259e-d954-4651-a4cd-9a21736dea04-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-292ac4d2-fkrkn\" (UID: \"9ee2259e-d954-4651-a4cd-9a21736dea04\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-292ac4d2-fkrkn"
Jul 25 00:02:23.275642 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:23.275607    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ee2259e-d954-4651-a4cd-9a21736dea04-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-292ac4d2-fkrkn\" (UID: \"9ee2259e-d954-4651-a4cd-9a21736dea04\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-292ac4d2-fkrkn"
Jul 25 00:02:23.276234 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:23.276215    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ee2259e-d954-4651-a4cd-9a21736dea04-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-292ac4d2-fkrkn\" (UID: \"9ee2259e-d954-4651-a4cd-9a21736dea04\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-292ac4d2-fkrkn"
Jul 25 00:02:23.381912 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:23.381841    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-292ac4d2-fkrkn"
Jul 25 00:02:23.510347 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:23.510325    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-292ac4d2-fkrkn"]
Jul 25 00:02:23.512011 ip-10-0-142-133 kubenswrapper[2558]: W0725 00:02:23.511978    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ee2259e_d954_4651_a4cd_9a21736dea04.slice/crio-b8b90a74a7a1ffd2d4870216ff55b0eaa05ef9a3b7a2b05204cae7c3a2969481 WatchSource:0}: Error finding container b8b90a74a7a1ffd2d4870216ff55b0eaa05ef9a3b7a2b05204cae7c3a2969481: Status 404 returned error can't find the container with id b8b90a74a7a1ffd2d4870216ff55b0eaa05ef9a3b7a2b05204cae7c3a2969481
Jul 25 00:02:24.196770 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:24.196741    2558 generic.go:356] "Generic (PLEG): container finished" podID="9ee2259e-d954-4651-a4cd-9a21736dea04" containerID="d921ba465882c9ba3e2eee667d789069539d690247c9aaa8f0cd623528ef147a" exitCode=0
Jul 25 00:02:24.197109 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:24.196827    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-292ac4d2-fkrkn" event={"ID":"9ee2259e-d954-4651-a4cd-9a21736dea04","Type":"ContainerDied","Data":"d921ba465882c9ba3e2eee667d789069539d690247c9aaa8f0cd623528ef147a"}
Jul 25 00:02:24.197109 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:24.196883    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-292ac4d2-fkrkn" event={"ID":"9ee2259e-d954-4651-a4cd-9a21736dea04","Type":"ContainerStarted","Data":"b8b90a74a7a1ffd2d4870216ff55b0eaa05ef9a3b7a2b05204cae7c3a2969481"}
Jul 25 00:02:25.333027 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:25.332998    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-292ac4d2-fkrkn"
Jul 25 00:02:25.495864 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:25.495836    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ee2259e-d954-4651-a4cd-9a21736dea04-maas-api-service-ca\") pod \"9ee2259e-d954-4651-a4cd-9a21736dea04\" (UID: \"9ee2259e-d954-4651-a4cd-9a21736dea04\") "
Jul 25 00:02:25.496180 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:25.496153    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ee2259e-d954-4651-a4cd-9a21736dea04-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9ee2259e-d954-4651-a4cd-9a21736dea04" (UID: "9ee2259e-d954-4651-a4cd-9a21736dea04"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 00:02:25.596731 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:25.596701    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ee2259e-d954-4651-a4cd-9a21736dea04-maas-api-service-ca\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 25 00:02:26.208944 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:26.208912    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-292ac4d2-fkrkn"
Jul 25 00:02:26.208944 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:26.208925    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-292ac4d2-fkrkn" event={"ID":"9ee2259e-d954-4651-a4cd-9a21736dea04","Type":"ContainerDied","Data":"b8b90a74a7a1ffd2d4870216ff55b0eaa05ef9a3b7a2b05204cae7c3a2969481"}
Jul 25 00:02:26.209147 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:26.208969    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b8b90a74a7a1ffd2d4870216ff55b0eaa05ef9a3b7a2b05204cae7c3a2969481"
Jul 25 00:02:38.513013 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:38.512978    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-292ac4d2-858cccdb94-jz2gc"]
Jul 25 00:02:38.513626 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:38.513182    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-292ac4d2-858cccdb94-jz2gc" podUID="b45771b6-e23b-49bf-b2bc-f898498abe90" containerName="maas-api" containerID="cri-o://25a4b8fb1d074e4138e7d03a4e1a31e6896aa0c6f4c312a3b1b8f6d22affd851" gracePeriod=30
Jul 25 00:02:38.545480 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:38.545433    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:02:38.566568 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:38.566523    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-292ac4d2-ffdf5cb-xpdc8"]
Jul 25 00:02:38.566783 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:38.566755    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-292ac4d2-ffdf5cb-xpdc8" podUID="43aaea7f-ff29-49c8-ba36-05503d6e1e37" containerName="payload-pre-processing" containerID="cri-o://70949d0128b371609024b7326a99e607e10f094a04474e34b851f6fe5f156e9d" gracePeriod=30
Jul 25 00:02:38.815745 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:38.815721    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-292ac4d2-858cccdb94-jz2gc"
Jul 25 00:02:38.847279 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:38.847251    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-292ac4d2-ffdf5cb-xpdc8"
Jul 25 00:02:38.905210 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:38.905178    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/43aaea7f-ff29-49c8-ba36-05503d6e1e37-plugins-config-volume\") pod \"43aaea7f-ff29-49c8-ba36-05503d6e1e37\" (UID: \"43aaea7f-ff29-49c8-ba36-05503d6e1e37\") "
Jul 25 00:02:38.905379 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:38.905229    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tqm8x\" (UniqueName: \"kubernetes.io/projected/b45771b6-e23b-49bf-b2bc-f898498abe90-kube-api-access-tqm8x\") pod \"b45771b6-e23b-49bf-b2bc-f898498abe90\" (UID: \"b45771b6-e23b-49bf-b2bc-f898498abe90\") "
Jul 25 00:02:38.905634 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:38.905612    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43aaea7f-ff29-49c8-ba36-05503d6e1e37-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "43aaea7f-ff29-49c8-ba36-05503d6e1e37" (UID: "43aaea7f-ff29-49c8-ba36-05503d6e1e37"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 00:02:38.907232 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:38.907210    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b45771b6-e23b-49bf-b2bc-f898498abe90-kube-api-access-tqm8x" (OuterVolumeSpecName: "kube-api-access-tqm8x") pod "b45771b6-e23b-49bf-b2bc-f898498abe90" (UID: "b45771b6-e23b-49bf-b2bc-f898498abe90"). InnerVolumeSpecName "kube-api-access-tqm8x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 00:02:39.006480 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:39.006449    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b45771b6-e23b-49bf-b2bc-f898498abe90-openshift-service-ca\") pod \"b45771b6-e23b-49bf-b2bc-f898498abe90\" (UID: \"b45771b6-e23b-49bf-b2bc-f898498abe90\") "
Jul 25 00:02:39.006480 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:39.006485    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b45771b6-e23b-49bf-b2bc-f898498abe90-maas-api-tls\") pod \"b45771b6-e23b-49bf-b2bc-f898498abe90\" (UID: \"b45771b6-e23b-49bf-b2bc-f898498abe90\") "
Jul 25 00:02:39.006717 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:39.006507    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-prm6w\" (UniqueName: \"kubernetes.io/projected/43aaea7f-ff29-49c8-ba36-05503d6e1e37-kube-api-access-prm6w\") pod \"43aaea7f-ff29-49c8-ba36-05503d6e1e37\" (UID: \"43aaea7f-ff29-49c8-ba36-05503d6e1e37\") "
Jul 25 00:02:39.006784 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:39.006760    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/43aaea7f-ff29-49c8-ba36-05503d6e1e37-plugins-config-volume\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 25 00:02:39.006784 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:39.006779    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tqm8x\" (UniqueName: \"kubernetes.io/projected/b45771b6-e23b-49bf-b2bc-f898498abe90-kube-api-access-tqm8x\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 25 00:02:39.006928 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:39.006910    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b45771b6-e23b-49bf-b2bc-f898498abe90-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b45771b6-e23b-49bf-b2bc-f898498abe90" (UID: "b45771b6-e23b-49bf-b2bc-f898498abe90"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 00:02:39.008522 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:39.008500    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43aaea7f-ff29-49c8-ba36-05503d6e1e37-kube-api-access-prm6w" (OuterVolumeSpecName: "kube-api-access-prm6w") pod "43aaea7f-ff29-49c8-ba36-05503d6e1e37" (UID: "43aaea7f-ff29-49c8-ba36-05503d6e1e37"). InnerVolumeSpecName "kube-api-access-prm6w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 00:02:39.008600 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:39.008550    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b45771b6-e23b-49bf-b2bc-f898498abe90-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b45771b6-e23b-49bf-b2bc-f898498abe90" (UID: "b45771b6-e23b-49bf-b2bc-f898498abe90"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 00:02:39.107617 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:39.107526    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b45771b6-e23b-49bf-b2bc-f898498abe90-openshift-service-ca\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 25 00:02:39.107617 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:39.107573    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b45771b6-e23b-49bf-b2bc-f898498abe90-maas-api-tls\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 25 00:02:39.107617 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:39.107584    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-prm6w\" (UniqueName: \"kubernetes.io/projected/43aaea7f-ff29-49c8-ba36-05503d6e1e37-kube-api-access-prm6w\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 25 00:02:39.279549 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:39.279508    2558 generic.go:356] "Generic (PLEG): container finished" podID="43aaea7f-ff29-49c8-ba36-05503d6e1e37" containerID="70949d0128b371609024b7326a99e607e10f094a04474e34b851f6fe5f156e9d" exitCode=0
Jul 25 00:02:39.279741 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:39.279581    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-292ac4d2-ffdf5cb-xpdc8"
Jul 25 00:02:39.279741 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:39.279597    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-292ac4d2-ffdf5cb-xpdc8" event={"ID":"43aaea7f-ff29-49c8-ba36-05503d6e1e37","Type":"ContainerDied","Data":"70949d0128b371609024b7326a99e607e10f094a04474e34b851f6fe5f156e9d"}
Jul 25 00:02:39.279741 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:39.279650    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-292ac4d2-ffdf5cb-xpdc8" event={"ID":"43aaea7f-ff29-49c8-ba36-05503d6e1e37","Type":"ContainerDied","Data":"e0ddb54ffe7b1af869c94ecdeefdf92599b8035a588b5ec0598a5b479e851026"}
Jul 25 00:02:39.279741 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:39.279674    2558 scope.go:117] "RemoveContainer" containerID="70949d0128b371609024b7326a99e607e10f094a04474e34b851f6fe5f156e9d"
Jul 25 00:02:39.281055 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:39.281027    2558 generic.go:356] "Generic (PLEG): container finished" podID="b45771b6-e23b-49bf-b2bc-f898498abe90" containerID="25a4b8fb1d074e4138e7d03a4e1a31e6896aa0c6f4c312a3b1b8f6d22affd851" exitCode=0
Jul 25 00:02:39.281189 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:39.281070    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-292ac4d2-858cccdb94-jz2gc"
Jul 25 00:02:39.281189 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:39.281086    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-292ac4d2-858cccdb94-jz2gc" event={"ID":"b45771b6-e23b-49bf-b2bc-f898498abe90","Type":"ContainerDied","Data":"25a4b8fb1d074e4138e7d03a4e1a31e6896aa0c6f4c312a3b1b8f6d22affd851"}
Jul 25 00:02:39.281189 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:39.281130    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-292ac4d2-858cccdb94-jz2gc" event={"ID":"b45771b6-e23b-49bf-b2bc-f898498abe90","Type":"ContainerDied","Data":"b6ea80106f4dae144293ee8f5bbe08e34583d117607b209af06c5f492681ecde"}
Jul 25 00:02:39.288313 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:39.288300    2558 scope.go:117] "RemoveContainer" containerID="70949d0128b371609024b7326a99e607e10f094a04474e34b851f6fe5f156e9d"
Jul 25 00:02:39.288596 ip-10-0-142-133 kubenswrapper[2558]: E0725 00:02:39.288574    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70949d0128b371609024b7326a99e607e10f094a04474e34b851f6fe5f156e9d\": container with ID starting with 70949d0128b371609024b7326a99e607e10f094a04474e34b851f6fe5f156e9d not found: ID does not exist" containerID="70949d0128b371609024b7326a99e607e10f094a04474e34b851f6fe5f156e9d"
Jul 25 00:02:39.288699 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:39.288605    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70949d0128b371609024b7326a99e607e10f094a04474e34b851f6fe5f156e9d"} err="failed to get container status \"70949d0128b371609024b7326a99e607e10f094a04474e34b851f6fe5f156e9d\": rpc error: code = NotFound desc = could not find container \"70949d0128b371609024b7326a99e607e10f094a04474e34b851f6fe5f156e9d\": container with ID starting with 70949d0128b371609024b7326a99e607e10f094a04474e34b851f6fe5f156e9d not found: ID does not exist"
Jul 25 00:02:39.288699 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:39.288623    2558 scope.go:117] "RemoveContainer" containerID="25a4b8fb1d074e4138e7d03a4e1a31e6896aa0c6f4c312a3b1b8f6d22affd851"
Jul 25 00:02:39.297024 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:39.297006    2558 scope.go:117] "RemoveContainer" containerID="25a4b8fb1d074e4138e7d03a4e1a31e6896aa0c6f4c312a3b1b8f6d22affd851"
Jul 25 00:02:39.297278 ip-10-0-142-133 kubenswrapper[2558]: E0725 00:02:39.297258    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"25a4b8fb1d074e4138e7d03a4e1a31e6896aa0c6f4c312a3b1b8f6d22affd851\": container with ID starting with 25a4b8fb1d074e4138e7d03a4e1a31e6896aa0c6f4c312a3b1b8f6d22affd851 not found: ID does not exist" containerID="25a4b8fb1d074e4138e7d03a4e1a31e6896aa0c6f4c312a3b1b8f6d22affd851"
Jul 25 00:02:39.297336 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:39.297287    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25a4b8fb1d074e4138e7d03a4e1a31e6896aa0c6f4c312a3b1b8f6d22affd851"} err="failed to get container status \"25a4b8fb1d074e4138e7d03a4e1a31e6896aa0c6f4c312a3b1b8f6d22affd851\": rpc error: code = NotFound desc = could not find container \"25a4b8fb1d074e4138e7d03a4e1a31e6896aa0c6f4c312a3b1b8f6d22affd851\": container with ID starting with 25a4b8fb1d074e4138e7d03a4e1a31e6896aa0c6f4c312a3b1b8f6d22affd851 not found: ID does not exist"
Jul 25 00:02:39.303745 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:39.303721    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-292ac4d2-ffdf5cb-xpdc8"]
Jul 25 00:02:39.307182 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:39.307157    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-292ac4d2-ffdf5cb-xpdc8"]
Jul 25 00:02:39.316289 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:39.316264    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-292ac4d2-858cccdb94-jz2gc"]
Jul 25 00:02:39.319552 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:39.319518    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-292ac4d2-858cccdb94-jz2gc"]
Jul 25 00:02:39.825927 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:39.825895    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:02:40.204394 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:40.204363    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43aaea7f-ff29-49c8-ba36-05503d6e1e37" path="/var/lib/kubelet/pods/43aaea7f-ff29-49c8-ba36-05503d6e1e37/volumes"
Jul 25 00:02:40.204779 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:40.204764    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b45771b6-e23b-49bf-b2bc-f898498abe90" path="/var/lib/kubelet/pods/b45771b6-e23b-49bf-b2bc-f898498abe90/volumes"
Jul 25 00:02:43.136947 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:43.136911    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-292ac4d2-fkrkn"]
Jul 25 00:02:43.138191 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:43.138160    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:02:43.139023 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:43.139001    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-292ac4d2-fkrkn"]
Jul 25 00:02:44.204252 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:44.204219    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ee2259e-d954-4651-a4cd-9a21736dea04" path="/var/lib/kubelet/pods/9ee2259e-d954-4651-a4cd-9a21736dea04/volumes"
Jul 25 00:02:46.584209 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:46.584178    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:02:51.658082 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:02:51.658044    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:03:00.993047 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:03:00.993003    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:03:20.328126 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:03:20.328091    2558 scope.go:117] "RemoveContainer" containerID="843234444c4bc6d98104b0158965a4ddbb6f99a054ce43a4ca2c023560c015a5"
Jul 25 00:03:28.153281 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:03:28.153245    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:03:28.585677 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:03:28.585639    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:03:33.953634 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:03:33.953600    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:03:34.054803 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:03:34.054767    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:03:34.237776 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:03:34.237728    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:03:34.501575 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:03:34.501469    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:03:35.600339 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:03:35.600301    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:03:35.684763 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:03:35.684733    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:03:35.786437 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:03:35.786406    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:03:36.547165 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:03:36.547127    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:03:36.645598 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:03:36.645560    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:03:36.790647 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:03:36.790614    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:03:38.195722 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:03:38.195686    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:03:43.897029 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:03:43.896945    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:03:43.956582 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:03:43.956525    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:03:44.627401 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:03:44.627359    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:03:50.654014 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:03:50.653981    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:03:50.760283 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:03:50.760251    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:03:50.856927 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:03:50.856896    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:04:01.411501 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:01.411447    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:04:01.799439 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:01.799398    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:04:20.355318 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:20.355287    2558 scope.go:117] "RemoveContainer" containerID="84db80ba39035df403d0de589328fddc88a45b5cdf08c13f632c31290f73be59"
Jul 25 00:04:22.825925 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:22.825888    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:04:28.797802 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:28.797762    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:04:28.898168 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:28.898136    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:04:29.109693 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:29.109610    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:04:29.438786 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:29.438706    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:04:29.574728 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:29.574690    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:04:32.601684 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:32.601654    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:04:32.698114 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:32.698080    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:04:32.699812 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:32.699788    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-820f12-7dbd55d7bb-6vv5l"]
Jul 25 00:04:32.700160 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:32.700142    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43aaea7f-ff29-49c8-ba36-05503d6e1e37" containerName="payload-pre-processing"
Jul 25 00:04:32.700211 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:32.700163    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="43aaea7f-ff29-49c8-ba36-05503d6e1e37" containerName="payload-pre-processing"
Jul 25 00:04:32.700211 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:32.700180    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b45771b6-e23b-49bf-b2bc-f898498abe90" containerName="maas-api"
Jul 25 00:04:32.700211 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:32.700191    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="b45771b6-e23b-49bf-b2bc-f898498abe90" containerName="maas-api"
Jul 25 00:04:32.700326 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:32.700232    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ee2259e-d954-4651-a4cd-9a21736dea04" containerName="revoke-keys"
Jul 25 00:04:32.700326 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:32.700243    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ee2259e-d954-4651-a4cd-9a21736dea04" containerName="revoke-keys"
Jul 25 00:04:32.700326 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:32.700317    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="43aaea7f-ff29-49c8-ba36-05503d6e1e37" containerName="payload-pre-processing"
Jul 25 00:04:32.700421 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:32.700329    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="b45771b6-e23b-49bf-b2bc-f898498abe90" containerName="maas-api"
Jul 25 00:04:32.700421 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:32.700337    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ee2259e-d954-4651-a4cd-9a21736dea04" containerName="revoke-keys"
Jul 25 00:04:32.703320 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:32.703299    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-820f12-7dbd55d7bb-6vv5l"
Jul 25 00:04:32.706099 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:32.706082    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-820f12-serving-cert\""
Jul 25 00:04:32.713930 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:32.713908    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-820f12-7dbd55d7bb-6vv5l"]
Jul 25 00:04:32.771198 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:32.771171    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0737648e-5145-4ffe-9344-275b82712ce3-openshift-service-ca\") pod \"maas-api-e2e-inf-a-820f12-7dbd55d7bb-6vv5l\" (UID: \"0737648e-5145-4ffe-9344-275b82712ce3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-820f12-7dbd55d7bb-6vv5l"
Jul 25 00:04:32.771338 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:32.771212    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0737648e-5145-4ffe-9344-275b82712ce3-maas-api-tls\") pod \"maas-api-e2e-inf-a-820f12-7dbd55d7bb-6vv5l\" (UID: \"0737648e-5145-4ffe-9344-275b82712ce3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-820f12-7dbd55d7bb-6vv5l"
Jul 25 00:04:32.771391 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:32.771345    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kk4tn\" (UniqueName: \"kubernetes.io/projected/0737648e-5145-4ffe-9344-275b82712ce3-kube-api-access-kk4tn\") pod \"maas-api-e2e-inf-a-820f12-7dbd55d7bb-6vv5l\" (UID: \"0737648e-5145-4ffe-9344-275b82712ce3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-820f12-7dbd55d7bb-6vv5l"
Jul 25 00:04:32.840599 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:32.840555    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:04:32.871849 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:32.871777    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0737648e-5145-4ffe-9344-275b82712ce3-openshift-service-ca\") pod \"maas-api-e2e-inf-a-820f12-7dbd55d7bb-6vv5l\" (UID: \"0737648e-5145-4ffe-9344-275b82712ce3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-820f12-7dbd55d7bb-6vv5l"
Jul 25 00:04:32.871849 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:32.871809    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0737648e-5145-4ffe-9344-275b82712ce3-maas-api-tls\") pod \"maas-api-e2e-inf-a-820f12-7dbd55d7bb-6vv5l\" (UID: \"0737648e-5145-4ffe-9344-275b82712ce3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-820f12-7dbd55d7bb-6vv5l"
Jul 25 00:04:32.871849 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:32.871847    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kk4tn\" (UniqueName: \"kubernetes.io/projected/0737648e-5145-4ffe-9344-275b82712ce3-kube-api-access-kk4tn\") pod \"maas-api-e2e-inf-a-820f12-7dbd55d7bb-6vv5l\" (UID: \"0737648e-5145-4ffe-9344-275b82712ce3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-820f12-7dbd55d7bb-6vv5l"
Jul 25 00:04:32.872061 ip-10-0-142-133 kubenswrapper[2558]: E0725 00:04:32.871955    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-820f12-serving-cert: secret "maas-api-e2e-inf-a-820f12-serving-cert" not found
Jul 25 00:04:32.872061 ip-10-0-142-133 kubenswrapper[2558]: E0725 00:04:32.872035    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0737648e-5145-4ffe-9344-275b82712ce3-maas-api-tls podName:0737648e-5145-4ffe-9344-275b82712ce3 nodeName:}" failed. No retries permitted until 2026-07-25 00:04:33.372014449 +0000 UTC m=+2113.758445158 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0737648e-5145-4ffe-9344-275b82712ce3-maas-api-tls") pod "maas-api-e2e-inf-a-820f12-7dbd55d7bb-6vv5l" (UID: "0737648e-5145-4ffe-9344-275b82712ce3") : secret "maas-api-e2e-inf-a-820f12-serving-cert" not found
Jul 25 00:04:32.872393 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:32.872375    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0737648e-5145-4ffe-9344-275b82712ce3-openshift-service-ca\") pod \"maas-api-e2e-inf-a-820f12-7dbd55d7bb-6vv5l\" (UID: \"0737648e-5145-4ffe-9344-275b82712ce3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-820f12-7dbd55d7bb-6vv5l"
Jul 25 00:04:32.883003 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:32.882981    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kk4tn\" (UniqueName: \"kubernetes.io/projected/0737648e-5145-4ffe-9344-275b82712ce3-kube-api-access-kk4tn\") pod \"maas-api-e2e-inf-a-820f12-7dbd55d7bb-6vv5l\" (UID: \"0737648e-5145-4ffe-9344-275b82712ce3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-820f12-7dbd55d7bb-6vv5l"
Jul 25 00:04:33.109031 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:33.109000    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:04:33.280830 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:33.280799    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:04:33.377321 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:33.377293    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0737648e-5145-4ffe-9344-275b82712ce3-maas-api-tls\") pod \"maas-api-e2e-inf-a-820f12-7dbd55d7bb-6vv5l\" (UID: \"0737648e-5145-4ffe-9344-275b82712ce3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-820f12-7dbd55d7bb-6vv5l"
Jul 25 00:04:33.379726 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:33.379696    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0737648e-5145-4ffe-9344-275b82712ce3-maas-api-tls\") pod \"maas-api-e2e-inf-a-820f12-7dbd55d7bb-6vv5l\" (UID: \"0737648e-5145-4ffe-9344-275b82712ce3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-820f12-7dbd55d7bb-6vv5l"
Jul 25 00:04:33.385431 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:33.385407    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:04:33.387884 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:33.387861    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-820f12-745f95844d-pdj5h"]
Jul 25 00:04:33.391632 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:33.391617    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-820f12-745f95844d-pdj5h"
Jul 25 00:04:33.394045 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:33.394026    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-820f12\""
Jul 25 00:04:33.394153 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:33.394105    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-820f12-dockercfg-wp5l9\""
Jul 25 00:04:33.398640 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:33.398620    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-820f12-745f95844d-pdj5h"]
Jul 25 00:04:33.478133 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:33.478096    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6551de2-8a80-400c-a830-82f63930b5f4-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-820f12-745f95844d-pdj5h\" (UID: \"b6551de2-8a80-400c-a830-82f63930b5f4\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-820f12-745f95844d-pdj5h"
Jul 25 00:04:33.478316 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:33.478178    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvmt4\" (UniqueName: \"kubernetes.io/projected/b6551de2-8a80-400c-a830-82f63930b5f4-kube-api-access-xvmt4\") pod \"payload-pre-processing-e2e-inf-a-820f12-745f95844d-pdj5h\" (UID: \"b6551de2-8a80-400c-a830-82f63930b5f4\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-820f12-745f95844d-pdj5h"
Jul 25 00:04:33.579357 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:33.579278    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvmt4\" (UniqueName: \"kubernetes.io/projected/b6551de2-8a80-400c-a830-82f63930b5f4-kube-api-access-xvmt4\") pod \"payload-pre-processing-e2e-inf-a-820f12-745f95844d-pdj5h\" (UID: \"b6551de2-8a80-400c-a830-82f63930b5f4\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-820f12-745f95844d-pdj5h"
Jul 25 00:04:33.579357 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:33.579348    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6551de2-8a80-400c-a830-82f63930b5f4-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-820f12-745f95844d-pdj5h\" (UID: \"b6551de2-8a80-400c-a830-82f63930b5f4\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-820f12-745f95844d-pdj5h"
Jul 25 00:04:33.579995 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:33.579976    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6551de2-8a80-400c-a830-82f63930b5f4-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-820f12-745f95844d-pdj5h\" (UID: \"b6551de2-8a80-400c-a830-82f63930b5f4\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-820f12-745f95844d-pdj5h"
Jul 25 00:04:33.587282 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:33.587256    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvmt4\" (UniqueName: \"kubernetes.io/projected/b6551de2-8a80-400c-a830-82f63930b5f4-kube-api-access-xvmt4\") pod \"payload-pre-processing-e2e-inf-a-820f12-745f95844d-pdj5h\" (UID: \"b6551de2-8a80-400c-a830-82f63930b5f4\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-820f12-745f95844d-pdj5h"
Jul 25 00:04:33.615958 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:33.615936    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-820f12-7dbd55d7bb-6vv5l"
Jul 25 00:04:33.703968 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:33.703945    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-820f12-745f95844d-pdj5h"
Jul 25 00:04:33.756080 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:33.754436    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-820f12-7dbd55d7bb-6vv5l"]
Jul 25 00:04:33.846477 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:33.846450    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-820f12-745f95844d-pdj5h"]
Jul 25 00:04:33.847677 ip-10-0-142-133 kubenswrapper[2558]: W0725 00:04:33.847647    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6551de2_8a80_400c_a830_82f63930b5f4.slice/crio-f6d69340db8d240296cab5f1a09d188a3a50cf1f30beef8a2838ddc887c6297a WatchSource:0}: Error finding container f6d69340db8d240296cab5f1a09d188a3a50cf1f30beef8a2838ddc887c6297a: Status 404 returned error can't find the container with id f6d69340db8d240296cab5f1a09d188a3a50cf1f30beef8a2838ddc887c6297a
Jul 25 00:04:33.879294 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:33.879267    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-820f12-745f95844d-pdj5h" event={"ID":"b6551de2-8a80-400c-a830-82f63930b5f4","Type":"ContainerStarted","Data":"f6d69340db8d240296cab5f1a09d188a3a50cf1f30beef8a2838ddc887c6297a"}
Jul 25 00:04:33.881089 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:33.880777    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-820f12-7dbd55d7bb-6vv5l" event={"ID":"0737648e-5145-4ffe-9344-275b82712ce3","Type":"ContainerStarted","Data":"ce760a31a1c25f0766de102ea3c1311c6035328f9e859337f97afb3dfe35e25c"}
Jul 25 00:04:34.887307 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:34.887273    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-820f12-745f95844d-pdj5h" event={"ID":"b6551de2-8a80-400c-a830-82f63930b5f4","Type":"ContainerStarted","Data":"34d6a7c22b97e09c4e9e3287ab071ab4e3c793906b67f50adb26d4258a00b632"}
Jul 25 00:04:34.887786 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:34.887385    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-820f12-745f95844d-pdj5h"
Jul 25 00:04:34.888911 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:34.888889    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-820f12-7dbd55d7bb-6vv5l" event={"ID":"0737648e-5145-4ffe-9344-275b82712ce3","Type":"ContainerStarted","Data":"3b314c3c75043db5a21334f7f9a6f5295631666600a35f802868b4631d97561a"}
Jul 25 00:04:34.889026 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:34.888937    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-820f12-7dbd55d7bb-6vv5l"
Jul 25 00:04:34.902001 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:34.901945    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-820f12-745f95844d-pdj5h" podStartSLOduration=1.901927004 podStartE2EDuration="1.901927004s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:04:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:04:34.901095681 +0000 UTC m=+2115.287526423" watchObservedRunningTime="2026-07-25 00:04:34.901927004 +0000 UTC m=+2115.288357731"
Jul 25 00:04:34.916263 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:34.916223    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-820f12-7dbd55d7bb-6vv5l" podStartSLOduration=2.311973469 podStartE2EDuration="2.916211501s" totalImagesPullingTime="604.238032ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:04:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:04:34.914860006 +0000 UTC m=+2115.301290745" watchObservedRunningTime="2026-07-25 00:04:34.916211501 +0000 UTC m=+2115.302642229"
Jul 25 00:04:37.145851 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:37.145818    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:04:37.502203 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:37.502163    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:04:37.823462 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:37.823355    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:04:38.015007 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:38.014967    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:04:40.683011 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:40.682976    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:04:40.754936 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:40.754900    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-107131-7747f587f-4d6mp"]
Jul 25 00:04:40.758765 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:40.758744    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-107131-7747f587f-4d6mp"
Jul 25 00:04:40.759317 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:40.759285    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:04:40.761025 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:40.761003    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-107131-serving-cert\""
Jul 25 00:04:40.769764 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:40.769666    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-107131-7747f587f-4d6mp"]
Jul 25 00:04:40.828449 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:40.828415    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:04:40.838718 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:40.838685    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6a6cfdf-a9b4-444e-a6c7-10823ff1f242-openshift-service-ca\") pod \"maas-api-e2e-inf-b-107131-7747f587f-4d6mp\" (UID: \"e6a6cfdf-a9b4-444e-a6c7-10823ff1f242\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-107131-7747f587f-4d6mp"
Jul 25 00:04:40.838883 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:40.838749    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddkb7\" (UniqueName: \"kubernetes.io/projected/e6a6cfdf-a9b4-444e-a6c7-10823ff1f242-kube-api-access-ddkb7\") pod \"maas-api-e2e-inf-b-107131-7747f587f-4d6mp\" (UID: \"e6a6cfdf-a9b4-444e-a6c7-10823ff1f242\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-107131-7747f587f-4d6mp"
Jul 25 00:04:40.838883 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:40.838828    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6a6cfdf-a9b4-444e-a6c7-10823ff1f242-maas-api-tls\") pod \"maas-api-e2e-inf-b-107131-7747f587f-4d6mp\" (UID: \"e6a6cfdf-a9b4-444e-a6c7-10823ff1f242\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-107131-7747f587f-4d6mp"
Jul 25 00:04:40.900035 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:40.900008    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-820f12-7dbd55d7bb-6vv5l"
Jul 25 00:04:40.940315 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:40.940223    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6a6cfdf-a9b4-444e-a6c7-10823ff1f242-openshift-service-ca\") pod \"maas-api-e2e-inf-b-107131-7747f587f-4d6mp\" (UID: \"e6a6cfdf-a9b4-444e-a6c7-10823ff1f242\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-107131-7747f587f-4d6mp"
Jul 25 00:04:40.941860 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:40.941833    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddkb7\" (UniqueName: \"kubernetes.io/projected/e6a6cfdf-a9b4-444e-a6c7-10823ff1f242-kube-api-access-ddkb7\") pod \"maas-api-e2e-inf-b-107131-7747f587f-4d6mp\" (UID: \"e6a6cfdf-a9b4-444e-a6c7-10823ff1f242\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-107131-7747f587f-4d6mp"
Jul 25 00:04:40.942158 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:40.942141    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6a6cfdf-a9b4-444e-a6c7-10823ff1f242-maas-api-tls\") pod \"maas-api-e2e-inf-b-107131-7747f587f-4d6mp\" (UID: \"e6a6cfdf-a9b4-444e-a6c7-10823ff1f242\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-107131-7747f587f-4d6mp"
Jul 25 00:04:40.942391 ip-10-0-142-133 kubenswrapper[2558]: E0725 00:04:40.942377    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-107131-serving-cert: secret "maas-api-e2e-inf-b-107131-serving-cert" not found
Jul 25 00:04:40.942522 ip-10-0-142-133 kubenswrapper[2558]: E0725 00:04:40.942513    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6a6cfdf-a9b4-444e-a6c7-10823ff1f242-maas-api-tls podName:e6a6cfdf-a9b4-444e-a6c7-10823ff1f242 nodeName:}" failed. No retries permitted until 2026-07-25 00:04:41.442493285 +0000 UTC m=+2121.828923998 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e6a6cfdf-a9b4-444e-a6c7-10823ff1f242-maas-api-tls") pod "maas-api-e2e-inf-b-107131-7747f587f-4d6mp" (UID: "e6a6cfdf-a9b4-444e-a6c7-10823ff1f242") : secret "maas-api-e2e-inf-b-107131-serving-cert" not found
Jul 25 00:04:40.942886 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:40.941715    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6a6cfdf-a9b4-444e-a6c7-10823ff1f242-openshift-service-ca\") pod \"maas-api-e2e-inf-b-107131-7747f587f-4d6mp\" (UID: \"e6a6cfdf-a9b4-444e-a6c7-10823ff1f242\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-107131-7747f587f-4d6mp"
Jul 25 00:04:40.954340 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:40.954312    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddkb7\" (UniqueName: \"kubernetes.io/projected/e6a6cfdf-a9b4-444e-a6c7-10823ff1f242-kube-api-access-ddkb7\") pod \"maas-api-e2e-inf-b-107131-7747f587f-4d6mp\" (UID: \"e6a6cfdf-a9b4-444e-a6c7-10823ff1f242\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-107131-7747f587f-4d6mp"
Jul 25 00:04:41.167973 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:41.167927    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:04:41.273468 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:41.273434    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:04:41.371942 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:41.371912    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:04:41.447139 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:41.447105    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6a6cfdf-a9b4-444e-a6c7-10823ff1f242-maas-api-tls\") pod \"maas-api-e2e-inf-b-107131-7747f587f-4d6mp\" (UID: \"e6a6cfdf-a9b4-444e-a6c7-10823ff1f242\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-107131-7747f587f-4d6mp"
Jul 25 00:04:41.449563 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:41.449525    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6a6cfdf-a9b4-444e-a6c7-10823ff1f242-maas-api-tls\") pod \"maas-api-e2e-inf-b-107131-7747f587f-4d6mp\" (UID: \"e6a6cfdf-a9b4-444e-a6c7-10823ff1f242\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-107131-7747f587f-4d6mp"
Jul 25 00:04:41.670376 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:41.670292    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-107131-7747f587f-4d6mp"
Jul 25 00:04:41.808482 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:41.808446    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-107131-7747f587f-4d6mp"]
Jul 25 00:04:41.809582 ip-10-0-142-133 kubenswrapper[2558]: W0725 00:04:41.809553    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6a6cfdf_a9b4_444e_a6c7_10823ff1f242.slice/crio-5eabc6c9e3bfe9d80a7ad9b0b2eef2105ad74fb8c4e21d4b64521f1baf7ac4b2 WatchSource:0}: Error finding container 5eabc6c9e3bfe9d80a7ad9b0b2eef2105ad74fb8c4e21d4b64521f1baf7ac4b2: Status 404 returned error can't find the container with id 5eabc6c9e3bfe9d80a7ad9b0b2eef2105ad74fb8c4e21d4b64521f1baf7ac4b2
Jul 25 00:04:41.930320 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:41.930238    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-107131-7747f587f-4d6mp" event={"ID":"e6a6cfdf-a9b4-444e-a6c7-10823ff1f242","Type":"ContainerStarted","Data":"5eabc6c9e3bfe9d80a7ad9b0b2eef2105ad74fb8c4e21d4b64521f1baf7ac4b2"}
Jul 25 00:04:42.937991 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:42.937955    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-107131-7747f587f-4d6mp" event={"ID":"e6a6cfdf-a9b4-444e-a6c7-10823ff1f242","Type":"ContainerStarted","Data":"ec48c12a939404f7ee0a82c56230210f90baeb3400c338f16027017f78d17c8b"}
Jul 25 00:04:42.938383 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:42.938088    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-107131-7747f587f-4d6mp"
Jul 25 00:04:42.953053 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:42.953000    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-107131-7747f587f-4d6mp" podStartSLOduration=2.454390009 podStartE2EDuration="2.952983722s" totalImagesPullingTime="498.593713ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:04:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:04:42.952133073 +0000 UTC m=+2123.338563800" watchObservedRunningTime="2026-07-25 00:04:42.952983722 +0000 UTC m=+2123.339414452"
Jul 25 00:04:45.208503 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:45.208465    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:04:45.897082 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:45.897052    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-820f12-745f95844d-pdj5h"
Jul 25 00:04:48.948361 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:48.948332    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-107131-7747f587f-4d6mp"
Jul 25 00:04:49.846609 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:04:49.846575    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:05:12.915630 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:05:12.915553    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:05:54.842799 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:05:54.842761    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:06:04.382825 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:04.382787    2558 generic.go:356] "Generic (PLEG): container finished" podID="1d634e98-014a-43c3-a840-efab61ea3a24" containerID="411cec1c986f33b1ebfda85de90dfd594ca4d440c88c4fc13e809daefd7e1bba" exitCode=1
Jul 25 00:06:04.383248 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:04.382862    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-s48c7" event={"ID":"1d634e98-014a-43c3-a840-efab61ea3a24","Type":"ContainerDied","Data":"411cec1c986f33b1ebfda85de90dfd594ca4d440c88c4fc13e809daefd7e1bba"}
Jul 25 00:06:04.383248 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:04.383184    2558 scope.go:117] "RemoveContainer" containerID="411cec1c986f33b1ebfda85de90dfd594ca4d440c88c4fc13e809daefd7e1bba"
Jul 25 00:06:05.390197 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:05.390107    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-s48c7" event={"ID":"1d634e98-014a-43c3-a840-efab61ea3a24","Type":"ContainerStarted","Data":"f405ffd4f71a95ce915690162602b9eff40834e6f9613e8ff8ba713e325e77b8"}
Jul 25 00:06:05.390622 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:05.390366    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-s48c7"
Jul 25 00:06:05.899827 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:05.899797    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:06:06.021996 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:06.021963    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-820f12-tjnjr"]
Jul 25 00:06:06.025386 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:06.025370    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-820f12-tjnjr"
Jul 25 00:06:06.027727 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:06.027704    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bxr46\""
Jul 25 00:06:06.032199 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:06.032175    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-820f12-tjnjr"]
Jul 25 00:06:06.190821 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:06.190724    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/93fd0bdc-7370-472f-96ca-4a0b8d4df4c5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-820f12-tjnjr\" (UID: \"93fd0bdc-7370-472f-96ca-4a0b8d4df4c5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-820f12-tjnjr"
Jul 25 00:06:06.291555 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:06.291500    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/93fd0bdc-7370-472f-96ca-4a0b8d4df4c5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-820f12-tjnjr\" (UID: \"93fd0bdc-7370-472f-96ca-4a0b8d4df4c5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-820f12-tjnjr"
Jul 25 00:06:06.292171 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:06.292145    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/93fd0bdc-7370-472f-96ca-4a0b8d4df4c5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-820f12-tjnjr\" (UID: \"93fd0bdc-7370-472f-96ca-4a0b8d4df4c5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-820f12-tjnjr"
Jul 25 00:06:06.339542 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:06.339494    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-820f12-tjnjr"
Jul 25 00:06:06.478846 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:06.478822    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-820f12-tjnjr"]
Jul 25 00:06:06.480887 ip-10-0-142-133 kubenswrapper[2558]: W0725 00:06:06.480862    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93fd0bdc_7370_472f_96ca_4a0b8d4df4c5.slice/crio-f99d2b177dc5ab77647c7bcf10d0272f1cbdeac14ca56eaf467ad88da53ca014 WatchSource:0}: Error finding container f99d2b177dc5ab77647c7bcf10d0272f1cbdeac14ca56eaf467ad88da53ca014: Status 404 returned error can't find the container with id f99d2b177dc5ab77647c7bcf10d0272f1cbdeac14ca56eaf467ad88da53ca014
Jul 25 00:06:07.404804 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:07.404768    2558 generic.go:356] "Generic (PLEG): container finished" podID="93fd0bdc-7370-472f-96ca-4a0b8d4df4c5" containerID="c110f808af16c0fb37742baa54510825ee6c68c3115bfbaeeb203e490938d9e0" exitCode=0
Jul 25 00:06:07.404985 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:07.404847    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-820f12-tjnjr" event={"ID":"93fd0bdc-7370-472f-96ca-4a0b8d4df4c5","Type":"ContainerDied","Data":"c110f808af16c0fb37742baa54510825ee6c68c3115bfbaeeb203e490938d9e0"}
Jul 25 00:06:07.404985 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:07.404899    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-820f12-tjnjr" event={"ID":"93fd0bdc-7370-472f-96ca-4a0b8d4df4c5","Type":"ContainerStarted","Data":"f99d2b177dc5ab77647c7bcf10d0272f1cbdeac14ca56eaf467ad88da53ca014"}
Jul 25 00:06:08.539867 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:08.539844    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-820f12-tjnjr"
Jul 25 00:06:08.714250 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:08.714165    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/93fd0bdc-7370-472f-96ca-4a0b8d4df4c5-maas-api-service-ca\") pod \"93fd0bdc-7370-472f-96ca-4a0b8d4df4c5\" (UID: \"93fd0bdc-7370-472f-96ca-4a0b8d4df4c5\") "
Jul 25 00:06:08.714565 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:08.714517    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93fd0bdc-7370-472f-96ca-4a0b8d4df4c5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "93fd0bdc-7370-472f-96ca-4a0b8d4df4c5" (UID: "93fd0bdc-7370-472f-96ca-4a0b8d4df4c5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 00:06:08.815495 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:08.815462    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/93fd0bdc-7370-472f-96ca-4a0b8d4df4c5-maas-api-service-ca\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 25 00:06:09.422882 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:09.422854    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-820f12-tjnjr"
Jul 25 00:06:09.423059 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:09.422885    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-820f12-tjnjr" event={"ID":"93fd0bdc-7370-472f-96ca-4a0b8d4df4c5","Type":"ContainerDied","Data":"f99d2b177dc5ab77647c7bcf10d0272f1cbdeac14ca56eaf467ad88da53ca014"}
Jul 25 00:06:09.423059 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:09.422960    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f99d2b177dc5ab77647c7bcf10d0272f1cbdeac14ca56eaf467ad88da53ca014"
Jul 25 00:06:16.400256 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:16.400227    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-s48c7"
Jul 25 00:06:16.512017 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:16.511974    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-820f12-7dbd55d7bb-6vv5l"]
Jul 25 00:06:16.512272 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:16.512219    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-820f12-7dbd55d7bb-6vv5l" podUID="0737648e-5145-4ffe-9344-275b82712ce3" containerName="maas-api" containerID="cri-o://3b314c3c75043db5a21334f7f9a6f5295631666600a35f802868b4631d97561a" gracePeriod=30
Jul 25 00:06:16.536780 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:16.536756    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:06:16.567924 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:16.567894    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-820f12-745f95844d-pdj5h"]
Jul 25 00:06:16.568137 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:16.568111    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-820f12-745f95844d-pdj5h" podUID="b6551de2-8a80-400c-a830-82f63930b5f4" containerName="payload-pre-processing" containerID="cri-o://34d6a7c22b97e09c4e9e3287ab071ab4e3c793906b67f50adb26d4258a00b632" gracePeriod=30
Jul 25 00:06:16.637263 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:16.637228    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:06:16.687742 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:16.687703    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:06:16.813476 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:16.813451    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-820f12-7dbd55d7bb-6vv5l"
Jul 25 00:06:16.843094 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:16.843070    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-820f12-745f95844d-pdj5h"
Jul 25 00:06:16.887949 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:16.887914    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0737648e-5145-4ffe-9344-275b82712ce3-openshift-service-ca\") pod \"0737648e-5145-4ffe-9344-275b82712ce3\" (UID: \"0737648e-5145-4ffe-9344-275b82712ce3\") "
Jul 25 00:06:16.888151 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:16.887963    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xvmt4\" (UniqueName: \"kubernetes.io/projected/b6551de2-8a80-400c-a830-82f63930b5f4-kube-api-access-xvmt4\") pod \"b6551de2-8a80-400c-a830-82f63930b5f4\" (UID: \"b6551de2-8a80-400c-a830-82f63930b5f4\") "
Jul 25 00:06:16.888151 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:16.887991    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6551de2-8a80-400c-a830-82f63930b5f4-plugins-config-volume\") pod \"b6551de2-8a80-400c-a830-82f63930b5f4\" (UID: \"b6551de2-8a80-400c-a830-82f63930b5f4\") "
Jul 25 00:06:16.888151 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:16.888075    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0737648e-5145-4ffe-9344-275b82712ce3-maas-api-tls\") pod \"0737648e-5145-4ffe-9344-275b82712ce3\" (UID: \"0737648e-5145-4ffe-9344-275b82712ce3\") "
Jul 25 00:06:16.888151 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:16.888100    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kk4tn\" (UniqueName: \"kubernetes.io/projected/0737648e-5145-4ffe-9344-275b82712ce3-kube-api-access-kk4tn\") pod \"0737648e-5145-4ffe-9344-275b82712ce3\" (UID: \"0737648e-5145-4ffe-9344-275b82712ce3\") "
Jul 25 00:06:16.888362 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:16.888326    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0737648e-5145-4ffe-9344-275b82712ce3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0737648e-5145-4ffe-9344-275b82712ce3" (UID: "0737648e-5145-4ffe-9344-275b82712ce3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 00:06:16.888423 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:16.888370    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6551de2-8a80-400c-a830-82f63930b5f4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b6551de2-8a80-400c-a830-82f63930b5f4" (UID: "b6551de2-8a80-400c-a830-82f63930b5f4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 00:06:16.890168 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:16.890139    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0737648e-5145-4ffe-9344-275b82712ce3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0737648e-5145-4ffe-9344-275b82712ce3" (UID: "0737648e-5145-4ffe-9344-275b82712ce3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 00:06:16.890289 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:16.890191    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6551de2-8a80-400c-a830-82f63930b5f4-kube-api-access-xvmt4" (OuterVolumeSpecName: "kube-api-access-xvmt4") pod "b6551de2-8a80-400c-a830-82f63930b5f4" (UID: "b6551de2-8a80-400c-a830-82f63930b5f4"). InnerVolumeSpecName "kube-api-access-xvmt4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 00:06:16.890330 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:16.890282    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0737648e-5145-4ffe-9344-275b82712ce3-kube-api-access-kk4tn" (OuterVolumeSpecName: "kube-api-access-kk4tn") pod "0737648e-5145-4ffe-9344-275b82712ce3" (UID: "0737648e-5145-4ffe-9344-275b82712ce3"). InnerVolumeSpecName "kube-api-access-kk4tn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 00:06:16.989583 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:16.989526    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0737648e-5145-4ffe-9344-275b82712ce3-openshift-service-ca\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 25 00:06:16.989583 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:16.989583    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xvmt4\" (UniqueName: \"kubernetes.io/projected/b6551de2-8a80-400c-a830-82f63930b5f4-kube-api-access-xvmt4\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 25 00:06:16.989783 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:16.989592    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6551de2-8a80-400c-a830-82f63930b5f4-plugins-config-volume\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 25 00:06:16.989783 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:16.989602    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0737648e-5145-4ffe-9344-275b82712ce3-maas-api-tls\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 25 00:06:16.989783 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:16.989612    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kk4tn\" (UniqueName: \"kubernetes.io/projected/0737648e-5145-4ffe-9344-275b82712ce3-kube-api-access-kk4tn\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 25 00:06:17.465773 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:17.465668    2558 generic.go:356] "Generic (PLEG): container finished" podID="b6551de2-8a80-400c-a830-82f63930b5f4" containerID="34d6a7c22b97e09c4e9e3287ab071ab4e3c793906b67f50adb26d4258a00b632" exitCode=0
Jul 25 00:06:17.465773 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:17.465714    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-820f12-745f95844d-pdj5h"
Jul 25 00:06:17.465773 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:17.465756    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-820f12-745f95844d-pdj5h" event={"ID":"b6551de2-8a80-400c-a830-82f63930b5f4","Type":"ContainerDied","Data":"34d6a7c22b97e09c4e9e3287ab071ab4e3c793906b67f50adb26d4258a00b632"}
Jul 25 00:06:17.466374 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:17.465812    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-820f12-745f95844d-pdj5h" event={"ID":"b6551de2-8a80-400c-a830-82f63930b5f4","Type":"ContainerDied","Data":"f6d69340db8d240296cab5f1a09d188a3a50cf1f30beef8a2838ddc887c6297a"}
Jul 25 00:06:17.466374 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:17.465843    2558 scope.go:117] "RemoveContainer" containerID="34d6a7c22b97e09c4e9e3287ab071ab4e3c793906b67f50adb26d4258a00b632"
Jul 25 00:06:17.467091 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:17.467068    2558 generic.go:356] "Generic (PLEG): container finished" podID="0737648e-5145-4ffe-9344-275b82712ce3" containerID="3b314c3c75043db5a21334f7f9a6f5295631666600a35f802868b4631d97561a" exitCode=0
Jul 25 00:06:17.467177 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:17.467112    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-820f12-7dbd55d7bb-6vv5l" event={"ID":"0737648e-5145-4ffe-9344-275b82712ce3","Type":"ContainerDied","Data":"3b314c3c75043db5a21334f7f9a6f5295631666600a35f802868b4631d97561a"}
Jul 25 00:06:17.467177 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:17.467148    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-820f12-7dbd55d7bb-6vv5l" event={"ID":"0737648e-5145-4ffe-9344-275b82712ce3","Type":"ContainerDied","Data":"ce760a31a1c25f0766de102ea3c1311c6035328f9e859337f97afb3dfe35e25c"}
Jul 25 00:06:17.467177 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:17.467115    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-820f12-7dbd55d7bb-6vv5l"
Jul 25 00:06:17.475411 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:17.475301    2558 scope.go:117] "RemoveContainer" containerID="34d6a7c22b97e09c4e9e3287ab071ab4e3c793906b67f50adb26d4258a00b632"
Jul 25 00:06:17.475666 ip-10-0-142-133 kubenswrapper[2558]: E0725 00:06:17.475643    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34d6a7c22b97e09c4e9e3287ab071ab4e3c793906b67f50adb26d4258a00b632\": container with ID starting with 34d6a7c22b97e09c4e9e3287ab071ab4e3c793906b67f50adb26d4258a00b632 not found: ID does not exist" containerID="34d6a7c22b97e09c4e9e3287ab071ab4e3c793906b67f50adb26d4258a00b632"
Jul 25 00:06:17.475784 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:17.475672    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34d6a7c22b97e09c4e9e3287ab071ab4e3c793906b67f50adb26d4258a00b632"} err="failed to get container status \"34d6a7c22b97e09c4e9e3287ab071ab4e3c793906b67f50adb26d4258a00b632\": rpc error: code = NotFound desc = could not find container \"34d6a7c22b97e09c4e9e3287ab071ab4e3c793906b67f50adb26d4258a00b632\": container with ID starting with 34d6a7c22b97e09c4e9e3287ab071ab4e3c793906b67f50adb26d4258a00b632 not found: ID does not exist"
Jul 25 00:06:17.475784 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:17.475695    2558 scope.go:117] "RemoveContainer" containerID="3b314c3c75043db5a21334f7f9a6f5295631666600a35f802868b4631d97561a"
Jul 25 00:06:17.484317 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:17.484301    2558 scope.go:117] "RemoveContainer" containerID="3b314c3c75043db5a21334f7f9a6f5295631666600a35f802868b4631d97561a"
Jul 25 00:06:17.484597 ip-10-0-142-133 kubenswrapper[2558]: E0725 00:06:17.484576    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b314c3c75043db5a21334f7f9a6f5295631666600a35f802868b4631d97561a\": container with ID starting with 3b314c3c75043db5a21334f7f9a6f5295631666600a35f802868b4631d97561a not found: ID does not exist" containerID="3b314c3c75043db5a21334f7f9a6f5295631666600a35f802868b4631d97561a"
Jul 25 00:06:17.484665 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:17.484603    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b314c3c75043db5a21334f7f9a6f5295631666600a35f802868b4631d97561a"} err="failed to get container status \"3b314c3c75043db5a21334f7f9a6f5295631666600a35f802868b4631d97561a\": rpc error: code = NotFound desc = could not find container \"3b314c3c75043db5a21334f7f9a6f5295631666600a35f802868b4631d97561a\": container with ID starting with 3b314c3c75043db5a21334f7f9a6f5295631666600a35f802868b4631d97561a not found: ID does not exist"
Jul 25 00:06:17.489725 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:17.489704    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-820f12-7dbd55d7bb-6vv5l"]
Jul 25 00:06:17.493058 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:17.493028    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-820f12-7dbd55d7bb-6vv5l"]
Jul 25 00:06:17.501272 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:17.501253    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-820f12-745f95844d-pdj5h"]
Jul 25 00:06:17.504902 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:17.504879    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-820f12-745f95844d-pdj5h"]
Jul 25 00:06:18.203904 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:18.203874    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0737648e-5145-4ffe-9344-275b82712ce3" path="/var/lib/kubelet/pods/0737648e-5145-4ffe-9344-275b82712ce3/volumes"
Jul 25 00:06:18.204241 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:18.204226    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6551de2-8a80-400c-a830-82f63930b5f4" path="/var/lib/kubelet/pods/b6551de2-8a80-400c-a830-82f63930b5f4/volumes"
Jul 25 00:06:21.137513 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:21.137477    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-820f12-tjnjr"]
Jul 25 00:06:21.139576 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:21.139554    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-820f12-tjnjr"]
Jul 25 00:06:21.188566 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:21.188515    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:06:22.005152 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:22.005090    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:06:22.204916 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:22.204880    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93fd0bdc-7370-472f-96ca-4a0b8d4df4c5" path="/var/lib/kubelet/pods/93fd0bdc-7370-472f-96ca-4a0b8d4df4c5/volumes"
Jul 25 00:06:25.868433 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:25.868396    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:06:26.342988 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:26.342947    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:06:27.331273 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:27.331243    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:06:27.432652 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:27.432616    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:06:27.617018 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:27.616938    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:06:27.860524 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:27.860490    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:06:27.991839 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:27.991802    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-107131-tkhgp"]
Jul 25 00:06:27.992363 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:27.992343    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93fd0bdc-7370-472f-96ca-4a0b8d4df4c5" containerName="revoke-keys"
Jul 25 00:06:27.992423 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:27.992367    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="93fd0bdc-7370-472f-96ca-4a0b8d4df4c5" containerName="revoke-keys"
Jul 25 00:06:27.992423 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:27.992411    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0737648e-5145-4ffe-9344-275b82712ce3" containerName="maas-api"
Jul 25 00:06:27.992423 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:27.992420    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="0737648e-5145-4ffe-9344-275b82712ce3" containerName="maas-api"
Jul 25 00:06:27.992518 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:27.992432    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6551de2-8a80-400c-a830-82f63930b5f4" containerName="payload-pre-processing"
Jul 25 00:06:27.992518 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:27.992441    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6551de2-8a80-400c-a830-82f63930b5f4" containerName="payload-pre-processing"
Jul 25 00:06:27.992608 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:27.992558    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="93fd0bdc-7370-472f-96ca-4a0b8d4df4c5" containerName="revoke-keys"
Jul 25 00:06:27.992608 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:27.992574    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="0737648e-5145-4ffe-9344-275b82712ce3" containerName="maas-api"
Jul 25 00:06:27.992608 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:27.992584    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6551de2-8a80-400c-a830-82f63930b5f4" containerName="payload-pre-processing"
Jul 25 00:06:27.996176 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:27.996156    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-107131-tkhgp"
Jul 25 00:06:27.998956 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:27.998925    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bxr46\""
Jul 25 00:06:28.001056 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:28.001032    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-107131-tkhgp"]
Jul 25 00:06:28.092366 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:28.092322    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5a91628-2d78-4a85-918e-12037c19ea78-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-107131-tkhgp\" (UID: \"a5a91628-2d78-4a85-918e-12037c19ea78\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-107131-tkhgp"
Jul 25 00:06:28.193012 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:28.192980    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5a91628-2d78-4a85-918e-12037c19ea78-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-107131-tkhgp\" (UID: \"a5a91628-2d78-4a85-918e-12037c19ea78\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-107131-tkhgp"
Jul 25 00:06:28.193658 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:28.193641    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5a91628-2d78-4a85-918e-12037c19ea78-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-107131-tkhgp\" (UID: \"a5a91628-2d78-4a85-918e-12037c19ea78\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-107131-tkhgp"
Jul 25 00:06:28.309592 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:28.309492    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-107131-tkhgp"
Jul 25 00:06:28.445211 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:28.445180    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-107131-tkhgp"]
Jul 25 00:06:28.446556 ip-10-0-142-133 kubenswrapper[2558]: W0725 00:06:28.446515    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5a91628_2d78_4a85_918e_12037c19ea78.slice/crio-82744ab5eab88988a6c6ed75982cf0388840795c65cb2c4c78285c3b143a2e8f WatchSource:0}: Error finding container 82744ab5eab88988a6c6ed75982cf0388840795c65cb2c4c78285c3b143a2e8f: Status 404 returned error can't find the container with id 82744ab5eab88988a6c6ed75982cf0388840795c65cb2c4c78285c3b143a2e8f
Jul 25 00:06:28.529251 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:28.529215    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-107131-tkhgp" event={"ID":"a5a91628-2d78-4a85-918e-12037c19ea78","Type":"ContainerStarted","Data":"82744ab5eab88988a6c6ed75982cf0388840795c65cb2c4c78285c3b143a2e8f"}
Jul 25 00:06:29.536573 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:29.536513    2558 generic.go:356] "Generic (PLEG): container finished" podID="a5a91628-2d78-4a85-918e-12037c19ea78" containerID="542ab75d42482826de52900be0987b132a125c00b5706ef65d26bfe2d7872f21" exitCode=0
Jul 25 00:06:29.536994 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:29.536600    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-107131-tkhgp" event={"ID":"a5a91628-2d78-4a85-918e-12037c19ea78","Type":"ContainerDied","Data":"542ab75d42482826de52900be0987b132a125c00b5706ef65d26bfe2d7872f21"}
Jul 25 00:06:30.675914 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:30.675890    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-107131-tkhgp"
Jul 25 00:06:30.718930 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:30.718899    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5a91628-2d78-4a85-918e-12037c19ea78-maas-api-service-ca\") pod \"a5a91628-2d78-4a85-918e-12037c19ea78\" (UID: \"a5a91628-2d78-4a85-918e-12037c19ea78\") "
Jul 25 00:06:30.719248 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:30.719224    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5a91628-2d78-4a85-918e-12037c19ea78-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a5a91628-2d78-4a85-918e-12037c19ea78" (UID: "a5a91628-2d78-4a85-918e-12037c19ea78"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 00:06:30.819891 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:30.819812    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5a91628-2d78-4a85-918e-12037c19ea78-maas-api-service-ca\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 25 00:06:31.549407 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:31.549371    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-107131-tkhgp" event={"ID":"a5a91628-2d78-4a85-918e-12037c19ea78","Type":"ContainerDied","Data":"82744ab5eab88988a6c6ed75982cf0388840795c65cb2c4c78285c3b143a2e8f"}
Jul 25 00:06:31.549407 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:31.549389    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-107131-tkhgp"
Jul 25 00:06:31.549646 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:31.549422    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="82744ab5eab88988a6c6ed75982cf0388840795c65cb2c4c78285c3b143a2e8f"
Jul 25 00:06:38.389128 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:38.389094    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-107131-7747f587f-4d6mp"]
Jul 25 00:06:38.389612 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:38.389311    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-107131-7747f587f-4d6mp" podUID="e6a6cfdf-a9b4-444e-a6c7-10823ff1f242" containerName="maas-api" containerID="cri-o://ec48c12a939404f7ee0a82c56230210f90baeb3400c338f16027017f78d17c8b" gracePeriod=30
Jul 25 00:06:38.596268 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:38.596235    2558 generic.go:356] "Generic (PLEG): container finished" podID="e6a6cfdf-a9b4-444e-a6c7-10823ff1f242" containerID="ec48c12a939404f7ee0a82c56230210f90baeb3400c338f16027017f78d17c8b" exitCode=0
Jul 25 00:06:38.596416 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:38.596278    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-107131-7747f587f-4d6mp" event={"ID":"e6a6cfdf-a9b4-444e-a6c7-10823ff1f242","Type":"ContainerDied","Data":"ec48c12a939404f7ee0a82c56230210f90baeb3400c338f16027017f78d17c8b"}
Jul 25 00:06:38.643738 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:38.643676    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-107131-7747f587f-4d6mp"
Jul 25 00:06:38.669118 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:38.669086    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:06:38.718376 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:38.718339    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:06:38.793156 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:38.793126    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6a6cfdf-a9b4-444e-a6c7-10823ff1f242-openshift-service-ca\") pod \"e6a6cfdf-a9b4-444e-a6c7-10823ff1f242\" (UID: \"e6a6cfdf-a9b4-444e-a6c7-10823ff1f242\") "
Jul 25 00:06:38.793346 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:38.793178    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6a6cfdf-a9b4-444e-a6c7-10823ff1f242-maas-api-tls\") pod \"e6a6cfdf-a9b4-444e-a6c7-10823ff1f242\" (UID: \"e6a6cfdf-a9b4-444e-a6c7-10823ff1f242\") "
Jul 25 00:06:38.793346 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:38.793222    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ddkb7\" (UniqueName: \"kubernetes.io/projected/e6a6cfdf-a9b4-444e-a6c7-10823ff1f242-kube-api-access-ddkb7\") pod \"e6a6cfdf-a9b4-444e-a6c7-10823ff1f242\" (UID: \"e6a6cfdf-a9b4-444e-a6c7-10823ff1f242\") "
Jul 25 00:06:38.793519 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:38.793498    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6a6cfdf-a9b4-444e-a6c7-10823ff1f242-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e6a6cfdf-a9b4-444e-a6c7-10823ff1f242" (UID: "e6a6cfdf-a9b4-444e-a6c7-10823ff1f242"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 00:06:38.795226 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:38.795198    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6a6cfdf-a9b4-444e-a6c7-10823ff1f242-kube-api-access-ddkb7" (OuterVolumeSpecName: "kube-api-access-ddkb7") pod "e6a6cfdf-a9b4-444e-a6c7-10823ff1f242" (UID: "e6a6cfdf-a9b4-444e-a6c7-10823ff1f242"). InnerVolumeSpecName "kube-api-access-ddkb7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 00:06:38.795394 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:38.795375    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e6a6cfdf-a9b4-444e-a6c7-10823ff1f242-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e6a6cfdf-a9b4-444e-a6c7-10823ff1f242" (UID: "e6a6cfdf-a9b4-444e-a6c7-10823ff1f242"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 00:06:38.894777 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:38.894682    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6a6cfdf-a9b4-444e-a6c7-10823ff1f242-maas-api-tls\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 25 00:06:38.894777 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:38.894720    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ddkb7\" (UniqueName: \"kubernetes.io/projected/e6a6cfdf-a9b4-444e-a6c7-10823ff1f242-kube-api-access-ddkb7\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 25 00:06:38.894777 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:38.894733    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6a6cfdf-a9b4-444e-a6c7-10823ff1f242-openshift-service-ca\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 25 00:06:39.602415 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:39.602383    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-107131-7747f587f-4d6mp"
Jul 25 00:06:39.602415 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:39.602407    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-107131-7747f587f-4d6mp" event={"ID":"e6a6cfdf-a9b4-444e-a6c7-10823ff1f242","Type":"ContainerDied","Data":"5eabc6c9e3bfe9d80a7ad9b0b2eef2105ad74fb8c4e21d4b64521f1baf7ac4b2"}
Jul 25 00:06:39.602906 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:39.602465    2558 scope.go:117] "RemoveContainer" containerID="ec48c12a939404f7ee0a82c56230210f90baeb3400c338f16027017f78d17c8b"
Jul 25 00:06:39.630147 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:39.630100    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-107131-7747f587f-4d6mp"]
Jul 25 00:06:39.631367 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:39.631344    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-107131-7747f587f-4d6mp"]
Jul 25 00:06:40.204836 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:40.204798    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6a6cfdf-a9b4-444e-a6c7-10823ff1f242" path="/var/lib/kubelet/pods/e6a6cfdf-a9b4-444e-a6c7-10823ff1f242/volumes"
Jul 25 00:06:43.065216 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:43.065129    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-107131-tkhgp"]
Jul 25 00:06:43.067737 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:43.067009    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:06:43.067897 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:43.067877    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-107131-tkhgp"]
Jul 25 00:06:43.472515 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:43.472430    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:06:44.204453 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:44.204419    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5a91628-2d78-4a85-918e-12037c19ea78" path="/var/lib/kubelet/pods/a5a91628-2d78-4a85-918e-12037c19ea78/volumes"
Jul 25 00:06:49.368973 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:49.368935    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:06:49.466890 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:49.466849    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:06:49.643762 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:49.643677    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:06:49.877790 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:49.877754    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:06:50.248280 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:50.248244    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:06:50.554509 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:50.554427    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:06:50.824747 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:50.824655    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:06:53.451017 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:53.450982    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:06:53.498716 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:53.498682    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-97aaee-75599d9f48-tdhmb"]
Jul 25 00:06:53.499077 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:53.499064    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5a91628-2d78-4a85-918e-12037c19ea78" containerName="revoke-keys"
Jul 25 00:06:53.499122 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:53.499079    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5a91628-2d78-4a85-918e-12037c19ea78" containerName="revoke-keys"
Jul 25 00:06:53.499122 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:53.499111    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6a6cfdf-a9b4-444e-a6c7-10823ff1f242" containerName="maas-api"
Jul 25 00:06:53.499122 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:53.499118    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6a6cfdf-a9b4-444e-a6c7-10823ff1f242" containerName="maas-api"
Jul 25 00:06:53.499227 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:53.499179    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6a6cfdf-a9b4-444e-a6c7-10823ff1f242" containerName="maas-api"
Jul 25 00:06:53.499227 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:53.499189    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5a91628-2d78-4a85-918e-12037c19ea78" containerName="revoke-keys"
Jul 25 00:06:53.502594 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:53.502522    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-97aaee-75599d9f48-tdhmb"
Jul 25 00:06:53.505067 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:53.505043    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-97aaee-serving-cert\""
Jul 25 00:06:53.519698 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:53.519659    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-97aaee-75599d9f48-tdhmb"]
Jul 25 00:06:53.526517 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:53.526485    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2f9gh\" (UniqueName: \"kubernetes.io/projected/e3ee0052-5f20-4d2d-b7d1-115a82a249ae-kube-api-access-2f9gh\") pod \"maas-api-e2e-api-a-97aaee-75599d9f48-tdhmb\" (UID: \"e3ee0052-5f20-4d2d-b7d1-115a82a249ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-97aaee-75599d9f48-tdhmb"
Jul 25 00:06:53.526690 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:53.526562    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e3ee0052-5f20-4d2d-b7d1-115a82a249ae-maas-api-tls\") pod \"maas-api-e2e-api-a-97aaee-75599d9f48-tdhmb\" (UID: \"e3ee0052-5f20-4d2d-b7d1-115a82a249ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-97aaee-75599d9f48-tdhmb"
Jul 25 00:06:53.526690 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:53.526588    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3ee0052-5f20-4d2d-b7d1-115a82a249ae-openshift-service-ca\") pod \"maas-api-e2e-api-a-97aaee-75599d9f48-tdhmb\" (UID: \"e3ee0052-5f20-4d2d-b7d1-115a82a249ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-97aaee-75599d9f48-tdhmb"
Jul 25 00:06:53.527989 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:53.527963    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:06:53.595706 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:53.595667    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:06:53.627349 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:53.627316    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2f9gh\" (UniqueName: \"kubernetes.io/projected/e3ee0052-5f20-4d2d-b7d1-115a82a249ae-kube-api-access-2f9gh\") pod \"maas-api-e2e-api-a-97aaee-75599d9f48-tdhmb\" (UID: \"e3ee0052-5f20-4d2d-b7d1-115a82a249ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-97aaee-75599d9f48-tdhmb"
Jul 25 00:06:53.627521 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:53.627367    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e3ee0052-5f20-4d2d-b7d1-115a82a249ae-maas-api-tls\") pod \"maas-api-e2e-api-a-97aaee-75599d9f48-tdhmb\" (UID: \"e3ee0052-5f20-4d2d-b7d1-115a82a249ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-97aaee-75599d9f48-tdhmb"
Jul 25 00:06:53.627521 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:53.627387    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3ee0052-5f20-4d2d-b7d1-115a82a249ae-openshift-service-ca\") pod \"maas-api-e2e-api-a-97aaee-75599d9f48-tdhmb\" (UID: \"e3ee0052-5f20-4d2d-b7d1-115a82a249ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-97aaee-75599d9f48-tdhmb"
Jul 25 00:06:53.628058 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:53.628038    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3ee0052-5f20-4d2d-b7d1-115a82a249ae-openshift-service-ca\") pod \"maas-api-e2e-api-a-97aaee-75599d9f48-tdhmb\" (UID: \"e3ee0052-5f20-4d2d-b7d1-115a82a249ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-97aaee-75599d9f48-tdhmb"
Jul 25 00:06:53.629993 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:53.629959    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e3ee0052-5f20-4d2d-b7d1-115a82a249ae-maas-api-tls\") pod \"maas-api-e2e-api-a-97aaee-75599d9f48-tdhmb\" (UID: \"e3ee0052-5f20-4d2d-b7d1-115a82a249ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-97aaee-75599d9f48-tdhmb"
Jul 25 00:06:53.637151 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:53.637125    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2f9gh\" (UniqueName: \"kubernetes.io/projected/e3ee0052-5f20-4d2d-b7d1-115a82a249ae-kube-api-access-2f9gh\") pod \"maas-api-e2e-api-a-97aaee-75599d9f48-tdhmb\" (UID: \"e3ee0052-5f20-4d2d-b7d1-115a82a249ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-97aaee-75599d9f48-tdhmb"
Jul 25 00:06:53.825958 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:53.825919    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-97aaee-75599d9f48-tdhmb"
Jul 25 00:06:53.960730 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:53.960705    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-97aaee-75599d9f48-tdhmb"]
Jul 25 00:06:53.962785 ip-10-0-142-133 kubenswrapper[2558]: W0725 00:06:53.962759    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3ee0052_5f20_4d2d_b7d1_115a82a249ae.slice/crio-260bf5e4a4a42c2b976e403f10e12404d6b1b65c9086941e81e363f5fa1cd196 WatchSource:0}: Error finding container 260bf5e4a4a42c2b976e403f10e12404d6b1b65c9086941e81e363f5fa1cd196: Status 404 returned error can't find the container with id 260bf5e4a4a42c2b976e403f10e12404d6b1b65c9086941e81e363f5fa1cd196
Jul 25 00:06:53.964121 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:53.964103    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 25 00:06:54.112303 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:54.112216    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:06:54.245232 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:54.245194    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-97aaee-5f5d8858d-dpm4t"]
Jul 25 00:06:54.249721 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:54.249702    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-97aaee-5f5d8858d-dpm4t"
Jul 25 00:06:54.252047 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:54.252024    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-97aaee\""
Jul 25 00:06:54.252196 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:54.252062    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-97aaee-dockercfg-2zb4v\""
Jul 25 00:06:54.259616 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:54.259557    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-97aaee-5f5d8858d-dpm4t"]
Jul 25 00:06:54.271721 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:54.271671    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:06:54.286010 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:54.285978    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-97aaee-ff695db96-mpzst"]
Jul 25 00:06:54.289522 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:54.289498    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-97aaee-ff695db96-mpzst"
Jul 25 00:06:54.296908 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:54.296880    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-97aaee-ff695db96-mpzst"]
Jul 25 00:06:54.332025 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:54.331989    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gqjh\" (UniqueName: \"kubernetes.io/projected/b82bb394-2149-4726-8463-2080ec8d06ab-kube-api-access-6gqjh\") pod \"payload-processing-e2e-api-a-97aaee-5f5d8858d-dpm4t\" (UID: \"b82bb394-2149-4726-8463-2080ec8d06ab\") " pod="openshift-ingress/payload-processing-e2e-api-a-97aaee-5f5d8858d-dpm4t"
Jul 25 00:06:54.332204 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:54.332050    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b82bb394-2149-4726-8463-2080ec8d06ab-plugins-config-volume\") pod \"payload-processing-e2e-api-a-97aaee-5f5d8858d-dpm4t\" (UID: \"b82bb394-2149-4726-8463-2080ec8d06ab\") " pod="openshift-ingress/payload-processing-e2e-api-a-97aaee-5f5d8858d-dpm4t"
Jul 25 00:06:54.433409 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:54.433323    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b82bb394-2149-4726-8463-2080ec8d06ab-plugins-config-volume\") pod \"payload-processing-e2e-api-a-97aaee-5f5d8858d-dpm4t\" (UID: \"b82bb394-2149-4726-8463-2080ec8d06ab\") " pod="openshift-ingress/payload-processing-e2e-api-a-97aaee-5f5d8858d-dpm4t"
Jul 25 00:06:54.433409 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:54.433363    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35765ccc-e347-4adb-af0a-f6ec31c12a45-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-97aaee-ff695db96-mpzst\" (UID: \"35765ccc-e347-4adb-af0a-f6ec31c12a45\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-97aaee-ff695db96-mpzst"
Jul 25 00:06:54.433409 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:54.433402    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9vm6\" (UniqueName: \"kubernetes.io/projected/35765ccc-e347-4adb-af0a-f6ec31c12a45-kube-api-access-s9vm6\") pod \"payload-pre-processing-e2e-api-a-97aaee-ff695db96-mpzst\" (UID: \"35765ccc-e347-4adb-af0a-f6ec31c12a45\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-97aaee-ff695db96-mpzst"
Jul 25 00:06:54.433978 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:54.433492    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gqjh\" (UniqueName: \"kubernetes.io/projected/b82bb394-2149-4726-8463-2080ec8d06ab-kube-api-access-6gqjh\") pod \"payload-processing-e2e-api-a-97aaee-5f5d8858d-dpm4t\" (UID: \"b82bb394-2149-4726-8463-2080ec8d06ab\") " pod="openshift-ingress/payload-processing-e2e-api-a-97aaee-5f5d8858d-dpm4t"
Jul 25 00:06:54.434061 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:54.434039    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b82bb394-2149-4726-8463-2080ec8d06ab-plugins-config-volume\") pod \"payload-processing-e2e-api-a-97aaee-5f5d8858d-dpm4t\" (UID: \"b82bb394-2149-4726-8463-2080ec8d06ab\") " pod="openshift-ingress/payload-processing-e2e-api-a-97aaee-5f5d8858d-dpm4t"
Jul 25 00:06:54.441371 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:54.441337    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gqjh\" (UniqueName: \"kubernetes.io/projected/b82bb394-2149-4726-8463-2080ec8d06ab-kube-api-access-6gqjh\") pod \"payload-processing-e2e-api-a-97aaee-5f5d8858d-dpm4t\" (UID: \"b82bb394-2149-4726-8463-2080ec8d06ab\") " pod="openshift-ingress/payload-processing-e2e-api-a-97aaee-5f5d8858d-dpm4t"
Jul 25 00:06:54.534700 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:54.534657    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35765ccc-e347-4adb-af0a-f6ec31c12a45-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-97aaee-ff695db96-mpzst\" (UID: \"35765ccc-e347-4adb-af0a-f6ec31c12a45\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-97aaee-ff695db96-mpzst"
Jul 25 00:06:54.535159 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:54.534728    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9vm6\" (UniqueName: \"kubernetes.io/projected/35765ccc-e347-4adb-af0a-f6ec31c12a45-kube-api-access-s9vm6\") pod \"payload-pre-processing-e2e-api-a-97aaee-ff695db96-mpzst\" (UID: \"35765ccc-e347-4adb-af0a-f6ec31c12a45\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-97aaee-ff695db96-mpzst"
Jul 25 00:06:54.535346 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:54.535323    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35765ccc-e347-4adb-af0a-f6ec31c12a45-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-97aaee-ff695db96-mpzst\" (UID: \"35765ccc-e347-4adb-af0a-f6ec31c12a45\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-97aaee-ff695db96-mpzst"
Jul 25 00:06:54.542324 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:54.542301    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9vm6\" (UniqueName: \"kubernetes.io/projected/35765ccc-e347-4adb-af0a-f6ec31c12a45-kube-api-access-s9vm6\") pod \"payload-pre-processing-e2e-api-a-97aaee-ff695db96-mpzst\" (UID: \"35765ccc-e347-4adb-af0a-f6ec31c12a45\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-97aaee-ff695db96-mpzst"
Jul 25 00:06:54.563190 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:54.563160    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-97aaee-5f5d8858d-dpm4t"
Jul 25 00:06:54.601983 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:54.601955    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-97aaee-ff695db96-mpzst"
Jul 25 00:06:54.691253 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:54.691137    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-97aaee-75599d9f48-tdhmb" event={"ID":"e3ee0052-5f20-4d2d-b7d1-115a82a249ae","Type":"ContainerStarted","Data":"ece7835689562602acdd60e42747c5a0a1319202a65d8aebab81f86cc272438f"}
Jul 25 00:06:54.691410 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:54.691260    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-97aaee-75599d9f48-tdhmb" event={"ID":"e3ee0052-5f20-4d2d-b7d1-115a82a249ae","Type":"ContainerStarted","Data":"260bf5e4a4a42c2b976e403f10e12404d6b1b65c9086941e81e363f5fa1cd196"}
Jul 25 00:06:54.692107 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:54.691923    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-97aaee-75599d9f48-tdhmb"
Jul 25 00:06:54.709275 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:54.709201    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-97aaee-75599d9f48-tdhmb" podStartSLOduration=1.084841251 podStartE2EDuration="1.709183552s" totalImagesPullingTime="624.342301ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:06:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:06:54.708490563 +0000 UTC m=+2255.094921293" watchObservedRunningTime="2026-07-25 00:06:54.709183552 +0000 UTC m=+2255.095614279"
Jul 25 00:06:54.718000 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:54.717969    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-97aaee-5f5d8858d-dpm4t"]
Jul 25 00:06:54.719593 ip-10-0-142-133 kubenswrapper[2558]: W0725 00:06:54.719558    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb82bb394_2149_4726_8463_2080ec8d06ab.slice/crio-4c1b724223845aec777bd98cb473da775be4d8f5d938a0f6c5027debdbc02a12 WatchSource:0}: Error finding container 4c1b724223845aec777bd98cb473da775be4d8f5d938a0f6c5027debdbc02a12: Status 404 returned error can't find the container with id 4c1b724223845aec777bd98cb473da775be4d8f5d938a0f6c5027debdbc02a12
Jul 25 00:06:54.767766 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:54.767742    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-97aaee-ff695db96-mpzst"]
Jul 25 00:06:54.773701 ip-10-0-142-133 kubenswrapper[2558]: W0725 00:06:54.773671    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35765ccc_e347_4adb_af0a_f6ec31c12a45.slice/crio-45d0265e0fa1ebbea53d41ef599b47be8aab59c90236ebdbd0984317cc36ae16 WatchSource:0}: Error finding container 45d0265e0fa1ebbea53d41ef599b47be8aab59c90236ebdbd0984317cc36ae16: Status 404 returned error can't find the container with id 45d0265e0fa1ebbea53d41ef599b47be8aab59c90236ebdbd0984317cc36ae16
Jul 25 00:06:55.698098 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:55.698049    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-97aaee-ff695db96-mpzst" event={"ID":"35765ccc-e347-4adb-af0a-f6ec31c12a45","Type":"ContainerStarted","Data":"7e7e15ffdeb665553512995afd389915ccc2b194fea488f9542052a5973175a1"}
Jul 25 00:06:55.698582 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:55.698124    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-97aaee-ff695db96-mpzst"
Jul 25 00:06:55.698582 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:55.698147    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-97aaee-ff695db96-mpzst" event={"ID":"35765ccc-e347-4adb-af0a-f6ec31c12a45","Type":"ContainerStarted","Data":"45d0265e0fa1ebbea53d41ef599b47be8aab59c90236ebdbd0984317cc36ae16"}
Jul 25 00:06:55.699764 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:55.699737    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-97aaee-5f5d8858d-dpm4t" event={"ID":"b82bb394-2149-4726-8463-2080ec8d06ab","Type":"ContainerStarted","Data":"16e7e79932f66278e733a59ec78ffc19386d0f043aaeda69591f32e1a8f86120"}
Jul 25 00:06:55.699872 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:55.699780    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-97aaee-5f5d8858d-dpm4t" event={"ID":"b82bb394-2149-4726-8463-2080ec8d06ab","Type":"ContainerStarted","Data":"4c1b724223845aec777bd98cb473da775be4d8f5d938a0f6c5027debdbc02a12"}
Jul 25 00:06:55.699872 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:55.699809    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-97aaee-5f5d8858d-dpm4t"
Jul 25 00:06:55.714506 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:55.714458    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-97aaee-ff695db96-mpzst" podStartSLOduration=1.71444402 podStartE2EDuration="1.71444402s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:06:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:06:55.711772578 +0000 UTC m=+2256.098203306" watchObservedRunningTime="2026-07-25 00:06:55.71444402 +0000 UTC m=+2256.100874765"
Jul 25 00:06:55.727190 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:55.727144    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-97aaee-5f5d8858d-dpm4t" podStartSLOduration=1.727131161 podStartE2EDuration="1.727131161s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:06:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:06:55.724917663 +0000 UTC m=+2256.111348392" watchObservedRunningTime="2026-07-25 00:06:55.727131161 +0000 UTC m=+2256.113561930"
Jul 25 00:06:57.965218 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:57.965170    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:06:58.456681 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:58.456645    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:06:58.774289 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:58.774257    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:06:59.154495 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:06:59.154404    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:07:00.705806 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:00.705775    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-97aaee-75599d9f48-tdhmb"
Jul 25 00:07:01.453038 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:01.452996    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:07:01.551651 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:01.551596    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:07:01.697717 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:01.697683    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-7d3cec-dc47dd986-t2hxp"]
Jul 25 00:07:01.701504 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:01.701479    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7d3cec-dc47dd986-t2hxp"
Jul 25 00:07:01.703773 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:01.703706    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-7d3cec-serving-cert\""
Jul 25 00:07:01.712786 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:01.712759    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-7d3cec-dc47dd986-t2hxp"]
Jul 25 00:07:01.730280 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:01.730250    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:07:01.804975 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:01.804938    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35a0cd19-77c0-4812-ba82-a60e1fa318b8-openshift-service-ca\") pod \"maas-api-e2e-api-b-7d3cec-dc47dd986-t2hxp\" (UID: \"35a0cd19-77c0-4812-ba82-a60e1fa318b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7d3cec-dc47dd986-t2hxp"
Jul 25 00:07:01.804975 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:01.804981    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qsctt\" (UniqueName: \"kubernetes.io/projected/35a0cd19-77c0-4812-ba82-a60e1fa318b8-kube-api-access-qsctt\") pod \"maas-api-e2e-api-b-7d3cec-dc47dd986-t2hxp\" (UID: \"35a0cd19-77c0-4812-ba82-a60e1fa318b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7d3cec-dc47dd986-t2hxp"
Jul 25 00:07:01.805225 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:01.804999    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35a0cd19-77c0-4812-ba82-a60e1fa318b8-maas-api-tls\") pod \"maas-api-e2e-api-b-7d3cec-dc47dd986-t2hxp\" (UID: \"35a0cd19-77c0-4812-ba82-a60e1fa318b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7d3cec-dc47dd986-t2hxp"
Jul 25 00:07:01.906042 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:01.906000    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35a0cd19-77c0-4812-ba82-a60e1fa318b8-openshift-service-ca\") pod \"maas-api-e2e-api-b-7d3cec-dc47dd986-t2hxp\" (UID: \"35a0cd19-77c0-4812-ba82-a60e1fa318b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7d3cec-dc47dd986-t2hxp"
Jul 25 00:07:01.906246 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:01.906050    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsctt\" (UniqueName: \"kubernetes.io/projected/35a0cd19-77c0-4812-ba82-a60e1fa318b8-kube-api-access-qsctt\") pod \"maas-api-e2e-api-b-7d3cec-dc47dd986-t2hxp\" (UID: \"35a0cd19-77c0-4812-ba82-a60e1fa318b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7d3cec-dc47dd986-t2hxp"
Jul 25 00:07:01.906246 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:01.906078    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35a0cd19-77c0-4812-ba82-a60e1fa318b8-maas-api-tls\") pod \"maas-api-e2e-api-b-7d3cec-dc47dd986-t2hxp\" (UID: \"35a0cd19-77c0-4812-ba82-a60e1fa318b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7d3cec-dc47dd986-t2hxp"
Jul 25 00:07:01.906246 ip-10-0-142-133 kubenswrapper[2558]: E0725 00:07:01.906197    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-7d3cec-serving-cert: secret "maas-api-e2e-api-b-7d3cec-serving-cert" not found
Jul 25 00:07:01.906426 ip-10-0-142-133 kubenswrapper[2558]: E0725 00:07:01.906257    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35a0cd19-77c0-4812-ba82-a60e1fa318b8-maas-api-tls podName:35a0cd19-77c0-4812-ba82-a60e1fa318b8 nodeName:}" failed. No retries permitted until 2026-07-25 00:07:02.406237345 +0000 UTC m=+2262.792668050 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/35a0cd19-77c0-4812-ba82-a60e1fa318b8-maas-api-tls") pod "maas-api-e2e-api-b-7d3cec-dc47dd986-t2hxp" (UID: "35a0cd19-77c0-4812-ba82-a60e1fa318b8") : secret "maas-api-e2e-api-b-7d3cec-serving-cert" not found
Jul 25 00:07:01.906734 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:01.906713    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35a0cd19-77c0-4812-ba82-a60e1fa318b8-openshift-service-ca\") pod \"maas-api-e2e-api-b-7d3cec-dc47dd986-t2hxp\" (UID: \"35a0cd19-77c0-4812-ba82-a60e1fa318b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7d3cec-dc47dd986-t2hxp"
Jul 25 00:07:01.916459 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:01.916432    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qsctt\" (UniqueName: \"kubernetes.io/projected/35a0cd19-77c0-4812-ba82-a60e1fa318b8-kube-api-access-qsctt\") pod \"maas-api-e2e-api-b-7d3cec-dc47dd986-t2hxp\" (UID: \"35a0cd19-77c0-4812-ba82-a60e1fa318b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7d3cec-dc47dd986-t2hxp"
Jul 25 00:07:02.352307 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:02.352272    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:07:02.411212 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:02.411175    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35a0cd19-77c0-4812-ba82-a60e1fa318b8-maas-api-tls\") pod \"maas-api-e2e-api-b-7d3cec-dc47dd986-t2hxp\" (UID: \"35a0cd19-77c0-4812-ba82-a60e1fa318b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7d3cec-dc47dd986-t2hxp"
Jul 25 00:07:02.411428 ip-10-0-142-133 kubenswrapper[2558]: E0725 00:07:02.411325    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-7d3cec-serving-cert: secret "maas-api-e2e-api-b-7d3cec-serving-cert" not found
Jul 25 00:07:02.411428 ip-10-0-142-133 kubenswrapper[2558]: E0725 00:07:02.411393    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35a0cd19-77c0-4812-ba82-a60e1fa318b8-maas-api-tls podName:35a0cd19-77c0-4812-ba82-a60e1fa318b8 nodeName:}" failed. No retries permitted until 2026-07-25 00:07:03.411376575 +0000 UTC m=+2263.797807281 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/35a0cd19-77c0-4812-ba82-a60e1fa318b8-maas-api-tls") pod "maas-api-e2e-api-b-7d3cec-dc47dd986-t2hxp" (UID: "35a0cd19-77c0-4812-ba82-a60e1fa318b8") : secret "maas-api-e2e-api-b-7d3cec-serving-cert" not found
Jul 25 00:07:02.470664 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:02.470633    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:07:02.528657 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:02.528620    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:07:03.419862 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:03.419820    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35a0cd19-77c0-4812-ba82-a60e1fa318b8-maas-api-tls\") pod \"maas-api-e2e-api-b-7d3cec-dc47dd986-t2hxp\" (UID: \"35a0cd19-77c0-4812-ba82-a60e1fa318b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7d3cec-dc47dd986-t2hxp"
Jul 25 00:07:03.422309 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:03.422286    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35a0cd19-77c0-4812-ba82-a60e1fa318b8-maas-api-tls\") pod \"maas-api-e2e-api-b-7d3cec-dc47dd986-t2hxp\" (UID: \"35a0cd19-77c0-4812-ba82-a60e1fa318b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7d3cec-dc47dd986-t2hxp"
Jul 25 00:07:03.513787 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:03.513752    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7d3cec-dc47dd986-t2hxp"
Jul 25 00:07:03.654515 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:03.654492    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-7d3cec-dc47dd986-t2hxp"]
Jul 25 00:07:03.655434 ip-10-0-142-133 kubenswrapper[2558]: W0725 00:07:03.655405    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35a0cd19_77c0_4812_ba82_a60e1fa318b8.slice/crio-d9646f1ea96091d03c14aea46b687b8fb3e27c2d7e7d2b5a863acd09df44b0ea WatchSource:0}: Error finding container d9646f1ea96091d03c14aea46b687b8fb3e27c2d7e7d2b5a863acd09df44b0ea: Status 404 returned error can't find the container with id d9646f1ea96091d03c14aea46b687b8fb3e27c2d7e7d2b5a863acd09df44b0ea
Jul 25 00:07:03.746303 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:03.746268    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7d3cec-dc47dd986-t2hxp" event={"ID":"35a0cd19-77c0-4812-ba82-a60e1fa318b8","Type":"ContainerStarted","Data":"d9646f1ea96091d03c14aea46b687b8fb3e27c2d7e7d2b5a863acd09df44b0ea"}
Jul 25 00:07:04.753574 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:04.753521    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7d3cec-dc47dd986-t2hxp" event={"ID":"35a0cd19-77c0-4812-ba82-a60e1fa318b8","Type":"ContainerStarted","Data":"378495e9a38dfa233284e32f0eadb7520da087c6d87ffc085054878630a9443c"}
Jul 25 00:07:04.754006 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:04.753602    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7d3cec-dc47dd986-t2hxp"
Jul 25 00:07:04.768819 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:04.768767    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7d3cec-dc47dd986-t2hxp" podStartSLOduration=3.364104133 podStartE2EDuration="3.768751458s" totalImagesPullingTime="404.647325ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:07:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:07:04.767927968 +0000 UTC m=+2265.154358693" watchObservedRunningTime="2026-07-25 00:07:04.768751458 +0000 UTC m=+2265.155182187"
Jul 25 00:07:05.972310 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:05.972273    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:07:06.707960 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:06.707929    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-97aaee-5f5d8858d-dpm4t"
Jul 25 00:07:06.708135 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:06.707998    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-97aaee-ff695db96-mpzst"
Jul 25 00:07:10.766155 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:10.766122    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7d3cec-dc47dd986-t2hxp"
Jul 25 00:07:20.401505 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:20.401467    2558 scope.go:117] "RemoveContainer" containerID="2f61ad50527631baf70d6f36a1fbdb42fa5e7bd5a38a524ce3979eba30a68bba"
Jul 25 00:07:39.965900 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:39.965860    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-97aaee-75599d9f48-tdhmb"]
Jul 25 00:07:39.966560 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:39.966113    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-97aaee-75599d9f48-tdhmb" podUID="e3ee0052-5f20-4d2d-b7d1-115a82a249ae" containerName="maas-api" containerID="cri-o://ece7835689562602acdd60e42747c5a0a1319202a65d8aebab81f86cc272438f" gracePeriod=30
Jul 25 00:07:39.991245 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:39.991208    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:07:40.010466 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:40.010433    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-97aaee-5f5d8858d-dpm4t"]
Jul 25 00:07:40.010704 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:40.010677    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-97aaee-5f5d8858d-dpm4t" podUID="b82bb394-2149-4726-8463-2080ec8d06ab" containerName="payload-processing" containerID="cri-o://16e7e79932f66278e733a59ec78ffc19386d0f043aaeda69591f32e1a8f86120" gracePeriod=30
Jul 25 00:07:40.017988 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:40.017942    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-97aaee-ff695db96-mpzst"]
Jul 25 00:07:40.018218 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:40.018186    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-97aaee-ff695db96-mpzst" podUID="35765ccc-e347-4adb-af0a-f6ec31c12a45" containerName="payload-pre-processing" containerID="cri-o://7e7e15ffdeb665553512995afd389915ccc2b194fea488f9542052a5973175a1" gracePeriod=30
Jul 25 00:07:40.274687 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:40.274655    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-97aaee-75599d9f48-tdhmb"
Jul 25 00:07:40.317863 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:40.317831    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-97aaee-5f5d8858d-dpm4t"
Jul 25 00:07:40.321387 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:40.321364    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-97aaee-ff695db96-mpzst"
Jul 25 00:07:40.353884 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:40.353854    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2f9gh\" (UniqueName: \"kubernetes.io/projected/e3ee0052-5f20-4d2d-b7d1-115a82a249ae-kube-api-access-2f9gh\") pod \"e3ee0052-5f20-4d2d-b7d1-115a82a249ae\" (UID: \"e3ee0052-5f20-4d2d-b7d1-115a82a249ae\") "
Jul 25 00:07:40.353884 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:40.353888    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3ee0052-5f20-4d2d-b7d1-115a82a249ae-openshift-service-ca\") pod \"e3ee0052-5f20-4d2d-b7d1-115a82a249ae\" (UID: \"e3ee0052-5f20-4d2d-b7d1-115a82a249ae\") "
Jul 25 00:07:40.354129 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:40.353913    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b82bb394-2149-4726-8463-2080ec8d06ab-plugins-config-volume\") pod \"b82bb394-2149-4726-8463-2080ec8d06ab\" (UID: \"b82bb394-2149-4726-8463-2080ec8d06ab\") "
Jul 25 00:07:40.354129 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:40.353951    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s9vm6\" (UniqueName: \"kubernetes.io/projected/35765ccc-e347-4adb-af0a-f6ec31c12a45-kube-api-access-s9vm6\") pod \"35765ccc-e347-4adb-af0a-f6ec31c12a45\" (UID: \"35765ccc-e347-4adb-af0a-f6ec31c12a45\") "
Jul 25 00:07:40.354129 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:40.353981    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35765ccc-e347-4adb-af0a-f6ec31c12a45-plugins-config-volume\") pod \"35765ccc-e347-4adb-af0a-f6ec31c12a45\" (UID: \"35765ccc-e347-4adb-af0a-f6ec31c12a45\") "
Jul 25 00:07:40.354129 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:40.354046    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6gqjh\" (UniqueName: \"kubernetes.io/projected/b82bb394-2149-4726-8463-2080ec8d06ab-kube-api-access-6gqjh\") pod \"b82bb394-2149-4726-8463-2080ec8d06ab\" (UID: \"b82bb394-2149-4726-8463-2080ec8d06ab\") "
Jul 25 00:07:40.354129 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:40.354081    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e3ee0052-5f20-4d2d-b7d1-115a82a249ae-maas-api-tls\") pod \"e3ee0052-5f20-4d2d-b7d1-115a82a249ae\" (UID: \"e3ee0052-5f20-4d2d-b7d1-115a82a249ae\") "
Jul 25 00:07:40.354422 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:40.354281    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3ee0052-5f20-4d2d-b7d1-115a82a249ae-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e3ee0052-5f20-4d2d-b7d1-115a82a249ae" (UID: "e3ee0052-5f20-4d2d-b7d1-115a82a249ae"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 00:07:40.354422 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:40.354360    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b82bb394-2149-4726-8463-2080ec8d06ab-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b82bb394-2149-4726-8463-2080ec8d06ab" (UID: "b82bb394-2149-4726-8463-2080ec8d06ab"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 00:07:40.354422 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:40.354385    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3ee0052-5f20-4d2d-b7d1-115a82a249ae-openshift-service-ca\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 25 00:07:40.354839 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:40.354783    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35765ccc-e347-4adb-af0a-f6ec31c12a45-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "35765ccc-e347-4adb-af0a-f6ec31c12a45" (UID: "35765ccc-e347-4adb-af0a-f6ec31c12a45"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 00:07:40.356207 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:40.356178    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3ee0052-5f20-4d2d-b7d1-115a82a249ae-kube-api-access-2f9gh" (OuterVolumeSpecName: "kube-api-access-2f9gh") pod "e3ee0052-5f20-4d2d-b7d1-115a82a249ae" (UID: "e3ee0052-5f20-4d2d-b7d1-115a82a249ae"). InnerVolumeSpecName "kube-api-access-2f9gh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 00:07:40.356353 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:40.356309    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35765ccc-e347-4adb-af0a-f6ec31c12a45-kube-api-access-s9vm6" (OuterVolumeSpecName: "kube-api-access-s9vm6") pod "35765ccc-e347-4adb-af0a-f6ec31c12a45" (UID: "35765ccc-e347-4adb-af0a-f6ec31c12a45"). InnerVolumeSpecName "kube-api-access-s9vm6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 00:07:40.356647 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:40.356622    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3ee0052-5f20-4d2d-b7d1-115a82a249ae-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e3ee0052-5f20-4d2d-b7d1-115a82a249ae" (UID: "e3ee0052-5f20-4d2d-b7d1-115a82a249ae"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 00:07:40.356722 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:40.356668    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b82bb394-2149-4726-8463-2080ec8d06ab-kube-api-access-6gqjh" (OuterVolumeSpecName: "kube-api-access-6gqjh") pod "b82bb394-2149-4726-8463-2080ec8d06ab" (UID: "b82bb394-2149-4726-8463-2080ec8d06ab"). InnerVolumeSpecName "kube-api-access-6gqjh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 00:07:40.455702 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:40.455665    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35765ccc-e347-4adb-af0a-f6ec31c12a45-plugins-config-volume\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 25 00:07:40.455702 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:40.455695    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6gqjh\" (UniqueName: \"kubernetes.io/projected/b82bb394-2149-4726-8463-2080ec8d06ab-kube-api-access-6gqjh\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 25 00:07:40.455702 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:40.455707    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e3ee0052-5f20-4d2d-b7d1-115a82a249ae-maas-api-tls\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 25 00:07:40.455947 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:40.455716    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2f9gh\" (UniqueName: \"kubernetes.io/projected/e3ee0052-5f20-4d2d-b7d1-115a82a249ae-kube-api-access-2f9gh\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 25 00:07:40.455947 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:40.455725    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b82bb394-2149-4726-8463-2080ec8d06ab-plugins-config-volume\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 25 00:07:40.455947 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:40.455734    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s9vm6\" (UniqueName: \"kubernetes.io/projected/35765ccc-e347-4adb-af0a-f6ec31c12a45-kube-api-access-s9vm6\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 25 00:07:40.951096 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:40.951063    2558 generic.go:356] "Generic (PLEG): container finished" podID="e3ee0052-5f20-4d2d-b7d1-115a82a249ae" containerID="ece7835689562602acdd60e42747c5a0a1319202a65d8aebab81f86cc272438f" exitCode=0
Jul 25 00:07:40.951298 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:40.951108    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-97aaee-75599d9f48-tdhmb"
Jul 25 00:07:40.951298 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:40.951144    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-97aaee-75599d9f48-tdhmb" event={"ID":"e3ee0052-5f20-4d2d-b7d1-115a82a249ae","Type":"ContainerDied","Data":"ece7835689562602acdd60e42747c5a0a1319202a65d8aebab81f86cc272438f"}
Jul 25 00:07:40.951298 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:40.951215    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-97aaee-75599d9f48-tdhmb" event={"ID":"e3ee0052-5f20-4d2d-b7d1-115a82a249ae","Type":"ContainerDied","Data":"260bf5e4a4a42c2b976e403f10e12404d6b1b65c9086941e81e363f5fa1cd196"}
Jul 25 00:07:40.951298 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:40.951233    2558 scope.go:117] "RemoveContainer" containerID="ece7835689562602acdd60e42747c5a0a1319202a65d8aebab81f86cc272438f"
Jul 25 00:07:40.952453 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:40.952429    2558 generic.go:356] "Generic (PLEG): container finished" podID="b82bb394-2149-4726-8463-2080ec8d06ab" containerID="16e7e79932f66278e733a59ec78ffc19386d0f043aaeda69591f32e1a8f86120" exitCode=0
Jul 25 00:07:40.952606 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:40.952475    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-97aaee-5f5d8858d-dpm4t"
Jul 25 00:07:40.952606 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:40.952523    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-97aaee-5f5d8858d-dpm4t" event={"ID":"b82bb394-2149-4726-8463-2080ec8d06ab","Type":"ContainerDied","Data":"16e7e79932f66278e733a59ec78ffc19386d0f043aaeda69591f32e1a8f86120"}
Jul 25 00:07:40.952728 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:40.952604    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-97aaee-5f5d8858d-dpm4t" event={"ID":"b82bb394-2149-4726-8463-2080ec8d06ab","Type":"ContainerDied","Data":"4c1b724223845aec777bd98cb473da775be4d8f5d938a0f6c5027debdbc02a12"}
Jul 25 00:07:40.954023 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:40.953997    2558 generic.go:356] "Generic (PLEG): container finished" podID="35765ccc-e347-4adb-af0a-f6ec31c12a45" containerID="7e7e15ffdeb665553512995afd389915ccc2b194fea488f9542052a5973175a1" exitCode=0
Jul 25 00:07:40.954151 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:40.954029    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-97aaee-ff695db96-mpzst"
Jul 25 00:07:40.954151 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:40.954038    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-97aaee-ff695db96-mpzst" event={"ID":"35765ccc-e347-4adb-af0a-f6ec31c12a45","Type":"ContainerDied","Data":"7e7e15ffdeb665553512995afd389915ccc2b194fea488f9542052a5973175a1"}
Jul 25 00:07:40.954151 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:40.954146    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-97aaee-ff695db96-mpzst" event={"ID":"35765ccc-e347-4adb-af0a-f6ec31c12a45","Type":"ContainerDied","Data":"45d0265e0fa1ebbea53d41ef599b47be8aab59c90236ebdbd0984317cc36ae16"}
Jul 25 00:07:40.961308 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:40.961288    2558 scope.go:117] "RemoveContainer" containerID="ece7835689562602acdd60e42747c5a0a1319202a65d8aebab81f86cc272438f"
Jul 25 00:07:40.961701 ip-10-0-142-133 kubenswrapper[2558]: E0725 00:07:40.961591    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ece7835689562602acdd60e42747c5a0a1319202a65d8aebab81f86cc272438f\": container with ID starting with ece7835689562602acdd60e42747c5a0a1319202a65d8aebab81f86cc272438f not found: ID does not exist" containerID="ece7835689562602acdd60e42747c5a0a1319202a65d8aebab81f86cc272438f"
Jul 25 00:07:40.961701 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:40.961613    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ece7835689562602acdd60e42747c5a0a1319202a65d8aebab81f86cc272438f"} err="failed to get container status \"ece7835689562602acdd60e42747c5a0a1319202a65d8aebab81f86cc272438f\": rpc error: code = NotFound desc = could not find container \"ece7835689562602acdd60e42747c5a0a1319202a65d8aebab81f86cc272438f\": container with ID starting with ece7835689562602acdd60e42747c5a0a1319202a65d8aebab81f86cc272438f not found: ID does not exist"
Jul 25 00:07:40.961701 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:40.961629    2558 scope.go:117] "RemoveContainer" containerID="16e7e79932f66278e733a59ec78ffc19386d0f043aaeda69591f32e1a8f86120"
Jul 25 00:07:40.971002 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:40.970865    2558 scope.go:117] "RemoveContainer" containerID="16e7e79932f66278e733a59ec78ffc19386d0f043aaeda69591f32e1a8f86120"
Jul 25 00:07:40.971228 ip-10-0-142-133 kubenswrapper[2558]: E0725 00:07:40.971108    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"16e7e79932f66278e733a59ec78ffc19386d0f043aaeda69591f32e1a8f86120\": container with ID starting with 16e7e79932f66278e733a59ec78ffc19386d0f043aaeda69591f32e1a8f86120 not found: ID does not exist" containerID="16e7e79932f66278e733a59ec78ffc19386d0f043aaeda69591f32e1a8f86120"
Jul 25 00:07:40.971228 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:40.971134    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16e7e79932f66278e733a59ec78ffc19386d0f043aaeda69591f32e1a8f86120"} err="failed to get container status \"16e7e79932f66278e733a59ec78ffc19386d0f043aaeda69591f32e1a8f86120\": rpc error: code = NotFound desc = could not find container \"16e7e79932f66278e733a59ec78ffc19386d0f043aaeda69591f32e1a8f86120\": container with ID starting with 16e7e79932f66278e733a59ec78ffc19386d0f043aaeda69591f32e1a8f86120 not found: ID does not exist"
Jul 25 00:07:40.971228 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:40.971150    2558 scope.go:117] "RemoveContainer" containerID="7e7e15ffdeb665553512995afd389915ccc2b194fea488f9542052a5973175a1"
Jul 25 00:07:40.982314 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:40.979936    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-97aaee-75599d9f48-tdhmb"]
Jul 25 00:07:40.983164 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:40.982408    2558 scope.go:117] "RemoveContainer" containerID="7e7e15ffdeb665553512995afd389915ccc2b194fea488f9542052a5973175a1"
Jul 25 00:07:40.983164 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:40.982781    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-97aaee-75599d9f48-tdhmb"]
Jul 25 00:07:40.983492 ip-10-0-142-133 kubenswrapper[2558]: E0725 00:07:40.983336    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e7e15ffdeb665553512995afd389915ccc2b194fea488f9542052a5973175a1\": container with ID starting with 7e7e15ffdeb665553512995afd389915ccc2b194fea488f9542052a5973175a1 not found: ID does not exist" containerID="7e7e15ffdeb665553512995afd389915ccc2b194fea488f9542052a5973175a1"
Jul 25 00:07:40.983492 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:40.983369    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e7e15ffdeb665553512995afd389915ccc2b194fea488f9542052a5973175a1"} err="failed to get container status \"7e7e15ffdeb665553512995afd389915ccc2b194fea488f9542052a5973175a1\": rpc error: code = NotFound desc = could not find container \"7e7e15ffdeb665553512995afd389915ccc2b194fea488f9542052a5973175a1\": container with ID starting with 7e7e15ffdeb665553512995afd389915ccc2b194fea488f9542052a5973175a1 not found: ID does not exist"
Jul 25 00:07:40.991985 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:40.991961    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-97aaee-5f5d8858d-dpm4t"]
Jul 25 00:07:40.994169 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:40.994147    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-97aaee-5f5d8858d-dpm4t"]
Jul 25 00:07:41.004224 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:41.004181    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-97aaee-ff695db96-mpzst"]
Jul 25 00:07:41.006337 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:41.006314    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-97aaee-ff695db96-mpzst"]
Jul 25 00:07:41.783703 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:41.783665    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:07:42.204794 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:42.204712    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35765ccc-e347-4adb-af0a-f6ec31c12a45" path="/var/lib/kubelet/pods/35765ccc-e347-4adb-af0a-f6ec31c12a45/volumes"
Jul 25 00:07:42.205165 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:42.205060    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b82bb394-2149-4726-8463-2080ec8d06ab" path="/var/lib/kubelet/pods/b82bb394-2149-4726-8463-2080ec8d06ab/volumes"
Jul 25 00:07:42.205399 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:42.205386    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3ee0052-5f20-4d2d-b7d1-115a82a249ae" path="/var/lib/kubelet/pods/e3ee0052-5f20-4d2d-b7d1-115a82a249ae/volumes"
Jul 25 00:07:44.237944 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:44.237909    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:07:44.631183 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:44.631100    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:07:50.306561 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:50.306500    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:07:50.418168 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:50.418129    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:07:50.663966 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:50.663888    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:07:50.825816 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:50.825780    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:07:50.954571 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:50.954462    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7d3cec-9cmxv"]
Jul 25 00:07:50.955094 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:50.955070    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35765ccc-e347-4adb-af0a-f6ec31c12a45" containerName="payload-pre-processing"
Jul 25 00:07:50.955179 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:50.955098    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="35765ccc-e347-4adb-af0a-f6ec31c12a45" containerName="payload-pre-processing"
Jul 25 00:07:50.955179 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:50.955119    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b82bb394-2149-4726-8463-2080ec8d06ab" containerName="payload-processing"
Jul 25 00:07:50.955179 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:50.955126    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="b82bb394-2149-4726-8463-2080ec8d06ab" containerName="payload-processing"
Jul 25 00:07:50.955179 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:50.955136    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3ee0052-5f20-4d2d-b7d1-115a82a249ae" containerName="maas-api"
Jul 25 00:07:50.955179 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:50.955143    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3ee0052-5f20-4d2d-b7d1-115a82a249ae" containerName="maas-api"
Jul 25 00:07:50.955447 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:50.955206    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3ee0052-5f20-4d2d-b7d1-115a82a249ae" containerName="maas-api"
Jul 25 00:07:50.955447 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:50.955217    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="b82bb394-2149-4726-8463-2080ec8d06ab" containerName="payload-processing"
Jul 25 00:07:50.955447 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:50.955227    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="35765ccc-e347-4adb-af0a-f6ec31c12a45" containerName="payload-pre-processing"
Jul 25 00:07:50.960300 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:50.960272    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7d3cec-9cmxv"
Jul 25 00:07:50.962634 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:50.962610    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bxr46\""
Jul 25 00:07:50.962858 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:50.962833    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7d3cec-9cmxv"]
Jul 25 00:07:51.049226 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:51.049185    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/44085ef6-6023-4083-beb2-ddcd7c793a9e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-7d3cec-9cmxv\" (UID: \"44085ef6-6023-4083-beb2-ddcd7c793a9e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7d3cec-9cmxv"
Jul 25 00:07:51.150230 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:51.150191    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/44085ef6-6023-4083-beb2-ddcd7c793a9e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-7d3cec-9cmxv\" (UID: \"44085ef6-6023-4083-beb2-ddcd7c793a9e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7d3cec-9cmxv"
Jul 25 00:07:51.150931 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:51.150906    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/44085ef6-6023-4083-beb2-ddcd7c793a9e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-7d3cec-9cmxv\" (UID: \"44085ef6-6023-4083-beb2-ddcd7c793a9e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7d3cec-9cmxv"
Jul 25 00:07:51.273100 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:51.273054    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7d3cec-9cmxv"
Jul 25 00:07:51.411091 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:51.411055    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7d3cec-9cmxv"]
Jul 25 00:07:51.414523 ip-10-0-142-133 kubenswrapper[2558]: W0725 00:07:51.414491    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44085ef6_6023_4083_beb2_ddcd7c793a9e.slice/crio-6cce8a7960d02e223522febe88c96bc6679e550199373e6c64c5b7a52a97e725 WatchSource:0}: Error finding container 6cce8a7960d02e223522febe88c96bc6679e550199373e6c64c5b7a52a97e725: Status 404 returned error can't find the container with id 6cce8a7960d02e223522febe88c96bc6679e550199373e6c64c5b7a52a97e725
Jul 25 00:07:52.018577 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:52.018518    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7d3cec-9cmxv" event={"ID":"44085ef6-6023-4083-beb2-ddcd7c793a9e","Type":"ContainerStarted","Data":"b54dc53e897fca0033f7b5eb95d294672e5901233b7b8ec346fa58fe0074bbdc"}
Jul 25 00:07:52.018775 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:52.018595    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7d3cec-9cmxv" event={"ID":"44085ef6-6023-4083-beb2-ddcd7c793a9e","Type":"ContainerStarted","Data":"6cce8a7960d02e223522febe88c96bc6679e550199373e6c64c5b7a52a97e725"}
Jul 25 00:07:52.034449 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:52.034399    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7d3cec-9cmxv" podStartSLOduration=2.034378698 podStartE2EDuration="2.034378698s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:07:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:07:52.032857445 +0000 UTC m=+2312.419288170" watchObservedRunningTime="2026-07-25 00:07:52.034378698 +0000 UTC m=+2312.420809426"
Jul 25 00:07:53.025655 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:53.025618    2558 generic.go:356] "Generic (PLEG): container finished" podID="44085ef6-6023-4083-beb2-ddcd7c793a9e" containerID="b54dc53e897fca0033f7b5eb95d294672e5901233b7b8ec346fa58fe0074bbdc" exitCode=0
Jul 25 00:07:53.026104 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:53.025702    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7d3cec-9cmxv" event={"ID":"44085ef6-6023-4083-beb2-ddcd7c793a9e","Type":"ContainerDied","Data":"b54dc53e897fca0033f7b5eb95d294672e5901233b7b8ec346fa58fe0074bbdc"}
Jul 25 00:07:54.164367 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:54.164343    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7d3cec-9cmxv"
Jul 25 00:07:54.278085 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:54.278049    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/44085ef6-6023-4083-beb2-ddcd7c793a9e-maas-api-service-ca\") pod \"44085ef6-6023-4083-beb2-ddcd7c793a9e\" (UID: \"44085ef6-6023-4083-beb2-ddcd7c793a9e\") "
Jul 25 00:07:54.278491 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:54.278469    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44085ef6-6023-4083-beb2-ddcd7c793a9e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "44085ef6-6023-4083-beb2-ddcd7c793a9e" (UID: "44085ef6-6023-4083-beb2-ddcd7c793a9e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 00:07:54.379687 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:54.379590    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/44085ef6-6023-4083-beb2-ddcd7c793a9e-maas-api-service-ca\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 25 00:07:55.037096 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:55.037062    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7d3cec-9cmxv"
Jul 25 00:07:55.037251 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:55.037091    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7d3cec-9cmxv" event={"ID":"44085ef6-6023-4083-beb2-ddcd7c793a9e","Type":"ContainerDied","Data":"6cce8a7960d02e223522febe88c96bc6679e550199373e6c64c5b7a52a97e725"}
Jul 25 00:07:55.037251 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:07:55.037160    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6cce8a7960d02e223522febe88c96bc6679e550199373e6c64c5b7a52a97e725"
Jul 25 00:08:06.377805 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:06.377525    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-7d3cec-dc47dd986-t2hxp"]
Jul 25 00:08:06.378328 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:06.377820    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7d3cec-dc47dd986-t2hxp" podUID="35a0cd19-77c0-4812-ba82-a60e1fa318b8" containerName="maas-api" containerID="cri-o://378495e9a38dfa233284e32f0eadb7520da087c6d87ffc085054878630a9443c" gracePeriod=30
Jul 25 00:08:06.407576 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:06.407514    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:08:06.478708 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:06.478670    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:08:06.535674 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:06.535638    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:08:06.627079 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:06.627051    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7d3cec-dc47dd986-t2hxp"
Jul 25 00:08:06.789485 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:06.789449    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35a0cd19-77c0-4812-ba82-a60e1fa318b8-maas-api-tls\") pod \"35a0cd19-77c0-4812-ba82-a60e1fa318b8\" (UID: \"35a0cd19-77c0-4812-ba82-a60e1fa318b8\") "
Jul 25 00:08:06.789662 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:06.789508    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qsctt\" (UniqueName: \"kubernetes.io/projected/35a0cd19-77c0-4812-ba82-a60e1fa318b8-kube-api-access-qsctt\") pod \"35a0cd19-77c0-4812-ba82-a60e1fa318b8\" (UID: \"35a0cd19-77c0-4812-ba82-a60e1fa318b8\") "
Jul 25 00:08:06.789662 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:06.789583    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35a0cd19-77c0-4812-ba82-a60e1fa318b8-openshift-service-ca\") pod \"35a0cd19-77c0-4812-ba82-a60e1fa318b8\" (UID: \"35a0cd19-77c0-4812-ba82-a60e1fa318b8\") "
Jul 25 00:08:06.790014 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:06.789990    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35a0cd19-77c0-4812-ba82-a60e1fa318b8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "35a0cd19-77c0-4812-ba82-a60e1fa318b8" (UID: "35a0cd19-77c0-4812-ba82-a60e1fa318b8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 00:08:06.791704 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:06.791675    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/35a0cd19-77c0-4812-ba82-a60e1fa318b8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "35a0cd19-77c0-4812-ba82-a60e1fa318b8" (UID: "35a0cd19-77c0-4812-ba82-a60e1fa318b8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 00:08:06.791803 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:06.791784    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35a0cd19-77c0-4812-ba82-a60e1fa318b8-kube-api-access-qsctt" (OuterVolumeSpecName: "kube-api-access-qsctt") pod "35a0cd19-77c0-4812-ba82-a60e1fa318b8" (UID: "35a0cd19-77c0-4812-ba82-a60e1fa318b8"). InnerVolumeSpecName "kube-api-access-qsctt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 00:08:06.890408 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:06.890368    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35a0cd19-77c0-4812-ba82-a60e1fa318b8-maas-api-tls\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 25 00:08:06.890408 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:06.890402    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qsctt\" (UniqueName: \"kubernetes.io/projected/35a0cd19-77c0-4812-ba82-a60e1fa318b8-kube-api-access-qsctt\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 25 00:08:06.890408 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:06.890412    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35a0cd19-77c0-4812-ba82-a60e1fa318b8-openshift-service-ca\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 25 00:08:07.109201 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:07.109115    2558 generic.go:356] "Generic (PLEG): container finished" podID="35a0cd19-77c0-4812-ba82-a60e1fa318b8" containerID="378495e9a38dfa233284e32f0eadb7520da087c6d87ffc085054878630a9443c" exitCode=0
Jul 25 00:08:07.109201 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:07.109157    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7d3cec-dc47dd986-t2hxp"
Jul 25 00:08:07.109376 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:07.109194    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7d3cec-dc47dd986-t2hxp" event={"ID":"35a0cd19-77c0-4812-ba82-a60e1fa318b8","Type":"ContainerDied","Data":"378495e9a38dfa233284e32f0eadb7520da087c6d87ffc085054878630a9443c"}
Jul 25 00:08:07.109376 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:07.109247    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7d3cec-dc47dd986-t2hxp" event={"ID":"35a0cd19-77c0-4812-ba82-a60e1fa318b8","Type":"ContainerDied","Data":"d9646f1ea96091d03c14aea46b687b8fb3e27c2d7e7d2b5a863acd09df44b0ea"}
Jul 25 00:08:07.109376 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:07.109271    2558 scope.go:117] "RemoveContainer" containerID="378495e9a38dfa233284e32f0eadb7520da087c6d87ffc085054878630a9443c"
Jul 25 00:08:07.118884 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:07.118866    2558 scope.go:117] "RemoveContainer" containerID="378495e9a38dfa233284e32f0eadb7520da087c6d87ffc085054878630a9443c"
Jul 25 00:08:07.119168 ip-10-0-142-133 kubenswrapper[2558]: E0725 00:08:07.119149    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"378495e9a38dfa233284e32f0eadb7520da087c6d87ffc085054878630a9443c\": container with ID starting with 378495e9a38dfa233284e32f0eadb7520da087c6d87ffc085054878630a9443c not found: ID does not exist" containerID="378495e9a38dfa233284e32f0eadb7520da087c6d87ffc085054878630a9443c"
Jul 25 00:08:07.119212 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:07.119177    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"378495e9a38dfa233284e32f0eadb7520da087c6d87ffc085054878630a9443c"} err="failed to get container status \"378495e9a38dfa233284e32f0eadb7520da087c6d87ffc085054878630a9443c\": rpc error: code = NotFound desc = could not find container \"378495e9a38dfa233284e32f0eadb7520da087c6d87ffc085054878630a9443c\": container with ID starting with 378495e9a38dfa233284e32f0eadb7520da087c6d87ffc085054878630a9443c not found: ID does not exist"
Jul 25 00:08:07.130302 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:07.130274    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-7d3cec-dc47dd986-t2hxp"]
Jul 25 00:08:07.134072 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:07.134040    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-7d3cec-dc47dd986-t2hxp"]
Jul 25 00:08:08.204341 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:08.204304    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35a0cd19-77c0-4812-ba82-a60e1fa318b8" path="/var/lib/kubelet/pods/35a0cd19-77c0-4812-ba82-a60e1fa318b8/volumes"
Jul 25 00:08:11.020845 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:11.020811    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:08:11.035306 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:11.035275    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7d3cec-9cmxv"]
Jul 25 00:08:11.038917 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:11.038889    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7d3cec-9cmxv"]
Jul 25 00:08:11.423142 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:11.423065    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:08:12.204224 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:12.204142    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44085ef6-6023-4083-beb2-ddcd7c793a9e" path="/var/lib/kubelet/pods/44085ef6-6023-4083-beb2-ddcd7c793a9e/volumes"
Jul 25 00:08:17.049907 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:17.049870    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:08:17.150154 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:17.150121    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:08:17.420332 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:17.420235    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:08:17.663440 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:17.663403    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:08:18.034584 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:18.034521    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:08:18.358844 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:18.358767    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:08:18.715152 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:18.715061    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:08:20.954676 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:20.954627    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:08:21.030660 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:21.030609    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:08:21.035146 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:21.035106    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-92df89-77ddf56b67-68mcv"]
Jul 25 00:08:21.035687 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:21.035664    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35a0cd19-77c0-4812-ba82-a60e1fa318b8" containerName="maas-api"
Jul 25 00:08:21.035687 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:21.035688    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="35a0cd19-77c0-4812-ba82-a60e1fa318b8" containerName="maas-api"
Jul 25 00:08:21.035884 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:21.035711    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44085ef6-6023-4083-beb2-ddcd7c793a9e" containerName="revoke-keys"
Jul 25 00:08:21.035884 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:21.035721    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="44085ef6-6023-4083-beb2-ddcd7c793a9e" containerName="revoke-keys"
Jul 25 00:08:21.035884 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:21.035834    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="44085ef6-6023-4083-beb2-ddcd7c793a9e" containerName="revoke-keys"
Jul 25 00:08:21.035884 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:21.035845    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="35a0cd19-77c0-4812-ba82-a60e1fa318b8" containerName="maas-api"
Jul 25 00:08:21.040470 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:21.040448    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-92df89-77ddf56b67-68mcv"
Jul 25 00:08:21.042734 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:21.042708    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-92df89-serving-cert\""
Jul 25 00:08:21.049950 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:21.049926    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-92df89-77ddf56b67-68mcv"]
Jul 25 00:08:21.098262 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:21.098224    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:08:21.221968 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:21.221880    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/febc96ee-61d4-4220-885c-45d1fa1edc59-maas-api-tls\") pod \"maas-api-e2e-shared-a-92df89-77ddf56b67-68mcv\" (UID: \"febc96ee-61d4-4220-885c-45d1fa1edc59\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-92df89-77ddf56b67-68mcv"
Jul 25 00:08:21.221968 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:21.221913    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/febc96ee-61d4-4220-885c-45d1fa1edc59-openshift-service-ca\") pod \"maas-api-e2e-shared-a-92df89-77ddf56b67-68mcv\" (UID: \"febc96ee-61d4-4220-885c-45d1fa1edc59\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-92df89-77ddf56b67-68mcv"
Jul 25 00:08:21.222163 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:21.221997    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qrn6\" (UniqueName: \"kubernetes.io/projected/febc96ee-61d4-4220-885c-45d1fa1edc59-kube-api-access-8qrn6\") pod \"maas-api-e2e-shared-a-92df89-77ddf56b67-68mcv\" (UID: \"febc96ee-61d4-4220-885c-45d1fa1edc59\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-92df89-77ddf56b67-68mcv"
Jul 25 00:08:21.323400 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:21.323354    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/febc96ee-61d4-4220-885c-45d1fa1edc59-openshift-service-ca\") pod \"maas-api-e2e-shared-a-92df89-77ddf56b67-68mcv\" (UID: \"febc96ee-61d4-4220-885c-45d1fa1edc59\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-92df89-77ddf56b67-68mcv"
Jul 25 00:08:21.323638 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:21.323448    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qrn6\" (UniqueName: \"kubernetes.io/projected/febc96ee-61d4-4220-885c-45d1fa1edc59-kube-api-access-8qrn6\") pod \"maas-api-e2e-shared-a-92df89-77ddf56b67-68mcv\" (UID: \"febc96ee-61d4-4220-885c-45d1fa1edc59\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-92df89-77ddf56b67-68mcv"
Jul 25 00:08:21.323841 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:21.323802    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/febc96ee-61d4-4220-885c-45d1fa1edc59-maas-api-tls\") pod \"maas-api-e2e-shared-a-92df89-77ddf56b67-68mcv\" (UID: \"febc96ee-61d4-4220-885c-45d1fa1edc59\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-92df89-77ddf56b67-68mcv"
Jul 25 00:08:21.323951 ip-10-0-142-133 kubenswrapper[2558]: E0725 00:08:21.323934    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-92df89-serving-cert: secret "maas-api-e2e-shared-a-92df89-serving-cert" not found
Jul 25 00:08:21.324018 ip-10-0-142-133 kubenswrapper[2558]: E0725 00:08:21.324006    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/febc96ee-61d4-4220-885c-45d1fa1edc59-maas-api-tls podName:febc96ee-61d4-4220-885c-45d1fa1edc59 nodeName:}" failed. No retries permitted until 2026-07-25 00:08:21.82398551 +0000 UTC m=+2342.210416235 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/febc96ee-61d4-4220-885c-45d1fa1edc59-maas-api-tls") pod "maas-api-e2e-shared-a-92df89-77ddf56b67-68mcv" (UID: "febc96ee-61d4-4220-885c-45d1fa1edc59") : secret "maas-api-e2e-shared-a-92df89-serving-cert" not found
Jul 25 00:08:21.324171 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:21.324150    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/febc96ee-61d4-4220-885c-45d1fa1edc59-openshift-service-ca\") pod \"maas-api-e2e-shared-a-92df89-77ddf56b67-68mcv\" (UID: \"febc96ee-61d4-4220-885c-45d1fa1edc59\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-92df89-77ddf56b67-68mcv"
Jul 25 00:08:21.342761 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:21.342727    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qrn6\" (UniqueName: \"kubernetes.io/projected/febc96ee-61d4-4220-885c-45d1fa1edc59-kube-api-access-8qrn6\") pod \"maas-api-e2e-shared-a-92df89-77ddf56b67-68mcv\" (UID: \"febc96ee-61d4-4220-885c-45d1fa1edc59\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-92df89-77ddf56b67-68mcv"
Jul 25 00:08:21.686497 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:21.686458    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:08:21.786825 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:21.786779    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:08:21.828636 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:21.828598    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/febc96ee-61d4-4220-885c-45d1fa1edc59-maas-api-tls\") pod \"maas-api-e2e-shared-a-92df89-77ddf56b67-68mcv\" (UID: \"febc96ee-61d4-4220-885c-45d1fa1edc59\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-92df89-77ddf56b67-68mcv"
Jul 25 00:08:21.831107 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:21.831073    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/febc96ee-61d4-4220-885c-45d1fa1edc59-maas-api-tls\") pod \"maas-api-e2e-shared-a-92df89-77ddf56b67-68mcv\" (UID: \"febc96ee-61d4-4220-885c-45d1fa1edc59\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-92df89-77ddf56b67-68mcv"
Jul 25 00:08:21.881907 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:21.881872    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:08:21.886095 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:21.886068    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-92df89-5694c5d774-zm222"]
Jul 25 00:08:21.889615 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:21.889599    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-92df89-5694c5d774-zm222"
Jul 25 00:08:21.892881 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:21.892861    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-92df89-dockercfg-qfztv\""
Jul 25 00:08:21.893007 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:21.892861    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-92df89\""
Jul 25 00:08:21.899573 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:21.899543    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-92df89-5694c5d774-zm222"]
Jul 25 00:08:21.918798 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:21.918769    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-92df89-7fd7485b8-td6zk"]
Jul 25 00:08:21.922173 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:21.922155    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-92df89-7fd7485b8-td6zk"
Jul 25 00:08:21.929424 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:21.929224    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b3546a5-0243-4529-9f03-878cdb41941a-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-92df89-5694c5d774-zm222\" (UID: \"6b3546a5-0243-4529-9f03-878cdb41941a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-92df89-5694c5d774-zm222"
Jul 25 00:08:21.929424 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:21.929294    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/47c7d4d4-fd24-4083-9eab-87560dea56ea-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-92df89-7fd7485b8-td6zk\" (UID: \"47c7d4d4-fd24-4083-9eab-87560dea56ea\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-92df89-7fd7485b8-td6zk"
Jul 25 00:08:21.929424 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:21.929324    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jt5k7\" (UniqueName: \"kubernetes.io/projected/47c7d4d4-fd24-4083-9eab-87560dea56ea-kube-api-access-jt5k7\") pod \"payload-pre-processing-e2e-shared-a-92df89-7fd7485b8-td6zk\" (UID: \"47c7d4d4-fd24-4083-9eab-87560dea56ea\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-92df89-7fd7485b8-td6zk"
Jul 25 00:08:21.929424 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:21.929352    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fcqlf\" (UniqueName: \"kubernetes.io/projected/6b3546a5-0243-4529-9f03-878cdb41941a-kube-api-access-fcqlf\") pod \"payload-processing-e2e-shared-a-92df89-5694c5d774-zm222\" (UID: \"6b3546a5-0243-4529-9f03-878cdb41941a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-92df89-5694c5d774-zm222"
Jul 25 00:08:21.931357 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:21.931330    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-92df89-7fd7485b8-td6zk"]
Jul 25 00:08:21.952845 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:21.952754    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-92df89-77ddf56b67-68mcv"
Jul 25 00:08:22.030769 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:22.030736    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/47c7d4d4-fd24-4083-9eab-87560dea56ea-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-92df89-7fd7485b8-td6zk\" (UID: \"47c7d4d4-fd24-4083-9eab-87560dea56ea\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-92df89-7fd7485b8-td6zk"
Jul 25 00:08:22.031334 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:22.030779    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jt5k7\" (UniqueName: \"kubernetes.io/projected/47c7d4d4-fd24-4083-9eab-87560dea56ea-kube-api-access-jt5k7\") pod \"payload-pre-processing-e2e-shared-a-92df89-7fd7485b8-td6zk\" (UID: \"47c7d4d4-fd24-4083-9eab-87560dea56ea\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-92df89-7fd7485b8-td6zk"
Jul 25 00:08:22.031334 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:22.030808    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fcqlf\" (UniqueName: \"kubernetes.io/projected/6b3546a5-0243-4529-9f03-878cdb41941a-kube-api-access-fcqlf\") pod \"payload-processing-e2e-shared-a-92df89-5694c5d774-zm222\" (UID: \"6b3546a5-0243-4529-9f03-878cdb41941a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-92df89-5694c5d774-zm222"
Jul 25 00:08:22.031334 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:22.030883    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b3546a5-0243-4529-9f03-878cdb41941a-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-92df89-5694c5d774-zm222\" (UID: \"6b3546a5-0243-4529-9f03-878cdb41941a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-92df89-5694c5d774-zm222"
Jul 25 00:08:22.031606 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:22.031584    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b3546a5-0243-4529-9f03-878cdb41941a-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-92df89-5694c5d774-zm222\" (UID: \"6b3546a5-0243-4529-9f03-878cdb41941a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-92df89-5694c5d774-zm222"
Jul 25 00:08:22.031709 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:22.031688    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/47c7d4d4-fd24-4083-9eab-87560dea56ea-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-92df89-7fd7485b8-td6zk\" (UID: \"47c7d4d4-fd24-4083-9eab-87560dea56ea\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-92df89-7fd7485b8-td6zk"
Jul 25 00:08:22.038540 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:22.038497    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jt5k7\" (UniqueName: \"kubernetes.io/projected/47c7d4d4-fd24-4083-9eab-87560dea56ea-kube-api-access-jt5k7\") pod \"payload-pre-processing-e2e-shared-a-92df89-7fd7485b8-td6zk\" (UID: \"47c7d4d4-fd24-4083-9eab-87560dea56ea\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-92df89-7fd7485b8-td6zk"
Jul 25 00:08:22.039045 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:22.039021    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fcqlf\" (UniqueName: \"kubernetes.io/projected/6b3546a5-0243-4529-9f03-878cdb41941a-kube-api-access-fcqlf\") pod \"payload-processing-e2e-shared-a-92df89-5694c5d774-zm222\" (UID: \"6b3546a5-0243-4529-9f03-878cdb41941a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-92df89-5694c5d774-zm222"
Jul 25 00:08:22.099761 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:22.099722    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-92df89-77ddf56b67-68mcv"]
Jul 25 00:08:22.101248 ip-10-0-142-133 kubenswrapper[2558]: W0725 00:08:22.101197    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfebc96ee_61d4_4220_885c_45d1fa1edc59.slice/crio-6416b1b3bb24a8a69a145cad5fbd8bff39c41d3c0d48e7b61c875b7cad140559 WatchSource:0}: Error finding container 6416b1b3bb24a8a69a145cad5fbd8bff39c41d3c0d48e7b61c875b7cad140559: Status 404 returned error can't find the container with id 6416b1b3bb24a8a69a145cad5fbd8bff39c41d3c0d48e7b61c875b7cad140559
Jul 25 00:08:22.197218 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:22.197170    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-92df89-77ddf56b67-68mcv" event={"ID":"febc96ee-61d4-4220-885c-45d1fa1edc59","Type":"ContainerStarted","Data":"6416b1b3bb24a8a69a145cad5fbd8bff39c41d3c0d48e7b61c875b7cad140559"}
Jul 25 00:08:22.201449 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:22.201419    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-92df89-5694c5d774-zm222"
Jul 25 00:08:22.234004 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:22.233964    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-92df89-7fd7485b8-td6zk"
Jul 25 00:08:22.348617 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:22.348588    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-92df89-5694c5d774-zm222"]
Jul 25 00:08:22.351419 ip-10-0-142-133 kubenswrapper[2558]: W0725 00:08:22.351384    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b3546a5_0243_4529_9f03_878cdb41941a.slice/crio-62e0417ba0fb03d2d371256d78a36c1a5feb7ade81a32cc78d8ce3587b40f626 WatchSource:0}: Error finding container 62e0417ba0fb03d2d371256d78a36c1a5feb7ade81a32cc78d8ce3587b40f626: Status 404 returned error can't find the container with id 62e0417ba0fb03d2d371256d78a36c1a5feb7ade81a32cc78d8ce3587b40f626
Jul 25 00:08:22.390036 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:22.390013    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-92df89-7fd7485b8-td6zk"]
Jul 25 00:08:22.393870 ip-10-0-142-133 kubenswrapper[2558]: W0725 00:08:22.393839    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47c7d4d4_fd24_4083_9eab_87560dea56ea.slice/crio-bf89b0652e4192cda76e14a4ae96bfa6b8eadb07d21ad0b833bd79495832c5f1 WatchSource:0}: Error finding container bf89b0652e4192cda76e14a4ae96bfa6b8eadb07d21ad0b833bd79495832c5f1: Status 404 returned error can't find the container with id bf89b0652e4192cda76e14a4ae96bfa6b8eadb07d21ad0b833bd79495832c5f1
Jul 25 00:08:23.204420 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:23.204382    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-92df89-5694c5d774-zm222" event={"ID":"6b3546a5-0243-4529-9f03-878cdb41941a","Type":"ContainerStarted","Data":"c743116504268adf37d1469088653762d48a586ba058ec4290c177b938423277"}
Jul 25 00:08:23.204965 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:23.204446    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-92df89-5694c5d774-zm222" event={"ID":"6b3546a5-0243-4529-9f03-878cdb41941a","Type":"ContainerStarted","Data":"62e0417ba0fb03d2d371256d78a36c1a5feb7ade81a32cc78d8ce3587b40f626"}
Jul 25 00:08:23.204965 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:23.204479    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-92df89-5694c5d774-zm222"
Jul 25 00:08:23.206074 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:23.206047    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-92df89-77ddf56b67-68mcv" event={"ID":"febc96ee-61d4-4220-885c-45d1fa1edc59","Type":"ContainerStarted","Data":"c1d74db7bd104ae4705bb7567cccee396f4f9f62e2938b6f45dd3ed45afd9ba6"}
Jul 25 00:08:23.206173 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:23.206092    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-92df89-77ddf56b67-68mcv"
Jul 25 00:08:23.207592 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:23.207573    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-92df89-7fd7485b8-td6zk" event={"ID":"47c7d4d4-fd24-4083-9eab-87560dea56ea","Type":"ContainerStarted","Data":"9fc686ae5a3c81136ea33f91f979bc78fafa5d824eff3384e3c5bae6bc64a69f"}
Jul 25 00:08:23.207705 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:23.207608    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-92df89-7fd7485b8-td6zk" event={"ID":"47c7d4d4-fd24-4083-9eab-87560dea56ea","Type":"ContainerStarted","Data":"bf89b0652e4192cda76e14a4ae96bfa6b8eadb07d21ad0b833bd79495832c5f1"}
Jul 25 00:08:23.207705 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:23.207666    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-92df89-7fd7485b8-td6zk"
Jul 25 00:08:23.225008 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:23.224967    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-92df89-5694c5d774-zm222" podStartSLOduration=2.224953732 podStartE2EDuration="2.224953732s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:08:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:08:23.221580737 +0000 UTC m=+2343.608011465" watchObservedRunningTime="2026-07-25 00:08:23.224953732 +0000 UTC m=+2343.611384460"
Jul 25 00:08:23.236429 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:23.236386    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-92df89-7fd7485b8-td6zk" podStartSLOduration=2.236370786 podStartE2EDuration="2.236370786s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:08:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:08:23.234827927 +0000 UTC m=+2343.621258654" watchObservedRunningTime="2026-07-25 00:08:23.236370786 +0000 UTC m=+2343.622801514"
Jul 25 00:08:23.251160 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:23.251109    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-92df89-77ddf56b67-68mcv" podStartSLOduration=1.820294122 podStartE2EDuration="2.251091827s" totalImagesPullingTime="430.797705ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:08:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:08:23.248390401 +0000 UTC m=+2343.634821133" watchObservedRunningTime="2026-07-25 00:08:23.251091827 +0000 UTC m=+2343.637522555"
Jul 25 00:08:25.478670 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:25.478620    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:08:25.827884 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:25.827767    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:08:26.148402 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:26.148301    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:08:26.553104 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:26.553069    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:08:28.959591 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:28.959553    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:08:29.073979 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:29.073932    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:08:29.211680 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:29.211579    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-413844-549c74bd78-vnhdv"]
Jul 25 00:08:29.219814 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:29.219626    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-413844-549c74bd78-vnhdv"
Jul 25 00:08:29.222207 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:29.222183    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-92df89-77ddf56b67-68mcv"
Jul 25 00:08:29.222856 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:29.222829    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-413844-serving-cert\""
Jul 25 00:08:29.223306 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:29.223278    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-413844-549c74bd78-vnhdv"]
Jul 25 00:08:29.245439 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:29.245414    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:08:29.306685 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:29.306653    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vwm5\" (UniqueName: \"kubernetes.io/projected/0a1b7baa-e633-4042-ac62-1ec8df05750a-kube-api-access-6vwm5\") pod \"maas-api-e2e-shared-b-413844-549c74bd78-vnhdv\" (UID: \"0a1b7baa-e633-4042-ac62-1ec8df05750a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-413844-549c74bd78-vnhdv"
Jul 25 00:08:29.306859 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:29.306692    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a1b7baa-e633-4042-ac62-1ec8df05750a-openshift-service-ca\") pod \"maas-api-e2e-shared-b-413844-549c74bd78-vnhdv\" (UID: \"0a1b7baa-e633-4042-ac62-1ec8df05750a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-413844-549c74bd78-vnhdv"
Jul 25 00:08:29.306859 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:29.306805    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0a1b7baa-e633-4042-ac62-1ec8df05750a-maas-api-tls\") pod \"maas-api-e2e-shared-b-413844-549c74bd78-vnhdv\" (UID: \"0a1b7baa-e633-4042-ac62-1ec8df05750a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-413844-549c74bd78-vnhdv"
Jul 25 00:08:29.408316 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:29.408281    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a1b7baa-e633-4042-ac62-1ec8df05750a-openshift-service-ca\") pod \"maas-api-e2e-shared-b-413844-549c74bd78-vnhdv\" (UID: \"0a1b7baa-e633-4042-ac62-1ec8df05750a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-413844-549c74bd78-vnhdv"
Jul 25 00:08:29.408316 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:29.408318    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0a1b7baa-e633-4042-ac62-1ec8df05750a-maas-api-tls\") pod \"maas-api-e2e-shared-b-413844-549c74bd78-vnhdv\" (UID: \"0a1b7baa-e633-4042-ac62-1ec8df05750a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-413844-549c74bd78-vnhdv"
Jul 25 00:08:29.408520 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:29.408403    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vwm5\" (UniqueName: \"kubernetes.io/projected/0a1b7baa-e633-4042-ac62-1ec8df05750a-kube-api-access-6vwm5\") pod \"maas-api-e2e-shared-b-413844-549c74bd78-vnhdv\" (UID: \"0a1b7baa-e633-4042-ac62-1ec8df05750a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-413844-549c74bd78-vnhdv"
Jul 25 00:08:29.408520 ip-10-0-142-133 kubenswrapper[2558]: E0725 00:08:29.408513    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-413844-serving-cert: secret "maas-api-e2e-shared-b-413844-serving-cert" not found
Jul 25 00:08:29.408615 ip-10-0-142-133 kubenswrapper[2558]: E0725 00:08:29.408602    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a1b7baa-e633-4042-ac62-1ec8df05750a-maas-api-tls podName:0a1b7baa-e633-4042-ac62-1ec8df05750a nodeName:}" failed. No retries permitted until 2026-07-25 00:08:29.908580634 +0000 UTC m=+2350.295011357 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0a1b7baa-e633-4042-ac62-1ec8df05750a-maas-api-tls") pod "maas-api-e2e-shared-b-413844-549c74bd78-vnhdv" (UID: "0a1b7baa-e633-4042-ac62-1ec8df05750a") : secret "maas-api-e2e-shared-b-413844-serving-cert" not found
Jul 25 00:08:29.408973 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:29.408952    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a1b7baa-e633-4042-ac62-1ec8df05750a-openshift-service-ca\") pod \"maas-api-e2e-shared-b-413844-549c74bd78-vnhdv\" (UID: \"0a1b7baa-e633-4042-ac62-1ec8df05750a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-413844-549c74bd78-vnhdv"
Jul 25 00:08:29.416848 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:29.416825    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vwm5\" (UniqueName: \"kubernetes.io/projected/0a1b7baa-e633-4042-ac62-1ec8df05750a-kube-api-access-6vwm5\") pod \"maas-api-e2e-shared-b-413844-549c74bd78-vnhdv\" (UID: \"0a1b7baa-e633-4042-ac62-1ec8df05750a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-413844-549c74bd78-vnhdv"
Jul 25 00:08:29.661591 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:29.661522    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:08:29.755668 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:29.755633    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:08:29.822857 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:29.822818    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:08:29.912684 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:29.912610    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0a1b7baa-e633-4042-ac62-1ec8df05750a-maas-api-tls\") pod \"maas-api-e2e-shared-b-413844-549c74bd78-vnhdv\" (UID: \"0a1b7baa-e633-4042-ac62-1ec8df05750a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-413844-549c74bd78-vnhdv"
Jul 25 00:08:29.915113 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:29.915088    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0a1b7baa-e633-4042-ac62-1ec8df05750a-maas-api-tls\") pod \"maas-api-e2e-shared-b-413844-549c74bd78-vnhdv\" (UID: \"0a1b7baa-e633-4042-ac62-1ec8df05750a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-413844-549c74bd78-vnhdv"
Jul 25 00:08:30.133123 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:30.133082    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-413844-549c74bd78-vnhdv"
Jul 25 00:08:30.278894 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:30.278866    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-413844-549c74bd78-vnhdv"]
Jul 25 00:08:30.280506 ip-10-0-142-133 kubenswrapper[2558]: W0725 00:08:30.280480    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a1b7baa_e633_4042_ac62_1ec8df05750a.slice/crio-80bc3a5595023b5de42d23f8bcda89c874ba41a0246ec477cd7d2537ea4bcbfd WatchSource:0}: Error finding container 80bc3a5595023b5de42d23f8bcda89c874ba41a0246ec477cd7d2537ea4bcbfd: Status 404 returned error can't find the container with id 80bc3a5595023b5de42d23f8bcda89c874ba41a0246ec477cd7d2537ea4bcbfd
Jul 25 00:08:31.263401 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:31.263362    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-413844-549c74bd78-vnhdv" event={"ID":"0a1b7baa-e633-4042-ac62-1ec8df05750a","Type":"ContainerStarted","Data":"4297c0d6dfd92567eb6b3120a3130e9967f12da03114228012f82d8f7e735ed1"}
Jul 25 00:08:31.263884 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:31.263604    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-413844-549c74bd78-vnhdv" event={"ID":"0a1b7baa-e633-4042-ac62-1ec8df05750a","Type":"ContainerStarted","Data":"80bc3a5595023b5de42d23f8bcda89c874ba41a0246ec477cd7d2537ea4bcbfd"}
Jul 25 00:08:31.263884 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:31.263690    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-413844-549c74bd78-vnhdv"
Jul 25 00:08:31.278767 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:31.278720    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-413844-549c74bd78-vnhdv" podStartSLOduration=1.946242363 podStartE2EDuration="2.278704648s" totalImagesPullingTime="332.462285ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:08:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:08:31.277650951 +0000 UTC m=+2351.664081678" watchObservedRunningTime="2026-07-25 00:08:31.278704648 +0000 UTC m=+2351.665135376"
Jul 25 00:08:33.492046 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:33.492009    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:08:34.215055 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:34.215027    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-92df89-5694c5d774-zm222"
Jul 25 00:08:34.215248 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:34.215196    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-92df89-7fd7485b8-td6zk"
Jul 25 00:08:36.480708 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:36.480667    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:08:37.275792 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:37.275762    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-413844-549c74bd78-vnhdv"
Jul 25 00:08:44.627607 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:08:44.627566    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:09:02.110550 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:09:02.110498    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:09:20.453567 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:09:20.453516    2558 scope.go:117] "RemoveContainer" containerID="d921ba465882c9ba3e2eee667d789069539d690247c9aaa8f0cd623528ef147a"
Jul 25 00:09:40.877013 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:09:40.876973    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:10:17.968456 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:10:17.968419    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:10:19.291044 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:10:19.291006    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:10:19.391676 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:10:19.391638    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:10:34.686415 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:10:34.686375    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:10:34.734313 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:10:34.734278    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:10:44.699000 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:10:44.698965    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:10:44.747910 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:10:44.747876    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:10:49.960836 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:10:49.960795    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:12:20.499590 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:12:20.499555    2558 scope.go:117] "RemoveContainer" containerID="c110f808af16c0fb37742baa54510825ee6c68c3115bfbaeeb203e490938d9e0"
Jul 25 00:12:39.096475 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:12:39.096430    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:13:20.523841 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:13:20.523757    2558 scope.go:117] "RemoveContainer" containerID="542ab75d42482826de52900be0987b132a125c00b5706ef65d26bfe2d7872f21"
Jul 25 00:13:24.393733 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:13:24.393691    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:13:26.095639 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:13:26.095595    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:13:30.873609 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:13:30.873572    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:14:01.023583 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:14:01.023548    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:14:20.544675 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:14:20.544600    2558 scope.go:117] "RemoveContainer" containerID="b54dc53e897fca0033f7b5eb95d294672e5901233b7b8ec346fa58fe0074bbdc"
Jul 25 00:15:00.131178 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:15:00.131140    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-413844-29748975-dmkl2"]
Jul 25 00:15:00.134651 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:15:00.134626    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-413844-29748975-dmkl2"
Jul 25 00:15:00.137045 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:15:00.137021    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bxr46\""
Jul 25 00:15:00.143774 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:15:00.143743    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-413844-29748975-dmkl2"]
Jul 25 00:15:00.146590 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:15:00.146562    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-413844-29748975-dmkl2"
Jul 25 00:15:00.290773 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:15:00.290743    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-413844-29748975-dmkl2"]
Jul 25 00:15:00.295992 ip-10-0-142-133 kubenswrapper[2558]: W0725 00:15:00.294810    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e921ad2_25d8_4708_8fe7_de17be837bbb.slice/crio-0299365cd621cc63a7fcc253e54f685c5d8c413e22d107b847210aa8cc25fa11 WatchSource:0}: Error finding container 0299365cd621cc63a7fcc253e54f685c5d8c413e22d107b847210aa8cc25fa11: Status 404 returned error can't find the container with id 0299365cd621cc63a7fcc253e54f685c5d8c413e22d107b847210aa8cc25fa11
Jul 25 00:15:00.412495 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:15:00.412402    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-413844-29748975-dmkl2" event={"ID":"7e921ad2-25d8-4708-8fe7-de17be837bbb","Type":"ContainerStarted","Data":"0299365cd621cc63a7fcc253e54f685c5d8c413e22d107b847210aa8cc25fa11"}
Jul 25 00:15:01.419594 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:15:01.419505    2558 generic.go:356] "Generic (PLEG): container finished" podID="7e921ad2-25d8-4708-8fe7-de17be837bbb" containerID="c24f0a320f566ab1ffb4f98bf7571a63222f0f68a524848c2d4afe62f4db0ae6" exitCode=0
Jul 25 00:15:01.419963 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:15:01.419581    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-413844-29748975-dmkl2" event={"ID":"7e921ad2-25d8-4708-8fe7-de17be837bbb","Type":"ContainerDied","Data":"c24f0a320f566ab1ffb4f98bf7571a63222f0f68a524848c2d4afe62f4db0ae6"}
Jul 25 00:15:02.556818 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:15:02.556796    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-413844-29748975-dmkl2"
Jul 25 00:15:03.431259 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:15:03.431228    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-413844-29748975-dmkl2"
Jul 25 00:15:03.431259 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:15:03.431248    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-413844-29748975-dmkl2" event={"ID":"7e921ad2-25d8-4708-8fe7-de17be837bbb","Type":"ContainerDied","Data":"0299365cd621cc63a7fcc253e54f685c5d8c413e22d107b847210aa8cc25fa11"}
Jul 25 00:15:03.431486 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:15:03.431290    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0299365cd621cc63a7fcc253e54f685c5d8c413e22d107b847210aa8cc25fa11"
Jul 25 00:15:09.292304 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:15:09.292269    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:15:10.311221 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:15:10.311186    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:15:24.925782 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:15:24.925741    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:16:07.914666 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:16:07.914566    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:16:29.418648 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:16:29.418606    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:16:48.650757 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:16:48.650720    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:17:22.614324 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:17:22.614269    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:17:23.612105 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:17:23.612073    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:17:43.670615 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:17:43.670576    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:17:44.011085 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:17:44.011047    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:17:44.269216 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:17:44.269122    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:17:47.062836 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:17:47.062770    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:17:47.161161 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:17:47.161122    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:17:48.001410 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:17:48.001374    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-be7457-5bc5998667-n5sls"]
Jul 25 00:17:48.001771 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:17:48.001748    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e921ad2-25d8-4708-8fe7-de17be837bbb" containerName="cleanup"
Jul 25 00:17:48.001771 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:17:48.001771    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e921ad2-25d8-4708-8fe7-de17be837bbb" containerName="cleanup"
Jul 25 00:17:48.001897 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:17:48.001845    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e921ad2-25d8-4708-8fe7-de17be837bbb" containerName="cleanup"
Jul 25 00:17:48.004691 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:17:48.004670    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-be7457-5bc5998667-n5sls"
Jul 25 00:17:48.013520 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:17:48.013497    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-be7457-serving-cert\""
Jul 25 00:17:48.017633 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:17:48.017611    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-be7457-5bc5998667-n5sls"]
Jul 25 00:17:48.037491 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:17:48.037451    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:17:48.137758 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:17:48.137718    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/185a8961-c049-4476-90e5-ee9350c33f6d-maas-api-tls\") pod \"maas-api-e2e-ipp-a-be7457-5bc5998667-n5sls\" (UID: \"185a8961-c049-4476-90e5-ee9350c33f6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-be7457-5bc5998667-n5sls"
Jul 25 00:17:48.138205 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:17:48.137849    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kcntp\" (UniqueName: \"kubernetes.io/projected/185a8961-c049-4476-90e5-ee9350c33f6d-kube-api-access-kcntp\") pod \"maas-api-e2e-ipp-a-be7457-5bc5998667-n5sls\" (UID: \"185a8961-c049-4476-90e5-ee9350c33f6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-be7457-5bc5998667-n5sls"
Jul 25 00:17:48.138205 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:17:48.137906    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/185a8961-c049-4476-90e5-ee9350c33f6d-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-be7457-5bc5998667-n5sls\" (UID: \"185a8961-c049-4476-90e5-ee9350c33f6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-be7457-5bc5998667-n5sls"
Jul 25 00:17:48.238628 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:17:48.238591    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/185a8961-c049-4476-90e5-ee9350c33f6d-maas-api-tls\") pod \"maas-api-e2e-ipp-a-be7457-5bc5998667-n5sls\" (UID: \"185a8961-c049-4476-90e5-ee9350c33f6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-be7457-5bc5998667-n5sls"
Jul 25 00:17:48.238820 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:17:48.238685    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kcntp\" (UniqueName: \"kubernetes.io/projected/185a8961-c049-4476-90e5-ee9350c33f6d-kube-api-access-kcntp\") pod \"maas-api-e2e-ipp-a-be7457-5bc5998667-n5sls\" (UID: \"185a8961-c049-4476-90e5-ee9350c33f6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-be7457-5bc5998667-n5sls"
Jul 25 00:17:48.238820 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:17:48.238720    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/185a8961-c049-4476-90e5-ee9350c33f6d-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-be7457-5bc5998667-n5sls\" (UID: \"185a8961-c049-4476-90e5-ee9350c33f6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-be7457-5bc5998667-n5sls"
Jul 25 00:17:48.238820 ip-10-0-142-133 kubenswrapper[2558]: E0725 00:17:48.238748    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-be7457-serving-cert: secret "maas-api-e2e-ipp-a-be7457-serving-cert" not found
Jul 25 00:17:48.239026 ip-10-0-142-133 kubenswrapper[2558]: E0725 00:17:48.238832    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/185a8961-c049-4476-90e5-ee9350c33f6d-maas-api-tls podName:185a8961-c049-4476-90e5-ee9350c33f6d nodeName:}" failed. No retries permitted until 2026-07-25 00:17:48.738802812 +0000 UTC m=+2909.125233527 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/185a8961-c049-4476-90e5-ee9350c33f6d-maas-api-tls") pod "maas-api-e2e-ipp-a-be7457-5bc5998667-n5sls" (UID: "185a8961-c049-4476-90e5-ee9350c33f6d") : secret "maas-api-e2e-ipp-a-be7457-serving-cert" not found
Jul 25 00:17:48.239373 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:17:48.239353    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/185a8961-c049-4476-90e5-ee9350c33f6d-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-be7457-5bc5998667-n5sls\" (UID: \"185a8961-c049-4476-90e5-ee9350c33f6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-be7457-5bc5998667-n5sls"
Jul 25 00:17:48.251175 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:17:48.251148    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kcntp\" (UniqueName: \"kubernetes.io/projected/185a8961-c049-4476-90e5-ee9350c33f6d-kube-api-access-kcntp\") pod \"maas-api-e2e-ipp-a-be7457-5bc5998667-n5sls\" (UID: \"185a8961-c049-4476-90e5-ee9350c33f6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-be7457-5bc5998667-n5sls"
Jul 25 00:17:48.606503 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:17:48.606472    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:17:48.700071 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:17:48.700034    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:17:48.704583 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:17:48.704555    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-be7457-647cc885bd-nlcdw"]
Jul 25 00:17:48.707778 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:17:48.707760    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-be7457-647cc885bd-nlcdw"
Jul 25 00:17:48.710207 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:17:48.710185    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-be7457\""
Jul 25 00:17:48.710420 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:17:48.710378    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-be7457-dockercfg-vs7wn\""
Jul 25 00:17:48.716346 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:17:48.716326    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-be7457-647cc885bd-nlcdw"]
Jul 25 00:17:48.742980 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:17:48.742954    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/185a8961-c049-4476-90e5-ee9350c33f6d-maas-api-tls\") pod \"maas-api-e2e-ipp-a-be7457-5bc5998667-n5sls\" (UID: \"185a8961-c049-4476-90e5-ee9350c33f6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-be7457-5bc5998667-n5sls"
Jul 25 00:17:48.745502 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:17:48.745478    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/185a8961-c049-4476-90e5-ee9350c33f6d-maas-api-tls\") pod \"maas-api-e2e-ipp-a-be7457-5bc5998667-n5sls\" (UID: \"185a8961-c049-4476-90e5-ee9350c33f6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-be7457-5bc5998667-n5sls"
Jul 25 00:17:48.843831 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:17:48.843785    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a32a3941-53c4-4f1b-886e-3440592322e8-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-be7457-647cc885bd-nlcdw\" (UID: \"a32a3941-53c4-4f1b-886e-3440592322e8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-be7457-647cc885bd-nlcdw"
Jul 25 00:17:48.844040 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:17:48.843938    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzhg5\" (UniqueName: \"kubernetes.io/projected/a32a3941-53c4-4f1b-886e-3440592322e8-kube-api-access-kzhg5\") pod \"payload-pre-processing-e2e-ipp-a-be7457-647cc885bd-nlcdw\" (UID: \"a32a3941-53c4-4f1b-886e-3440592322e8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-be7457-647cc885bd-nlcdw"
Jul 25 00:17:48.915760 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:17:48.915673    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-be7457-5bc5998667-n5sls"
Jul 25 00:17:48.944834 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:17:48.944799    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzhg5\" (UniqueName: \"kubernetes.io/projected/a32a3941-53c4-4f1b-886e-3440592322e8-kube-api-access-kzhg5\") pod \"payload-pre-processing-e2e-ipp-a-be7457-647cc885bd-nlcdw\" (UID: \"a32a3941-53c4-4f1b-886e-3440592322e8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-be7457-647cc885bd-nlcdw"
Jul 25 00:17:48.945006 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:17:48.944884    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a32a3941-53c4-4f1b-886e-3440592322e8-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-be7457-647cc885bd-nlcdw\" (UID: \"a32a3941-53c4-4f1b-886e-3440592322e8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-be7457-647cc885bd-nlcdw"
Jul 25 00:17:48.945617 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:17:48.945520    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a32a3941-53c4-4f1b-886e-3440592322e8-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-be7457-647cc885bd-nlcdw\" (UID: \"a32a3941-53c4-4f1b-886e-3440592322e8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-be7457-647cc885bd-nlcdw"
Jul 25 00:17:48.953201 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:17:48.953170    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzhg5\" (UniqueName: \"kubernetes.io/projected/a32a3941-53c4-4f1b-886e-3440592322e8-kube-api-access-kzhg5\") pod \"payload-pre-processing-e2e-ipp-a-be7457-647cc885bd-nlcdw\" (UID: \"a32a3941-53c4-4f1b-886e-3440592322e8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-be7457-647cc885bd-nlcdw"
Jul 25 00:17:49.020876 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:17:49.020843    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-be7457-647cc885bd-nlcdw"
Jul 25 00:17:49.059315 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:17:49.059286    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-be7457-5bc5998667-n5sls"]
Jul 25 00:17:49.060608 ip-10-0-142-133 kubenswrapper[2558]: W0725 00:17:49.060583    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod185a8961_c049_4476_90e5_ee9350c33f6d.slice/crio-894704fef2e55db868341c347c7d058bf50ad409f764a345450bb9e03339b6dc WatchSource:0}: Error finding container 894704fef2e55db868341c347c7d058bf50ad409f764a345450bb9e03339b6dc: Status 404 returned error can't find the container with id 894704fef2e55db868341c347c7d058bf50ad409f764a345450bb9e03339b6dc
Jul 25 00:17:49.062204 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:17:49.062183    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 25 00:17:49.161126 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:17:49.161102    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-be7457-647cc885bd-nlcdw"]
Jul 25 00:17:49.162946 ip-10-0-142-133 kubenswrapper[2558]: W0725 00:17:49.162920    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda32a3941_53c4_4f1b_886e_3440592322e8.slice/crio-8f7db9720ffe52482af918680f76496b31c419dc83152cf47591e71b6d852f3e WatchSource:0}: Error finding container 8f7db9720ffe52482af918680f76496b31c419dc83152cf47591e71b6d852f3e: Status 404 returned error can't find the container with id 8f7db9720ffe52482af918680f76496b31c419dc83152cf47591e71b6d852f3e
Jul 25 00:17:49.362401 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:17:49.362366    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-be7457-5bc5998667-n5sls" event={"ID":"185a8961-c049-4476-90e5-ee9350c33f6d","Type":"ContainerStarted","Data":"894704fef2e55db868341c347c7d058bf50ad409f764a345450bb9e03339b6dc"}
Jul 25 00:17:49.364136 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:17:49.364109    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-be7457-647cc885bd-nlcdw" event={"ID":"a32a3941-53c4-4f1b-886e-3440592322e8","Type":"ContainerStarted","Data":"30a3ff3ad716b4da04f4ddb3d00ff56d3b0e2bdf3010e52d99797d4a230e96c1"}
Jul 25 00:17:49.364300 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:17:49.364151    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-be7457-647cc885bd-nlcdw" event={"ID":"a32a3941-53c4-4f1b-886e-3440592322e8","Type":"ContainerStarted","Data":"8f7db9720ffe52482af918680f76496b31c419dc83152cf47591e71b6d852f3e"}
Jul 25 00:17:49.364300 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:17:49.364196    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-be7457-647cc885bd-nlcdw"
Jul 25 00:17:49.380393 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:17:49.380340    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-be7457-647cc885bd-nlcdw" podStartSLOduration=1.380325278 podStartE2EDuration="1.380325278s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:17:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:17:49.379609361 +0000 UTC m=+2909.766040090" watchObservedRunningTime="2026-07-25 00:17:49.380325278 +0000 UTC m=+2909.766756006"
Jul 25 00:17:50.371163 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:17:50.371128    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-be7457-5bc5998667-n5sls" event={"ID":"185a8961-c049-4476-90e5-ee9350c33f6d","Type":"ContainerStarted","Data":"8741d67bb2ee35fa8786c6ca75b48967c688752f1b2a2dc9d737b6d7a7ff7e60"}
Jul 25 00:17:50.371599 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:17:50.371285    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-be7457-5bc5998667-n5sls"
Jul 25 00:17:50.391289 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:17:50.391229    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-be7457-5bc5998667-n5sls" podStartSLOduration=2.84261755 podStartE2EDuration="3.391214904s" totalImagesPullingTime="548.597354ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:17:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:17:50.389950046 +0000 UTC m=+2910.776380775" watchObservedRunningTime="2026-07-25 00:17:50.391214904 +0000 UTC m=+2910.777645631"
Jul 25 00:17:51.464378 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:17:51.464339    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:17:56.382626 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:17:56.382599    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-be7457-5bc5998667-n5sls"
Jul 25 00:18:00.373589 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:18:00.373559    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-be7457-647cc885bd-nlcdw"
Jul 25 00:18:02.404064 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:18:02.404019    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:18:02.737495 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:18:02.737458    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:18:02.865300 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:18:02.865237    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:18:06.056950 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:18:06.056894    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:18:06.152880 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:18:06.152842    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:18:07.884129 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:18:07.884093    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-899559-fcf548c5c-znksx"]
Jul 25 00:18:07.888837 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:18:07.888811    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-899559-fcf548c5c-znksx"
Jul 25 00:18:07.891137 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:18:07.891112    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-899559-serving-cert\""
Jul 25 00:18:07.897486 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:18:07.897374    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-899559-fcf548c5c-znksx"]
Jul 25 00:18:07.918779 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:18:07.918747    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:18:07.919850 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:18:07.919812    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/59409783-5c60-4731-9e8c-78de0f0639d3-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-899559-fcf548c5c-znksx\" (UID: \"59409783-5c60-4731-9e8c-78de0f0639d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-899559-fcf548c5c-znksx"
Jul 25 00:18:07.919953 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:18:07.919854    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59409783-5c60-4731-9e8c-78de0f0639d3-maas-api-tls\") pod \"maas-api-e2e-ipp-b-899559-fcf548c5c-znksx\" (UID: \"59409783-5c60-4731-9e8c-78de0f0639d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-899559-fcf548c5c-znksx"
Jul 25 00:18:07.919953 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:18:07.919901    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxctx\" (UniqueName: \"kubernetes.io/projected/59409783-5c60-4731-9e8c-78de0f0639d3-kube-api-access-vxctx\") pod \"maas-api-e2e-ipp-b-899559-fcf548c5c-znksx\" (UID: \"59409783-5c60-4731-9e8c-78de0f0639d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-899559-fcf548c5c-znksx"
Jul 25 00:18:08.020781 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:18:08.020745    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/59409783-5c60-4731-9e8c-78de0f0639d3-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-899559-fcf548c5c-znksx\" (UID: \"59409783-5c60-4731-9e8c-78de0f0639d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-899559-fcf548c5c-znksx"
Jul 25 00:18:08.021046 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:18:08.020799    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59409783-5c60-4731-9e8c-78de0f0639d3-maas-api-tls\") pod \"maas-api-e2e-ipp-b-899559-fcf548c5c-znksx\" (UID: \"59409783-5c60-4731-9e8c-78de0f0639d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-899559-fcf548c5c-znksx"
Jul 25 00:18:08.021046 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:18:08.020848    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxctx\" (UniqueName: \"kubernetes.io/projected/59409783-5c60-4731-9e8c-78de0f0639d3-kube-api-access-vxctx\") pod \"maas-api-e2e-ipp-b-899559-fcf548c5c-znksx\" (UID: \"59409783-5c60-4731-9e8c-78de0f0639d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-899559-fcf548c5c-znksx"
Jul 25 00:18:08.021046 ip-10-0-142-133 kubenswrapper[2558]: E0725 00:18:08.020978    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-899559-serving-cert: secret "maas-api-e2e-ipp-b-899559-serving-cert" not found
Jul 25 00:18:08.021243 ip-10-0-142-133 kubenswrapper[2558]: E0725 00:18:08.021071    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59409783-5c60-4731-9e8c-78de0f0639d3-maas-api-tls podName:59409783-5c60-4731-9e8c-78de0f0639d3 nodeName:}" failed. No retries permitted until 2026-07-25 00:18:08.521050475 +0000 UTC m=+2928.907481180 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/59409783-5c60-4731-9e8c-78de0f0639d3-maas-api-tls") pod "maas-api-e2e-ipp-b-899559-fcf548c5c-znksx" (UID: "59409783-5c60-4731-9e8c-78de0f0639d3") : secret "maas-api-e2e-ipp-b-899559-serving-cert" not found
Jul 25 00:18:08.021490 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:18:08.021472    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/59409783-5c60-4731-9e8c-78de0f0639d3-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-899559-fcf548c5c-znksx\" (UID: \"59409783-5c60-4731-9e8c-78de0f0639d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-899559-fcf548c5c-znksx"
Jul 25 00:18:08.029732 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:18:08.029702    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxctx\" (UniqueName: \"kubernetes.io/projected/59409783-5c60-4731-9e8c-78de0f0639d3-kube-api-access-vxctx\") pod \"maas-api-e2e-ipp-b-899559-fcf548c5c-znksx\" (UID: \"59409783-5c60-4731-9e8c-78de0f0639d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-899559-fcf548c5c-znksx"
Jul 25 00:18:08.183306 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:18:08.183206    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:18:08.390579 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:18:08.390520    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:18:08.492568 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:18:08.489810    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:18:08.524429 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:18:08.524390    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59409783-5c60-4731-9e8c-78de0f0639d3-maas-api-tls\") pod \"maas-api-e2e-ipp-b-899559-fcf548c5c-znksx\" (UID: \"59409783-5c60-4731-9e8c-78de0f0639d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-899559-fcf548c5c-znksx"
Jul 25 00:18:08.527337 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:18:08.527311    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59409783-5c60-4731-9e8c-78de0f0639d3-maas-api-tls\") pod \"maas-api-e2e-ipp-b-899559-fcf548c5c-znksx\" (UID: \"59409783-5c60-4731-9e8c-78de0f0639d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-899559-fcf548c5c-znksx"
Jul 25 00:18:08.539168 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:18:08.539126    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-899559-69bb66b758-v8h2h"]
Jul 25 00:18:08.543484 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:18:08.543462    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-899559-69bb66b758-v8h2h"
Jul 25 00:18:08.545764 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:18:08.545725    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-899559\""
Jul 25 00:18:08.545980 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:18:08.545958    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-899559-dockercfg-bkk58\""
Jul 25 00:18:08.552412 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:18:08.552381    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-899559-69bb66b758-v8h2h"]
Jul 25 00:18:08.625890 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:18:08.625852    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhsgf\" (UniqueName: \"kubernetes.io/projected/fe0db738-ccda-4179-a1ee-2a305492fac2-kube-api-access-bhsgf\") pod \"payload-processing-e2e-ipp-b-899559-69bb66b758-v8h2h\" (UID: \"fe0db738-ccda-4179-a1ee-2a305492fac2\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-899559-69bb66b758-v8h2h"
Jul 25 00:18:08.626076 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:18:08.625942    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe0db738-ccda-4179-a1ee-2a305492fac2-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-899559-69bb66b758-v8h2h\" (UID: \"fe0db738-ccda-4179-a1ee-2a305492fac2\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-899559-69bb66b758-v8h2h"
Jul 25 00:18:08.726690 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:18:08.726657    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhsgf\" (UniqueName: \"kubernetes.io/projected/fe0db738-ccda-4179-a1ee-2a305492fac2-kube-api-access-bhsgf\") pod \"payload-processing-e2e-ipp-b-899559-69bb66b758-v8h2h\" (UID: \"fe0db738-ccda-4179-a1ee-2a305492fac2\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-899559-69bb66b758-v8h2h"
Jul 25 00:18:08.726690 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:18:08.726703    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe0db738-ccda-4179-a1ee-2a305492fac2-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-899559-69bb66b758-v8h2h\" (UID: \"fe0db738-ccda-4179-a1ee-2a305492fac2\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-899559-69bb66b758-v8h2h"
Jul 25 00:18:08.727419 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:18:08.727391    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe0db738-ccda-4179-a1ee-2a305492fac2-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-899559-69bb66b758-v8h2h\" (UID: \"fe0db738-ccda-4179-a1ee-2a305492fac2\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-899559-69bb66b758-v8h2h"
Jul 25 00:18:08.733690 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:18:08.733661    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhsgf\" (UniqueName: \"kubernetes.io/projected/fe0db738-ccda-4179-a1ee-2a305492fac2-kube-api-access-bhsgf\") pod \"payload-processing-e2e-ipp-b-899559-69bb66b758-v8h2h\" (UID: \"fe0db738-ccda-4179-a1ee-2a305492fac2\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-899559-69bb66b758-v8h2h"
Jul 25 00:18:08.802400 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:18:08.802304    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-899559-fcf548c5c-znksx"
Jul 25 00:18:08.859406 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:18:08.859379    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-899559-69bb66b758-v8h2h"
Jul 25 00:18:08.945268 ip-10-0-142-133 kubenswrapper[2558]: W0725 00:18:08.945236    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59409783_5c60_4731_9e8c_78de0f0639d3.slice/crio-7ddea76b196e0e83704c9a38645900af6d2458e6ae11be567f72aec42bb87046 WatchSource:0}: Error finding container 7ddea76b196e0e83704c9a38645900af6d2458e6ae11be567f72aec42bb87046: Status 404 returned error can't find the container with id 7ddea76b196e0e83704c9a38645900af6d2458e6ae11be567f72aec42bb87046
Jul 25 00:18:08.945606 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:18:08.945317    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-899559-fcf548c5c-znksx"]
Jul 25 00:18:09.007315 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:18:09.007276    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-899559-69bb66b758-v8h2h"]
Jul 25 00:18:09.009441 ip-10-0-142-133 kubenswrapper[2558]: W0725 00:18:09.009414    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe0db738_ccda_4179_a1ee_2a305492fac2.slice/crio-d1f16f45b5476d234f92007f9c84c7289b6a44eab00c4a4c99160d4940480903 WatchSource:0}: Error finding container d1f16f45b5476d234f92007f9c84c7289b6a44eab00c4a4c99160d4940480903: Status 404 returned error can't find the container with id d1f16f45b5476d234f92007f9c84c7289b6a44eab00c4a4c99160d4940480903
Jul 25 00:18:09.475418 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:18:09.475381    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-899559-69bb66b758-v8h2h" event={"ID":"fe0db738-ccda-4179-a1ee-2a305492fac2","Type":"ContainerStarted","Data":"28f79a1a4811d96bbfb12b4b2adbc9b2d7ea3380c074aaef13117ea65964f807"}
Jul 25 00:18:09.475639 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:18:09.475453    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-899559-69bb66b758-v8h2h"
Jul 25 00:18:09.475639 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:18:09.475479    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-899559-69bb66b758-v8h2h" event={"ID":"fe0db738-ccda-4179-a1ee-2a305492fac2","Type":"ContainerStarted","Data":"d1f16f45b5476d234f92007f9c84c7289b6a44eab00c4a4c99160d4940480903"}
Jul 25 00:18:09.477238 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:18:09.477205    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-899559-fcf548c5c-znksx" event={"ID":"59409783-5c60-4731-9e8c-78de0f0639d3","Type":"ContainerStarted","Data":"70b3a4b5b2feb7f48ceef1442295abc29980eb1e941500108c49ea495caf917a"}
Jul 25 00:18:09.477435 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:18:09.477254    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-899559-fcf548c5c-znksx" event={"ID":"59409783-5c60-4731-9e8c-78de0f0639d3","Type":"ContainerStarted","Data":"7ddea76b196e0e83704c9a38645900af6d2458e6ae11be567f72aec42bb87046"}
Jul 25 00:18:09.477435 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:18:09.477290    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-899559-fcf548c5c-znksx"
Jul 25 00:18:09.490441 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:18:09.490383    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-899559-69bb66b758-v8h2h" podStartSLOduration=1.490365891 podStartE2EDuration="1.490365891s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:18:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:18:09.489501163 +0000 UTC m=+2929.875931887" watchObservedRunningTime="2026-07-25 00:18:09.490365891 +0000 UTC m=+2929.876796619"
Jul 25 00:18:09.505038 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:18:09.504992    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-899559-fcf548c5c-znksx" podStartSLOduration=2.189725054 podStartE2EDuration="2.504976072s" totalImagesPullingTime="315.251018ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:18:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:18:09.50420503 +0000 UTC m=+2929.890635758" watchObservedRunningTime="2026-07-25 00:18:09.504976072 +0000 UTC m=+2929.891406798"
Jul 25 00:18:10.562904 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:18:10.562871    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:18:15.489225 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:18:15.489196    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-899559-fcf548c5c-znksx"
Jul 25 00:18:20.485028 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:18:20.484996    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-899559-69bb66b758-v8h2h"
Jul 25 00:18:27.852841 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:18:27.852806    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-8ddmv_180a4542-a43a-4d03-bd07-9380a02de282/payload-processing/0.log"
Jul 25 00:18:27.968292 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:18:27.968263    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-899559-69bb66b758-v8h2h_fe0db738-ccda-4179-a1ee-2a305492fac2/payload-processing/0.log"
Jul 25 00:18:33.513464 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:18:33.513425    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:19:15.061351 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:15.061272    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-8ddmv_180a4542-a43a-4d03-bd07-9380a02de282/payload-processing/0.log"
Jul 25 00:19:15.179275 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:15.179250    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-899559-69bb66b758-v8h2h_fe0db738-ccda-4179-a1ee-2a305492fac2/payload-processing/0.log"
Jul 25 00:19:15.441701 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:15.441618    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:19:15.756605 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:15.756571    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:19:16.007804 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:16.007719    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:19:16.150062 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:16.150030    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:19:18.538386 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:18.538342    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:19:18.643003 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:18.642958    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:19:21.003821 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:21.003785    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-85930d-7b577d6f5-bk9qt"]
Jul 25 00:19:21.008191 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:21.008168    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-85930d-7b577d6f5-bk9qt"
Jul 25 00:19:21.010470 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:21.010436    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-85930d-serving-cert\""
Jul 25 00:19:21.017516 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:21.017491    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-85930d-7b577d6f5-bk9qt"]
Jul 25 00:19:21.036033 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:21.035988    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:19:21.101822 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:21.101732    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffcf9e76-8b9d-4339-940e-5e8696724e23-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-85930d-7b577d6f5-bk9qt\" (UID: \"ffcf9e76-8b9d-4339-940e-5e8696724e23\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-85930d-7b577d6f5-bk9qt"
Jul 25 00:19:21.102008 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:21.101935    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ffcf9e76-8b9d-4339-940e-5e8696724e23-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-85930d-7b577d6f5-bk9qt\" (UID: \"ffcf9e76-8b9d-4339-940e-5e8696724e23\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-85930d-7b577d6f5-bk9qt"
Jul 25 00:19:21.102008 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:21.101981    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwwbz\" (UniqueName: \"kubernetes.io/projected/ffcf9e76-8b9d-4339-940e-5e8696724e23-kube-api-access-fwwbz\") pod \"maas-api-e2e-ipp-cleanup-85930d-7b577d6f5-bk9qt\" (UID: \"ffcf9e76-8b9d-4339-940e-5e8696724e23\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-85930d-7b577d6f5-bk9qt"
Jul 25 00:19:21.145931 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:21.145898    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:19:21.202955 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:21.202925    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffcf9e76-8b9d-4339-940e-5e8696724e23-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-85930d-7b577d6f5-bk9qt\" (UID: \"ffcf9e76-8b9d-4339-940e-5e8696724e23\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-85930d-7b577d6f5-bk9qt"
Jul 25 00:19:21.203138 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:21.203013    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ffcf9e76-8b9d-4339-940e-5e8696724e23-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-85930d-7b577d6f5-bk9qt\" (UID: \"ffcf9e76-8b9d-4339-940e-5e8696724e23\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-85930d-7b577d6f5-bk9qt"
Jul 25 00:19:21.203138 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:21.203039    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwwbz\" (UniqueName: \"kubernetes.io/projected/ffcf9e76-8b9d-4339-940e-5e8696724e23-kube-api-access-fwwbz\") pod \"maas-api-e2e-ipp-cleanup-85930d-7b577d6f5-bk9qt\" (UID: \"ffcf9e76-8b9d-4339-940e-5e8696724e23\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-85930d-7b577d6f5-bk9qt"
Jul 25 00:19:21.203572 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:21.203552    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffcf9e76-8b9d-4339-940e-5e8696724e23-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-85930d-7b577d6f5-bk9qt\" (UID: \"ffcf9e76-8b9d-4339-940e-5e8696724e23\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-85930d-7b577d6f5-bk9qt"
Jul 25 00:19:21.205323 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:21.205304    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ffcf9e76-8b9d-4339-940e-5e8696724e23-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-85930d-7b577d6f5-bk9qt\" (UID: \"ffcf9e76-8b9d-4339-940e-5e8696724e23\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-85930d-7b577d6f5-bk9qt"
Jul 25 00:19:21.210491 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:21.210465    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwwbz\" (UniqueName: \"kubernetes.io/projected/ffcf9e76-8b9d-4339-940e-5e8696724e23-kube-api-access-fwwbz\") pod \"maas-api-e2e-ipp-cleanup-85930d-7b577d6f5-bk9qt\" (UID: \"ffcf9e76-8b9d-4339-940e-5e8696724e23\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-85930d-7b577d6f5-bk9qt"
Jul 25 00:19:21.321749 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:21.321658    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-85930d-7b577d6f5-bk9qt"
Jul 25 00:19:21.467924 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:21.467894    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-85930d-7b577d6f5-bk9qt"]
Jul 25 00:19:21.470779 ip-10-0-142-133 kubenswrapper[2558]: W0725 00:19:21.470748    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffcf9e76_8b9d_4339_940e_5e8696724e23.slice/crio-b1982a5908ff2e1ba5bbb9e35a722d3a31fe0940264fe8aabb6b58363f7f64da WatchSource:0}: Error finding container b1982a5908ff2e1ba5bbb9e35a722d3a31fe0940264fe8aabb6b58363f7f64da: Status 404 returned error can't find the container with id b1982a5908ff2e1ba5bbb9e35a722d3a31fe0940264fe8aabb6b58363f7f64da
Jul 25 00:19:21.485095 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:21.485066    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:19:21.587367 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:21.587282    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-85930d-5fb4d7b596-6wxv9"]
Jul 25 00:19:21.588692 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:21.588662    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:19:21.592376 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:21.592356    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-85930d-5fb4d7b596-6wxv9"
Jul 25 00:19:21.594637 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:21.594614    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-85930d\""
Jul 25 00:19:21.594847 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:21.594831    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-85930d-dockercfg-5vc66\""
Jul 25 00:19:21.598642 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:21.598618    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-85930d-5fb4d7b596-6wxv9"]
Jul 25 00:19:21.626171 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:21.626137    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-85930d-dfdb86f4b-fcmsn"]
Jul 25 00:19:21.629935 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:21.629916    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-85930d-dfdb86f4b-fcmsn"
Jul 25 00:19:21.645525 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:21.645484    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-85930d-dfdb86f4b-fcmsn"]
Jul 25 00:19:21.707606 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:21.707569    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5a41adfe-3254-4af1-a217-bc323d2ffc07-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-85930d-5fb4d7b596-6wxv9\" (UID: \"5a41adfe-3254-4af1-a217-bc323d2ffc07\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-85930d-5fb4d7b596-6wxv9"
Jul 25 00:19:21.707606 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:21.707611    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rt692\" (UniqueName: \"kubernetes.io/projected/5a41adfe-3254-4af1-a217-bc323d2ffc07-kube-api-access-rt692\") pod \"payload-pre-processing-e2e-ipp-cleanup-85930d-5fb4d7b596-6wxv9\" (UID: \"5a41adfe-3254-4af1-a217-bc323d2ffc07\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-85930d-5fb4d7b596-6wxv9"
Jul 25 00:19:21.707810 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:21.707649    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3795189b-994c-4522-9607-0276cc4ada00-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-85930d-dfdb86f4b-fcmsn\" (UID: \"3795189b-994c-4522-9607-0276cc4ada00\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-85930d-dfdb86f4b-fcmsn"
Jul 25 00:19:21.707846 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:21.707791    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8m72h\" (UniqueName: \"kubernetes.io/projected/3795189b-994c-4522-9607-0276cc4ada00-kube-api-access-8m72h\") pod \"payload-processing-e2e-ipp-cleanup-85930d-dfdb86f4b-fcmsn\" (UID: \"3795189b-994c-4522-9607-0276cc4ada00\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-85930d-dfdb86f4b-fcmsn"
Jul 25 00:19:21.808520 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:21.808485    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5a41adfe-3254-4af1-a217-bc323d2ffc07-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-85930d-5fb4d7b596-6wxv9\" (UID: \"5a41adfe-3254-4af1-a217-bc323d2ffc07\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-85930d-5fb4d7b596-6wxv9"
Jul 25 00:19:21.808520 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:21.808520    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rt692\" (UniqueName: \"kubernetes.io/projected/5a41adfe-3254-4af1-a217-bc323d2ffc07-kube-api-access-rt692\") pod \"payload-pre-processing-e2e-ipp-cleanup-85930d-5fb4d7b596-6wxv9\" (UID: \"5a41adfe-3254-4af1-a217-bc323d2ffc07\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-85930d-5fb4d7b596-6wxv9"
Jul 25 00:19:21.808809 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:21.808571    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3795189b-994c-4522-9607-0276cc4ada00-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-85930d-dfdb86f4b-fcmsn\" (UID: \"3795189b-994c-4522-9607-0276cc4ada00\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-85930d-dfdb86f4b-fcmsn"
Jul 25 00:19:21.808809 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:21.808630    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8m72h\" (UniqueName: \"kubernetes.io/projected/3795189b-994c-4522-9607-0276cc4ada00-kube-api-access-8m72h\") pod \"payload-processing-e2e-ipp-cleanup-85930d-dfdb86f4b-fcmsn\" (UID: \"3795189b-994c-4522-9607-0276cc4ada00\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-85930d-dfdb86f4b-fcmsn"
Jul 25 00:19:21.809248 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:21.809224    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5a41adfe-3254-4af1-a217-bc323d2ffc07-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-85930d-5fb4d7b596-6wxv9\" (UID: \"5a41adfe-3254-4af1-a217-bc323d2ffc07\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-85930d-5fb4d7b596-6wxv9"
Jul 25 00:19:21.809302 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:21.809251    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3795189b-994c-4522-9607-0276cc4ada00-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-85930d-dfdb86f4b-fcmsn\" (UID: \"3795189b-994c-4522-9607-0276cc4ada00\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-85930d-dfdb86f4b-fcmsn"
Jul 25 00:19:21.816575 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:21.816546    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8m72h\" (UniqueName: \"kubernetes.io/projected/3795189b-994c-4522-9607-0276cc4ada00-kube-api-access-8m72h\") pod \"payload-processing-e2e-ipp-cleanup-85930d-dfdb86f4b-fcmsn\" (UID: \"3795189b-994c-4522-9607-0276cc4ada00\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-85930d-dfdb86f4b-fcmsn"
Jul 25 00:19:21.817196 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:21.817175    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rt692\" (UniqueName: \"kubernetes.io/projected/5a41adfe-3254-4af1-a217-bc323d2ffc07-kube-api-access-rt692\") pod \"payload-pre-processing-e2e-ipp-cleanup-85930d-5fb4d7b596-6wxv9\" (UID: \"5a41adfe-3254-4af1-a217-bc323d2ffc07\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-85930d-5fb4d7b596-6wxv9"
Jul 25 00:19:21.880901 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:21.880871    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-85930d-7b577d6f5-bk9qt" event={"ID":"ffcf9e76-8b9d-4339-940e-5e8696724e23","Type":"ContainerStarted","Data":"b1982a5908ff2e1ba5bbb9e35a722d3a31fe0940264fe8aabb6b58363f7f64da"}
Jul 25 00:19:21.905918 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:21.905895    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-85930d-5fb4d7b596-6wxv9"
Jul 25 00:19:21.950846 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:21.950795    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-85930d-dfdb86f4b-fcmsn"
Jul 25 00:19:22.055325 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:22.055291    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-85930d-5fb4d7b596-6wxv9"]
Jul 25 00:19:22.057302 ip-10-0-142-133 kubenswrapper[2558]: W0725 00:19:22.057232    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a41adfe_3254_4af1_a217_bc323d2ffc07.slice/crio-9707add5abc847d4969257ee92a3bd6c245e5833a27c52285638b05553aad35f WatchSource:0}: Error finding container 9707add5abc847d4969257ee92a3bd6c245e5833a27c52285638b05553aad35f: Status 404 returned error can't find the container with id 9707add5abc847d4969257ee92a3bd6c245e5833a27c52285638b05553aad35f
Jul 25 00:19:22.113860 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:22.113820    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-85930d-dfdb86f4b-fcmsn"]
Jul 25 00:19:22.115849 ip-10-0-142-133 kubenswrapper[2558]: W0725 00:19:22.115820    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3795189b_994c_4522_9607_0276cc4ada00.slice/crio-483651bf502248996133db468fbb2e7a9e866815a65ff686fce390955663213a WatchSource:0}: Error finding container 483651bf502248996133db468fbb2e7a9e866815a65ff686fce390955663213a: Status 404 returned error can't find the container with id 483651bf502248996133db468fbb2e7a9e866815a65ff686fce390955663213a
Jul 25 00:19:22.890324 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:22.890286    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-85930d-5fb4d7b596-6wxv9" event={"ID":"5a41adfe-3254-4af1-a217-bc323d2ffc07","Type":"ContainerStarted","Data":"593602e4017ebda22cc89809b89822243a14c84e85ca63484ba2c5547949f1bf"}
Jul 25 00:19:22.890515 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:22.890343    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-85930d-5fb4d7b596-6wxv9" event={"ID":"5a41adfe-3254-4af1-a217-bc323d2ffc07","Type":"ContainerStarted","Data":"9707add5abc847d4969257ee92a3bd6c245e5833a27c52285638b05553aad35f"}
Jul 25 00:19:22.890515 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:22.890379    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-85930d-5fb4d7b596-6wxv9"
Jul 25 00:19:22.892040 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:22.892013    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-85930d-dfdb86f4b-fcmsn" event={"ID":"3795189b-994c-4522-9607-0276cc4ada00","Type":"ContainerStarted","Data":"c76e1bb04b13284f3f68a2a586b96f9ab254e005558923bd4e9d761e162954b1"}
Jul 25 00:19:22.892128 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:22.892061    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-85930d-dfdb86f4b-fcmsn" event={"ID":"3795189b-994c-4522-9607-0276cc4ada00","Type":"ContainerStarted","Data":"483651bf502248996133db468fbb2e7a9e866815a65ff686fce390955663213a"}
Jul 25 00:19:22.892128 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:22.892091    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-85930d-dfdb86f4b-fcmsn"
Jul 25 00:19:22.893405 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:22.893380    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-85930d-7b577d6f5-bk9qt" event={"ID":"ffcf9e76-8b9d-4339-940e-5e8696724e23","Type":"ContainerStarted","Data":"847cd103ebd181716e1caa30a62a97654b534dff87d01aeda4b7293f4e8b94ce"}
Jul 25 00:19:22.893514 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:22.893506    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-85930d-7b577d6f5-bk9qt"
Jul 25 00:19:22.904859 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:22.904805    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-85930d-5fb4d7b596-6wxv9" podStartSLOduration=1.904788857 podStartE2EDuration="1.904788857s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:19:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:19:22.904041466 +0000 UTC m=+3003.290472191" watchObservedRunningTime="2026-07-25 00:19:22.904788857 +0000 UTC m=+3003.291219586"
Jul 25 00:19:22.921293 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:22.921239    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-85930d-7b577d6f5-bk9qt" podStartSLOduration=2.541421056 podStartE2EDuration="2.921225744s" totalImagesPullingTime="379.804688ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:19:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:19:22.918416745 +0000 UTC m=+3003.304847475" watchObservedRunningTime="2026-07-25 00:19:22.921225744 +0000 UTC m=+3003.307656472"
Jul 25 00:19:22.936366 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:22.936325    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-85930d-dfdb86f4b-fcmsn" podStartSLOduration=1.936311769 podStartE2EDuration="1.936311769s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:19:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:19:22.934707239 +0000 UTC m=+3003.321137968" watchObservedRunningTime="2026-07-25 00:19:22.936311769 +0000 UTC m=+3003.322742496"
Jul 25 00:19:23.048438 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:23.048394    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:19:28.905402 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:28.905372    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-85930d-7b577d6f5-bk9qt"
Jul 25 00:19:33.902123 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:33.902092    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-85930d-5fb4d7b596-6wxv9"
Jul 25 00:19:33.902563 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:33.902174    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-85930d-dfdb86f4b-fcmsn"
Jul 25 00:19:49.417608 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:49.417574    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-85930d-7b577d6f5-bk9qt"]
Jul 25 00:19:49.418066 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:49.417850    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-85930d-7b577d6f5-bk9qt" podUID="ffcf9e76-8b9d-4339-940e-5e8696724e23" containerName="maas-api" containerID="cri-o://847cd103ebd181716e1caa30a62a97654b534dff87d01aeda4b7293f4e8b94ce" gracePeriod=30
Jul 25 00:19:49.439160 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:49.439126    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:19:49.474405 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:49.474368    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-85930d-dfdb86f4b-fcmsn"]
Jul 25 00:19:49.474715 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:49.474659    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-85930d-dfdb86f4b-fcmsn" podUID="3795189b-994c-4522-9607-0276cc4ada00" containerName="payload-processing" containerID="cri-o://c76e1bb04b13284f3f68a2a586b96f9ab254e005558923bd4e9d761e162954b1" gracePeriod=30
Jul 25 00:19:49.480986 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:49.480866    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-85930d-5fb4d7b596-6wxv9"]
Jul 25 00:19:49.481127 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:49.481085    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-85930d-5fb4d7b596-6wxv9" podUID="5a41adfe-3254-4af1-a217-bc323d2ffc07" containerName="payload-pre-processing" containerID="cri-o://593602e4017ebda22cc89809b89822243a14c84e85ca63484ba2c5547949f1bf" gracePeriod=30
Jul 25 00:19:49.515793 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:49.515760    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:19:49.580587 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:49.580514    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:19:49.752669 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:49.752642    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-85930d-7b577d6f5-bk9qt"
Jul 25 00:19:49.800922 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:49.800895    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-85930d-5fb4d7b596-6wxv9"
Jul 25 00:19:49.804797 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:49.804769    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-85930d-dfdb86f4b-fcmsn"
Jul 25 00:19:49.868621 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:49.868588    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ffcf9e76-8b9d-4339-940e-5e8696724e23-maas-api-tls\") pod \"ffcf9e76-8b9d-4339-940e-5e8696724e23\" (UID: \"ffcf9e76-8b9d-4339-940e-5e8696724e23\") "
Jul 25 00:19:49.868794 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:49.868666    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fwwbz\" (UniqueName: \"kubernetes.io/projected/ffcf9e76-8b9d-4339-940e-5e8696724e23-kube-api-access-fwwbz\") pod \"ffcf9e76-8b9d-4339-940e-5e8696724e23\" (UID: \"ffcf9e76-8b9d-4339-940e-5e8696724e23\") "
Jul 25 00:19:49.868794 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:49.868687    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3795189b-994c-4522-9607-0276cc4ada00-plugins-config-volume\") pod \"3795189b-994c-4522-9607-0276cc4ada00\" (UID: \"3795189b-994c-4522-9607-0276cc4ada00\") "
Jul 25 00:19:49.868794 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:49.868720    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5a41adfe-3254-4af1-a217-bc323d2ffc07-plugins-config-volume\") pod \"5a41adfe-3254-4af1-a217-bc323d2ffc07\" (UID: \"5a41adfe-3254-4af1-a217-bc323d2ffc07\") "
Jul 25 00:19:49.868794 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:49.868762    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rt692\" (UniqueName: \"kubernetes.io/projected/5a41adfe-3254-4af1-a217-bc323d2ffc07-kube-api-access-rt692\") pod \"5a41adfe-3254-4af1-a217-bc323d2ffc07\" (UID: \"5a41adfe-3254-4af1-a217-bc323d2ffc07\") "
Jul 25 00:19:49.869016 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:49.868804    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffcf9e76-8b9d-4339-940e-5e8696724e23-openshift-service-ca\") pod \"ffcf9e76-8b9d-4339-940e-5e8696724e23\" (UID: \"ffcf9e76-8b9d-4339-940e-5e8696724e23\") "
Jul 25 00:19:49.869016 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:49.868830    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8m72h\" (UniqueName: \"kubernetes.io/projected/3795189b-994c-4522-9607-0276cc4ada00-kube-api-access-8m72h\") pod \"3795189b-994c-4522-9607-0276cc4ada00\" (UID: \"3795189b-994c-4522-9607-0276cc4ada00\") "
Jul 25 00:19:49.869132 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:49.869056    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3795189b-994c-4522-9607-0276cc4ada00-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3795189b-994c-4522-9607-0276cc4ada00" (UID: "3795189b-994c-4522-9607-0276cc4ada00"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 00:19:49.869183 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:49.869128    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a41adfe-3254-4af1-a217-bc323d2ffc07-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5a41adfe-3254-4af1-a217-bc323d2ffc07" (UID: "5a41adfe-3254-4af1-a217-bc323d2ffc07"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 00:19:49.869183 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:49.869152    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ffcf9e76-8b9d-4339-940e-5e8696724e23-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ffcf9e76-8b9d-4339-940e-5e8696724e23" (UID: "ffcf9e76-8b9d-4339-940e-5e8696724e23"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 00:19:49.869272 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:49.869178    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3795189b-994c-4522-9607-0276cc4ada00-plugins-config-volume\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 25 00:19:49.870821 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:49.870799    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ffcf9e76-8b9d-4339-940e-5e8696724e23-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ffcf9e76-8b9d-4339-940e-5e8696724e23" (UID: "ffcf9e76-8b9d-4339-940e-5e8696724e23"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 00:19:49.871139 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:49.871116    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3795189b-994c-4522-9607-0276cc4ada00-kube-api-access-8m72h" (OuterVolumeSpecName: "kube-api-access-8m72h") pod "3795189b-994c-4522-9607-0276cc4ada00" (UID: "3795189b-994c-4522-9607-0276cc4ada00"). InnerVolumeSpecName "kube-api-access-8m72h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 00:19:49.871306 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:49.871285    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ffcf9e76-8b9d-4339-940e-5e8696724e23-kube-api-access-fwwbz" (OuterVolumeSpecName: "kube-api-access-fwwbz") pod "ffcf9e76-8b9d-4339-940e-5e8696724e23" (UID: "ffcf9e76-8b9d-4339-940e-5e8696724e23"). InnerVolumeSpecName "kube-api-access-fwwbz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 00:19:49.871350 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:49.871288    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a41adfe-3254-4af1-a217-bc323d2ffc07-kube-api-access-rt692" (OuterVolumeSpecName: "kube-api-access-rt692") pod "5a41adfe-3254-4af1-a217-bc323d2ffc07" (UID: "5a41adfe-3254-4af1-a217-bc323d2ffc07"). InnerVolumeSpecName "kube-api-access-rt692". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 00:19:49.970329 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:49.970247    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ffcf9e76-8b9d-4339-940e-5e8696724e23-maas-api-tls\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 25 00:19:49.970329 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:49.970277    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fwwbz\" (UniqueName: \"kubernetes.io/projected/ffcf9e76-8b9d-4339-940e-5e8696724e23-kube-api-access-fwwbz\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 25 00:19:49.970329 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:49.970290    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5a41adfe-3254-4af1-a217-bc323d2ffc07-plugins-config-volume\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 25 00:19:49.970329 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:49.970299    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rt692\" (UniqueName: \"kubernetes.io/projected/5a41adfe-3254-4af1-a217-bc323d2ffc07-kube-api-access-rt692\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 25 00:19:49.970329 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:49.970308    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffcf9e76-8b9d-4339-940e-5e8696724e23-openshift-service-ca\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 25 00:19:49.970329 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:49.970316    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8m72h\" (UniqueName: \"kubernetes.io/projected/3795189b-994c-4522-9607-0276cc4ada00-kube-api-access-8m72h\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 25 00:19:50.054018 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:50.053981    2558 generic.go:356] "Generic (PLEG): container finished" podID="5a41adfe-3254-4af1-a217-bc323d2ffc07" containerID="593602e4017ebda22cc89809b89822243a14c84e85ca63484ba2c5547949f1bf" exitCode=0
Jul 25 00:19:50.054182 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:50.054029    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-85930d-5fb4d7b596-6wxv9"
Jul 25 00:19:50.054182 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:50.054073    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-85930d-5fb4d7b596-6wxv9" event={"ID":"5a41adfe-3254-4af1-a217-bc323d2ffc07","Type":"ContainerDied","Data":"593602e4017ebda22cc89809b89822243a14c84e85ca63484ba2c5547949f1bf"}
Jul 25 00:19:50.054182 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:50.054128    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-85930d-5fb4d7b596-6wxv9" event={"ID":"5a41adfe-3254-4af1-a217-bc323d2ffc07","Type":"ContainerDied","Data":"9707add5abc847d4969257ee92a3bd6c245e5833a27c52285638b05553aad35f"}
Jul 25 00:19:50.054182 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:50.054156    2558 scope.go:117] "RemoveContainer" containerID="593602e4017ebda22cc89809b89822243a14c84e85ca63484ba2c5547949f1bf"
Jul 25 00:19:50.055456 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:50.055435    2558 generic.go:356] "Generic (PLEG): container finished" podID="3795189b-994c-4522-9607-0276cc4ada00" containerID="c76e1bb04b13284f3f68a2a586b96f9ab254e005558923bd4e9d761e162954b1" exitCode=0
Jul 25 00:19:50.055545 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:50.055487    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-85930d-dfdb86f4b-fcmsn"
Jul 25 00:19:50.055545 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:50.055518    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-85930d-dfdb86f4b-fcmsn" event={"ID":"3795189b-994c-4522-9607-0276cc4ada00","Type":"ContainerDied","Data":"c76e1bb04b13284f3f68a2a586b96f9ab254e005558923bd4e9d761e162954b1"}
Jul 25 00:19:50.055639 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:50.055582    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-85930d-dfdb86f4b-fcmsn" event={"ID":"3795189b-994c-4522-9607-0276cc4ada00","Type":"ContainerDied","Data":"483651bf502248996133db468fbb2e7a9e866815a65ff686fce390955663213a"}
Jul 25 00:19:50.056982 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:50.056959    2558 generic.go:356] "Generic (PLEG): container finished" podID="ffcf9e76-8b9d-4339-940e-5e8696724e23" containerID="847cd103ebd181716e1caa30a62a97654b534dff87d01aeda4b7293f4e8b94ce" exitCode=0
Jul 25 00:19:50.057120 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:50.056999    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-85930d-7b577d6f5-bk9qt"
Jul 25 00:19:50.057120 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:50.057011    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-85930d-7b577d6f5-bk9qt" event={"ID":"ffcf9e76-8b9d-4339-940e-5e8696724e23","Type":"ContainerDied","Data":"847cd103ebd181716e1caa30a62a97654b534dff87d01aeda4b7293f4e8b94ce"}
Jul 25 00:19:50.057120 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:50.057049    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-85930d-7b577d6f5-bk9qt" event={"ID":"ffcf9e76-8b9d-4339-940e-5e8696724e23","Type":"ContainerDied","Data":"b1982a5908ff2e1ba5bbb9e35a722d3a31fe0940264fe8aabb6b58363f7f64da"}
Jul 25 00:19:50.066165 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:50.066143    2558 scope.go:117] "RemoveContainer" containerID="593602e4017ebda22cc89809b89822243a14c84e85ca63484ba2c5547949f1bf"
Jul 25 00:19:50.066973 ip-10-0-142-133 kubenswrapper[2558]: E0725 00:19:50.066946    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"593602e4017ebda22cc89809b89822243a14c84e85ca63484ba2c5547949f1bf\": container with ID starting with 593602e4017ebda22cc89809b89822243a14c84e85ca63484ba2c5547949f1bf not found: ID does not exist" containerID="593602e4017ebda22cc89809b89822243a14c84e85ca63484ba2c5547949f1bf"
Jul 25 00:19:50.067106 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:50.067083    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"593602e4017ebda22cc89809b89822243a14c84e85ca63484ba2c5547949f1bf"} err="failed to get container status \"593602e4017ebda22cc89809b89822243a14c84e85ca63484ba2c5547949f1bf\": rpc error: code = NotFound desc = could not find container \"593602e4017ebda22cc89809b89822243a14c84e85ca63484ba2c5547949f1bf\": container with ID starting with 593602e4017ebda22cc89809b89822243a14c84e85ca63484ba2c5547949f1bf not found: ID does not exist"
Jul 25 00:19:50.067202 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:50.067191    2558 scope.go:117] "RemoveContainer" containerID="c76e1bb04b13284f3f68a2a586b96f9ab254e005558923bd4e9d761e162954b1"
Jul 25 00:19:50.079021 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:50.078984    2558 scope.go:117] "RemoveContainer" containerID="c76e1bb04b13284f3f68a2a586b96f9ab254e005558923bd4e9d761e162954b1"
Jul 25 00:19:50.079314 ip-10-0-142-133 kubenswrapper[2558]: E0725 00:19:50.079291    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c76e1bb04b13284f3f68a2a586b96f9ab254e005558923bd4e9d761e162954b1\": container with ID starting with c76e1bb04b13284f3f68a2a586b96f9ab254e005558923bd4e9d761e162954b1 not found: ID does not exist" containerID="c76e1bb04b13284f3f68a2a586b96f9ab254e005558923bd4e9d761e162954b1"
Jul 25 00:19:50.079382 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:50.079321    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c76e1bb04b13284f3f68a2a586b96f9ab254e005558923bd4e9d761e162954b1"} err="failed to get container status \"c76e1bb04b13284f3f68a2a586b96f9ab254e005558923bd4e9d761e162954b1\": rpc error: code = NotFound desc = could not find container \"c76e1bb04b13284f3f68a2a586b96f9ab254e005558923bd4e9d761e162954b1\": container with ID starting with c76e1bb04b13284f3f68a2a586b96f9ab254e005558923bd4e9d761e162954b1 not found: ID does not exist"
Jul 25 00:19:50.079382 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:50.079336    2558 scope.go:117] "RemoveContainer" containerID="847cd103ebd181716e1caa30a62a97654b534dff87d01aeda4b7293f4e8b94ce"
Jul 25 00:19:50.086198 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:50.086175    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-85930d-5fb4d7b596-6wxv9"]
Jul 25 00:19:50.088515 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:50.088490    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-85930d-5fb4d7b596-6wxv9"]
Jul 25 00:19:50.088701 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:50.088495    2558 scope.go:117] "RemoveContainer" containerID="847cd103ebd181716e1caa30a62a97654b534dff87d01aeda4b7293f4e8b94ce"
Jul 25 00:19:50.088835 ip-10-0-142-133 kubenswrapper[2558]: E0725 00:19:50.088818    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"847cd103ebd181716e1caa30a62a97654b534dff87d01aeda4b7293f4e8b94ce\": container with ID starting with 847cd103ebd181716e1caa30a62a97654b534dff87d01aeda4b7293f4e8b94ce not found: ID does not exist" containerID="847cd103ebd181716e1caa30a62a97654b534dff87d01aeda4b7293f4e8b94ce"
Jul 25 00:19:50.088876 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:50.088842    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"847cd103ebd181716e1caa30a62a97654b534dff87d01aeda4b7293f4e8b94ce"} err="failed to get container status \"847cd103ebd181716e1caa30a62a97654b534dff87d01aeda4b7293f4e8b94ce\": rpc error: code = NotFound desc = could not find container \"847cd103ebd181716e1caa30a62a97654b534dff87d01aeda4b7293f4e8b94ce\": container with ID starting with 847cd103ebd181716e1caa30a62a97654b534dff87d01aeda4b7293f4e8b94ce not found: ID does not exist"
Jul 25 00:19:50.096665 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:50.096643    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-85930d-dfdb86f4b-fcmsn"]
Jul 25 00:19:50.100966 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:50.100939    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-85930d-dfdb86f4b-fcmsn"]
Jul 25 00:19:50.109150 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:50.109129    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-85930d-7b577d6f5-bk9qt"]
Jul 25 00:19:50.113012 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:50.112990    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-85930d-7b577d6f5-bk9qt"]
Jul 25 00:19:50.205859 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:50.205825    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3795189b-994c-4522-9607-0276cc4ada00" path="/var/lib/kubelet/pods/3795189b-994c-4522-9607-0276cc4ada00/volumes"
Jul 25 00:19:50.206195 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:50.206181    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a41adfe-3254-4af1-a217-bc323d2ffc07" path="/var/lib/kubelet/pods/5a41adfe-3254-4af1-a217-bc323d2ffc07/volumes"
Jul 25 00:19:50.206541 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:50.206517    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ffcf9e76-8b9d-4339-940e-5e8696724e23" path="/var/lib/kubelet/pods/ffcf9e76-8b9d-4339-940e-5e8696724e23/volumes"
Jul 25 00:19:54.214093 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:54.214055    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:19:54.612955 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:19:54.612908    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:20:00.865818 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:00.865756    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:20:00.966149 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:00.966112    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:20:01.124124 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:01.124024    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:20:01.344019 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:01.343971    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:20:04.844679 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:04.844645    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-be7457-bhtgl"]
Jul 25 00:20:04.845058 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:04.845038    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a41adfe-3254-4af1-a217-bc323d2ffc07" containerName="payload-pre-processing"
Jul 25 00:20:04.845058 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:04.845050    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a41adfe-3254-4af1-a217-bc323d2ffc07" containerName="payload-pre-processing"
Jul 25 00:20:04.845134 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:04.845069    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3795189b-994c-4522-9607-0276cc4ada00" containerName="payload-processing"
Jul 25 00:20:04.845134 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:04.845075    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="3795189b-994c-4522-9607-0276cc4ada00" containerName="payload-processing"
Jul 25 00:20:04.845134 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:04.845085    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffcf9e76-8b9d-4339-940e-5e8696724e23" containerName="maas-api"
Jul 25 00:20:04.845134 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:04.845091    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffcf9e76-8b9d-4339-940e-5e8696724e23" containerName="maas-api"
Jul 25 00:20:04.845253 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:04.845159    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a41adfe-3254-4af1-a217-bc323d2ffc07" containerName="payload-pre-processing"
Jul 25 00:20:04.845253 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:04.845167    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="3795189b-994c-4522-9607-0276cc4ada00" containerName="payload-processing"
Jul 25 00:20:04.845253 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:04.845172    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="ffcf9e76-8b9d-4339-940e-5e8696724e23" containerName="maas-api"
Jul 25 00:20:04.848290 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:04.848263    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-be7457-bhtgl"
Jul 25 00:20:04.850591 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:04.850569    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bxr46\""
Jul 25 00:20:04.855131 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:04.855065    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-be7457-bhtgl"]
Jul 25 00:20:04.915839 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:04.915799    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0304483-18c0-450b-98da-82201c990557-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-be7457-bhtgl\" (UID: \"f0304483-18c0-450b-98da-82201c990557\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-be7457-bhtgl"
Jul 25 00:20:05.016994 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:05.016945    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0304483-18c0-450b-98da-82201c990557-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-be7457-bhtgl\" (UID: \"f0304483-18c0-450b-98da-82201c990557\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-be7457-bhtgl"
Jul 25 00:20:05.017587 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:05.017566    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0304483-18c0-450b-98da-82201c990557-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-be7457-bhtgl\" (UID: \"f0304483-18c0-450b-98da-82201c990557\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-be7457-bhtgl"
Jul 25 00:20:05.162401 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:05.162309    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-be7457-bhtgl"
Jul 25 00:20:05.297777 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:05.297738    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-be7457-bhtgl"]
Jul 25 00:20:05.299286 ip-10-0-142-133 kubenswrapper[2558]: W0725 00:20:05.299258    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0304483_18c0_450b_98da_82201c990557.slice/crio-90ac8774d36de2d6e0c3ac60c67e2e61254e61e50913b3d3130fe8f418dd6250 WatchSource:0}: Error finding container 90ac8774d36de2d6e0c3ac60c67e2e61254e61e50913b3d3130fe8f418dd6250: Status 404 returned error can't find the container with id 90ac8774d36de2d6e0c3ac60c67e2e61254e61e50913b3d3130fe8f418dd6250
Jul 25 00:20:06.154920 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:06.154882    2558 generic.go:356] "Generic (PLEG): container finished" podID="f0304483-18c0-450b-98da-82201c990557" containerID="072fca65de1bd8fec6ec47574a19709670ee57418faa14957972966726d083a2" exitCode=0
Jul 25 00:20:06.155322 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:06.154919    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-be7457-bhtgl" event={"ID":"f0304483-18c0-450b-98da-82201c990557","Type":"ContainerDied","Data":"072fca65de1bd8fec6ec47574a19709670ee57418faa14957972966726d083a2"}
Jul 25 00:20:06.155322 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:06.154982    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-be7457-bhtgl" event={"ID":"f0304483-18c0-450b-98da-82201c990557","Type":"ContainerStarted","Data":"90ac8774d36de2d6e0c3ac60c67e2e61254e61e50913b3d3130fe8f418dd6250"}
Jul 25 00:20:07.295453 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:07.295428    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-be7457-bhtgl"
Jul 25 00:20:07.436655 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:07.436572    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0304483-18c0-450b-98da-82201c990557-maas-api-service-ca\") pod \"f0304483-18c0-450b-98da-82201c990557\" (UID: \"f0304483-18c0-450b-98da-82201c990557\") "
Jul 25 00:20:07.436925 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:07.436904    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0304483-18c0-450b-98da-82201c990557-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f0304483-18c0-450b-98da-82201c990557" (UID: "f0304483-18c0-450b-98da-82201c990557"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 00:20:07.537890 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:07.537855    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0304483-18c0-450b-98da-82201c990557-maas-api-service-ca\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 25 00:20:08.167693 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:08.167661    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-be7457-bhtgl"
Jul 25 00:20:08.167878 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:08.167696    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-be7457-bhtgl" event={"ID":"f0304483-18c0-450b-98da-82201c990557","Type":"ContainerDied","Data":"90ac8774d36de2d6e0c3ac60c67e2e61254e61e50913b3d3130fe8f418dd6250"}
Jul 25 00:20:08.167878 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:08.167745    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="90ac8774d36de2d6e0c3ac60c67e2e61254e61e50913b3d3130fe8f418dd6250"
Jul 25 00:20:13.195886 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:13.195805    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6zf8m"]
Jul 25 00:20:13.196355 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:13.196263    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0304483-18c0-450b-98da-82201c990557" containerName="revoke-keys"
Jul 25 00:20:13.196355 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:13.196275    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0304483-18c0-450b-98da-82201c990557" containerName="revoke-keys"
Jul 25 00:20:13.196355 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:13.196341    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0304483-18c0-450b-98da-82201c990557" containerName="revoke-keys"
Jul 25 00:20:13.199751 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:13.199732    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-6zf8m"
Jul 25 00:20:13.202744 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:13.202722    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 25 00:20:13.203716 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:13.203517    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 25 00:20:13.203984 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:13.203962    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-b5w5s\""
Jul 25 00:20:13.207508 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:13.207486    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6zf8m"]
Jul 25 00:20:13.290467 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:13.290436    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxpfm\" (UniqueName: \"kubernetes.io/projected/a0e5fb07-b2d1-44b5-ba9c-21ad483f6b30-kube-api-access-jxpfm\") pod \"kuadrant-operator-catalog-6zf8m\" (UID: \"a0e5fb07-b2d1-44b5-ba9c-21ad483f6b30\") " pod="kuadrant-system/kuadrant-operator-catalog-6zf8m"
Jul 25 00:20:13.391897 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:13.391861    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxpfm\" (UniqueName: \"kubernetes.io/projected/a0e5fb07-b2d1-44b5-ba9c-21ad483f6b30-kube-api-access-jxpfm\") pod \"kuadrant-operator-catalog-6zf8m\" (UID: \"a0e5fb07-b2d1-44b5-ba9c-21ad483f6b30\") " pod="kuadrant-system/kuadrant-operator-catalog-6zf8m"
Jul 25 00:20:13.402187 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:13.402158    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxpfm\" (UniqueName: \"kubernetes.io/projected/a0e5fb07-b2d1-44b5-ba9c-21ad483f6b30-kube-api-access-jxpfm\") pod \"kuadrant-operator-catalog-6zf8m\" (UID: \"a0e5fb07-b2d1-44b5-ba9c-21ad483f6b30\") " pod="kuadrant-system/kuadrant-operator-catalog-6zf8m"
Jul 25 00:20:13.512504 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:13.512470    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-6zf8m"
Jul 25 00:20:13.669510 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:13.669480    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6zf8m"]
Jul 25 00:20:13.670734 ip-10-0-142-133 kubenswrapper[2558]: W0725 00:20:13.670709    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0e5fb07_b2d1_44b5_ba9c_21ad483f6b30.slice/crio-4682b23c944e8ba6be2abaeddc87d65f4f741269296a048e1cec3686fdffd410 WatchSource:0}: Error finding container 4682b23c944e8ba6be2abaeddc87d65f4f741269296a048e1cec3686fdffd410: Status 404 returned error can't find the container with id 4682b23c944e8ba6be2abaeddc87d65f4f741269296a048e1cec3686fdffd410
Jul 25 00:20:13.780609 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:13.780509    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:20:14.204968 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:14.204885    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-6zf8m" event={"ID":"a0e5fb07-b2d1-44b5-ba9c-21ad483f6b30","Type":"ContainerStarted","Data":"4682b23c944e8ba6be2abaeddc87d65f4f741269296a048e1cec3686fdffd410"}
Jul 25 00:20:16.217031 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:16.216946    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-6zf8m" event={"ID":"a0e5fb07-b2d1-44b5-ba9c-21ad483f6b30","Type":"ContainerStarted","Data":"3445b1fadd5113eda1ef7a2dbe2a5709b42f41c0274699de28168e3b5cc0a1bd"}
Jul 25 00:20:16.231991 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:16.231941    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-6zf8m" podStartSLOduration=1.022812281 podStartE2EDuration="3.231926978s" totalImagesPullingTime="2.209114697s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:20:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:20:16.23055371 +0000 UTC m=+3056.616984431" watchObservedRunningTime="2026-07-25 00:20:16.231926978 +0000 UTC m=+3056.618357705"
Jul 25 00:20:23.513260 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:23.513223    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-6zf8m"
Jul 25 00:20:23.513828 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:23.513323    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-6zf8m"
Jul 25 00:20:23.535215 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:23.535187    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-6zf8m"
Jul 25 00:20:23.971722 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:23.971635    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-be7457-5bc5998667-n5sls"]
Jul 25 00:20:23.971994 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:23.971942    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-be7457-5bc5998667-n5sls" podUID="185a8961-c049-4476-90e5-ee9350c33f6d" containerName="maas-api" containerID="cri-o://8741d67bb2ee35fa8786c6ca75b48967c688752f1b2a2dc9d737b6d7a7ff7e60" gracePeriod=30
Jul 25 00:20:23.993746 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:23.993708    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:20:24.027906 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:24.027880    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-be7457-647cc885bd-nlcdw"]
Jul 25 00:20:24.028170 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:24.028137    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-be7457-647cc885bd-nlcdw" podUID="a32a3941-53c4-4f1b-886e-3440592322e8" containerName="payload-pre-processing" containerID="cri-o://30a3ff3ad716b4da04f4ddb3d00ff56d3b0e2bdf3010e52d99797d4a230e96c1" gracePeriod=30
Jul 25 00:20:24.093597 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:24.093562    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:20:24.194272 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:24.194175    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:20:24.262962 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:24.262925    2558 generic.go:356] "Generic (PLEG): container finished" podID="185a8961-c049-4476-90e5-ee9350c33f6d" containerID="8741d67bb2ee35fa8786c6ca75b48967c688752f1b2a2dc9d737b6d7a7ff7e60" exitCode=0
Jul 25 00:20:24.263115 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:24.263004    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-be7457-5bc5998667-n5sls" event={"ID":"185a8961-c049-4476-90e5-ee9350c33f6d","Type":"ContainerDied","Data":"8741d67bb2ee35fa8786c6ca75b48967c688752f1b2a2dc9d737b6d7a7ff7e60"}
Jul 25 00:20:24.264880 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:24.264853    2558 generic.go:356] "Generic (PLEG): container finished" podID="a32a3941-53c4-4f1b-886e-3440592322e8" containerID="30a3ff3ad716b4da04f4ddb3d00ff56d3b0e2bdf3010e52d99797d4a230e96c1" exitCode=0
Jul 25 00:20:24.265006 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:24.264935    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-be7457-647cc885bd-nlcdw" event={"ID":"a32a3941-53c4-4f1b-886e-3440592322e8","Type":"ContainerDied","Data":"30a3ff3ad716b4da04f4ddb3d00ff56d3b0e2bdf3010e52d99797d4a230e96c1"}
Jul 25 00:20:24.287783 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:24.287755    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-6zf8m"
Jul 25 00:20:24.297872 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:24.297852    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-be7457-5bc5998667-n5sls"
Jul 25 00:20:24.330126 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:24.330101    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-be7457-647cc885bd-nlcdw"
Jul 25 00:20:24.334497 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:24.334476    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6zf8m"]
Jul 25 00:20:24.392813 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:24.392785    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kzhg5\" (UniqueName: \"kubernetes.io/projected/a32a3941-53c4-4f1b-886e-3440592322e8-kube-api-access-kzhg5\") pod \"a32a3941-53c4-4f1b-886e-3440592322e8\" (UID: \"a32a3941-53c4-4f1b-886e-3440592322e8\") "
Jul 25 00:20:24.392962 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:24.392835    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/185a8961-c049-4476-90e5-ee9350c33f6d-maas-api-tls\") pod \"185a8961-c049-4476-90e5-ee9350c33f6d\" (UID: \"185a8961-c049-4476-90e5-ee9350c33f6d\") "
Jul 25 00:20:24.392962 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:24.392873    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/185a8961-c049-4476-90e5-ee9350c33f6d-openshift-service-ca\") pod \"185a8961-c049-4476-90e5-ee9350c33f6d\" (UID: \"185a8961-c049-4476-90e5-ee9350c33f6d\") "
Jul 25 00:20:24.392962 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:24.392898    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kcntp\" (UniqueName: \"kubernetes.io/projected/185a8961-c049-4476-90e5-ee9350c33f6d-kube-api-access-kcntp\") pod \"185a8961-c049-4476-90e5-ee9350c33f6d\" (UID: \"185a8961-c049-4476-90e5-ee9350c33f6d\") "
Jul 25 00:20:24.392962 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:24.392913    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a32a3941-53c4-4f1b-886e-3440592322e8-plugins-config-volume\") pod \"a32a3941-53c4-4f1b-886e-3440592322e8\" (UID: \"a32a3941-53c4-4f1b-886e-3440592322e8\") "
Jul 25 00:20:24.393300 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:24.393272    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/185a8961-c049-4476-90e5-ee9350c33f6d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "185a8961-c049-4476-90e5-ee9350c33f6d" (UID: "185a8961-c049-4476-90e5-ee9350c33f6d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 00:20:24.393407 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:24.393374    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a32a3941-53c4-4f1b-886e-3440592322e8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a32a3941-53c4-4f1b-886e-3440592322e8" (UID: "a32a3941-53c4-4f1b-886e-3440592322e8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 00:20:24.394988 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:24.394965    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/185a8961-c049-4476-90e5-ee9350c33f6d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "185a8961-c049-4476-90e5-ee9350c33f6d" (UID: "185a8961-c049-4476-90e5-ee9350c33f6d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 00:20:24.395105 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:24.395081    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/185a8961-c049-4476-90e5-ee9350c33f6d-kube-api-access-kcntp" (OuterVolumeSpecName: "kube-api-access-kcntp") pod "185a8961-c049-4476-90e5-ee9350c33f6d" (UID: "185a8961-c049-4476-90e5-ee9350c33f6d"). InnerVolumeSpecName "kube-api-access-kcntp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 00:20:24.395176 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:24.395105    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a32a3941-53c4-4f1b-886e-3440592322e8-kube-api-access-kzhg5" (OuterVolumeSpecName: "kube-api-access-kzhg5") pod "a32a3941-53c4-4f1b-886e-3440592322e8" (UID: "a32a3941-53c4-4f1b-886e-3440592322e8"). InnerVolumeSpecName "kube-api-access-kzhg5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 00:20:24.493882 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:24.493817    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kcntp\" (UniqueName: \"kubernetes.io/projected/185a8961-c049-4476-90e5-ee9350c33f6d-kube-api-access-kcntp\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 25 00:20:24.493882 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:24.493840    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a32a3941-53c4-4f1b-886e-3440592322e8-plugins-config-volume\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 25 00:20:24.493882 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:24.493850    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kzhg5\" (UniqueName: \"kubernetes.io/projected/a32a3941-53c4-4f1b-886e-3440592322e8-kube-api-access-kzhg5\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 25 00:20:24.493882 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:24.493860    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/185a8961-c049-4476-90e5-ee9350c33f6d-maas-api-tls\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 25 00:20:24.493882 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:24.493870    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/185a8961-c049-4476-90e5-ee9350c33f6d-openshift-service-ca\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 25 00:20:24.922303 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:24.922213    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-be7457-bhtgl"]
Jul 25 00:20:24.924389 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:24.924334    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:20:24.925469 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:24.925445    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-be7457-bhtgl"]
Jul 25 00:20:25.272152 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:25.272123    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-be7457-5bc5998667-n5sls"
Jul 25 00:20:25.272365 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:25.272116    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-be7457-5bc5998667-n5sls" event={"ID":"185a8961-c049-4476-90e5-ee9350c33f6d","Type":"ContainerDied","Data":"894704fef2e55db868341c347c7d058bf50ad409f764a345450bb9e03339b6dc"}
Jul 25 00:20:25.272365 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:25.272283    2558 scope.go:117] "RemoveContainer" containerID="8741d67bb2ee35fa8786c6ca75b48967c688752f1b2a2dc9d737b6d7a7ff7e60"
Jul 25 00:20:25.273770 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:25.273577    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-be7457-647cc885bd-nlcdw"
Jul 25 00:20:25.273770 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:25.273632    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-be7457-647cc885bd-nlcdw" event={"ID":"a32a3941-53c4-4f1b-886e-3440592322e8","Type":"ContainerDied","Data":"8f7db9720ffe52482af918680f76496b31c419dc83152cf47591e71b6d852f3e"}
Jul 25 00:20:25.282739 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:25.282718    2558 scope.go:117] "RemoveContainer" containerID="30a3ff3ad716b4da04f4ddb3d00ff56d3b0e2bdf3010e52d99797d4a230e96c1"
Jul 25 00:20:25.301436 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:25.301414    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-be7457-647cc885bd-nlcdw"]
Jul 25 00:20:25.304480 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:25.304457    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-be7457-647cc885bd-nlcdw"]
Jul 25 00:20:25.313766 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:25.313728    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-be7457-5bc5998667-n5sls"]
Jul 25 00:20:25.316971 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:25.316950    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-be7457-5bc5998667-n5sls"]
Jul 25 00:20:25.319392 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:25.319371    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:20:26.204901 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:26.204864    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="185a8961-c049-4476-90e5-ee9350c33f6d" path="/var/lib/kubelet/pods/185a8961-c049-4476-90e5-ee9350c33f6d/volumes"
Jul 25 00:20:26.205395 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:26.205233    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a32a3941-53c4-4f1b-886e-3440592322e8" path="/var/lib/kubelet/pods/a32a3941-53c4-4f1b-886e-3440592322e8/volumes"
Jul 25 00:20:26.205518 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:26.205506    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0304483-18c0-450b-98da-82201c990557" path="/var/lib/kubelet/pods/f0304483-18c0-450b-98da-82201c990557/volumes"
Jul 25 00:20:26.281782 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:26.281734    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-6zf8m" podUID="a0e5fb07-b2d1-44b5-ba9c-21ad483f6b30" containerName="registry-server" containerID="cri-o://3445b1fadd5113eda1ef7a2dbe2a5709b42f41c0274699de28168e3b5cc0a1bd" gracePeriod=2
Jul 25 00:20:26.539818 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:26.539795    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-6zf8m"
Jul 25 00:20:26.610318 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:26.610285    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jxpfm\" (UniqueName: \"kubernetes.io/projected/a0e5fb07-b2d1-44b5-ba9c-21ad483f6b30-kube-api-access-jxpfm\") pod \"a0e5fb07-b2d1-44b5-ba9c-21ad483f6b30\" (UID: \"a0e5fb07-b2d1-44b5-ba9c-21ad483f6b30\") "
Jul 25 00:20:26.612374 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:26.612347    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0e5fb07-b2d1-44b5-ba9c-21ad483f6b30-kube-api-access-jxpfm" (OuterVolumeSpecName: "kube-api-access-jxpfm") pod "a0e5fb07-b2d1-44b5-ba9c-21ad483f6b30" (UID: "a0e5fb07-b2d1-44b5-ba9c-21ad483f6b30"). InnerVolumeSpecName "kube-api-access-jxpfm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 00:20:26.710962 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:26.710927    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jxpfm\" (UniqueName: \"kubernetes.io/projected/a0e5fb07-b2d1-44b5-ba9c-21ad483f6b30-kube-api-access-jxpfm\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 25 00:20:27.288450 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:27.288410    2558 generic.go:356] "Generic (PLEG): container finished" podID="a0e5fb07-b2d1-44b5-ba9c-21ad483f6b30" containerID="3445b1fadd5113eda1ef7a2dbe2a5709b42f41c0274699de28168e3b5cc0a1bd" exitCode=0
Jul 25 00:20:27.288969 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:27.288460    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-6zf8m"
Jul 25 00:20:27.288969 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:27.288503    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-6zf8m" event={"ID":"a0e5fb07-b2d1-44b5-ba9c-21ad483f6b30","Type":"ContainerDied","Data":"3445b1fadd5113eda1ef7a2dbe2a5709b42f41c0274699de28168e3b5cc0a1bd"}
Jul 25 00:20:27.288969 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:27.288586    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-6zf8m" event={"ID":"a0e5fb07-b2d1-44b5-ba9c-21ad483f6b30","Type":"ContainerDied","Data":"4682b23c944e8ba6be2abaeddc87d65f4f741269296a048e1cec3686fdffd410"}
Jul 25 00:20:27.288969 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:27.288623    2558 scope.go:117] "RemoveContainer" containerID="3445b1fadd5113eda1ef7a2dbe2a5709b42f41c0274699de28168e3b5cc0a1bd"
Jul 25 00:20:27.298177 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:27.298150    2558 scope.go:117] "RemoveContainer" containerID="3445b1fadd5113eda1ef7a2dbe2a5709b42f41c0274699de28168e3b5cc0a1bd"
Jul 25 00:20:27.298476 ip-10-0-142-133 kubenswrapper[2558]: E0725 00:20:27.298449    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3445b1fadd5113eda1ef7a2dbe2a5709b42f41c0274699de28168e3b5cc0a1bd\": container with ID starting with 3445b1fadd5113eda1ef7a2dbe2a5709b42f41c0274699de28168e3b5cc0a1bd not found: ID does not exist" containerID="3445b1fadd5113eda1ef7a2dbe2a5709b42f41c0274699de28168e3b5cc0a1bd"
Jul 25 00:20:27.298571 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:27.298477    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3445b1fadd5113eda1ef7a2dbe2a5709b42f41c0274699de28168e3b5cc0a1bd"} err="failed to get container status \"3445b1fadd5113eda1ef7a2dbe2a5709b42f41c0274699de28168e3b5cc0a1bd\": rpc error: code = NotFound desc = could not find container \"3445b1fadd5113eda1ef7a2dbe2a5709b42f41c0274699de28168e3b5cc0a1bd\": container with ID starting with 3445b1fadd5113eda1ef7a2dbe2a5709b42f41c0274699de28168e3b5cc0a1bd not found: ID does not exist"
Jul 25 00:20:27.309867 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:27.309836    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6zf8m"]
Jul 25 00:20:27.312175 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:27.312154    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6zf8m"]
Jul 25 00:20:28.204851 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:28.204817    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0e5fb07-b2d1-44b5-ba9c-21ad483f6b30" path="/var/lib/kubelet/pods/a0e5fb07-b2d1-44b5-ba9c-21ad483f6b30/volumes"
Jul 25 00:20:30.389817 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:20:30.389777    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:21:06.747113 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:06.747073    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:21:07.582827 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:07.582786    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:21:13.132662 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:13.132620    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:21:13.233429 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:13.233390    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:21:13.423206 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:13.423105    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:21:13.649541 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:13.649486    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:21:13.787996 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:13.787957    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-899559-qnmgg"]
Jul 25 00:21:13.788419 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:13.788404    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="185a8961-c049-4476-90e5-ee9350c33f6d" containerName="maas-api"
Jul 25 00:21:13.788468 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:13.788421    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="185a8961-c049-4476-90e5-ee9350c33f6d" containerName="maas-api"
Jul 25 00:21:13.788468 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:13.788429    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0e5fb07-b2d1-44b5-ba9c-21ad483f6b30" containerName="registry-server"
Jul 25 00:21:13.788468 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:13.788434    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0e5fb07-b2d1-44b5-ba9c-21ad483f6b30" containerName="registry-server"
Jul 25 00:21:13.788468 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:13.788449    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a32a3941-53c4-4f1b-886e-3440592322e8" containerName="payload-pre-processing"
Jul 25 00:21:13.788468 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:13.788454    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="a32a3941-53c4-4f1b-886e-3440592322e8" containerName="payload-pre-processing"
Jul 25 00:21:13.788670 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:13.788519    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="185a8961-c049-4476-90e5-ee9350c33f6d" containerName="maas-api"
Jul 25 00:21:13.788670 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:13.788541    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0e5fb07-b2d1-44b5-ba9c-21ad483f6b30" containerName="registry-server"
Jul 25 00:21:13.788670 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:13.788549    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="a32a3941-53c4-4f1b-886e-3440592322e8" containerName="payload-pre-processing"
Jul 25 00:21:13.791590 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:13.791564    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-899559-qnmgg"
Jul 25 00:21:13.793897 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:13.793875    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bxr46\""
Jul 25 00:21:13.798490 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:13.798446    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-899559-qnmgg"]
Jul 25 00:21:13.856085 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:13.856046    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c12cd7c8-0d1c-4d76-a1b5-d9b44fbcf3cd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-899559-qnmgg\" (UID: \"c12cd7c8-0d1c-4d76-a1b5-d9b44fbcf3cd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-899559-qnmgg"
Jul 25 00:21:13.957382 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:13.957339    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c12cd7c8-0d1c-4d76-a1b5-d9b44fbcf3cd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-899559-qnmgg\" (UID: \"c12cd7c8-0d1c-4d76-a1b5-d9b44fbcf3cd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-899559-qnmgg"
Jul 25 00:21:13.958102 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:13.958079    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c12cd7c8-0d1c-4d76-a1b5-d9b44fbcf3cd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-899559-qnmgg\" (UID: \"c12cd7c8-0d1c-4d76-a1b5-d9b44fbcf3cd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-899559-qnmgg"
Jul 25 00:21:14.104871 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:14.104785    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-899559-qnmgg"
Jul 25 00:21:14.251816 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:14.251776    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-899559-qnmgg"]
Jul 25 00:21:14.253239 ip-10-0-142-133 kubenswrapper[2558]: W0725 00:21:14.253210    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc12cd7c8_0d1c_4d76_a1b5_d9b44fbcf3cd.slice/crio-33ff7ad160263f2c87450fc2d8f8fa26d1f66029c76173cfa0c636470b177c7d WatchSource:0}: Error finding container 33ff7ad160263f2c87450fc2d8f8fa26d1f66029c76173cfa0c636470b177c7d: Status 404 returned error can't find the container with id 33ff7ad160263f2c87450fc2d8f8fa26d1f66029c76173cfa0c636470b177c7d
Jul 25 00:21:14.573671 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:14.573639    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-899559-qnmgg" event={"ID":"c12cd7c8-0d1c-4d76-a1b5-d9b44fbcf3cd","Type":"ContainerStarted","Data":"d5b7cf474c46564b6efd6b373af35ef1cd24c67f9cad7275964b49876ff8c912"}
Jul 25 00:21:14.573875 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:14.573692    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-899559-qnmgg" event={"ID":"c12cd7c8-0d1c-4d76-a1b5-d9b44fbcf3cd","Type":"ContainerStarted","Data":"33ff7ad160263f2c87450fc2d8f8fa26d1f66029c76173cfa0c636470b177c7d"}
Jul 25 00:21:14.587314 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:14.587267    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-899559-qnmgg" podStartSLOduration=1.5872502590000002 podStartE2EDuration="1.587250259s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:21:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:21:14.586309597 +0000 UTC m=+3114.972740325" watchObservedRunningTime="2026-07-25 00:21:14.587250259 +0000 UTC m=+3114.973680987"
Jul 25 00:21:15.580914 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:15.580875    2558 generic.go:356] "Generic (PLEG): container finished" podID="c12cd7c8-0d1c-4d76-a1b5-d9b44fbcf3cd" containerID="d5b7cf474c46564b6efd6b373af35ef1cd24c67f9cad7275964b49876ff8c912" exitCode=0
Jul 25 00:21:15.581377 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:15.580968    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-899559-qnmgg" event={"ID":"c12cd7c8-0d1c-4d76-a1b5-d9b44fbcf3cd","Type":"ContainerDied","Data":"d5b7cf474c46564b6efd6b373af35ef1cd24c67f9cad7275964b49876ff8c912"}
Jul 25 00:21:16.726494 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:16.726470    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-899559-qnmgg"
Jul 25 00:21:16.785046 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:16.785015    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c12cd7c8-0d1c-4d76-a1b5-d9b44fbcf3cd-maas-api-service-ca\") pod \"c12cd7c8-0d1c-4d76-a1b5-d9b44fbcf3cd\" (UID: \"c12cd7c8-0d1c-4d76-a1b5-d9b44fbcf3cd\") "
Jul 25 00:21:16.785382 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:16.785360    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c12cd7c8-0d1c-4d76-a1b5-d9b44fbcf3cd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c12cd7c8-0d1c-4d76-a1b5-d9b44fbcf3cd" (UID: "c12cd7c8-0d1c-4d76-a1b5-d9b44fbcf3cd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 00:21:16.885943 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:16.885852    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c12cd7c8-0d1c-4d76-a1b5-d9b44fbcf3cd-maas-api-service-ca\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 25 00:21:17.593263 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:17.593232    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-899559-qnmgg"
Jul 25 00:21:17.593263 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:17.593254    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-899559-qnmgg" event={"ID":"c12cd7c8-0d1c-4d76-a1b5-d9b44fbcf3cd","Type":"ContainerDied","Data":"33ff7ad160263f2c87450fc2d8f8fa26d1f66029c76173cfa0c636470b177c7d"}
Jul 25 00:21:17.593471 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:17.593301    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="33ff7ad160263f2c87450fc2d8f8fa26d1f66029c76173cfa0c636470b177c7d"
Jul 25 00:21:24.252236 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:24.251881    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-899559-fcf548c5c-znksx"]
Jul 25 00:21:24.252236 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:24.252185    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-899559-fcf548c5c-znksx" podUID="59409783-5c60-4731-9e8c-78de0f0639d3" containerName="maas-api" containerID="cri-o://70b3a4b5b2feb7f48ceef1442295abc29980eb1e941500108c49ea495caf917a" gracePeriod=30
Jul 25 00:21:24.278016 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:24.277976    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:21:24.313688 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:24.313653    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-899559-69bb66b758-v8h2h"]
Jul 25 00:21:24.314062 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:24.314030    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-899559-69bb66b758-v8h2h" podUID="fe0db738-ccda-4179-a1ee-2a305492fac2" containerName="payload-processing" containerID="cri-o://28f79a1a4811d96bbfb12b4b2adbc9b2d7ea3380c074aaef13117ea65964f807" gracePeriod=30
Jul 25 00:21:24.348154 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:24.347956    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:21:24.401682 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:24.401643    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:21:24.581887 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:24.581858    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-899559-fcf548c5c-znksx"
Jul 25 00:21:24.604493 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:24.604465    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-899559-69bb66b758-v8h2h"
Jul 25 00:21:24.636208 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:24.636171    2558 generic.go:356] "Generic (PLEG): container finished" podID="fe0db738-ccda-4179-a1ee-2a305492fac2" containerID="28f79a1a4811d96bbfb12b4b2adbc9b2d7ea3380c074aaef13117ea65964f807" exitCode=0
Jul 25 00:21:24.636432 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:24.636224    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-899559-69bb66b758-v8h2h"
Jul 25 00:21:24.636432 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:24.636259    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-899559-69bb66b758-v8h2h" event={"ID":"fe0db738-ccda-4179-a1ee-2a305492fac2","Type":"ContainerDied","Data":"28f79a1a4811d96bbfb12b4b2adbc9b2d7ea3380c074aaef13117ea65964f807"}
Jul 25 00:21:24.636432 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:24.636338    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-899559-69bb66b758-v8h2h" event={"ID":"fe0db738-ccda-4179-a1ee-2a305492fac2","Type":"ContainerDied","Data":"d1f16f45b5476d234f92007f9c84c7289b6a44eab00c4a4c99160d4940480903"}
Jul 25 00:21:24.636432 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:24.636366    2558 scope.go:117] "RemoveContainer" containerID="28f79a1a4811d96bbfb12b4b2adbc9b2d7ea3380c074aaef13117ea65964f807"
Jul 25 00:21:24.637692 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:24.637669    2558 generic.go:356] "Generic (PLEG): container finished" podID="59409783-5c60-4731-9e8c-78de0f0639d3" containerID="70b3a4b5b2feb7f48ceef1442295abc29980eb1e941500108c49ea495caf917a" exitCode=0
Jul 25 00:21:24.637814 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:24.637742    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-899559-fcf548c5c-znksx" event={"ID":"59409783-5c60-4731-9e8c-78de0f0639d3","Type":"ContainerDied","Data":"70b3a4b5b2feb7f48ceef1442295abc29980eb1e941500108c49ea495caf917a"}
Jul 25 00:21:24.637814 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:24.637800    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-899559-fcf548c5c-znksx" event={"ID":"59409783-5c60-4731-9e8c-78de0f0639d3","Type":"ContainerDied","Data":"7ddea76b196e0e83704c9a38645900af6d2458e6ae11be567f72aec42bb87046"}
Jul 25 00:21:24.637902 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:24.637750    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-899559-fcf548c5c-znksx"
Jul 25 00:21:24.645345 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:24.645325    2558 scope.go:117] "RemoveContainer" containerID="28f79a1a4811d96bbfb12b4b2adbc9b2d7ea3380c074aaef13117ea65964f807"
Jul 25 00:21:24.645649 ip-10-0-142-133 kubenswrapper[2558]: E0725 00:21:24.645622    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28f79a1a4811d96bbfb12b4b2adbc9b2d7ea3380c074aaef13117ea65964f807\": container with ID starting with 28f79a1a4811d96bbfb12b4b2adbc9b2d7ea3380c074aaef13117ea65964f807 not found: ID does not exist" containerID="28f79a1a4811d96bbfb12b4b2adbc9b2d7ea3380c074aaef13117ea65964f807"
Jul 25 00:21:24.645753 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:24.645658    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28f79a1a4811d96bbfb12b4b2adbc9b2d7ea3380c074aaef13117ea65964f807"} err="failed to get container status \"28f79a1a4811d96bbfb12b4b2adbc9b2d7ea3380c074aaef13117ea65964f807\": rpc error: code = NotFound desc = could not find container \"28f79a1a4811d96bbfb12b4b2adbc9b2d7ea3380c074aaef13117ea65964f807\": container with ID starting with 28f79a1a4811d96bbfb12b4b2adbc9b2d7ea3380c074aaef13117ea65964f807 not found: ID does not exist"
Jul 25 00:21:24.645753 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:24.645676    2558 scope.go:117] "RemoveContainer" containerID="70b3a4b5b2feb7f48ceef1442295abc29980eb1e941500108c49ea495caf917a"
Jul 25 00:21:24.653996 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:24.653968    2558 scope.go:117] "RemoveContainer" containerID="70b3a4b5b2feb7f48ceef1442295abc29980eb1e941500108c49ea495caf917a"
Jul 25 00:21:24.654262 ip-10-0-142-133 kubenswrapper[2558]: E0725 00:21:24.654240    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70b3a4b5b2feb7f48ceef1442295abc29980eb1e941500108c49ea495caf917a\": container with ID starting with 70b3a4b5b2feb7f48ceef1442295abc29980eb1e941500108c49ea495caf917a not found: ID does not exist" containerID="70b3a4b5b2feb7f48ceef1442295abc29980eb1e941500108c49ea495caf917a"
Jul 25 00:21:24.654334 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:24.654271    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70b3a4b5b2feb7f48ceef1442295abc29980eb1e941500108c49ea495caf917a"} err="failed to get container status \"70b3a4b5b2feb7f48ceef1442295abc29980eb1e941500108c49ea495caf917a\": rpc error: code = NotFound desc = could not find container \"70b3a4b5b2feb7f48ceef1442295abc29980eb1e941500108c49ea495caf917a\": container with ID starting with 70b3a4b5b2feb7f48ceef1442295abc29980eb1e941500108c49ea495caf917a not found: ID does not exist"
Jul 25 00:21:24.655503 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:24.655484    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bhsgf\" (UniqueName: \"kubernetes.io/projected/fe0db738-ccda-4179-a1ee-2a305492fac2-kube-api-access-bhsgf\") pod \"fe0db738-ccda-4179-a1ee-2a305492fac2\" (UID: \"fe0db738-ccda-4179-a1ee-2a305492fac2\") "
Jul 25 00:21:24.655604 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:24.655518    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59409783-5c60-4731-9e8c-78de0f0639d3-maas-api-tls\") pod \"59409783-5c60-4731-9e8c-78de0f0639d3\" (UID: \"59409783-5c60-4731-9e8c-78de0f0639d3\") "
Jul 25 00:21:24.655604 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:24.655575    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/59409783-5c60-4731-9e8c-78de0f0639d3-openshift-service-ca\") pod \"59409783-5c60-4731-9e8c-78de0f0639d3\" (UID: \"59409783-5c60-4731-9e8c-78de0f0639d3\") "
Jul 25 00:21:24.655720 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:24.655655    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe0db738-ccda-4179-a1ee-2a305492fac2-plugins-config-volume\") pod \"fe0db738-ccda-4179-a1ee-2a305492fac2\" (UID: \"fe0db738-ccda-4179-a1ee-2a305492fac2\") "
Jul 25 00:21:24.655720 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:24.655687    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vxctx\" (UniqueName: \"kubernetes.io/projected/59409783-5c60-4731-9e8c-78de0f0639d3-kube-api-access-vxctx\") pod \"59409783-5c60-4731-9e8c-78de0f0639d3\" (UID: \"59409783-5c60-4731-9e8c-78de0f0639d3\") "
Jul 25 00:21:24.656052 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:24.656018    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/59409783-5c60-4731-9e8c-78de0f0639d3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "59409783-5c60-4731-9e8c-78de0f0639d3" (UID: "59409783-5c60-4731-9e8c-78de0f0639d3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 00:21:24.656137 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:24.656045    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe0db738-ccda-4179-a1ee-2a305492fac2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fe0db738-ccda-4179-a1ee-2a305492fac2" (UID: "fe0db738-ccda-4179-a1ee-2a305492fac2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 00:21:24.657685 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:24.657659    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59409783-5c60-4731-9e8c-78de0f0639d3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "59409783-5c60-4731-9e8c-78de0f0639d3" (UID: "59409783-5c60-4731-9e8c-78de0f0639d3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 00:21:24.657770 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:24.657712    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59409783-5c60-4731-9e8c-78de0f0639d3-kube-api-access-vxctx" (OuterVolumeSpecName: "kube-api-access-vxctx") pod "59409783-5c60-4731-9e8c-78de0f0639d3" (UID: "59409783-5c60-4731-9e8c-78de0f0639d3"). InnerVolumeSpecName "kube-api-access-vxctx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 00:21:24.657770 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:24.657745    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe0db738-ccda-4179-a1ee-2a305492fac2-kube-api-access-bhsgf" (OuterVolumeSpecName: "kube-api-access-bhsgf") pod "fe0db738-ccda-4179-a1ee-2a305492fac2" (UID: "fe0db738-ccda-4179-a1ee-2a305492fac2"). InnerVolumeSpecName "kube-api-access-bhsgf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 00:21:24.756703 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:24.756663    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe0db738-ccda-4179-a1ee-2a305492fac2-plugins-config-volume\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 25 00:21:24.756703 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:24.756695    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vxctx\" (UniqueName: \"kubernetes.io/projected/59409783-5c60-4731-9e8c-78de0f0639d3-kube-api-access-vxctx\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 25 00:21:24.756703 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:24.756704    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bhsgf\" (UniqueName: \"kubernetes.io/projected/fe0db738-ccda-4179-a1ee-2a305492fac2-kube-api-access-bhsgf\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 25 00:21:24.756703 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:24.756714    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59409783-5c60-4731-9e8c-78de0f0639d3-maas-api-tls\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 25 00:21:24.756999 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:24.756723    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/59409783-5c60-4731-9e8c-78de0f0639d3-openshift-service-ca\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 25 00:21:24.963029 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:24.962986    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-899559-fcf548c5c-znksx"]
Jul 25 00:21:24.967090 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:24.967047    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-899559-fcf548c5c-znksx"]
Jul 25 00:21:24.977764 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:24.977722    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-899559-69bb66b758-v8h2h"]
Jul 25 00:21:24.979941 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:24.979917    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-899559-69bb66b758-v8h2h"]
Jul 25 00:21:26.204467 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:26.204420    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59409783-5c60-4731-9e8c-78de0f0639d3" path="/var/lib/kubelet/pods/59409783-5c60-4731-9e8c-78de0f0639d3/volumes"
Jul 25 00:21:26.205009 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:26.204980    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe0db738-ccda-4179-a1ee-2a305492fac2" path="/var/lib/kubelet/pods/fe0db738-ccda-4179-a1ee-2a305492fac2/volumes"
Jul 25 00:21:28.862027 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:28.861983    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:21:28.863873 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:28.863848    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-899559-qnmgg"]
Jul 25 00:21:28.866054 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:28.866032    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-899559-qnmgg"]
Jul 25 00:21:29.319300 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:29.319263    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:21:30.205324 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:30.205288    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c12cd7c8-0d1c-4d76-a1b5-d9b44fbcf3cd" path="/var/lib/kubelet/pods/c12cd7c8-0d1c-4d76-a1b5-d9b44fbcf3cd/volumes"
Jul 25 00:21:35.133708 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:35.133670    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:21:35.234293 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:35.234250    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:21:35.414646 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:35.414488    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:21:35.649571 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:35.649520    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:21:35.798204 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:35.798171    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-92df89-jltvz"]
Jul 25 00:21:35.798619 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:35.798605    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe0db738-ccda-4179-a1ee-2a305492fac2" containerName="payload-processing"
Jul 25 00:21:35.798697 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:35.798620    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe0db738-ccda-4179-a1ee-2a305492fac2" containerName="payload-processing"
Jul 25 00:21:35.798697 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:35.798632    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c12cd7c8-0d1c-4d76-a1b5-d9b44fbcf3cd" containerName="revoke-keys"
Jul 25 00:21:35.798697 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:35.798639    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="c12cd7c8-0d1c-4d76-a1b5-d9b44fbcf3cd" containerName="revoke-keys"
Jul 25 00:21:35.798697 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:35.798646    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59409783-5c60-4731-9e8c-78de0f0639d3" containerName="maas-api"
Jul 25 00:21:35.798697 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:35.798651    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="59409783-5c60-4731-9e8c-78de0f0639d3" containerName="maas-api"
Jul 25 00:21:35.798893 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:35.798716    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="59409783-5c60-4731-9e8c-78de0f0639d3" containerName="maas-api"
Jul 25 00:21:35.798893 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:35.798730    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe0db738-ccda-4179-a1ee-2a305492fac2" containerName="payload-processing"
Jul 25 00:21:35.798893 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:35.798736    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="c12cd7c8-0d1c-4d76-a1b5-d9b44fbcf3cd" containerName="revoke-keys"
Jul 25 00:21:35.801814 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:35.801797    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-92df89-jltvz"
Jul 25 00:21:35.804111 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:35.804084    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bxr46\""
Jul 25 00:21:35.808338 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:35.808310    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-92df89-jltvz"]
Jul 25 00:21:35.958215 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:35.958180    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbfa28bf-7f57-4366-b3a3-3e1674212319-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-92df89-jltvz\" (UID: \"cbfa28bf-7f57-4366-b3a3-3e1674212319\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-92df89-jltvz"
Jul 25 00:21:36.059774 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:36.059688    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbfa28bf-7f57-4366-b3a3-3e1674212319-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-92df89-jltvz\" (UID: \"cbfa28bf-7f57-4366-b3a3-3e1674212319\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-92df89-jltvz"
Jul 25 00:21:36.060324 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:36.060304    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbfa28bf-7f57-4366-b3a3-3e1674212319-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-92df89-jltvz\" (UID: \"cbfa28bf-7f57-4366-b3a3-3e1674212319\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-92df89-jltvz"
Jul 25 00:21:36.113501 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:36.113463    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-92df89-jltvz"
Jul 25 00:21:36.268483 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:36.268452    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-92df89-jltvz"]
Jul 25 00:21:36.270974 ip-10-0-142-133 kubenswrapper[2558]: W0725 00:21:36.270945    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbfa28bf_7f57_4366_b3a3_3e1674212319.slice/crio-aa6cabf5f8ac3ce5a131e3625d65533e77421f964dd6a102fa5f19f59b0d7624 WatchSource:0}: Error finding container aa6cabf5f8ac3ce5a131e3625d65533e77421f964dd6a102fa5f19f59b0d7624: Status 404 returned error can't find the container with id aa6cabf5f8ac3ce5a131e3625d65533e77421f964dd6a102fa5f19f59b0d7624
Jul 25 00:21:36.708671 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:36.708579    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-92df89-jltvz" event={"ID":"cbfa28bf-7f57-4366-b3a3-3e1674212319","Type":"ContainerStarted","Data":"d23654fb45f808638e3c33d05881a38efdc37f7c8b00339ecb4318cb8ac84bf2"}
Jul 25 00:21:36.708671 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:36.708644    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-92df89-jltvz" event={"ID":"cbfa28bf-7f57-4366-b3a3-3e1674212319","Type":"ContainerStarted","Data":"aa6cabf5f8ac3ce5a131e3625d65533e77421f964dd6a102fa5f19f59b0d7624"}
Jul 25 00:21:36.722835 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:36.722785    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-92df89-jltvz" podStartSLOduration=1.722770205 podStartE2EDuration="1.722770205s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:21:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:21:36.720842453 +0000 UTC m=+3137.107273182" watchObservedRunningTime="2026-07-25 00:21:36.722770205 +0000 UTC m=+3137.109200932"
Jul 25 00:21:37.716682 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:37.716646    2558 generic.go:356] "Generic (PLEG): container finished" podID="cbfa28bf-7f57-4366-b3a3-3e1674212319" containerID="d23654fb45f808638e3c33d05881a38efdc37f7c8b00339ecb4318cb8ac84bf2" exitCode=0
Jul 25 00:21:37.717187 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:37.716737    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-92df89-jltvz" event={"ID":"cbfa28bf-7f57-4366-b3a3-3e1674212319","Type":"ContainerDied","Data":"d23654fb45f808638e3c33d05881a38efdc37f7c8b00339ecb4318cb8ac84bf2"}
Jul 25 00:21:38.850376 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:38.850346    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-92df89-jltvz"
Jul 25 00:21:38.988943 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:38.988913    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbfa28bf-7f57-4366-b3a3-3e1674212319-maas-api-service-ca\") pod \"cbfa28bf-7f57-4366-b3a3-3e1674212319\" (UID: \"cbfa28bf-7f57-4366-b3a3-3e1674212319\") "
Jul 25 00:21:38.989322 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:38.989296    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cbfa28bf-7f57-4366-b3a3-3e1674212319-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "cbfa28bf-7f57-4366-b3a3-3e1674212319" (UID: "cbfa28bf-7f57-4366-b3a3-3e1674212319"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 00:21:39.090561 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:39.090490    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbfa28bf-7f57-4366-b3a3-3e1674212319-maas-api-service-ca\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 25 00:21:39.728682 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:39.728645    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-92df89-jltvz"
Jul 25 00:21:39.728905 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:39.728650    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-92df89-jltvz" event={"ID":"cbfa28bf-7f57-4366-b3a3-3e1674212319","Type":"ContainerDied","Data":"aa6cabf5f8ac3ce5a131e3625d65533e77421f964dd6a102fa5f19f59b0d7624"}
Jul 25 00:21:39.728905 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:39.728780    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="aa6cabf5f8ac3ce5a131e3625d65533e77421f964dd6a102fa5f19f59b0d7624"
Jul 25 00:21:46.036095 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:46.035992    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-92df89-77ddf56b67-68mcv"]
Jul 25 00:21:46.036562 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:46.036254    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-92df89-77ddf56b67-68mcv" podUID="febc96ee-61d4-4220-885c-45d1fa1edc59" containerName="maas-api" containerID="cri-o://c1d74db7bd104ae4705bb7567cccee396f4f9f62e2938b6f45dd3ed45afd9ba6" gracePeriod=30
Jul 25 00:21:46.057026 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:46.056992    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:21:46.086984 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:46.086942    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-92df89-5694c5d774-zm222"]
Jul 25 00:21:46.087551 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:46.087494    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-92df89-5694c5d774-zm222" podUID="6b3546a5-0243-4529-9f03-878cdb41941a" containerName="payload-processing" containerID="cri-o://c743116504268adf37d1469088653762d48a586ba058ec4290c177b938423277" gracePeriod=30
Jul 25 00:21:46.096684 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:46.096652    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-92df89-7fd7485b8-td6zk"]
Jul 25 00:21:46.096888 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:46.096856    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-92df89-7fd7485b8-td6zk" podUID="47c7d4d4-fd24-4083-9eab-87560dea56ea" containerName="payload-pre-processing" containerID="cri-o://9fc686ae5a3c81136ea33f91f979bc78fafa5d824eff3384e3c5bae6bc64a69f" gracePeriod=30
Jul 25 00:21:46.127170 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:46.127138    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:21:46.177067 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:46.177033    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:21:46.367948 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:46.367918    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-92df89-77ddf56b67-68mcv"
Jul 25 00:21:46.408970 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:46.408940    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-92df89-5694c5d774-zm222"
Jul 25 00:21:46.417130 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:46.417099    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-92df89-7fd7485b8-td6zk"
Jul 25 00:21:46.462660 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:46.462619    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/febc96ee-61d4-4220-885c-45d1fa1edc59-maas-api-tls\") pod \"febc96ee-61d4-4220-885c-45d1fa1edc59\" (UID: \"febc96ee-61d4-4220-885c-45d1fa1edc59\") "
Jul 25 00:21:46.462851 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:46.462693    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8qrn6\" (UniqueName: \"kubernetes.io/projected/febc96ee-61d4-4220-885c-45d1fa1edc59-kube-api-access-8qrn6\") pod \"febc96ee-61d4-4220-885c-45d1fa1edc59\" (UID: \"febc96ee-61d4-4220-885c-45d1fa1edc59\") "
Jul 25 00:21:46.462851 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:46.462743    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/febc96ee-61d4-4220-885c-45d1fa1edc59-openshift-service-ca\") pod \"febc96ee-61d4-4220-885c-45d1fa1edc59\" (UID: \"febc96ee-61d4-4220-885c-45d1fa1edc59\") "
Jul 25 00:21:46.463209 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:46.463179    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/febc96ee-61d4-4220-885c-45d1fa1edc59-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "febc96ee-61d4-4220-885c-45d1fa1edc59" (UID: "febc96ee-61d4-4220-885c-45d1fa1edc59"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 00:21:46.464911 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:46.464883    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/febc96ee-61d4-4220-885c-45d1fa1edc59-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "febc96ee-61d4-4220-885c-45d1fa1edc59" (UID: "febc96ee-61d4-4220-885c-45d1fa1edc59"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 00:21:46.464998 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:46.464923    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/febc96ee-61d4-4220-885c-45d1fa1edc59-kube-api-access-8qrn6" (OuterVolumeSpecName: "kube-api-access-8qrn6") pod "febc96ee-61d4-4220-885c-45d1fa1edc59" (UID: "febc96ee-61d4-4220-885c-45d1fa1edc59"). InnerVolumeSpecName "kube-api-access-8qrn6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 00:21:46.563310 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:46.563261    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/47c7d4d4-fd24-4083-9eab-87560dea56ea-plugins-config-volume\") pod \"47c7d4d4-fd24-4083-9eab-87560dea56ea\" (UID: \"47c7d4d4-fd24-4083-9eab-87560dea56ea\") "
Jul 25 00:21:46.563517 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:46.563342    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fcqlf\" (UniqueName: \"kubernetes.io/projected/6b3546a5-0243-4529-9f03-878cdb41941a-kube-api-access-fcqlf\") pod \"6b3546a5-0243-4529-9f03-878cdb41941a\" (UID: \"6b3546a5-0243-4529-9f03-878cdb41941a\") "
Jul 25 00:21:46.563517 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:46.563381    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jt5k7\" (UniqueName: \"kubernetes.io/projected/47c7d4d4-fd24-4083-9eab-87560dea56ea-kube-api-access-jt5k7\") pod \"47c7d4d4-fd24-4083-9eab-87560dea56ea\" (UID: \"47c7d4d4-fd24-4083-9eab-87560dea56ea\") "
Jul 25 00:21:46.563517 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:46.563451    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b3546a5-0243-4529-9f03-878cdb41941a-plugins-config-volume\") pod \"6b3546a5-0243-4529-9f03-878cdb41941a\" (UID: \"6b3546a5-0243-4529-9f03-878cdb41941a\") "
Jul 25 00:21:46.563729 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:46.563675    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8qrn6\" (UniqueName: \"kubernetes.io/projected/febc96ee-61d4-4220-885c-45d1fa1edc59-kube-api-access-8qrn6\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 25 00:21:46.563729 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:46.563672    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/47c7d4d4-fd24-4083-9eab-87560dea56ea-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "47c7d4d4-fd24-4083-9eab-87560dea56ea" (UID: "47c7d4d4-fd24-4083-9eab-87560dea56ea"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 00:21:46.563823 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:46.563691    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/febc96ee-61d4-4220-885c-45d1fa1edc59-openshift-service-ca\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 25 00:21:46.563823 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:46.563805    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/febc96ee-61d4-4220-885c-45d1fa1edc59-maas-api-tls\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 25 00:21:46.563936 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:46.563860    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b3546a5-0243-4529-9f03-878cdb41941a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6b3546a5-0243-4529-9f03-878cdb41941a" (UID: "6b3546a5-0243-4529-9f03-878cdb41941a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 00:21:46.565509 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:46.565487    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47c7d4d4-fd24-4083-9eab-87560dea56ea-kube-api-access-jt5k7" (OuterVolumeSpecName: "kube-api-access-jt5k7") pod "47c7d4d4-fd24-4083-9eab-87560dea56ea" (UID: "47c7d4d4-fd24-4083-9eab-87560dea56ea"). InnerVolumeSpecName "kube-api-access-jt5k7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 00:21:46.565509 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:46.565500    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b3546a5-0243-4529-9f03-878cdb41941a-kube-api-access-fcqlf" (OuterVolumeSpecName: "kube-api-access-fcqlf") pod "6b3546a5-0243-4529-9f03-878cdb41941a" (UID: "6b3546a5-0243-4529-9f03-878cdb41941a"). InnerVolumeSpecName "kube-api-access-fcqlf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 00:21:46.664715 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:46.664619    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b3546a5-0243-4529-9f03-878cdb41941a-plugins-config-volume\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 25 00:21:46.664715 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:46.664653    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/47c7d4d4-fd24-4083-9eab-87560dea56ea-plugins-config-volume\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 25 00:21:46.664715 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:46.664662    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fcqlf\" (UniqueName: \"kubernetes.io/projected/6b3546a5-0243-4529-9f03-878cdb41941a-kube-api-access-fcqlf\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 25 00:21:46.664715 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:46.664675    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jt5k7\" (UniqueName: \"kubernetes.io/projected/47c7d4d4-fd24-4083-9eab-87560dea56ea-kube-api-access-jt5k7\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 25 00:21:46.776360 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:46.776321    2558 generic.go:356] "Generic (PLEG): container finished" podID="47c7d4d4-fd24-4083-9eab-87560dea56ea" containerID="9fc686ae5a3c81136ea33f91f979bc78fafa5d824eff3384e3c5bae6bc64a69f" exitCode=0
Jul 25 00:21:46.776578 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:46.776370    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-92df89-7fd7485b8-td6zk"
Jul 25 00:21:46.776578 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:46.776407    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-92df89-7fd7485b8-td6zk" event={"ID":"47c7d4d4-fd24-4083-9eab-87560dea56ea","Type":"ContainerDied","Data":"9fc686ae5a3c81136ea33f91f979bc78fafa5d824eff3384e3c5bae6bc64a69f"}
Jul 25 00:21:46.776578 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:46.776468    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-92df89-7fd7485b8-td6zk" event={"ID":"47c7d4d4-fd24-4083-9eab-87560dea56ea","Type":"ContainerDied","Data":"bf89b0652e4192cda76e14a4ae96bfa6b8eadb07d21ad0b833bd79495832c5f1"}
Jul 25 00:21:46.776578 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:46.776500    2558 scope.go:117] "RemoveContainer" containerID="9fc686ae5a3c81136ea33f91f979bc78fafa5d824eff3384e3c5bae6bc64a69f"
Jul 25 00:21:46.777894 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:46.777869    2558 generic.go:356] "Generic (PLEG): container finished" podID="6b3546a5-0243-4529-9f03-878cdb41941a" containerID="c743116504268adf37d1469088653762d48a586ba058ec4290c177b938423277" exitCode=0
Jul 25 00:21:46.778019 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:46.777894    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-92df89-5694c5d774-zm222" event={"ID":"6b3546a5-0243-4529-9f03-878cdb41941a","Type":"ContainerDied","Data":"c743116504268adf37d1469088653762d48a586ba058ec4290c177b938423277"}
Jul 25 00:21:46.778019 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:46.777917    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-92df89-5694c5d774-zm222"
Jul 25 00:21:46.778019 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:46.777931    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-92df89-5694c5d774-zm222" event={"ID":"6b3546a5-0243-4529-9f03-878cdb41941a","Type":"ContainerDied","Data":"62e0417ba0fb03d2d371256d78a36c1a5feb7ade81a32cc78d8ce3587b40f626"}
Jul 25 00:21:46.779311 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:46.779287    2558 generic.go:356] "Generic (PLEG): container finished" podID="febc96ee-61d4-4220-885c-45d1fa1edc59" containerID="c1d74db7bd104ae4705bb7567cccee396f4f9f62e2938b6f45dd3ed45afd9ba6" exitCode=0
Jul 25 00:21:46.779438 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:46.779320    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-92df89-77ddf56b67-68mcv"
Jul 25 00:21:46.779438 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:46.779358    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-92df89-77ddf56b67-68mcv" event={"ID":"febc96ee-61d4-4220-885c-45d1fa1edc59","Type":"ContainerDied","Data":"c1d74db7bd104ae4705bb7567cccee396f4f9f62e2938b6f45dd3ed45afd9ba6"}
Jul 25 00:21:46.779438 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:46.779397    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-92df89-77ddf56b67-68mcv" event={"ID":"febc96ee-61d4-4220-885c-45d1fa1edc59","Type":"ContainerDied","Data":"6416b1b3bb24a8a69a145cad5fbd8bff39c41d3c0d48e7b61c875b7cad140559"}
Jul 25 00:21:46.787742 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:46.787695    2558 scope.go:117] "RemoveContainer" containerID="9fc686ae5a3c81136ea33f91f979bc78fafa5d824eff3384e3c5bae6bc64a69f"
Jul 25 00:21:46.788093 ip-10-0-142-133 kubenswrapper[2558]: E0725 00:21:46.788073    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9fc686ae5a3c81136ea33f91f979bc78fafa5d824eff3384e3c5bae6bc64a69f\": container with ID starting with 9fc686ae5a3c81136ea33f91f979bc78fafa5d824eff3384e3c5bae6bc64a69f not found: ID does not exist" containerID="9fc686ae5a3c81136ea33f91f979bc78fafa5d824eff3384e3c5bae6bc64a69f"
Jul 25 00:21:46.788203 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:46.788104    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9fc686ae5a3c81136ea33f91f979bc78fafa5d824eff3384e3c5bae6bc64a69f"} err="failed to get container status \"9fc686ae5a3c81136ea33f91f979bc78fafa5d824eff3384e3c5bae6bc64a69f\": rpc error: code = NotFound desc = could not find container \"9fc686ae5a3c81136ea33f91f979bc78fafa5d824eff3384e3c5bae6bc64a69f\": container with ID starting with 9fc686ae5a3c81136ea33f91f979bc78fafa5d824eff3384e3c5bae6bc64a69f not found: ID does not exist"
Jul 25 00:21:46.788203 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:46.788128    2558 scope.go:117] "RemoveContainer" containerID="c743116504268adf37d1469088653762d48a586ba058ec4290c177b938423277"
Jul 25 00:21:46.797471 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:46.797449    2558 scope.go:117] "RemoveContainer" containerID="c743116504268adf37d1469088653762d48a586ba058ec4290c177b938423277"
Jul 25 00:21:46.797807 ip-10-0-142-133 kubenswrapper[2558]: E0725 00:21:46.797788    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c743116504268adf37d1469088653762d48a586ba058ec4290c177b938423277\": container with ID starting with c743116504268adf37d1469088653762d48a586ba058ec4290c177b938423277 not found: ID does not exist" containerID="c743116504268adf37d1469088653762d48a586ba058ec4290c177b938423277"
Jul 25 00:21:46.797895 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:46.797815    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c743116504268adf37d1469088653762d48a586ba058ec4290c177b938423277"} err="failed to get container status \"c743116504268adf37d1469088653762d48a586ba058ec4290c177b938423277\": rpc error: code = NotFound desc = could not find container \"c743116504268adf37d1469088653762d48a586ba058ec4290c177b938423277\": container with ID starting with c743116504268adf37d1469088653762d48a586ba058ec4290c177b938423277 not found: ID does not exist"
Jul 25 00:21:46.797895 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:46.797834    2558 scope.go:117] "RemoveContainer" containerID="c1d74db7bd104ae4705bb7567cccee396f4f9f62e2938b6f45dd3ed45afd9ba6"
Jul 25 00:21:46.801990 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:46.801964    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-92df89-7fd7485b8-td6zk"]
Jul 25 00:21:46.806467 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:46.806441    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-92df89-7fd7485b8-td6zk"]
Jul 25 00:21:46.808852 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:46.808833    2558 scope.go:117] "RemoveContainer" containerID="c1d74db7bd104ae4705bb7567cccee396f4f9f62e2938b6f45dd3ed45afd9ba6"
Jul 25 00:21:46.809214 ip-10-0-142-133 kubenswrapper[2558]: E0725 00:21:46.809192    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1d74db7bd104ae4705bb7567cccee396f4f9f62e2938b6f45dd3ed45afd9ba6\": container with ID starting with c1d74db7bd104ae4705bb7567cccee396f4f9f62e2938b6f45dd3ed45afd9ba6 not found: ID does not exist" containerID="c1d74db7bd104ae4705bb7567cccee396f4f9f62e2938b6f45dd3ed45afd9ba6"
Jul 25 00:21:46.809322 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:46.809220    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1d74db7bd104ae4705bb7567cccee396f4f9f62e2938b6f45dd3ed45afd9ba6"} err="failed to get container status \"c1d74db7bd104ae4705bb7567cccee396f4f9f62e2938b6f45dd3ed45afd9ba6\": rpc error: code = NotFound desc = could not find container \"c1d74db7bd104ae4705bb7567cccee396f4f9f62e2938b6f45dd3ed45afd9ba6\": container with ID starting with c1d74db7bd104ae4705bb7567cccee396f4f9f62e2938b6f45dd3ed45afd9ba6 not found: ID does not exist"
Jul 25 00:21:46.817443 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:46.817414    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-92df89-5694c5d774-zm222"]
Jul 25 00:21:46.821340 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:46.821310    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-92df89-5694c5d774-zm222"]
Jul 25 00:21:46.829986 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:46.829956    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-92df89-77ddf56b67-68mcv"]
Jul 25 00:21:46.833447 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:46.833420    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-92df89-77ddf56b67-68mcv"]
Jul 25 00:21:48.205162 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:48.205127    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="47c7d4d4-fd24-4083-9eab-87560dea56ea" path="/var/lib/kubelet/pods/47c7d4d4-fd24-4083-9eab-87560dea56ea/volumes"
Jul 25 00:21:48.205559 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:48.205484    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b3546a5-0243-4529-9f03-878cdb41941a" path="/var/lib/kubelet/pods/6b3546a5-0243-4529-9f03-878cdb41941a/volumes"
Jul 25 00:21:48.205897 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:48.205882    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="febc96ee-61d4-4220-885c-45d1fa1edc59" path="/var/lib/kubelet/pods/febc96ee-61d4-4220-885c-45d1fa1edc59/volumes"
Jul 25 00:21:50.878730 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:50.878675    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:21:50.879229 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:50.879202    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-92df89-jltvz"]
Jul 25 00:21:50.881568 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:50.881545    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-92df89-jltvz"]
Jul 25 00:21:51.298356 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:51.298323    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:21:52.206224 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:52.206188    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cbfa28bf-7f57-4366-b3a3-3e1674212319" path="/var/lib/kubelet/pods/cbfa28bf-7f57-4366-b3a3-3e1674212319/volumes"
Jul 25 00:21:57.262363 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:57.262325    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:21:57.317395 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:57.317359    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:21:57.534110 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:57.533994    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:21:57.768810 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:57.768769    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:21:57.883513 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:57.883421    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-413844-slksp"]
Jul 25 00:21:57.883827 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:57.883813    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cbfa28bf-7f57-4366-b3a3-3e1674212319" containerName="revoke-keys"
Jul 25 00:21:57.883871 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:57.883828    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="cbfa28bf-7f57-4366-b3a3-3e1674212319" containerName="revoke-keys"
Jul 25 00:21:57.883871 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:57.883842    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="febc96ee-61d4-4220-885c-45d1fa1edc59" containerName="maas-api"
Jul 25 00:21:57.883871 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:57.883848    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="febc96ee-61d4-4220-885c-45d1fa1edc59" containerName="maas-api"
Jul 25 00:21:57.883871 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:57.883869    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47c7d4d4-fd24-4083-9eab-87560dea56ea" containerName="payload-pre-processing"
Jul 25 00:21:57.884002 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:57.883875    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="47c7d4d4-fd24-4083-9eab-87560dea56ea" containerName="payload-pre-processing"
Jul 25 00:21:57.884002 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:57.883883    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b3546a5-0243-4529-9f03-878cdb41941a" containerName="payload-processing"
Jul 25 00:21:57.884002 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:57.883889    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b3546a5-0243-4529-9f03-878cdb41941a" containerName="payload-processing"
Jul 25 00:21:57.884002 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:57.883941    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="febc96ee-61d4-4220-885c-45d1fa1edc59" containerName="maas-api"
Jul 25 00:21:57.884002 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:57.883949    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="47c7d4d4-fd24-4083-9eab-87560dea56ea" containerName="payload-pre-processing"
Jul 25 00:21:57.884002 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:57.883956    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="cbfa28bf-7f57-4366-b3a3-3e1674212319" containerName="revoke-keys"
Jul 25 00:21:57.884002 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:57.883966    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b3546a5-0243-4529-9f03-878cdb41941a" containerName="payload-processing"
Jul 25 00:21:57.888687 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:57.888661    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-413844-slksp"
Jul 25 00:21:57.890990 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:57.890961    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bxr46\""
Jul 25 00:21:57.893320 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:57.893291    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-413844-slksp"]
Jul 25 00:21:57.955027 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:57.954969    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/86f2b971-dcfc-4acb-a096-03f2232f34e7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-413844-slksp\" (UID: \"86f2b971-dcfc-4acb-a096-03f2232f34e7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-413844-slksp"
Jul 25 00:21:58.056084 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:58.056048    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/86f2b971-dcfc-4acb-a096-03f2232f34e7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-413844-slksp\" (UID: \"86f2b971-dcfc-4acb-a096-03f2232f34e7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-413844-slksp"
Jul 25 00:21:58.056714 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:58.056692    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/86f2b971-dcfc-4acb-a096-03f2232f34e7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-413844-slksp\" (UID: \"86f2b971-dcfc-4acb-a096-03f2232f34e7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-413844-slksp"
Jul 25 00:21:58.201200 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:58.201104    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-413844-slksp"
Jul 25 00:21:58.336830 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:58.336803    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-413844-slksp"]
Jul 25 00:21:58.338377 ip-10-0-142-133 kubenswrapper[2558]: W0725 00:21:58.338349    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86f2b971_dcfc_4acb_a096_03f2232f34e7.slice/crio-20a88605d0d1e68c5df1f17f58dec04f7bcc9521b495ade2710e064462493a34 WatchSource:0}: Error finding container 20a88605d0d1e68c5df1f17f58dec04f7bcc9521b495ade2710e064462493a34: Status 404 returned error can't find the container with id 20a88605d0d1e68c5df1f17f58dec04f7bcc9521b495ade2710e064462493a34
Jul 25 00:21:58.852654 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:58.852573    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-413844-slksp" event={"ID":"86f2b971-dcfc-4acb-a096-03f2232f34e7","Type":"ContainerStarted","Data":"06a09c291ddc6a19f74199f4a676398599e81e23f022ca2eb34e287957bd41bb"}
Jul 25 00:21:58.852654 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:58.852628    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-413844-slksp" event={"ID":"86f2b971-dcfc-4acb-a096-03f2232f34e7","Type":"ContainerStarted","Data":"20a88605d0d1e68c5df1f17f58dec04f7bcc9521b495ade2710e064462493a34"}
Jul 25 00:21:58.868232 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:58.868173    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-413844-slksp" podStartSLOduration=1.868151191 podStartE2EDuration="1.868151191s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 00:21:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:21:58.867741007 +0000 UTC m=+3159.254171735" watchObservedRunningTime="2026-07-25 00:21:58.868151191 +0000 UTC m=+3159.254581922"
Jul 25 00:21:59.859942 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:59.859898    2558 generic.go:356] "Generic (PLEG): container finished" podID="86f2b971-dcfc-4acb-a096-03f2232f34e7" containerID="06a09c291ddc6a19f74199f4a676398599e81e23f022ca2eb34e287957bd41bb" exitCode=0
Jul 25 00:21:59.860400 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:21:59.859982    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-413844-slksp" event={"ID":"86f2b971-dcfc-4acb-a096-03f2232f34e7","Type":"ContainerDied","Data":"06a09c291ddc6a19f74199f4a676398599e81e23f022ca2eb34e287957bd41bb"}
Jul 25 00:22:01.000636 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:01.000606    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-413844-slksp"
Jul 25 00:22:01.084836 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:01.084806    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/86f2b971-dcfc-4acb-a096-03f2232f34e7-maas-api-service-ca\") pod \"86f2b971-dcfc-4acb-a096-03f2232f34e7\" (UID: \"86f2b971-dcfc-4acb-a096-03f2232f34e7\") "
Jul 25 00:22:01.085208 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:01.085186    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86f2b971-dcfc-4acb-a096-03f2232f34e7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "86f2b971-dcfc-4acb-a096-03f2232f34e7" (UID: "86f2b971-dcfc-4acb-a096-03f2232f34e7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 00:22:01.185982 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:01.185901    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/86f2b971-dcfc-4acb-a096-03f2232f34e7-maas-api-service-ca\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 25 00:22:01.879991 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:01.879955    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-413844-slksp"
Jul 25 00:22:01.879991 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:01.879966    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-413844-slksp" event={"ID":"86f2b971-dcfc-4acb-a096-03f2232f34e7","Type":"ContainerDied","Data":"20a88605d0d1e68c5df1f17f58dec04f7bcc9521b495ade2710e064462493a34"}
Jul 25 00:22:01.880210 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:01.880041    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="20a88605d0d1e68c5df1f17f58dec04f7bcc9521b495ade2710e064462493a34"
Jul 25 00:22:08.943184 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:08.943145    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-413844-549c74bd78-vnhdv"]
Jul 25 00:22:08.943650 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:08.943359    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-413844-549c74bd78-vnhdv" podUID="0a1b7baa-e633-4042-ac62-1ec8df05750a" containerName="maas-api" containerID="cri-o://4297c0d6dfd92567eb6b3120a3130e9967f12da03114228012f82d8f7e735ed1" gracePeriod=30
Jul 25 00:22:08.954330 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:08.954295    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:22:08.960342 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:08.960239    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-413844-29748975-dmkl2"]
Jul 25 00:22:08.963743 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:08.963718    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-413844-29748975-dmkl2"]
Jul 25 00:22:09.035526 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:09.035483    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:22:09.094026 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:09.093989    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:22:09.213710 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:09.213683    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-413844-549c74bd78-vnhdv"
Jul 25 00:22:09.256711 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:09.256675    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a1b7baa-e633-4042-ac62-1ec8df05750a-openshift-service-ca\") pod \"0a1b7baa-e633-4042-ac62-1ec8df05750a\" (UID: \"0a1b7baa-e633-4042-ac62-1ec8df05750a\") "
Jul 25 00:22:09.256711 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:09.256714    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0a1b7baa-e633-4042-ac62-1ec8df05750a-maas-api-tls\") pod \"0a1b7baa-e633-4042-ac62-1ec8df05750a\" (UID: \"0a1b7baa-e633-4042-ac62-1ec8df05750a\") "
Jul 25 00:22:09.257014 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:09.256814    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6vwm5\" (UniqueName: \"kubernetes.io/projected/0a1b7baa-e633-4042-ac62-1ec8df05750a-kube-api-access-6vwm5\") pod \"0a1b7baa-e633-4042-ac62-1ec8df05750a\" (UID: \"0a1b7baa-e633-4042-ac62-1ec8df05750a\") "
Jul 25 00:22:09.257147 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:09.257110    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a1b7baa-e633-4042-ac62-1ec8df05750a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0a1b7baa-e633-4042-ac62-1ec8df05750a" (UID: "0a1b7baa-e633-4042-ac62-1ec8df05750a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 00:22:09.259224 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:09.259181    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a1b7baa-e633-4042-ac62-1ec8df05750a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0a1b7baa-e633-4042-ac62-1ec8df05750a" (UID: "0a1b7baa-e633-4042-ac62-1ec8df05750a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 00:22:09.259455 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:09.259415    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a1b7baa-e633-4042-ac62-1ec8df05750a-kube-api-access-6vwm5" (OuterVolumeSpecName: "kube-api-access-6vwm5") pod "0a1b7baa-e633-4042-ac62-1ec8df05750a" (UID: "0a1b7baa-e633-4042-ac62-1ec8df05750a"). InnerVolumeSpecName "kube-api-access-6vwm5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 00:22:09.358026 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:09.357986    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6vwm5\" (UniqueName: \"kubernetes.io/projected/0a1b7baa-e633-4042-ac62-1ec8df05750a-kube-api-access-6vwm5\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 25 00:22:09.358026 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:09.358017    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a1b7baa-e633-4042-ac62-1ec8df05750a-openshift-service-ca\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 25 00:22:09.358026 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:09.358027    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0a1b7baa-e633-4042-ac62-1ec8df05750a-maas-api-tls\") on node \"ip-10-0-142-133.ec2.internal\" DevicePath \"\""
Jul 25 00:22:09.930480 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:09.930442    2558 generic.go:356] "Generic (PLEG): container finished" podID="0a1b7baa-e633-4042-ac62-1ec8df05750a" containerID="4297c0d6dfd92567eb6b3120a3130e9967f12da03114228012f82d8f7e735ed1" exitCode=0
Jul 25 00:22:09.930711 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:09.930507    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-413844-549c74bd78-vnhdv"
Jul 25 00:22:09.930711 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:09.930507    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-413844-549c74bd78-vnhdv" event={"ID":"0a1b7baa-e633-4042-ac62-1ec8df05750a","Type":"ContainerDied","Data":"4297c0d6dfd92567eb6b3120a3130e9967f12da03114228012f82d8f7e735ed1"}
Jul 25 00:22:09.930711 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:09.930705    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-413844-549c74bd78-vnhdv" event={"ID":"0a1b7baa-e633-4042-ac62-1ec8df05750a","Type":"ContainerDied","Data":"80bc3a5595023b5de42d23f8bcda89c874ba41a0246ec477cd7d2537ea4bcbfd"}
Jul 25 00:22:09.930889 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:09.930741    2558 scope.go:117] "RemoveContainer" containerID="4297c0d6dfd92567eb6b3120a3130e9967f12da03114228012f82d8f7e735ed1"
Jul 25 00:22:09.941189 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:09.941166    2558 scope.go:117] "RemoveContainer" containerID="4297c0d6dfd92567eb6b3120a3130e9967f12da03114228012f82d8f7e735ed1"
Jul 25 00:22:09.941581 ip-10-0-142-133 kubenswrapper[2558]: E0725 00:22:09.941559    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4297c0d6dfd92567eb6b3120a3130e9967f12da03114228012f82d8f7e735ed1\": container with ID starting with 4297c0d6dfd92567eb6b3120a3130e9967f12da03114228012f82d8f7e735ed1 not found: ID does not exist" containerID="4297c0d6dfd92567eb6b3120a3130e9967f12da03114228012f82d8f7e735ed1"
Jul 25 00:22:09.941668 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:09.941592    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4297c0d6dfd92567eb6b3120a3130e9967f12da03114228012f82d8f7e735ed1"} err="failed to get container status \"4297c0d6dfd92567eb6b3120a3130e9967f12da03114228012f82d8f7e735ed1\": rpc error: code = NotFound desc = could not find container \"4297c0d6dfd92567eb6b3120a3130e9967f12da03114228012f82d8f7e735ed1\": container with ID starting with 4297c0d6dfd92567eb6b3120a3130e9967f12da03114228012f82d8f7e735ed1 not found: ID does not exist"
Jul 25 00:22:09.951981 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:09.951946    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-413844-549c74bd78-vnhdv"]
Jul 25 00:22:09.955369 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:09.955337    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-413844-549c74bd78-vnhdv"]
Jul 25 00:22:10.205333 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:10.205245    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a1b7baa-e633-4042-ac62-1ec8df05750a" path="/var/lib/kubelet/pods/0a1b7baa-e633-4042-ac62-1ec8df05750a/volumes"
Jul 25 00:22:10.205566 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:10.205552    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e921ad2-25d8-4708-8fe7-de17be837bbb" path="/var/lib/kubelet/pods/7e921ad2-25d8-4708-8fe7-de17be837bbb/volumes"
Jul 25 00:22:12.954572 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:12.954510    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-413844-slksp"]
Jul 25 00:22:12.956830 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:12.956803    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-413844-slksp"]
Jul 25 00:22:12.960170 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:12.960143    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:22:13.398933 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:13.398897    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:22:14.205154 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:14.205118    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86f2b971-dcfc-4acb-a096-03f2232f34e7" path="/var/lib/kubelet/pods/86f2b971-dcfc-4acb-a096-03f2232f34e7/volumes"
Jul 25 00:22:19.428506 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:19.428469    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:22:19.489699 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:19.489667    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:22:19.734570 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:19.734502    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:22:19.912456 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:19.912416    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:22:20.012785 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:20.012685    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:22:20.654814 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:20.654777    2558 scope.go:117] "RemoveContainer" containerID="c24f0a320f566ab1ffb4f98bf7571a63222f0f68a524848c2d4afe62f4db0ae6"
Jul 25 00:22:26.590056 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:26.590010    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-59bb55c686-v7p6f_e3d3d419-d577-41e6-952f-032abd72c77c/manager/0.log"
Jul 25 00:22:26.846960 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:26.846878    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-s48c7_1d634e98-014a-43c3-a840-efab61ea3a24/manager/1.log"
Jul 25 00:22:26.978787 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:26.978755    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-69778f9898-b4rzt_ffa2c446-0664-4bd8-be20-99c469570a69/copy-manifests/0.log"
Jul 25 00:22:26.986136 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:26.986104    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-69778f9898-b4rzt_ffa2c446-0664-4bd8-be20-99c469570a69/manager/0.log"
Jul 25 00:22:27.661369 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:27.661335    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-587494447-w87vs_8d30495b-e008-4ea6-951f-d1c97c6fbadf/maas-api/0.log"
Jul 25 00:22:27.906746 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:27.906714    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29748960-9n6jt_83c97281-d4ed-4be8-a187-776def664053/cleanup/0.log"
Jul 25 00:22:28.157105 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:28.157077    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-w9zls_73160574-b79c-4c1e-8f5f-8a11c814f3fe/postgres/0.log"
Jul 25 00:22:29.009482 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:29.009445    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75999hgt_bad54421-36a4-4e6f-be54-59d837f047c4/extract/0.log"
Jul 25 00:22:29.015960 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:29.015931    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75999hgt_bad54421-36a4-4e6f-be54-59d837f047c4/util/0.log"
Jul 25 00:22:29.022133 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:29.022108    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75999hgt_bad54421-36a4-4e6f-be54-59d837f047c4/pull/0.log"
Jul 25 00:22:29.139062 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:29.139030    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jp4jc_33ce1323-16e5-4514-8a41-26d0b876bb91/util/0.log"
Jul 25 00:22:29.145808 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:29.145780    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jp4jc_33ce1323-16e5-4514-8a41-26d0b876bb91/pull/0.log"
Jul 25 00:22:29.152416 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:29.152396    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jp4jc_33ce1323-16e5-4514-8a41-26d0b876bb91/extract/0.log"
Jul 25 00:22:29.279713 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:29.279626    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7394m4w_8db0cdbd-5d8c-4213-8aa2-cb0d25ffb425/util/0.log"
Jul 25 00:22:29.285986 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:29.285959    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7394m4w_8db0cdbd-5d8c-4213-8aa2-cb0d25ffb425/pull/0.log"
Jul 25 00:22:29.292212 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:29.292189    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7394m4w_8db0cdbd-5d8c-4213-8aa2-cb0d25ffb425/extract/0.log"
Jul 25 00:22:29.410364 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:29.410332    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kqwqt_650df1f9-9ba5-44bc-82a9-07cc28c371d2/util/0.log"
Jul 25 00:22:29.417011 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:29.416981    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kqwqt_650df1f9-9ba5-44bc-82a9-07cc28c371d2/pull/0.log"
Jul 25 00:22:29.423296 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:29.423275    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kqwqt_650df1f9-9ba5-44bc-82a9-07cc28c371d2/extract/0.log"
Jul 25 00:22:30.974655 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:30.974619    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-mn8nv_164984ed-ef05-4e6d-860b-38908ef71961/discovery/0.log"
Jul 25 00:22:31.620544 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:31.620494    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-8ddmv_180a4542-a43a-4d03-bd07-9380a02de282/payload-processing/0.log"
Jul 25 00:22:32.243270 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:32.243240    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-qtt6g_ebada3cb-11ae-457a-a140-b40306cbc4bf/main/0.log"
Jul 25 00:22:32.510903 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:32.510819    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbpwhk_7f5d3e73-8c38-4b2f-acd4-c91e986baaa6/main/0.log"
Jul 25 00:22:38.730459 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:38.730423    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:22:38.790107 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:38.790071    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log" limit=1024
Jul 25 00:22:41.596716 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:41.596687    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-rplqt_68864bf3-c771-4894-9f14-52ee9e8052f9/global-pull-secret-syncer/0.log"
Jul 25 00:22:41.649474 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:41.649441    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-hjb7w_1d4690c2-2cef-4ede-8db1-fe6d22add6e2/konnectivity-agent/0.log"
Jul 25 00:22:41.793547 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:41.793501    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-133.ec2.internal_b8b5e88d55d068e81c55bbf522837bfe/haproxy/0.log"
Jul 25 00:22:46.275487 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:46.275452    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75999hgt_bad54421-36a4-4e6f-be54-59d837f047c4/extract/0.log"
Jul 25 00:22:46.302208 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:46.302185    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75999hgt_bad54421-36a4-4e6f-be54-59d837f047c4/util/0.log"
Jul 25 00:22:46.322296 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:46.322266    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75999hgt_bad54421-36a4-4e6f-be54-59d837f047c4/pull/0.log"
Jul 25 00:22:46.345679 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:46.345648    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jp4jc_33ce1323-16e5-4514-8a41-26d0b876bb91/extract/0.log"
Jul 25 00:22:46.363803 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:46.363772    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jp4jc_33ce1323-16e5-4514-8a41-26d0b876bb91/util/0.log"
Jul 25 00:22:46.381409 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:46.381383    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jp4jc_33ce1323-16e5-4514-8a41-26d0b876bb91/pull/0.log"
Jul 25 00:22:46.408981 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:46.408956    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7394m4w_8db0cdbd-5d8c-4213-8aa2-cb0d25ffb425/extract/0.log"
Jul 25 00:22:46.430664 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:46.430639    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7394m4w_8db0cdbd-5d8c-4213-8aa2-cb0d25ffb425/util/0.log"
Jul 25 00:22:46.449844 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:46.449815    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed7394m4w_8db0cdbd-5d8c-4213-8aa2-cb0d25ffb425/pull/0.log"
Jul 25 00:22:46.507473 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:46.507438    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kqwqt_650df1f9-9ba5-44bc-82a9-07cc28c371d2/extract/0.log"
Jul 25 00:22:46.530494 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:46.530425    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kqwqt_650df1f9-9ba5-44bc-82a9-07cc28c371d2/util/0.log"
Jul 25 00:22:46.554134 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:46.554099    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kqwqt_650df1f9-9ba5-44bc-82a9-07cc28c371d2/pull/0.log"
Jul 25 00:22:49.291654 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:49.291620    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1dcf2959-0da6-410b-80cd-3408b640aa7d/alertmanager/0.log"
Jul 25 00:22:49.317702 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:49.317677    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1dcf2959-0da6-410b-80cd-3408b640aa7d/config-reloader/0.log"
Jul 25 00:22:49.339578 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:49.339506    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1dcf2959-0da6-410b-80cd-3408b640aa7d/kube-rbac-proxy-web/0.log"
Jul 25 00:22:49.359722 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:49.359694    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1dcf2959-0da6-410b-80cd-3408b640aa7d/kube-rbac-proxy/0.log"
Jul 25 00:22:49.377937 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:49.377906    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1dcf2959-0da6-410b-80cd-3408b640aa7d/kube-rbac-proxy-metric/0.log"
Jul 25 00:22:49.399406 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:49.399385    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1dcf2959-0da6-410b-80cd-3408b640aa7d/prom-label-proxy/0.log"
Jul 25 00:22:49.418386 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:49.418358    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1dcf2959-0da6-410b-80cd-3408b640aa7d/init-config-reloader/0.log"
Jul 25 00:22:49.496427 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:49.496394    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-9jpxt_18c4ebca-b677-451a-bb9a-a2acda06fd79/kube-state-metrics/0.log"
Jul 25 00:22:49.510718 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:49.510695    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-9jpxt_18c4ebca-b677-451a-bb9a-a2acda06fd79/kube-rbac-proxy-main/0.log"
Jul 25 00:22:49.528715 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:49.528690    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-9jpxt_18c4ebca-b677-451a-bb9a-a2acda06fd79/kube-rbac-proxy-self/0.log"
Jul 25 00:22:49.750308 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:49.750274    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rhrlf_39b86669-df55-45b0-ab05-f4a4df0926b1/node-exporter/0.log"
Jul 25 00:22:49.780917 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:49.780885    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rhrlf_39b86669-df55-45b0-ab05-f4a4df0926b1/kube-rbac-proxy/0.log"
Jul 25 00:22:49.798825 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:49.798801    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rhrlf_39b86669-df55-45b0-ab05-f4a4df0926b1/init-textfile/0.log"
Jul 25 00:22:50.142300 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:50.142210    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-z9hzm_7b3e8ae2-f6bb-4296-b607-9d144e32bb26/prometheus-operator/0.log"
Jul 25 00:22:50.155870 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:50.155839    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-z9hzm_7b3e8ae2-f6bb-4296-b607-9d144e32bb26/kube-rbac-proxy/0.log"
Jul 25 00:22:50.178363 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:50.178336    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7f8d78f54b-zvt4z_8cba79ff-e0ae-4e55-aca5-13f58f7b7541/prometheus-operator-admission-webhook/0.log"
Jul 25 00:22:51.503392 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:51.503356    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-t745v/perf-node-gather-daemonset-c2gnr"]
Jul 25 00:22:51.503900 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:51.503886    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a1b7baa-e633-4042-ac62-1ec8df05750a" containerName="maas-api"
Jul 25 00:22:51.503971 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:51.503905    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a1b7baa-e633-4042-ac62-1ec8df05750a" containerName="maas-api"
Jul 25 00:22:51.503971 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:51.503920    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86f2b971-dcfc-4acb-a096-03f2232f34e7" containerName="revoke-keys"
Jul 25 00:22:51.503971 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:51.503928    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="86f2b971-dcfc-4acb-a096-03f2232f34e7" containerName="revoke-keys"
Jul 25 00:22:51.504123 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:51.504029    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="86f2b971-dcfc-4acb-a096-03f2232f34e7" containerName="revoke-keys"
Jul 25 00:22:51.504123 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:51.504042    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a1b7baa-e633-4042-ac62-1ec8df05750a" containerName="maas-api"
Jul 25 00:22:51.507204 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:51.507184    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-t745v/perf-node-gather-daemonset-c2gnr"
Jul 25 00:22:51.509491 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:51.509473    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-t745v\"/\"default-dockercfg-f6z68\""
Jul 25 00:22:51.509491 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:51.509475    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-t745v\"/\"openshift-service-ca.crt\""
Jul 25 00:22:51.510439 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:51.510413    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-t745v\"/\"kube-root-ca.crt\""
Jul 25 00:22:51.513255 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:51.513180    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-t745v/perf-node-gather-daemonset-c2gnr"]
Jul 25 00:22:51.533057 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:51.533028    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9z78\" (UniqueName: \"kubernetes.io/projected/a06f9c0f-ba0f-4d9e-a2af-febb62ff67c5-kube-api-access-b9z78\") pod \"perf-node-gather-daemonset-c2gnr\" (UID: \"a06f9c0f-ba0f-4d9e-a2af-febb62ff67c5\") " pod="openshift-must-gather-t745v/perf-node-gather-daemonset-c2gnr"
Jul 25 00:22:51.533224 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:51.533070    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a06f9c0f-ba0f-4d9e-a2af-febb62ff67c5-sys\") pod \"perf-node-gather-daemonset-c2gnr\" (UID: \"a06f9c0f-ba0f-4d9e-a2af-febb62ff67c5\") " pod="openshift-must-gather-t745v/perf-node-gather-daemonset-c2gnr"
Jul 25 00:22:51.533224 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:51.533115    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a06f9c0f-ba0f-4d9e-a2af-febb62ff67c5-lib-modules\") pod \"perf-node-gather-daemonset-c2gnr\" (UID: \"a06f9c0f-ba0f-4d9e-a2af-febb62ff67c5\") " pod="openshift-must-gather-t745v/perf-node-gather-daemonset-c2gnr"
Jul 25 00:22:51.533224 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:51.533141    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a06f9c0f-ba0f-4d9e-a2af-febb62ff67c5-proc\") pod \"perf-node-gather-daemonset-c2gnr\" (UID: \"a06f9c0f-ba0f-4d9e-a2af-febb62ff67c5\") " pod="openshift-must-gather-t745v/perf-node-gather-daemonset-c2gnr"
Jul 25 00:22:51.533335 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:51.533230    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a06f9c0f-ba0f-4d9e-a2af-febb62ff67c5-podres\") pod \"perf-node-gather-daemonset-c2gnr\" (UID: \"a06f9c0f-ba0f-4d9e-a2af-febb62ff67c5\") " pod="openshift-must-gather-t745v/perf-node-gather-daemonset-c2gnr"
Jul 25 00:22:51.633925 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:51.633898    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a06f9c0f-ba0f-4d9e-a2af-febb62ff67c5-sys\") pod \"perf-node-gather-daemonset-c2gnr\" (UID: \"a06f9c0f-ba0f-4d9e-a2af-febb62ff67c5\") " pod="openshift-must-gather-t745v/perf-node-gather-daemonset-c2gnr"
Jul 25 00:22:51.633925 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:51.633927    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a06f9c0f-ba0f-4d9e-a2af-febb62ff67c5-lib-modules\") pod \"perf-node-gather-daemonset-c2gnr\" (UID: \"a06f9c0f-ba0f-4d9e-a2af-febb62ff67c5\") " pod="openshift-must-gather-t745v/perf-node-gather-daemonset-c2gnr"
Jul 25 00:22:51.634144 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:51.633952    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a06f9c0f-ba0f-4d9e-a2af-febb62ff67c5-proc\") pod \"perf-node-gather-daemonset-c2gnr\" (UID: \"a06f9c0f-ba0f-4d9e-a2af-febb62ff67c5\") " pod="openshift-must-gather-t745v/perf-node-gather-daemonset-c2gnr"
Jul 25 00:22:51.634144 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:51.634000    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a06f9c0f-ba0f-4d9e-a2af-febb62ff67c5-podres\") pod \"perf-node-gather-daemonset-c2gnr\" (UID: \"a06f9c0f-ba0f-4d9e-a2af-febb62ff67c5\") " pod="openshift-must-gather-t745v/perf-node-gather-daemonset-c2gnr"
Jul 25 00:22:51.634144 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:51.634024    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a06f9c0f-ba0f-4d9e-a2af-febb62ff67c5-sys\") pod \"perf-node-gather-daemonset-c2gnr\" (UID: \"a06f9c0f-ba0f-4d9e-a2af-febb62ff67c5\") " pod="openshift-must-gather-t745v/perf-node-gather-daemonset-c2gnr"
Jul 25 00:22:51.634144 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:51.634057    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9z78\" (UniqueName: \"kubernetes.io/projected/a06f9c0f-ba0f-4d9e-a2af-febb62ff67c5-kube-api-access-b9z78\") pod \"perf-node-gather-daemonset-c2gnr\" (UID: \"a06f9c0f-ba0f-4d9e-a2af-febb62ff67c5\") " pod="openshift-must-gather-t745v/perf-node-gather-daemonset-c2gnr"
Jul 25 00:22:51.634144 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:51.634096    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a06f9c0f-ba0f-4d9e-a2af-febb62ff67c5-proc\") pod \"perf-node-gather-daemonset-c2gnr\" (UID: \"a06f9c0f-ba0f-4d9e-a2af-febb62ff67c5\") " pod="openshift-must-gather-t745v/perf-node-gather-daemonset-c2gnr"
Jul 25 00:22:51.634144 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:51.634098    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a06f9c0f-ba0f-4d9e-a2af-febb62ff67c5-lib-modules\") pod \"perf-node-gather-daemonset-c2gnr\" (UID: \"a06f9c0f-ba0f-4d9e-a2af-febb62ff67c5\") " pod="openshift-must-gather-t745v/perf-node-gather-daemonset-c2gnr"
Jul 25 00:22:51.634144 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:51.634121    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a06f9c0f-ba0f-4d9e-a2af-febb62ff67c5-podres\") pod \"perf-node-gather-daemonset-c2gnr\" (UID: \"a06f9c0f-ba0f-4d9e-a2af-febb62ff67c5\") " pod="openshift-must-gather-t745v/perf-node-gather-daemonset-c2gnr"
Jul 25 00:22:51.641308 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:51.641280    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9z78\" (UniqueName: \"kubernetes.io/projected/a06f9c0f-ba0f-4d9e-a2af-febb62ff67c5-kube-api-access-b9z78\") pod \"perf-node-gather-daemonset-c2gnr\" (UID: \"a06f9c0f-ba0f-4d9e-a2af-febb62ff67c5\") " pod="openshift-must-gather-t745v/perf-node-gather-daemonset-c2gnr"
Jul 25 00:22:51.818423 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:51.818333    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-t745v/perf-node-gather-daemonset-c2gnr"
Jul 25 00:22:51.969745 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:51.969719    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-t745v/perf-node-gather-daemonset-c2gnr"]
Jul 25 00:22:51.971244 ip-10-0-142-133 kubenswrapper[2558]: W0725 00:22:51.971210    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda06f9c0f_ba0f_4d9e_a2af_febb62ff67c5.slice/crio-7c33c4316a0d93782201f2df37f09fe8c2c8871fa9ee5f0609a1d4ac0ef1f45b WatchSource:0}: Error finding container 7c33c4316a0d93782201f2df37f09fe8c2c8871fa9ee5f0609a1d4ac0ef1f45b: Status 404 returned error can't find the container with id 7c33c4316a0d93782201f2df37f09fe8c2c8871fa9ee5f0609a1d4ac0ef1f45b
Jul 25 00:22:52.155628 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:52.155543    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-t745v/perf-node-gather-daemonset-c2gnr" event={"ID":"a06f9c0f-ba0f-4d9e-a2af-febb62ff67c5","Type":"ContainerStarted","Data":"42b093a448c038dd159f6e9413dda67ba84f2efff8ddc4f0e2aa694c702861c8"}
Jul 25 00:22:52.155628 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:52.155601    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-t745v/perf-node-gather-daemonset-c2gnr" event={"ID":"a06f9c0f-ba0f-4d9e-a2af-febb62ff67c5","Type":"ContainerStarted","Data":"7c33c4316a0d93782201f2df37f09fe8c2c8871fa9ee5f0609a1d4ac0ef1f45b"}
Jul 25 00:22:52.155628 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:52.155630    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-t745v/perf-node-gather-daemonset-c2gnr"
Jul 25 00:22:52.171901 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:52.171850    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-t745v/perf-node-gather-daemonset-c2gnr" podStartSLOduration=1.171834958 podStartE2EDuration="1.171834958s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-25 00:22:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 00:22:52.170225921 +0000 UTC m=+3212.556656648" watchObservedRunningTime="2026-07-25 00:22:52.171834958 +0000 UTC m=+3212.558265699"
Jul 25 00:22:54.260611 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:54.260581    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jcc9s_d3b9e67d-4964-4049-975a-464a9fe43f23/dns/0.log"
Jul 25 00:22:54.280022 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:54.279993    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jcc9s_d3b9e67d-4964-4049-975a-464a9fe43f23/kube-rbac-proxy/0.log"
Jul 25 00:22:54.362751 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:54.362720    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7f7qw_3504a0bb-6580-4187-8f2d-7f71494675e4/dns-node-resolver/0.log"
Jul 25 00:22:54.930728 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:54.930694    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-544bc854f6-wqshw_7d4cc02f-cfdb-4f1e-aef4-5477808977aa/registry/0.log"
Jul 25 00:22:54.981551 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:54.981511    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-x8chk_77f6315f-d3d1-47ea-a83d-2b036ac39189/node-ca/0.log"
Jul 25 00:22:56.181139 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:56.181110    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-mn8nv_164984ed-ef05-4e6d-860b-38908ef71961/discovery/0.log"
Jul 25 00:22:56.380996 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:56.380956    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-8ddmv_180a4542-a43a-4d03-bd07-9380a02de282/payload-processing/0.log"
Jul 25 00:22:57.206109 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:57.206080    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-lrp6m_a1fb734a-d36c-4a41-a7f3-281f7cbacb5e/serve-healthcheck-canary/0.log"
Jul 25 00:22:57.827974 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:57.827934    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-n9sjh_b5d56d9a-1eab-409c-9c69-5939576e4836/kube-rbac-proxy/0.log"
Jul 25 00:22:57.847861 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:57.847831    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-n9sjh_b5d56d9a-1eab-409c-9c69-5939576e4836/exporter/0.log"
Jul 25 00:22:57.869443 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:57.869414    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-n9sjh_b5d56d9a-1eab-409c-9c69-5939576e4836/extractor/0.log"
Jul 25 00:22:58.170344 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:22:58.170270    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-t745v/perf-node-gather-daemonset-c2gnr"
Jul 25 00:23:00.676369 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:23:00.676246    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-59bb55c686-v7p6f_e3d3d419-d577-41e6-952f-032abd72c77c/manager/0.log"
Jul 25 00:23:00.803210 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:23:00.803174    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-s48c7_1d634e98-014a-43c3-a840-efab61ea3a24/manager/1.log"
Jul 25 00:23:00.813767 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:23:00.813732    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-s48c7_1d634e98-014a-43c3-a840-efab61ea3a24/manager/0.log"
Jul 25 00:23:00.838066 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:23:00.838033    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-69778f9898-b4rzt_ffa2c446-0664-4bd8-be20-99c469570a69/manager/0.log"
Jul 25 00:23:00.858064 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:23:00.858034    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-69778f9898-b4rzt_ffa2c446-0664-4bd8-be20-99c469570a69/copy-manifests/0.log"
Jul 25 00:23:02.307587 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:23:02.307553    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-54fc5fd468-qpp94_7ea2904c-c15e-4169-bbdd-bd4f3bf6956d/manager/0.log"
Jul 25 00:23:02.331935 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:23:02.331896    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-kkhvt_6696c8f5-c130-4c9d-a1be-b7a12be36a83/openshift-lws-operator/0.log"
Jul 25 00:23:02.449790 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:23:02.449754    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-mn8nv_164984ed-ef05-4e6d-860b-38908ef71961/discovery/0.log"
Jul 25 00:23:02.600116 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:23:02.600029    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-8ddmv_180a4542-a43a-4d03-bd07-9380a02de282/payload-processing/0.log"
Jul 25 00:23:04.209005 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:23:04.208975    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-mn8nv_164984ed-ef05-4e6d-860b-38908ef71961/discovery/0.log"
Jul 25 00:23:04.369442 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:23:04.369416    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-8ddmv_180a4542-a43a-4d03-bd07-9380a02de282/payload-processing/0.log"
Jul 25 00:23:08.655505 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:23:08.655476    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pck26_03bb2978-3f29-4167-8b41-c257a2b3878d/kube-multus-additional-cni-plugins/0.log"
Jul 25 00:23:08.672799 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:23:08.672770    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pck26_03bb2978-3f29-4167-8b41-c257a2b3878d/egress-router-binary-copy/0.log"
Jul 25 00:23:08.689942 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:23:08.689921    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pck26_03bb2978-3f29-4167-8b41-c257a2b3878d/cni-plugins/0.log"
Jul 25 00:23:08.706498 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:23:08.706469    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pck26_03bb2978-3f29-4167-8b41-c257a2b3878d/bond-cni-plugin/0.log"
Jul 25 00:23:08.723621 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:23:08.723594    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pck26_03bb2978-3f29-4167-8b41-c257a2b3878d/routeoverride-cni/0.log"
Jul 25 00:23:08.742392 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:23:08.742372    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pck26_03bb2978-3f29-4167-8b41-c257a2b3878d/whereabouts-cni-bincopy/0.log"
Jul 25 00:23:08.759373 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:23:08.759350    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pck26_03bb2978-3f29-4167-8b41-c257a2b3878d/whereabouts-cni/0.log"
Jul 25 00:23:08.957177 ip-10-0-142-133 kubenswrapper[2558]: I0725 00:23:08.957083    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-rf4fp_5e81d551-3cb1-4707-998f-713e083fefe8/kube-multus/0.log"